Data Collection and Analysis Key Events Schedule

К оглавлению
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 
136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 
153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 
170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 
187 188 189 190 191 192 

What must be done? Who will do the activity? When is the activity due?

Develop evaluation Senior Communications 2 weeks before kick-off

communications packet Specialist of the coaching initiative

Gain approval for all Initiative Manager (Jacqui) 2 weeks before kick-off

evaluation tools and timing of the coaching initiative

Administer Questionnaire A Evaluator (Michael) 6 weeks after coaching

initiated

Analyze Questionnaire A Evaluator (Michael) 8 weeks after coaching

data; prepare report with initiated

recommendations

Share report with Initiative manager (Jacqui) 9 weeks after coaching

recommendations to the initiated

initiative sponsor and

Leadership Advisory Board

Etc.

evaluation rests on one method of evaluation.Credibility of the

evaluation is a critical issue that is dealt with in more detail in

the last section of this chapter.

_ Sources of data. Coaching clients were to be interviewed to

produce the main body of data. These data include the clients’

initial reaction to their coaching experience, what they learned

from their coaching, how they applied what they learned to

their workplace, and how these actions produced value for the

business.

_ Data collection tools. Two data collection tools were required for

the evaluation. Questionnaire A (Figure 12.1 located in the

Evaluation Toolkit section of this chapter) is intended to

capture reaction and learning data from the coaching clients.

Interview Guide B (Figure 12.2 and also in this next section)

captures application and business impact data. The questionnaire

and interview guide, when combined, collect all of the

data required for an ROI analysis. This interview guide offers

a step-by-step process to successfully demonstrate ROI.

Readers are encouraged to review these documents and the

specifications that follow in the Evaluation Toolkit section.

Later we will see how these data collection tools were utilized

at OptiCom.

_ Analysis approach. Jacqui and Michael planned to administer

Questionnaire A via e-mail to each coaching client after his or

her fourth coaching session, which occurred about six weeks

into the coaching initiative. Clients were to receive the e-mail

message, complete the questionnaire, and then e-mail it back

to the evaluator. These data represent the early returns of the

coaching initiative: Are the clients responding well to their

coaching? Is rapport being established? Is the coaching being

conducted effectively? The answers to these questions are critical

to understand if any corrective actions are needed. Later on

we will see how OptiCom used this survey to spot and fix a

potential trouble point. This questionnaire also sheds light on

the initial learning experiences of the coaching clients: understanding

how to be more effective as a leader, engaging work

teams more effectively, and improving communication skills.

These and subsequent learning experiences form the foundation

for making significant personal and professional changes.

Interview Guide B was planned to be administered by the

evaluator, Michael, with each coaching client in a personal

interview setting about two months after completion of the

coaching. The external evaluator brings objectivity and privacy

to the data collection with a certain measure of credibility.

Coaching clients are naturally sensitive about their coaching

experience and the changes they would like to make, both personally

and professionally. Clients more readily open up to a

third-party evaluator who ensures that their individual data

will not be revealed to the sponsoring company. Only grouped

data were presented.

The two-month time lag for administering Interview Guide

B is important. An appropriate gestation time is needed for the

impact of coaching to be felt in the organization. Later on we

will look at a specific example of this kind of impact. Suffice it

to say at this point that leaders who make significant changes

in their behavior will likely have significant impact in the organization.

This impact takes time to manifest in the form of

monetary benefits. Generally, the higher up the leader is in the

organization, the longer this time frame becomes. In other

words, those leaders who are higher in the organization will

take actions that have greater strategic value and it is the strategic

nature of these actions that take longer to have impact on

the business. There are no hard-and-fast rules for timing the

data collection. Common sense is the best guide. Just ask yourself:

Has a reasonable amount of time elapsed for the leader’s

behavior changes to have impacted the organization?

_ Key events schedule. The last piece of the Data Collection and

Analysis Plan is the key events schedule. These key events refer

to everything that must be done to successfully execute the

plan: communicating the purpose for the evaluation, gaining

approval for the evaluation tools and approach, conducting the

data collection, preparing reports, and other activities. Table

Demonstrating the ROI of Coaching 207

12.1 presents a few of these activities. Each activity is described

in terms of what must be done, who will do it and when the

activity must be done. Ideally, these evaluation activities are

integrated into the overall plan for the coaching initiative and

not kept as a stand-alone plan. Having one, integrated project

plan enables the manager of the coaching initiative to more

easily make the connections of coaching activities and evaluation

activities.

The next section of this chapter presents the two evaluation tools

with detailed specifications. In the subsequent section, we learn how

OptiCom used these tools to evaluate the coaching initiative.

ROI Evaluation Toolkit

Questionnaire A: Reaction and Learning

Questionnaire A: Reaction and Learning: Specifications