Coding Scheme

К оглавлению
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 
136 137 138 139 140 141 142 143 144 145 146 

Identification of the patterns and frequencies of the construct connections is made with

the revealed causal mapping technique, resulting in a theoretical structure that is more

purely elicited from the data, not from predetermined biases (Robinson, 1950). This is

accomplished through the procedure of developing an initial concept-level coding

scheme, identifying the major evoked constructs, and organizing the concepts and

constructs.

Each causal statement is analyzed to identify specific concepts, and as new concepts are

elicited from the transcripts, a complete list is generated. This first part of the coding

process should be done by the members of the interview team when respondents are first

interviewed face-to-face. The interviewers are best able to do the initial coding since they

were in the room with the respondents and can interpret the interview transcripts with

knowledge of the body language, demeanor, and other characteristics of the respondent.

If the interviews are performed by telephone, it becomes less critical that the interviewers

do the initial coding, although it is still preferred. The number of transcripts chosen to

develop the initial coding scheme will vary both by number of respondents and the point

of redundancy in the study, which is reached when the number of constructs from

interviews converge and no new constructs are found. There is no specific percentage

of respondents that drives how many transcripts are used to form the initial coding

scheme. Rather, the initial coders will continue to do comparisons for inter-rater reliability

until an acceptable level of agreement is reached. The level of agreement between the

researchers can be measured using Kendall’s coefficient of concordance (Siegel, 1956).

The reliability of this process is improved through constant checks of inter-rater

concordance.

For the five studies I have done using this methodology in a discovery context, all with

fewer than 100 respondents, acceptable concordance was reached by having multiple

researchers (in my experience, two to four) each code ten spreadsheets (one spreadsheet

per interview transcript) containing the raw causal statements from the transcripts. This

should be done individually by each researcher without consultation from the others. No

formal coding scheme should be used during this phase of the process. This is done

deliberately to let concepts emerge as they are reported by the respondents. Whatever

the initial coders believe the cause or effect represents is written down as a code in one

to three words. Then the initial coders exchange transcripts and recode each others’,

checking for inconsistencies. The inconsistencies are worked out by going back to the

original transcripts and reading the cause or effect statement in context. This process

continues and is refined until all of the transcripts are coded. If ten spreadsheets do not

result in an acceptable coefficient of concordance, the process must be continued until

concordance. For example, in the IT Personnel Transition Study (Rice & Nelson, 2003)

the final coefficient of concordance was 0.75.

Coding Scheme Granularity

From this iterative process a first cut, an initial coding guide is developed that sorts the

causal statement codes into three levels of granularity: concepts, categories, and

constructs. A concept is the actual idea or information embodied in the statement. A

category is a grouping of related concepts that occur at a similar level of granularity (e.g.,

individual, group, organization). Constructs are the highest level of abstraction and

consist of related categories. Once the coding scheme has been determined, the rest of

the spreadsheets are coded, again by at least two individuals who cross-check each

other’s work to maintain reliability. Through trial and error (Nelson, Nadkarni, Narayanan,

& Ghods, 2000), we found that having an additional level of granularity — categories —

allowed us to group similar concepts that were revealed at different levels of analysis.

We call this final guide a construct/concept table, with the acknowledgement that the

“category” column of the table sorts out the levels of analysis for each construct.

For example, in the Information Technology Personnel Transition study (Rice & Nelson,

2003) we found that there were some categories that significantly impacted IT personnel

in different ways. Within the Environment construct at the individual level, concepts that

influenced the individual in the workplace had a different impact on transition than

concepts that concern home and family life. Therefore, these concepts were put in

different categories. Concepts such as autonomy, responsibility, flexibility and availability

of opportunities were put in the category of Job Environment (organization level).

Concepts such as family, mental health, and age were put in the category that was

labeled Individual Environmental factors (individual level). As can be seen, these are

both individual-level concepts, but their impact on an individual’s ability to transition

Construct Environment

Category Job Environment Individual Environmental Factors

Autonomy

Flexibility

Independence

Job Quality

Responsibility

Home

Family

Job Tenure

Mental Health

Time

Concept

Figure 1. Sample coding scheme hierarchy

is considerably different. These differences require the concepts to be categorized into

different groupings even though they are still part of the same construct — in this case,

the Environment. See Figure 1 for a representation of these relationships.