Analysis
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135
136 137 138 139 140 141 142 143 144 145 146
Once the construct-level scheme is complete, the concepts can be aggregated into higher
level (construct) categories. While some loss of information will be experienced, the ease
of interpretation is greatly increased. Table 6 lists a sample of raw statements, the
corresponding concept and construct-level categorization.
Validation of Concepts
Once the coding scheme has been developed the concepts should be validated to ensure
reliability of the scheme. The coding scheme approach (benchmarking or theory-driven)
will determine the most appropriate method of validation. When using the benchmarking
approach validation of the coding scheme with individuals who have domain expertise
but are not involved in a prior portion of the study is helpful. Validation can be
accomplished in multiple ways (e.g., card sort, electronic card sort). With a card sort, the
participants are provided the constructs and index cards with each concept. The
reliability between the participants is calculated by measuring the level of agreement on
the card sort. With the electronic card sort, participants are given an electronic spreadsheet
with all of the concepts listed on one sheet and the statements on another sheet.
The expert raters sort the concepts into the constructs. The results of each card sort are
compared to determine reliability of the coding scheme.
When using the theory-driven approach, the coding scheme should be validated against
the existing theoretical framework. When using the theory-driven approach validation
of the coding scheme can be accomplished using researchers who have knowledge of
the theoretical framework but are not involved in a prior portion of the study. The
researchers can validate the scheme by comparing the coding scheme with the theoretical
framework. Any discrepancies should be resolved through discussion.
Step 4: Recast “Raw” Maps into Revealed Causal Maps
Once the classification scheme is completed, the causal statements for each participant
are placed into the appropriate concept (and construct level) categories. The result is a
concept (and construct level) causal map for each participant. See Figures 4 and 5 for
concept and construct level maps respectively.
Figure 4. Concept level causal map
Method
Object
OO Development