Measure Definition
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135
136 137 138 139 140 141 142 143 144 145 146
Comprehensiveness Number of concepts included in the map (Carley and Palmquist, 1992); applicable
at the overall map level
Density Ratio of links between a concept and the total concepts in the map (Carley and
Palmquist, 1992); applicable at the overall map level
Centrality Reflects how central or involved the concept/construct is to the map; a ratio of the
aggregate of linkages involving the concept/construct divided by the total linkages
in the map (Knoke and Kuklinski, 1982); applicable at the concept/construct level
based on the research context used. The measures listed here are not exhaustive, but
exemplars for researchers to contemplate using in their causal mapping endeavors. Table
9 lists the measures and a brief description.
Comprehensiveness is a characteristic of the overall map and is a measure of the number
of concepts in the map (Carley & Palmquist, 1992). This measure can be used for
comparisons between maps. The more comprehensive the map, the more complex the
cognition (Nelson et al., 2000). Density is a characteristic of the overall maps and is a
measure of how connected the concepts in the map are. Density is a proportion that is
calculated as the number of linkages between the concepts divided by the number of
Figure 7. Sample density measure
Method
OO Development
Identifying
Objects
Object
Participant A
number of links in map
number of concepts in map
Density =
Density-1A =
3
4
= .75
Density-1B =
6
4
= 1.50
Participant B
Method
OO Development
Identifying
Objects
Object
Note: These measures can be used at both the concept and construct level
concepts in the map. There is another density measure that has been used which is a
proportion that is calculated as the number of all linkages occurring in the matrix divided
by the number of all possible linkages (Knoke & Kuklinski, 1982). In both cases, the higher
the ratio, the denser the map and the higher level of cognitive complexity (Nadkarni, 2003).
Figure 7 provides a sample density calculation.
Centrality is a measure used for the individual concepts/constructs within a map. It is
a measure of how central or involved the concept/construct is to the map, and reflects
the degree of hierarchy characterizing the map. Centrality is a ratio of the aggregate of
linkages involving the concept/construct divided by the total linkages in the matrix
(Knoke & Kuklinski, 1982). Figure 8 provides a sample centrality calculation.
As stated previously, the structural analysis of causal maps differs for each of the
research contexts. In the discovery context, the purpose of causal mapping is to identify
patterns and describe aspects of the phenomenon. In an evocative setting, the goal is
to develop domain specific theory. In theory testing the goal is to confirm/dispute/
expand existing theory. Lastly, in an intervention setting, the goal is to create consensus
around a course of action or issue at hand. With each research setting a different analysis
protocol is appropriate. In a discovery setting, the analysis would take on the form of
description, relying heavily on the content aspects and identifying which concepts are
linked. In an evocative setting, the analysis would be concerned with both the content
and the structural aspects. It is through understanding the linkages between the
1
-
1
0
0
1 2 3 4 Out-degree
In-degree
Identifying Objects
OO Development
Method
Object
1
0
0
-
1
0
0
0
0
- 1 2
0 0
- 1
1 3
0 0
In-degree + Out-degree
Total Number of Linkages
CObject = 0 + 2
3
= .66
CMethod = 0 + 1
3
= .33
Method
OO Development
Identifying
Objects
Object
Participant A
Centrality (C) =
Figure 8. Sample concept centrality measure
concepts (constructs) that theory can be developed. Basic measures such as density and
centrality may be used to develop theory. In a hypothesis testing context, the measures
would need to be much more robust and cover many aspects of the map’s structure.
Reporting Results
While the standards in reporting CM results have not yet evolved, there are some key
items that I have found reviewers will be looking for in your results. The first item is the
sample design. Reviewers will want to know what sampling frame was used, was the
sample population appropriate and was the sample adequate (point of redundancy). The
second item that should be included is a discussion of the coding process. Reviewers
will want to know what coding process was used as well as the reliability and validity of
the process. One thing to keep in mind is that most IS reviewers are not yet familiar with
the CM method. As with other research methods, you must prove that the research is well
designed and rigorously undertaken. Similar to other qualitative methods, examples and
quotes from the study are key to convincing the reviewer that what you report is an
accurate (and rich) representation of the data. Over time, the need for clearly articulating
the steps involved in CM research will diminish, but for now researchers may want you
to include the steps provided in this chapter in an appendix to substantiate the CM
process.
Summary of Key Decision Points
There are several issues discussed in this chapter that a researcher will want to consider
when designing a CM study. There are nine key decision points that will be summarized
here. See Table 10 for a listing of these decision points.
1. The first decision point is the selection of the research context (e.g., evocative,
hypothesis testing). The research context should be selected based on the fit with
the phenomenon under study and the research questions being addressed.
2. The second decision point is in the choice of data collection method (TBCM or
IECM). This decision should be driven by which method is appropriate for the
research question and the research context.
3. The third decision point is in the choice of which sampling method (e.g., random,
snowball, exhaustive) to use. The sampling method should be chosen based on the
data collection method (IECM versus TBCM) and in the IECM method also the
sample (participants versus experts).
4. The fourth decision point is with regard to the reliability of the causal statement
identification procedure. The level of agreement between the researchers should
be at least 0.75 to have an acceptable level of reliability. A reliability less than 0.75
indicates that the procedure is not robust enough for research purposes, and a
modified identification procedure will need to be developed.
5. The fifth decision point is in the choice of coding scheme development method
(benchmarking and theory-driven). This choice is primarily dependent on the
research context of the study (discovery versus hypothesis testing).
6. The sixth decision point is with regard to validating the concepts. Once the coding
scheme has been developed the concepts should be validated to ensure reliability
of the scheme. The coding scheme approach (benchmarking or theory-driven) will
determine the most appropriate method of validation.
7. The seventh decision point deals with the validation of the maps. The validation
method is determined by the data collection method (IECM or TBCM). For IECMs
one source of validation is a “member check,” whereas using TBCMs validation is
often accomplished via triangulation with other sources.
8. The eighth decision point deals with representation. Causal maps may be represented
via diagram or matrix. The only limitation on the choice of representation
may be in complexity of the map. The more complex the map, the more difficult to
represent and analyze via diagrammatic methods.
9. The last decision point deals with the analysis of the maps. When analyzing a
causal map the researcher should address both the content and structural aspects
of the map. Within the structural analysis there are many possible measures that
can be utilized to operationalize the structural properties of the causal maps (e.g.,
centrality). The applicability of each measure is based on the research context used
and research questions addressed.
Table 10. Key decision points
Decision Point Description
Research Context The research context (e.g., evocative, hypothesis testing) should be selected
based on the fit with the phenomenon under study and the research
questions being addressed.
Data Collection Method Choice of method (TBCM or IECM) is dependent on the research question
and the research context.
Choice of Sampling Method Choice of method (e.g., random, snowball, exhaustive) is dependent on data
collection method and research context.
Causal Statement Identification
Reliability
If reliability >= 0.80, then proceed with the study, if <= 0.80 the procedure
will need to be modified.
Coding Scheme Choice of method (benchmarking and theory-driven) is dependent on the
research context of the study.
Concept Validation The coding scheme approach (benchmarking or theory-driven) will
determine the most appropriate method of validation.
Map Validation The validation method is determined by the data collection method (IECM
or TBCM). For IECMs a source of validation is a ‘member check’. For
TBCMs a source of validation is via triangulation with other sources.
Representation Causal maps may be represented in two main forms: via diagram or matrix.
The choice of representation is only limited by the complexity of the maps.
Analysis Many measures can be utilized to operationalize the structural properties of
the causal maps (e.g., centrality). The applicability of each measure is based
on the research context used.