Measure Definition

Comprehensiveness Number of concepts included in the map (Carley and Palmquist, 1992); applicable

at the overall map level

Density Ratio of links between a concept and the total concepts in the map (Carley and

Palmquist, 1992); applicable at the overall map level

Centrality Reflects how central or involved the concept/construct is to the map; a ratio of the

aggregate of linkages involving the concept/construct divided by the total linkages

in the map (Knoke and Kuklinski, 1982); applicable at the concept/construct level

based on the research context used. The measures listed here are not exhaustive, but

exemplars for researchers to contemplate using in their causal mapping endeavors. Table

9 lists the measures and a brief description.

Comprehensiveness is a characteristic of the overall map and is a measure of the number

of concepts in the map (Carley & Palmquist, 1992). This measure can be used for

comparisons between maps. The more comprehensive the map, the more complex the

cognition (Nelson et al., 2000). Density is a characteristic of the overall maps and is a

measure of how connected the concepts in the map are. Density is a proportion that is

calculated as the number of linkages between the concepts divided by the number of

Figure 7. Sample density measure

Method

OO Development

Identifying

Objects

Object

Participant A

number of links in map

number of concepts in map

Density =

Density-1A =

3

4

= .75

Density-1B =

6

4

= 1.50

Participant B

Method

OO Development

Identifying

Objects

Object

Note: These measures can be used at both the concept and construct level

concepts in the map. There is another density measure that has been used which is a

proportion that is calculated as the number of all linkages occurring in the matrix divided

by the number of all possible linkages (Knoke & Kuklinski, 1982). In both cases, the higher

the ratio, the denser the map and the higher level of cognitive complexity (Nadkarni, 2003).

Figure 7 provides a sample density calculation.

Centrality is a measure used for the individual concepts/constructs within a map. It is

a measure of how central or involved the concept/construct is to the map, and reflects

the degree of hierarchy characterizing the map. Centrality is a ratio of the aggregate of

linkages involving the concept/construct divided by the total linkages in the matrix

(Knoke & Kuklinski, 1982). Figure 8 provides a sample centrality calculation.

As stated previously, the structural analysis of causal maps differs for each of the

research contexts. In the discovery context, the purpose of causal mapping is to identify

patterns and describe aspects of the phenomenon. In an evocative setting, the goal is

to develop domain specific theory. In theory testing the goal is to confirm/dispute/

expand existing theory. Lastly, in an intervention setting, the goal is to create consensus

around a course of action or issue at hand. With each research setting a different analysis

protocol is appropriate. In a discovery setting, the analysis would take on the form of

description, relying heavily on the content aspects and identifying which concepts are

linked. In an evocative setting, the analysis would be concerned with both the content

and the structural aspects. It is through understanding the linkages between the

1

-

1

0

0

1 2 3 4 Out-degree

In-degree

Identifying Objects

OO Development

Method

Object

1

0

0

-

1

0

0

0

0

- 1 2

0 0

- 1

1 3

0 0

In-degree + Out-degree

Total Number of Linkages

CObject = 0 + 2

3

= .66

CMethod = 0 + 1

3

= .33

Method

OO Development

Identifying

Objects

Object

Participant A

Centrality (C) =

Figure 8. Sample concept centrality measure

concepts (constructs) that theory can be developed. Basic measures such as density and

centrality may be used to develop theory. In a hypothesis testing context, the measures

would need to be much more robust and cover many aspects of the map’s structure.

Reporting Results

While the standards in reporting CM results have not yet evolved, there are some key

items that I have found reviewers will be looking for in your results. The first item is the

sample design. Reviewers will want to know what sampling frame was used, was the

sample population appropriate and was the sample adequate (point of redundancy). The

second item that should be included is a discussion of the coding process. Reviewers

will want to know what coding process was used as well as the reliability and validity of

the process. One thing to keep in mind is that most IS reviewers are not yet familiar with

the CM method. As with other research methods, you must prove that the research is well

designed and rigorously undertaken. Similar to other qualitative methods, examples and

quotes from the study are key to convincing the reviewer that what you report is an

accurate (and rich) representation of the data. Over time, the need for clearly articulating

the steps involved in CM research will diminish, but for now researchers may want you

to include the steps provided in this chapter in an appendix to substantiate the CM

process.

Summary of Key Decision Points

There are several issues discussed in this chapter that a researcher will want to consider

when designing a CM study. There are nine key decision points that will be summarized

here. See Table 10 for a listing of these decision points.

1. The first decision point is the selection of the research context (e.g., evocative,

hypothesis testing). The research context should be selected based on the fit with

the phenomenon under study and the research questions being addressed.

2. The second decision point is in the choice of data collection method (TBCM or

IECM). This decision should be driven by which method is appropriate for the

research question and the research context.

3. The third decision point is in the choice of which sampling method (e.g., random,

snowball, exhaustive) to use. The sampling method should be chosen based on the

data collection method (IECM versus TBCM) and in the IECM method also the

sample (participants versus experts).

4. The fourth decision point is with regard to the reliability of the causal statement

identification procedure. The level of agreement between the researchers should

be at least 0.75 to have an acceptable level of reliability. A reliability less than 0.75

indicates that the procedure is not robust enough for research purposes, and a

modified identification procedure will need to be developed.

5. The fifth decision point is in the choice of coding scheme development method

(benchmarking and theory-driven). This choice is primarily dependent on the

research context of the study (discovery versus hypothesis testing).

6. The sixth decision point is with regard to validating the concepts. Once the coding

scheme has been developed the concepts should be validated to ensure reliability

of the scheme. The coding scheme approach (benchmarking or theory-driven) will

determine the most appropriate method of validation.

7. The seventh decision point deals with the validation of the maps. The validation

method is determined by the data collection method (IECM or TBCM). For IECMs

one source of validation is a “member check,” whereas using TBCMs validation is

often accomplished via triangulation with other sources.

8. The eighth decision point deals with representation. Causal maps may be represented

via diagram or matrix. The only limitation on the choice of representation

may be in complexity of the map. The more complex the map, the more difficult to

represent and analyze via diagrammatic methods.

9. The last decision point deals with the analysis of the maps. When analyzing a

causal map the researcher should address both the content and structural aspects

of the map. Within the structural analysis there are many possible measures that

can be utilized to operationalize the structural properties of the causal maps (e.g.,

centrality). The applicability of each measure is based on the research context used

and research questions addressed.

Table 10. Key decision points

Decision Point Description

Research Context The research context (e.g., evocative, hypothesis testing) should be selected

based on the fit with the phenomenon under study and the research

questions being addressed.

Data Collection Method Choice of method (TBCM or IECM) is dependent on the research question

and the research context.

Choice of Sampling Method Choice of method (e.g., random, snowball, exhaustive) is dependent on data

collection method and research context.

Causal Statement Identification

Reliability

If reliability >= 0.80, then proceed with the study, if <= 0.80 the procedure

will need to be modified.

Coding Scheme Choice of method (benchmarking and theory-driven) is dependent on the

research context of the study.

Concept Validation The coding scheme approach (benchmarking or theory-driven) will

determine the most appropriate method of validation.

Map Validation The validation method is determined by the data collection method (IECM

or TBCM). For IECMs a source of validation is a ‘member check’. For

TBCMs a source of validation is via triangulation with other sources.

Representation Causal maps may be represented in two main forms: via diagram or matrix.

The choice of representation is only limited by the complexity of the maps.

Analysis Many measures can be utilized to operationalize the structural properties of

the causal maps (e.g., centrality). The applicability of each measure is based

on the research context used.