Deriving Revealed Causal Maps
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135
136 137 138 139 140 141 142 143 144 145 146
The causal mapping process used for this study is consistent with the steps provided
in Chapter II. We have expanded on those issues not previously addressed or unique to
this study.
Demographic Mean SD
Age 39.29 7.35
Gender (% male) 96.3% -
Years in IT 15.42 6.61
Years with current organization 3.35 2.91
Years of procedural experience 10.40 5.27
Number of procedural projects participated in 20.5 16.73
Years of object-oriented experience 4.73 1.54
Number of object-oriented projects participated in 6.79 9.46
Table 3 — Study I. OO interview demographic, N=24
Table 3 — Study II. Survey demographic, N= 142
Demographic Mean SD
Age (1 < 21, 2 = 21-30, 3=31-40, 4=41-50, 5=51-60, 6 > 60) 3.77 .870
Gender (% male) 85.9% -
Years in IT 19.98 8.90
Years with current organization 10.89 8.63
Years of procedural experience 16.52 7.76
Number of procedural projects participated in 32.01 34.86
Years of object-oriented experience 3.46 3.07
Number of object-oriented projects participated in 2.91 2.61
• Step 1: Identify Causal Statements. Due to the cognitive nature of this study it was
determined that in addition to explicit causal statements, implicit causal statements
should also be recorded. To establish the reliability of the identification procedure,
interview texts were coded by the researchers and one of two additional raters. The
raters were deemed appropriate to identify the causal statements because of their
familiarity with the technique, the domain under study, and they were not participants
in any portion of the study. There were two rounds of coding that covered
six OO interview texts chosen at random from the interviews. The reliability between
the researchers was calculated by measuring the level of agreement on terms and
linkages. The level of agreement between the researchers averaged 0.80, suggesting
an acceptable level of reliability.
• Step 2: Construct Raw Causal Maps. In the second step, the causal statements
identified in the first step were separated into “causes” and “effects” to construct
the “raw causal maps.”
• Step 3: Develop Coding Scheme. The relevant concepts were identified from the
statements. A software development expert reviewed the statements and independently
placed them into conceptual categories. The level of agreement between the
researcher(s) and rater averaged 0.87 and a total of 19 OO concepts were identified.
Once the conceptual level scheme was developed, the statements were placed into
the appropriate concepts.
Tying emergent concepts to established literature has been recommended to develop
categories that are distinct and uniform in breadth and level of abstraction (Carley &
Palmquist, 1992; Fahey & Narayanan, 1989; Priem, 1994). To do this, we assessed the
degree of congruence of the three theoretical frameworks identified in the literature
review to the evoked concepts.
Three object-oriented software development experts were asked to assess the congruence
of the emergent concepts with each theoretical framework. The developers were
provided the constructs from each theoretical framework (see Table 2) and a set of index
cards containing each of the 19 concepts, with a definition of the concept provided on
the back of the card for their reference. Each developer performed a card sort according
to each of the three theoretical frameworks. The card sort was conducted using a different
framework order for each developer. The average level of agreement among the developers
was 0.52 for Framework I, 0.33 for Framework II, and 0.70 for Framework III. Based
on the results of the card sort, we chose theoretical Framework III as the appropriate
construct level classification scheme. The correspondence between the 19 evoked
concepts and constructs is presented in Table 4.
• Step 4: Recast the “raw” maps into Revealed Causal Maps. Once the classification
scheme was completed, the causal statements for each respondent were placed into
the appropriate concept and construct level categories. The result is a concept and
construct level RCM for each respondent. The individual maps are then aggregated
(Axelrod, 1976; Bougon, Weick & Binkhorst, 1977; Nadkarni & Nah, 2003) into one
overall map.
• Step 5: Create Measures for the Maps. The analysis of the maps in this study was
based on past research in causal mapping (Bougon et al., 1977; Ford & Hegarty,
1984: Huff, 1990; Narayanan & Fahey, 1990; Nelson et al., 2000). The measures used
are borrowed from the social network analysis field (Knoke & Kuklinski, 1982) and
include the adjacency and reachability matrices, centrality and density measures
(see Chapter II for a detailed discussion of the measures).
Study II: Survey
• Data source. We developed an instrument to capture the concepts emerging from
the interviews. As a starting point, five statements per concept were selected for
inclusion in the instrument based on content and clarity. Internal validity and
construct validity were addressed through a pretest sort (Anderson & Gerbing,
1991). Based on the results of the pretest sort statements were re-worded or deleted.
The pilot instrument was developed from the remaining list of statements. The pilot
instrument was given to 31 software developers. We assessed scale internal
consistency reliability using Cronbach’s alpha (Huck, 2000). Correlations between
variables were analyzed, and questions were deleted or re-worded for clarity. The
final instrument covered 19 object-oriented concepts5.
Table 4. Construct and concept detail for theoretical Framework III
Construct Concept
Structure Abstraction
Attribute
Class
Encapsulation
Information Hiding
Inheritance
Instantiation
Object
Behavior Collaboration
Message Passing
Method
Polymorphism
Relationship
OO Modeling/Analysis Identifying Objects
Noun-Verb Analysis
Object Model
OO Development Concepts Patterns
Layer
OO Development
• Sample. The validated instrument was then given to a large sample of software
developers. The sample was drawn from organizations selected based on their
identification of available “software developers” and their willingness to participate.
Thirty-three organizations of diverse industries (e.g., telecommunications,
manufacturing, aerospace and financial services) provided access to their software
developers. Study respondents were chosen based on a key-informant method
(Bagozzi, Yi & Phillips, 1991; Seidler, 1974). Participants in this phase of the study
were different than the respondents used in the previous phase of the project. A
total of 177 responses were originally recorded with 35 being eliminated due to lack
of software development experience, duplicate responses, or other problems with
the data, leaving a sample of 142 respondents. A profile of the Study II (survey)
sample is presented in Table 3.
• Factor analysis. We used an exploratory principal components factor analysis
with varimax rotation to reduce the variables to a usable set of constructs for the
OO software development expertise framework. Past research has provided guidelines
for the minimum sample size needed to conduct factor analysis. Some have
suggested the ratio of sample size to number of variables as a criterion: the
recommendations range from 2:1 to 20:1 (e.g., Bryant & Yarnold, 1995). Others have
suggested using a minimum sample size as the criterion. For example, Lawley and
Construct Alpha Concept
Nesting 0.806 Inheritance
OO Development
Polymorphism
Object Characteristics 0.918 Attributes
Class
Encapsulation
Instantiation
Method
Behavior II 0.818 Collaboration
Message Passing
Relationship
OO Modeling/Analysis II 0.872 Identifying Objects
Noun-Verb Analysis
Object
OO Development Concepts II 0.883 Abstraction
Patterns
Information Hiding
Layer
Object Model
Table 5. Factor analysis for Framework IV
Maxwell (1971) suggest that there should be 51 more cases than the number of
variables. In their 1988 study, Guadagnoli and Velicer found that absolute sample
size was more important than functions of sample size in determining stable
solutions and recommend 100 to 200 observations. Our sample of 142 respondents
fell within the acceptable range for the analysis. The number of respondents
represents a sufficient sample to perform principal components analysis.
The resulting factor structure is summarized in Table 5, with the associated Cronbach’s
alpha reliability scores. As shown in the table, the object-oriented concepts loaded on
five factors that we named Nesting, Object Characteristics, Behavior II2, OO Modeling/
Analysis II, and OO Development Concepts II, using an eigen value >1.00 criterion.
Cumulatively, the factors accounted for 71% of variance in the sample. The Nesting
construct (a = 0.81) dealt with managing the complexity in the object-oriented mindset
(e.g., inheritance, polymorphism). Similarly, the Object Characteristics construct (a =
0.92) dealt with the packages that the basic concepts are put into (e.g., class, instantiation),
thus focusing on the development and functioning of an object. The Behavior II
construct (a = 0.82) consisted of the communication and relationships between sets of
objects (e.g., collaboration, message passing). The OO Modeling/Analysis II construct
(a = 0.87) consists of concepts that are the fundamental techniques for object-oriented
analysis (e.g., object, identifying things as objects). Finally, OO Development Concepts
II (a = 0.88) focused on how objects function within the larger system to make up an
object-oriented application (e.g., abstraction, layer), i.e., finding, modifying, and assembling
the classes and methods needed to support an entire application.
Results and Discussion
We present the results in three steps. First, we present the constructs revealed in the
interviews that are central to understanding OO software development expertise, highlighting
the characteristics of the revealed linkages between constructs in the aggregate
RCMs. Second, we summarize the results of the recasting of RCMs based on classification
suggested by the factor analysis of survey responses. Third, we provide an
assessment of the congruence between the original and the reconstructed RCMs.
Study I: Revealed Causal Maps
The aggregate level RCMs from the interview data represented OO software development
expertise as being constituted by four major constructs: Structure, Behavior, OO
Modeling/Analysis, and OO Development Concepts and is shown in Figure 2(a). The
overall density of the map is 1.75, indicating a relatively high interconnectedness. As
shown in Table 4, eight concepts comprise the Structure construct, which has a centrality
of 0.57, suggesting that it occupies a prominent role in OO software development. The
Behavior construct contains five concepts and the centrality is 0.57 suggesting a
prominent role in OO development. The OO Modeling/Analysis construct evoked in this
study contains three concepts and has a centrality of 0.43 suggesting a slightly lesser
role in OO software development. The fourth construct, OO Development contains three
concepts related to overall OO development. The centrality of the construct is 0.43,
suggesting a slightly lesser role in the OO approach. While the results are consistent with
those found by Sheetz and Tegarden (2001), this is the first study that has found the
importance of these additional concepts as components of OO. The data indicate that
the structure and behavior of objects are equally central to OO development, with
modeling and overall development slightly less central.
The reachability matrix of the aggregated construct level RCM is presented in Table 6.
It reveals that experts see OO Modeling/Analysis as a cause construct (all arrows from
the construct) and OO Development as an effect construct (all arrows into the construct).
The Structure and Behavior constructs are both cause and effect constructs. The highest
reachability occurs for the linkages in which OO Development Concepts is the effect
construct.