Introduction
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135
136 137 138 139 140 141 142 143 144 145 146
Research on causal mapping has been an active area in information systems (IS) research.
Causal mapping has been applied in information systems requirements analysis (Montazemi
and Conrath, 1986) and for planning network services (Dutta, 2001). Boland, Tenkasi &
Teeni (1994) argue that causal mapping can be used to capture subjects’ perspectives
for use in decision making, system design, and other activities. Several information
systems have been designed to support causal mapping (Eden, 1989; Zhang, Wang &
King, 1994; Boland, Tenkasi & Teeni, 1994; Kwahk and Kim, 1999; Hong and Han, 2002).
Sheetz, Tegarden, Kozar & Zigurs (1994) proposed a group support system as an aid in
uncovering causal maps of users.
Developing and evaluating group or collective causal maps has been the subject of
several recent IS studies (Tegarden and Sheetz, 2003; Lee, Courtney & O’Keefe, 1992;
Vennix, 1996; Kwahk and Kim, 1999). Ackerman, Eden & Williams (1997), Massey and
Wallace (1996) and Vennix (1996) have maintained that collective maps can be used to
broaden problem solvers’ perspectives by taking alternative views into account in the
definition of a messy problem situation. In their view collective maps can be viewed as
a means to access multiple perspectives for problems with no definitive formulation. This
is a particularly important application, because several researchers (Checkland, 1981;
Courtney, 2001; Linstone, 1984; Mitroff and Linstone, 1993; Senge, 1990) have proposed
that systems-based multiple perspective approaches are required to deal with problems
in organizations and society today. Different perspectives are assumed to hold different
models, and it is through the juxtaposition and combination of models that perspectives
can be mediated. To gain the precision necessary to compare, contrast, and combine
multiple perspectives, it is necessary to build models of the situation. However, it is by
no means straightforward to develop group or collective maps. Tegarden and Sheetz
(2003) found that merging causal maps of individuals into a collective causal map has
been problematic and argued that the creation of collective causal maps is impractical for
many organizational situations.
The research reported in this chapter focuses on the use of modeling to mediate multiple
perspectives on problems through the development of group causal maps. We focus on
the comparison of existing modeling approaches that are capable of representing multiple
perspectives through collective maps. Our research questions are: What approaches are
available to formulate collective causal maps based on multiple perspectives? Are some
approaches superior to others?
This chapter has three objectives. First, it discusses and compares at a conceptual level
three methods of constructing collective causal maps. These three approaches represent
fundamental distinctions in methods of collective causal mapping and each has some
track record of success. Second, it reports the results of two studies conducted to
compare the three methods in terms of various criteria. The criteria include objective
measures such as map complexity, density, and map distance ratio and subjective
measures such as user perceptions of the adequacy of problem representation, solution
implications, stakeholder implications, and degree to which the collective maps capture
different perspectives. These studies test two hypotheses regarding map complexity and
the perceived utility of collective maps for the aggregate, congregate, and workshop
mapping procedures. The results of these studies should help illuminate the relative
strengths and weaknesses of the different approaches to building collective causal maps
and suggest guidelines for selecting the best procedure to fit the modeler’s situation.
This chapter starts with a conceptual comparison of three methods for constructing
collective maps. We then detail a research design for two studies that compare the three
methods. In Study 1 we built collective causal maps to facilitate problem formulation for
a sales problem in a Vietnamese company and in Study 2 we built collective maps to
support the development of a research model of the impacts of infrastructure and
infrastructure projects on a large urban area. In each study we empirically compare
models derived with the three methods on both objective and subjective measures.
Methods for Constructing a Collective
Map/Model
A map is an aggregation of “interrelated information” (O’Keefe and Nadel, 1978). Maps
help represent people’s perceptions about their environments (Weick and Bougon,
1986). A causal map consists of nodes and links (or arrows) that one may use to
understand a situation (Axelrod, 1976). Nodes stand for factors, labels, concepts, or
variables. Links represent relationships or associations. If causal relationships are used,
the maps are called causal maps. Most researchers (Eden, Jones & Sims, 1981; Hart, 1977)
do not differentiate causal maps from cognitive maps. However, Weick and Bougon
(1986) believe that the concept of a cognitive map is broader than a causal map as the
former may have other relationships than causal—such as “contiguity, proximity,
resemblance, and implication.” For consistency, we use the term causal map throughout
the chapter.
Causal maps were originally devised to elicit mental models for individuals (Axelrod,
1976; Eden, 1989). A number of researchers (Landfield-Smith, 1992; Bougon, 1992; Weick
and Bougon, 1986; Schneider and Andgelmar, 1993; Nicolini, 1999; Laukkanen, 1994;
Lant and Shapira, 2001) have extended the application of the concept to a group,
collective or organization. Three common methods found in the literature for constructing
collective mental models are aggregate mapping, congregate mapping, and workshop
mapping. A conceptual comparison of these approaches is provided in Table 1.
In the aggregate mapping approach, the focus is on representing all individual maps
as fully as possible in the collective map. All labels and links from each individual causal
map are included in the collective map. As a result, the aggregate map may become quite
complex. The aggregate approach does not emphasize the causal loops in the collective
map. For this reason, it may not provide a full representation of social systems, because
typically social systems consist of many actors with significantly diverse viewpoints.
Aggregating is also referred to as “merging” or “overlaying” (Eden, 1989; Eden, et al.,
1983), or the “structural/relational join” operation (Lee et al., 1992), or “combination”
(Kwahk and Kim, 1999).
The congregate mapping approach centers on the identification of key causal loops that
drive system dynamics (Bougon, 1992). The study of causal loops or cycles in causal
mapping and causal modeling has been emphasized by systems dynamics researchers
(Bougon et al., 1990; Forrester, 1961; Senge, 1990). To these researchers, if a causal map
or model is used to represent a social system, causal loops are essential elements that
are responsible for the system’s identity and change (Bougon, 1992) and for the system’s
complex behaviors (Forrester, 1961). In the congregate approach, only labels and links
that contribute to forming loops are entered into the collective map. As a result, the
congregate map may be simpler than the sum of individual maps.
In the workshop mapping approach, the focus is on consensual model building at the
group level. Group members exchange their perceptions of a problem situation to foster
consensus (Vennix, 1996). Workshops are group meetings where the group as a whole
builds a model aided by a facilitator. The purpose of the workshop is to reach agreement
on what elements should be entered into the collective map. As a result of group
discussion and interaction in the workshop, the workshop collective model is expected
to be shared among group members. In some cases, individual maps are not used in the
workshop method, but the facilitator leads the group in building a collective map from
scratch.
Hypotheses
With respect to comparing the three methods for developing collective maps we propose
two hypotheses regarding map complexity and the perceived utility of collective maps.
Researchers in organizational cognition tend to use some simple analyses to measure
causal map complexity. The simplest form of map analysis is based on the number of
nodes. This approach suggests that the more nodes (or constructs) in a map, the more
complex is the map. Eden and Ackermann (1992) note that the number of nodes should
be treated with great care as a measure of complexity, because the number of concepts
surfaced depends on the interviewing skills of the map builders and the length of
interviews. The major weakness of this measure is that it does not include the total number
of links in a map, which is associated with the density of the map. Eden and Ackermann
(1992) suggest that links-to-nodes ratio (L/N) better represents a map’s density. A higher
links-to-nodes ratio “indicates a densely connected map and supposedly a higher level
of cognitive complexity.”
Hart (1977) proposed an alternative measure of map complexity—map density— as a
measure of degree of interconnection. It is measured by dividing the total number of links
by the maximum possible number of links (L/N(N-1)). Klein and Cooper (1982) use map
density to measure the cognitive complexity of decision makers. In a set of maps they
constructed, they found that maps with largest densities were also the three smallest
maps. In smaller maps the concepts tend to be of central importance to the situation, thus
the decision makers acknowledge many relationships between them, making the maps
dense.
Building a map by aggregation involves combining individual maps so that all concepts
and links in the individual maps are included. When the number of group members
increases, complexity of aggregate maps will increase enormously. Thus, the aggregate
mapping method is expected to produce collective maps with the highest degree of
complexity. In contrast, congregate mapping samples a set of links and nodes from
individual maps by identifying key loops or cycles in one or more individual maps, which
should result in a simpler representation than the aggregate map yields. When the number
of group members increases, the complexity of congregate maps will increase slowly.
Thus, the congregate mapping method is expected to produce collective maps with the
Table 1. A comparison of different approaches to building a collective map
Aggregation Congregation Workshop
Work that has
used the
approach
Lee et al. (1992),
Kwahk and Kim (1999),
Eden, et al. (1981), and
Eden (1989)
Bougon (1992)
Hall (1984),
Diffenbach (1982)
Langfield-Smith (1992),
Massey and Wallace (1996),
Vennix (1996)
Core processes Joining, merging
individual maps through
common concepts.
Looking for
congregating labels,
forming loops.
Workshop mapping, group
meeting/discussion, consensus
building, group facilitation.
Procedure Unique concepts are
merged directly to the
composite map while
common concepts are
merged taking care not
to introduce conflicts.
Merge two maps at a
time until the group
maps are exhausted.
Use common concepts
as coupling device to
combine two maps.
Individual maps are
connected to form loops
through “cryptic” labels,
which are repeatedly
used by the subjects.
Individual maps remain
separate, intact in the
congregate map.
Group members add concepts
(or labels) and relationships (or
connections) between concepts,
discuss and decide whether
they should be included in the
group map under the guidance
of a group facilitator.
Applications Organizational memory
(Lee et al., 1992),
Business Process
Reengineering (BPR)
(Kwahk and Kim,
1999). Distributed
decision making (Zhang
et al., 1994).
Strategic planning,
organizational identity
analysis (Bougon, 1992)
Understanding the
dynamics of
organizations (Hall,
1984).
Group decision support
systems (GDSS) (Eden, 1989),
solving messy problems
(Vennix, 1996).
Advantages Merging is simple and
straight forward; It can
be automated with an
algorithm; Conflict
detection.
Loops help understand
the dynamics of the
system; Better at
capturing multiple
perspectives on the
problem situation.
May have more beliefs than
individuals’ maps; Individual
biases can be overcome with
group interaction.
Disadvantages A simple merging of all
maps may not be a
“shared” map;
There is no chance to
mitigate biases in
individual maps.
Difficult to identify the
congregating labels;
Complex and difficult to
automate; Hard to apply
as it requires
perspectives of all
stakeholders.
Premature consensus
(groupthink) due to dominant
perspectives; Unresolvable
conflicts prevent convergence
on single map.
lowest degree of complexity. Workshop mapping relies on the efforts of a group to
identify nodes and links. In view of the limited information processing capacity of group
discussion, a workshop map should also be simpler than an aggregate map. The
workshop mapping method is expected to produce maps with intermediate degrees of
complexity, which depends on the nature and skill of the facilitator. Therefore, we
propose the first hypothesis:
Hypothesis 1: Aggregate maps will be more complex than either congregate or workshop
maps.
Subjective measures have been used in the literature to evaluate and compare maps. For
example, Nicolini (1999) used the subjects’ feedback/knowledge to compare maps.
Massey and Wallace (1996) used the knowledge of a panel of experts to judge the maps
using a Multi-Attribute Value (MAV) model (Massey and O’Keefe, 1993; Massey and
Wallace, 1996; Sakman, 1985). The MAV model consists of five attributes (or criteria):
structure, stakeholders, solution implication, level of focus, and clarity. The aggregate
measure was the weighted sum of the scores on these attributes. In this chapter, we use
three attributes: representation of the problem situation, solution implication or direction,
and representation of stakeholders or multiple perspectives to evaluate the
effectiveness of derived group maps.
Workshop mapping relies on a high level of member involvement. By contrast, aggregate
and congregate maps can be built through analysis of individual maps, and thus require
much lower levels of participant involvement. Even in cases when subjects play a role
in developing the aggregate and congregate maps, they must follow a well-codified set
of rules and procedures for developing the map and this will restrict their degree of
involvement in the process compared to workshop mapping. The second hypothesis is
based on the expectation that the degree to which members participate in building a map
is positively related to their evaluation of it:
Hypothesis 2: Subjects will evaluate a map developed using the workshop approach more
favorably than with those developed using the congregate or aggregate approaches.
We conducted two studies to compare the three mapping methods and to test the two
hypotheses.
Research Design
Two studies were conducted to carry out the comparison of the three approaches.
Research designs for the two cases were similar in general respects, but they differed
in some details. Some of the differences between the two cases are highlighted in
Table 2.
As the detailed designs were different for the two cases, we did not expect that the two
cases would produce exactly similar results, but we expected that the patterns fund in
the two cases would be comparable. We believe that to the extent similar patterns emerge
from different studies we can have more confidence in our conclusions. Conversely, if
we find different patterns for the two studies, this highlights areas that require more
nuanced judgment and further research. Study 1 commenced before Study 2. Some of the
lessons learned (such as the procedure, questionnaire design, etc.) from Study 1 were
incorporated into Study 2. It happened that Study 2 was completed before Study 1. Thus
some insights gained from Study 2 were also fed back to the conducting of Study 1.
Study 1
The objective in Study 1 was to develop an understanding of the causes of a problem
in HALONG, a Vietnamese retail organization, through building a collective causal map.
Established as a private company in 1986, HALONG has about 120 employees and
manufactures and distributes construction products in Vietnam. It has three plants, near
Ho Chi Minh City, in DaNang and in Can Tho and its annual revenue is 10 million USD.
According to the President of HALONG, the company grew steadily for the period from
1990 to 1996, but sales had been declining from 1996 to 2001. The objective of the mapping
Table 2. Comparison of the two studies
HALONG (Study 1) HOUSTON (Study 2)
Problem/issue A particular problem was identified
based on discussion with the
management team. The problem we
arrived at was: “Sales situation and
factors that affect sales at the
organization.”
The problem was developing a model of
the relationship of infrastructure growth
to quality of life of Houstonians based on
the expertise of the research team, which
was composed of scholars of different
disciplines.
Groups involved
in the study
Six groups were formed. Two groups
were assigned to each of three
treatments. Each treatment was one of
the methods of building collective
causal maps. Four subjects were in
each group.
There was one group, which was the
research team, but it was composed of
scholars with different perspectives on the
problem.
Limitations of
factors and
relationships
The number of factors in individual
maps was limited to 15 and the number
of relationships was limited to 35. We
wanted subjects to focus on important
factors and their relationships.
There was no attempt to limit the number
of factors and relationships in individual
maps and group maps. We wanted a rich
picture of the issue.
Data sources for
the models
Only subjects’ reports of their cognitive
constructs were used to build the
models.
A variety of data sources were used to
build the models, including relevant
literature, interview transcripts with
infrastructure decision makers, and the
subjects’ interdisciplinary knowledge.
Treatments The aggregate and congregate methods
were designed to include group model
construction. For all three methods,
group members worked together to
construct their collective map.
The aggregate and congregate methods
were designed not to include group model
construction. The aggregation and
congregation of individual maps were
carried out by the researcher. The subjects
only participated in the workshop
method.
process was to develop a collective understanding of the causes underlying the decline
in sales. Different members of HALONG were expected to have different degrees of
familiarity with the issue and to have different perspectives on the problem. After some
discussion with managers in HALONG, the issue was framed in terms of creating a causal
map depicting the “sales situation and factors that affect sales at HALONG.” The
relevance of this problem to the livelihoods of the subjects ensured their involvement
in the study.
All three methods were employed to construct the collective maps. The mapping process
consisted of two stages: individual causal mapping and group mapping. Individual maps
were obtained for three reasons. First, they provided the basis for comparison with the
collective maps. Second, the individual stage provided the subjects with an opportunity
to learn and become accustomed to the mapping method and the researchers. Finally, the
individual maps improved the quality of the group mapping process by encouraging
members to advance their own individual ideas.
The specific process was incorporated into a five-step experimental procedure: (1)
Subjects were asked to identify lists of factors important to understanding the problem;
(2) Based on these lists, subjects created their own causal maps; (3) The groups built
maps using the particular method assigned to them; (4) Subjects were then asked to
update their own causal maps if they wished; and (5) Subjects completed the questionnaire
and were debriefed. These steps will now be described in more detail.
(1) List of factors. Subjects were first asked to identify variables that may be used to
describe the problem. In the HALONG case, the problem variables most commonly
identified were sales, profit, and customer satisfaction. Subjects were asked next
to identify causal factors that have impacts on the problem variables and to identify
consequent factors that the problem may have impacts on. This was facilitated with
an open questionnaire that had blanks for the subjects to list factors in.
(2) Individual causal maps. Individual causal mapping sessions were held with all
subjects in the conference room at their workplace. We followed the procedure
described in Markoczy and Goldberg (1995) to capture individual causal maps1.
This procedure is well documented and easy to follow for the subjects. Subjects
were asked first to identify possible sensible influences among factors and specify
the type, the sign, and the strength of influences. After that they were asked to
review their maps and revise them wherever applicable. Specific guidelines for map
revision (also included in the questionnaire) were provided. Finally, subjects were
asked to identify important feedback loops in their maps. During the individual
mapping process, the researcher and two colleagues were available for questions
and guidance. The result of this step was a collection of individual causal maps
about the problem.
(3) Group maps. The subjects met for about 60 - 90 minutes to build group maps. Each
group member was given a detailed instruction sheet that they were expected to
follow. The researcher and two of his colleagues were present to answer questions
and to make sure subjects followed the instructions. Summaries of the group
process for each method are as follows:
The Aggregate method. The subjects took turns: i) introducing a factor with a brief
description if needed (determined by other members); ii) a member in charge wrote
down this factor on a sheet and asked other members whether they agreed to
include this factor in the group map; iii) the group then decided whether the factor
should be included (if all members agreed, the factor was entered into the group
map; if a majority did not favor it the factor was left out; if there were mixed opinions,
the factor was marked on the group map with a different color for a second round
of discussion); and iv) the process was repeated until all new factors were
exhausted. During the factor entry process, relationships were entered into the
group map in a similar manner. At the end, the groups were asked to revise their
maps by discussing the marked elements. These elements remained in the group
maps if group members agreed (including if they agreed that these elements were
relevant from other perspectives that they had no knowledge about). Otherwise,
the marked elements were left out. Groups were encouraged to add new information
into the group map as a result of group discussion and interaction.
The Congregate method. The subjects took turns: i) introducing a causal loop with
a brief description if needed (determined by other members); ii) a member in charge
recorded this loop on a blackboard and asked other members whether they agreed
this loop should be included in the group map; iii) the group then decided whether
the loop should be included (if all members agreed, the loop was entered into the
group map; if a majority did not favor it the loop was omitted; if there were mixed
opinions, the loop was marked on the group map with a different color for a second
round of discussion) and; iv) the process was repeated until no more loops were
identified. During the loop entry process both factors and relationships were
entered into the group map at the same time. At the end, the groups were asked to
revise their map by discussing the marked loops or elements. These loops remained
in the group maps if group members agreed (including if they agreed that these
elements were relevant from other perspectives that they had no knowledge about).
Otherwise these loops were omitted. Groups were encouraged to form new loops
in the group map as a result of group discussion and interaction.
The Workshop method. The subjects took turns describing the problem situation
by identifying problem variables, consequence factors, and causal factors that
affect the problem variables, and causal relationships between them. The facilitators
recorded these factors/variables and their relationships on blackboards and
asked other members whether they agreed to include these elements in the group
map and agreed with the story being told. If members all agreed, these elements were
put on the group map. If a majority did not advocate inclusion, they were left out.
If there were mixed opinions, they were marked with a different color for a second
round of discussion. The process was repeated until element entries were exhausted.
At the end, the groups were asked to revise their map by discussing the
marked elements and considering reducing the number of zero in-degree and outdegree
nodes. The agreed-upon elements remained in the group maps and elements
on which there was disagreement were left out.
(4) Individual map update. After their group meeting, the subjects were asked to
revise/update their individual maps (either on the questionnaire or on the diagram/
map) to include any insights they had from the group meeting and from viewing the
group map.
(5) Post experiment. The subjects were asked to give feedback on the experiment via
a questionnaire. This questionnaire gathered information that was used to calculate
the subjective measures to evaluate the collective causal maps, described
below.
The treatments for this experiment were the three methods of deriving collective maps:
aggregate mapping, congregate mapping, and workshop mapping. Subjects were selected
from employees of HALONG who work in the sales, production, and accounting
departments. They were randomly assigned to groups A, B, C and D. Groups A and B
were assigned to the aggregate method, and groups C and D to the congregate method.
Due to time restriction, sales reps were randomly assigned to groups E and F (the
workshop method).
Measures
The collective maps were compared in terms of both objective measures and subjective
measures. Objective measures consist of map complexity and distance ratios between
collective models and individual models. These were calculated by the researchers based
on comparison of the maps. Subjective measures were based on participants’ ratings.
These included the degree to which the maps gave a full and accurate representation of
the problem representation, the degree to which the maps suggested effective solutions
to the problem, and the degree to which the map fairly represented different stakeholders’
views. These subjective attributes are based on a Multi-Attribute-Value (MAV) model
developed for evaluating causal maps (Massey and O’Keefe, 1993; Massey and Wallace,
1996; Sakman, 1985). These responses were elicited from subjects with the questionnaire
that was filled out in the final step of the study. Indices calculated from these responses
were used as criteria to measure the effectiveness or utilities of the derived collective
maps. Because the two studies used quite different samples and addressed different
problems, the questionnaires and procedures for administering them differed, and details
of this are given in the description of Study 2.
To analyze the results of Study 1, we used ANOVA, nesting subjects within groups to
evaluate the impact of grouping and the impact of the method used on the effectiveness
of the composite map in understanding the problem at the individual level. At the group
level we were not able to employ statistical tests, but looked for patterns in results.
Table 3. Grouping and treatments
Treatments Aggregate method Congregate method Workshop method
Group A (5 subjects) Group C Grouping (5 subjects) Group E (5 subjects)
Group B (5 subjects) Group D (5 subjects) Group F (5 subjects)