Collective Causal Map Derivation and Analysis

К оглавлению
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 
136 137 138 139 140 141 142 143 144 145 146 

The overall group consensus map for this strategic planning team only contained two

categories (nodes) and a single causal relationship from Leadership to Growth giving a

link to node ratio of .5 (See Figure 4). Obviously, the complexity of this map is nonexistent.

On the other hand, the overall group total map contains all 12 categories and

119 causal relationships giving a link to node ratio of 9.9. As expected, the total map is

substantially more complex. However, what was not expected is the degree of difference

between the two maps. The consensus map only contains 17% of the categories and less

than 1% of the actual causal relationships found in the total map. The average cognitive

centrality is 1 and 6.7 for the consensus and total maps, respectively.

We also used cognitive centrality as a measure of the perceived importance of a node

in addressing the issues in the framing statement. As in the ratings analysis, we

converted the cognitive centrality of each node into a rank ordering of importance for

each participant. In this case, the participants reached only a low level of agreement on

the rank ordering of the importance of each category (Kendall’s W = .373, X2 = 53.406,

p = .000). Based on the degree of differences uncovered between the consensus and total

maps and the low level of agreement reached on the cognitive centrality based rank

orderings, we concluded that further analysis was required to determine whether

cognitive factions, i.e., subgroups of belief structures, existed or not.

Leadership Growth

Figure 4. Overall group consensus map

Category Name Definition

Growth Of the company, customer base, revenues.

Profitability ROI, stock value, fee, expectations.

Communication Internal vertically and horizontally.

Personnel Mgmt. Compensation, recruiting, HR, training, retention.

Organization Getting better organized, corporate structure, organized to meet goals.

External Image New logo, stakeholders, reputation, public image, name recognition.

Products Information technology, services, solutions, expansion.

Customers Anyone paying us money, internal functions.

Marketing/BusDev Business development, strategic posturing, how to get customers.

Leadership Accountability, ethics/corporate values, developing vision.

Quality How you do the job, meet customer expectations, internal/external, compliance.

Competitors Anyone who could take our work, internal component, anyone with work we want.

Table 4. Participant identified categories and definitions

Cognitive Faction Identification and Description

As described above, using the shared causal relationships, cluster analysis was used to

identify the cognitive factions. In our case, we used the hierarchical cluster analysis in

SPSS 12.0. The cluster analysis provided three potential sets of clusters: a three

([1,4,11,12], [2,5,6], [3,7,8,9,10,13]), four ([1,12], [2,5,6], [3,7,8,9,10,13], [4,11]), and five

([1,12], [2,5,6], [3,7,10,13], [4,11], [8,9]) cluster solution (See Table 5). The clusters were

Case 5 Clusters 4 Clusters 3 Clusters

1 1 1 1

2 2 2 2

3 3 3 3

4 4 4 1

5 2 2 2

6 2 2 2

7 3 3 3

8 5 3 3

9 5 3 3

10 3 3 3

11 4 4 1

12 1 1 1

13 3 3 3

Table 5. Cluster membership

Figure 5: Dendrogram using Ward Method

identified based on the mathematical similarity computed using the Jaccard coefficient

(Boyce et al., 1994). The higher the value of the coefficient, the more causal relationships

shared among the team members. Using the dendogram in Figure 5, we chose to use four

clusters based on the “closeness” or length of the line that merged similar groups

together. A shorter line indicates that the Euclidian distance between two groups is

smaller, indicating more similarity between the groups. For example, the shortness of the

horizontal lines of participant numbers 3, 10, and 13 (at the top of the diagram)

demonstrate that they share many causal relationships. Based on a visual inspection of

the dendogram, we chose the four cluster solution. This was based on the length of the

line that clusters participants 4 and 11 together. We decided that we would not allow any

“weaker” clusters to be formed. As such, four cognitive factions were uncovered.

Demographic Analysis

A variety of demographic characteristics (age, experience, education) were compared

across the four factions. None were significantly different across the factions. Even

though the planning team’s ages ranged from 32 to 54 and job tenure at the company

ranged from seven to 16 years, the differences were not associated with different belief

structures. In addition, there was considerable homogeneity among the managers

regarding education, gender and race. All but two of the managers had master’s degrees.

The other two had bachelor’s degrees. All but one manager was male and all were

Caucasian.

Cognitive Faction Company Function

Number Executive Support Business

Total

1 2 0 0 2

2 0 1 1 2

3 0 3 0 3

4 1 1 3 5

Total 3 5 4 12

Table 6. Cognitive faction membership description

Overall Cognitive Factions

Measures Group 1 2 3 4

Number of Group Members 13 2 2 3 6

# Concepts Generated 153 22 23 38 70

Avg # Concepts Generated per

Category per Member

.98

.92

.96

1.06

.97

Table 7. Concept generation within groups

The most distinguishing characteristic was type of functional area within the company

reported in Table 6. We defined company functions as executive (CEO, President and

board member), support functions (Director-HRM, Director-Marketing, Director-Finance,

and VP-Business Development), and business areas (VPs and Directors of IT,

Logistics and Facilities Management). The president and board member define cognitive

Faction 1. This represents two of the three top executives attending the strategic

planning session. Cognitive Faction 2 is comprised of the VP of IT and the Director of

Finance. IT represents the “newest” business division of the company. Cognitive

Faction 3 consists of three support function managers and cognitive Faction 4 consists

of logistics and transportation managers (the traditional line of business) as well as a top

executive and a supporting general manager.

Description of Factions

When we look at each faction’s productivity, as measured by the number of concepts

generated by the members of the factions, we find that all members generated about one

concept per category (Table 7). This can be viewed as evidence that all faction members

were about equally involved in the GCMS process. Next, we perform complexity, ratings,

and cognitive centrality analysis for each of the cognitive factions and compare them

among themselves and to the overall group. Following that, we present additional

evidence that there are four cognitive factions by looking at clusters of categories,

shared and idiosyncratic causal relationships, category analysis, and Givens-Means-

Ends (GME) analysis for the different cognitive faction consensus maps.

Complexity Analysis

Table 8 presents the complexity measures for the consensus maps for the overall group

and the cognitive factions. Visually inspecting Table 8 shows that the cognitive factions

had a much higher level of agreement as to the number of nodes and relationships that

existed in their consensus maps. Furthermore, there seems to be differences between the

cognitive faction groups as to the ratio of links to nodes and average cognitive centrality

of the consensus maps. This provides additional evidence that cognitive factions, i.e.,

subgroups of shared knowledge, existed within the top management team.

Overall Cognitive Factions

Measures Group 1 2 3 4

Number of Nodes 2 10 11 7 11

Number of Links 1 11 14 6 13

Ratio of Links to Nodes 0.5 1.1 1.3 .9 1.2

Avg Cognitive Centrality 1 2 2.5 1.7 2.4

Table 8. “Consensus” map complexity

Importance Ratings and Cognitive Centrality Analysis

The top part of Table 9 shows the importance ratings for the overall group and the four

cognitive factions. It is ordered by the overall group’s average importance ratings of

categories shown in the second column in the table. The rating values in the table indicate

there are differences among the factions and that the factions differ from the overall

group. For example, Faction 2 ranks Communication as the most important category, while

Faction 3 and 4 see Growth as the most important category. When we analyzed the

factions for within group agreement, we found that the level of agreement (Kendall’s W)

increased among the members of each faction in comparison to the level of agreement

reached in the overall group. The increased values range from .593 (Faction 3) to .823

(Faction 2) compared to .514 for the overall group. However, due to the small size of each

faction, the statistical significance of the level of agreement decreased. These results

provide additional evidence that the beliefs of the individuals in the factions were more

similar to each other than they were to the beliefs of the entire team or other combinations

of the cognitive factions.

Table 10 displays the average cognitive centrality of each node (category) in the

individual maps for the overall group and the cognitive factions. It is ordered by the

overall group’s average cognitive centrality. A review of the table indicates that the

cognitive factions disagree with one another and they all differ from the overall group.

Furthermore, the within group agreement is greater for the factions than for the overall

group. The level of agreement values ranged from .465 (Faction 4) to .716 (Faction 3) in

comparison to .373 for the overall group. However, like the ratings, the statistical

significance of the level of agreement decreased due to the small size of each faction. Like

the ratings, these results provide additional evidence for the identified cognitive

factions.

Overall Cognitive Factions

Category Group 1 2 3 4

Growth 8.31 (1) 8.00 (3) 6.50 (6) 8.67 (1) 8.83 (1)

Leadership 7.69 (2) 7.50 (4) 7.50 (2) 7.00 (4) 8.17 (2)

Personnel Management 7.69 (2) 8.50 (1) 7.00 (4) 7.00 (4) 8.00 (4)

Mkt/Bus Development 7.31 (4) 8.50 (1) 7.50 (3) 6.67 (6) 7.17 (5)

Communication 7.15 (5) 7.00 (5) 8.50 (1) 8.00 (2) 6.33 (7)

Customers 6.54 (6) 5.50 (9) 7.00 (4) 6.33 (7) 6.83 (6)

Quality 6.54 (6) 7.00 (5) 2.00 (10) 6.00 (8) 8.17 (2)

Organization 6.15 (8) 6.00 (8) 5.00 (7) 8.00 (2) 5.67 (9)

Profitability 5.69 (9) 7.00 (5) 5.00 (7) 4.67 (9) 6.00 (8)

Products 4.31 (10) 5.00 (10) 0.50 (11) 4.00 (11) 5.50 (10)

External Image 3.77 (11) 3.00 (11) 4.50 (9) 4.33 (10) 3.50 (11)

Competitors 2.00 (12) 1.00 (12) 0.50 (11) 1.00 (12) 3.33 (12)

Within Group

Rank Order Agreement

Kendall's W

X2

p

.514

73.572

.000

.718

15.785

.149

.823

18.099

.079

.593

19.556

.052

.620

40.937

.000

Table 9. Average category importance rating (and converted rank) by group

By reviewing the results reported in Tables 9 and 10, we see that the level of agreement

within the factions on both the explicit importance ratings and the cognitive centrality

of the categories are greater than what was reached by the overall group. This demonstrates

that the members within the cognitive factions agreed with one another more than

they agreed with members of the other cognitive factions. By reexamining the average

rating and converted rank order of the importance of each category (see Table 9), we see

that there is substantial disagreement between the cognitive factions. For example, the

Growth category is the most important category for Cognitive Factions 3 and 4, while it

is only the sixth ranked category for Cognitive Faction 2. Furthermore when inspecting

Table 10, we see that not only does the rank ordering of the categories by cognitive

centrality values differ between the cognitive factions, but we also see that the level of

connectivity among the categories are different. For example, the causal maps for

Cognitive Faction 4 are much more interconnected than the other cognitive factions.

Givens-Means-Ends Analysis

Both the overall group and cognitive faction maps can be evaluated using givens means

ends (GME) analysis to determine the flow of causality through the map (Bougon et al.,

1977; Eden et al., 1992; Weick & Bougon, 1986). Table 11 presents the Givens, Means,

and Ends of the overall group’s and cognitive faction’s consensus maps. Again, there

is little agreement among the cognitive factions. All cognitive factions only agreed to the

Growth category as playing the role of an end, or goal. The other six common categories

played different roles for different subsets of the cognitive factions. The Communication

category was perceived as a given by Cognitive Factions 2, 3, and 4 and as a means by

Cognitive Faction 1. The Leadership category was a given for Cognitive Factions 1, 2,

Overall Cognitive Factions

Category Group 1 2 3 4

Growth 12.31 (1) 8.50 (1) 10.00 (1) 5.67 (3) 16.00 (1)

Personnel Management 7.46 (2) 7.50 (2) 4.00 (5) 6.67 (1) 9.00 (7)

Quality 7.31 (3) 5.50 (5) 3.00 (10) 3.33 (6) 11.33 (2)

Leadership 7.08 (4) 6.00 (3) 3.50 (9) 4.00 (4) 10.17 (5)

Mkt/Bus Development 6.92 (5) 6.00 (3) 4.00 (5) 2.67 (7) 10.33 (4)

Profitability 6.92 (5) 5.00 (6) 5.00 (3) 2.33 (8) 10.50 (3)

Communication 6.77 (7) 5.00 (6) 4.00 (5) 4.00 (4) 9.67 (6)

Customers 6.15 (8) 4.00 (8) 6.50 (2) 1.67 (10) 9.00 (7)

Organization 5.77 (9) 4.00 (8) 4.50 (4) 6.00 (2) 6.67 (11)

External Image 5.62 (10) 4.00 (8) 4.00 (5) 2.00 (9) 8.50 (10)

Products 5.23 (11) 2.00 11) 2.50 (11) 1.67 (10) 9.00 (7)

Competitors 3.08 (12) 1.50 (12) 2.00 (12) 0.67 12) 5.17 (12)

Within Group

Rank Order Agreement

Kendall's W

X2

p

.373

53.406

.000

.536

1.800

.379

.660

14.513

.206

.716

23.627

.014

.465

30.711

.001

Table 10: Average cognitive centrality (and converted rank) by group

3 and a means for Cognitive Faction 4. The Organization category was a given for

Cognitive Factions 1, 3, and 4 and a means for Cognitive Faction 2. The GME analysis

provides additional support for the existence of cognitive factions and the use of causal

mapping to uncover them. We refer to the givens, means, and ends of the maps below

where we discuss the similarities and differences among the maps.

Discussion

The above results demonstrated that there were higher levels of agreement within the

cognitive factions than within the overall group and that there were differences between

the cognitive factions. As such, the results provided support for using causal mapping

to uncover cognitive diversity within a top management team. In this section, we describe

the perceptions and beliefs within each cognitive faction as well as the differences

between the cognitive factions.

Figures 4 and 6 through 9 show the actual consensus causal maps for the overall group

and the individual cognitive factions. The maps are drawn in a left to right order by

Givens-Means-Ends. Givens are shown as a lightly shaded box drawn with a solid outline,

Means are shown as an unshaded box drawn with a dashed outline, and Ends are shown

as a darker shaded box drawn with a dashed outline. Positive causal relationships are

shown with a solid arrow, while negative ones are shown with a dashed arrow. The width

of the relationship line portrays the strength (1, 2, or 3) of the relationship.

The overall group consensus map (see Figure 4), only contains a single strong positive

causal relationship. As such, there are no Means within this map. Based on the few

categories contained within this map, it is obvious that there is a lack of agreement among

the members of this strategic planning team.

Group Givens Means Ends

Overall Group Leadership Growth

Cognitive

Faction 1

Leadership

Mkt/Bus Development

Organization

Communication

Customers

Personnel Mgmt

External Image

Growth

Quality

Profitability

Cognitive

Faction 2

Communication

Competitors

External Image

Leadership

Products

Customers

Organization

Personnel Mgmt

Profitability

Growth

Mkt/Bus Development

Cognitive

Faction 3

Communication

Leadership

Mkt/Bus Development

Organization

Personnel Mgmt

Growth

Profitability

Cognitive

Faction 4

Communication

Mkt/Bus Development

Organization

Products

Customers

Leadership

Quality

External Image

Growth

Personnel Mgmt

Profitability

Table 11. Givens-means-ends for consensus maps

The consensus map for Cognitive Faction 1 is shown in Figure 6. This faction was

comprised of a board member and the president of the organization. As such, not

surprisingly, this map shows that this faction believes that Leadership is a very important

Given. In fact, the Leadership category causally affects five of the other categories either

directly or indirectly: Communication, Customers, Quality, Growth, and Profitability. In

fact, the only End that is not affected by Leadership is External Image. Additionally,

Growth is a very important End, or goal, for this faction. This faction also believes that

issues related to the Organization category, which was defined as “getting better

organized, corporate structure, and organized to meet goals” (see Table 4), has a

negative, or inverse, causal effect on Personnel Management which has a positive effect

on Growth. This faction also believes that the Organization category has a negative or

inverse relationship with Growth of the firm. This is due to the indirect relationship that

Organization category has on the Growth category via the Personnel Management

category. The causal effect from the Organization category to Growth is negative since

Organization has negative direct effect on Personnel Management which, in turn has a

positive direct effect on Growth. Therefore, if issues related to the Organization category

increase, they will cause a decrease in Personnel Management which then will cause a

decrease in Growth. This negative causal belief is counter to the other cognitive factions’

beliefs in which they feel that Organization has either a direct or an indirect positive

relationship to Growth (see Figures 7 through 9).

Figure 7 shows the consensus map for Cognitive Faction 2. This faction consisted of the

VP of information technology, a business line in this firm, and the director of finance. This

cognitive faction is the only one that included the Competitors category in their

Leadership

Growth

Personnel

Management

Marketing /

Business

Development

Communication Customers

Profitability

Quality

Organization

External Image

Figure 6. Cognitive Faction 1 consensus map

consensus map. They believe that Competitor issues will negatively affect the Growth

of the firm. They also have an internal and external causal theme that impacts Growth.

The internal theme is driven by the Communication category and mediated by the

Personnel Management and Organization categories. Based on this theme and the

definition of the Communication category (see Table 4), it is obvious that this faction

feels that internal organizational communication plays an important role in the growth

of the firm. The external theme is driven by the External Image and Products categories

and is mediated by the Customers and Profitability categories. This theme implies that

for the firm to grow, the firm must increase their customers which will only occur if the

firm’s external image is improved and the firm’s products are expanded. This external

theme is unique to this faction.

The consensus map for Cognitive Faction 3 (see Figure 8) is the simplest of the consensus

maps. The members of this faction did not include five of the twelve categories identified

by the strategic planning team. Table 8 shows that not only did this faction have the

fewest nodes in their consensus map, they also had the fewest number of causal

relationships, the smallest ratio of relationships to categories, and the smallest average

cognitive centrality. Furthermore, this is the only faction whose map does not have any

means. This faction was made up of three support function managers: VP of business

development, director of marketing, and a human resource manager. Of the four factions,

this one provides the least insight into what the firm needs to address and where the firm

Leadership

Growth

Personnel

Management

Marketing /

Business

Development

Customers

Communication

Profitability

Organization

External Image

Competitors

Products

Figure 7. Cognitive Faction 2 consensus map

will go into the future. What we can infer from their consensus map is that they believe

that the Givens (Personnel Management, Communication, Leadership, Organization, and

Mkt/Bus Development) directly affect the Growth category. And, based on the strength

of the relationships, they see issues related to the Organization and Mkt/Bus Development

categories contributing the most to that Growth. Based on the membership of this

faction, this should not be surprising. Based on the limited information contained in this

map, it is very difficult to use it to help set the future direction of the firm. However, it

can be used to reinforce ideas that are contained in the maps of the other cognitive

factions. For example, the positive causal effect that Organization has on Growth, adds

force to the similar belief contained in the consensus maps of Cognitive Factions 2 and

4 (see Figures 7 and 9).

The consensus map for the final cognitive faction, Cognitive Faction 4, is shown in Figure

9. This faction included the CEO, VP and General Manager, VP of Logistics, and directors

of logistics, technical services, and facility services—all of which are primary, traditional

lines of business of the firm. This faction, like Cognitive Faction 2, has an internal and

external set of causal themes. The internal theme shows that Communication issues

indirectly affect the Growth of the firm via the Leadership and Quality categories.

Interestingly, this is the only faction that did not see Leadership as a Given. Instead,

Leadership only mediates the effect of the Communication and Organization issues have

on the Growth of the firm. The external theme that this faction has identified is related

Leadership Growth

Personnel

Management

Marketing /

Business

Development

Communication

Profitability

Organization

Figure 8. Cognitive Faction 3 consensus map

to the one identified by Cognitive Faction 2. In both cases, the Products category affects

Growth via the Customers category. This again implies for Growth to occur for the firm,

the Customer base must be increased which can be done by increasing the Product

offerings. However, the two factions disagree as to the role that the External Image and

Mkt/Bus Development categories play, one believes they are Givens, the other Ends.

By careful review of the different causal maps of the cognitive factions, it is clear that

the different factions have different underlying belief structures. The identification of

the similarities and differences among the cognitive factions allowed the uncovering of

potentially important minority views of the form’s strategic position and future direction.

Without the identification of the cognitive factions within the strategic planning team,

these minority views may have been lost. As such, the identification and analysis of

cognitive factions is useful as a beginning point in the negotiating and bargaining

processes that are part of any strategic planning cycle.

This strategic planning team benefited from the information we uncovered regarding their

different beliefs about where and how this company should grow. The issues identified

from the factions in this study needed to be addressed by the planning team. It is our

contention that the minority views uncovered through the identification and analysis of

the cognitive factions would not have been heard if they had not been explicitly identified

for the strategic planning team. While the political nature of top management teams

Leadership

Growth

Personnel

Management

Marketing /

Business

Development

Communication

Profitability

Organization

Products

Quality

Customers

External Image

Figure 9. Cognitive Faction 4 consensus map

results in much bargaining and negotiation, the inclusion of strategic variables may not

be the foundation of negotiation. Instead, resource constraints and control by individual

managers become bargaining tools to gain a better position within the firm. In this case,

the explicit uncovering of different beliefs regarding strategy increased the attention

paid to the strategic aspects of the firm.

We also discovered, through discussion of the cognitive faction maps, that the planning

team found this approach useful in identifying the key issues associated with their future

direction. The friction between the different points of view was apparent throughout the

planning retreat by all involved. However, until the cognitive faction maps were

presented, it had remained below the surface. As such, the explicit representation

through the maps facilitated the strategic planning team in reaching a better understanding

of the different perspectives of their strategic situation. Again, the cognitive faction

maps ensured that the minority views received (more) attention.

Summary

The use of causal mapping provides an efficient and effective way to identify idiosyncratic

and shared knowledge among members of a top management team. By clustering

the individual causal maps, based on their shared causal relationships, we were able to

uncover a set of cognitive factions within the top management team. The number of

cognitive factions represents the level of cognitive diversity within the team. Since our

causal mapping approach forces the group to come to a common set of nodes or

congregating labels before causal relationships are identified, clustering the maps is very

straightforward. Furthermore, by forcing the group to identify the congregating labels,

it enabled the creation of group maps to be created without researcher intervention, thus

reducing the possibility of researcher bias.

We also provided a set of analyses that can be used to check the validity of the identified

factions. We looked at the importance ratings (and their corresponding ranks) of the

categories, the level of complexity of the causal maps, and the cognitive centrality (and

their corresponding ranks) of the causal maps. We also compared the consensus maps

using Givens-Means-Ends analysis. Finally, we compared the consensus maps based on

the causal themes contained in them. The identification of the similarities within each

cognitive faction and the differences between the cognitive factions is useful for a

strategic planning facilitator to have as a beginning point for the typical negotiating and

bargaining processes that are part of any strategic planning cycle.

The primary limitation for this approach to uncovering cognitive diversity is the

requirement that the individual maps can only be merged once sufficient congregating

labels have been identified. Depending on the causal mapping approach used, the

identification of the congregating labels can be very labor intensive. By using the

methodology incorporated in the GCMS, we were able to avoid this problem. However,

once the congregating labels have been identified, and the individual maps have been

recast using the congregating labels as the nodes in the causal maps, this approach is

straightforward. A second limitation of the reported research is that the results are based

on a single top management team. As such, any generalization of the results must be done

with care.

Our study did not show a relationship between demographic diversity and cognitive

diversity. Even though many aspects of this strategic planning team were homogeneous,

their belief structures were not. The assumption that demographic diversity measures

cognitive diversity needs further investigation. We did find the cognitive factions to be

related to the different functional areas of the organization. This relationship supports

the view that divisions operating autonomously will have different experiences and

decision contexts.

Further investigation into using causal mapping and cluster analysis to identify cognitive

factions in top management teams as a way to uncover cognitive diversity is needed.

Moreover, we believe that top management team research can benefit from cognitive

diversity measurement that enables the researchers to directly measure relationships

between team cognition with other organizational variables like structure, processes, and

firm performance. Currently, we are investigating the use of our approach with other teams.

Finally, a more complete comparison of our approach with other approaches to uncover

cognitive diversity is necessary. Specifically, how do the other collective cause mapping

approaches affect the cognitive diversity of a group? We expect that researcher-driven

approaches to either data capture or merging can reduce the diversity uncovered. For a

strategic planning session, this may result in consensus too early in the planning

process. With complex, diversified firms, the different perspectives of the factions can

enhance the analysis of the firm’s situation.

Appendix: Description of the Group

Cognitive Mapping System3

The Group Cognitive Mapping System (GCMS) is a multi-user, client-server system that

uses thick-client technology implemented in Java and SQL in conjunction with an Access

database. The user interface for the data collection aspect of the system is implemented

as a Java applet that runs within a WWW-browser. The data analysis portion of the

system is implemented in SQL, C++, VBA, Excel, and SPSS. The researcher can either set

up an ODBC connection from Excel or SPSS to the Access data base which allows the

researcher to simply execute the appropriate analysis query or they can simply copy the

results of a query in Access into an Excel table or SPSS data editor window and run the

appropriate analysis tool.

Data Collection Tools

The design of the data collection aspect of the system is based on designs typically

associated with group support systems (GSS). As such, the data collection part of the

system is organized around the idea of an agenda. Furthermore, all tools guarantee the

anonymity of the participant. This helps alleviate any power-type relationships. The

facilitator/researcher uses the agenda to control the deployment of the appropriate data

collection tools. Currently, there are seven tools that directly support the cause map

elicitation step in the methodology used in this study (see Figure 2 and Table 2).

Additionally, there is a log-on tool that participants use to get access to the system. Each

of the data collection tools is described below.

Concept Identification is supported with a distributed electronic brainstorming tool. In

this tool, the subject is presented with a framing statement and is asked to type concepts

into the system related to the framing statement. As they type their concepts in, the

system distributes them to the screens of all other participants. In the case of a subject

suffering from a mental block, the subject can also have the system display a stall diagram

(see Figure 3). The stall diagram is essentially a graphical depiction of the framing

statement.

Identify and Define Categories is supported using a tool that randomly chooses a

participant and asks them to propose a category name and definition that can be used

by all participants to categorize the concepts generated in the previous step. This tool

provides the “definer” with the list of concepts in the order that they were generated.

Once the definer creates a category and definition, the definer shares the proposed

category and its definition with the other participants. The tool also includes a “chat

room” type of facility that allows all participants to provide feedback on the proposed

category and definition. Once the “group” is comfortable with the proposed category and

definition, the definer saves them in the system. Next, the system chooses another

participant to play the role of definer. This process goes on using a round-robin type of

approach until the group is comfortable with the proposed set of categories.4

The Classify Concepts step was supported with a categorization tool in which the

participants placed each concept into a single category that was defined in the previous

step. This tool presents the list of concepts created in the first step, the categories and

definitions created in the second step, and the list of concepts that the participant has

placed in the current category (at the beginning, these lists are null). To categorize a

concept, the participant chooses a concept and category and tells the system to place

the concept in the category. It does not matter whether the participant chooses the

category or concept first. When the participant chooses a category, both the definition

of the category and the concepts currently placed in the category are displayed to the

participant. Once the concept has been categorized, it is removed from the list of concepts

to be categorized. Occasionally, a participant would like to reclassify a concept. The

system supports this action by allowing the participant to remove the concept from a

category by placing it back into the list of concepts to be categorized. At that point, the

concept can be placed into any category.

Rank Categories is actually supported by two related tools: the Category Rating tool and

the Category Rating Discussion tool. These tools support a Delphi-like process that

allows the participants to rate the categories, discuss the category ratings, and then rerate

the categories. The Category Rating tool provides the participant with the categories

and their definitions along with a slider that allows them to rate each category on a scale

of one (1) to nine (9). The tool displays the entire list of categories and their sliders

simultaneously. In this manner, the participant can perform both an absolute rating value,

and by viewing the pattern of values displayed over the sliders, the participants can

ensure that the individual category ratings are reasonable in a relative sense. As such,

the tool supports both absolute and relative judgment (Miller, 1956). The Category

Rating Discussion tool presents the individual participant ratings on their individual

screen. Additionally, the tool displays the average group rating for each category. Again,

this allows the participant to see the pattern of group rating values over the entire set

of categories, by focusing on the sliders as a set, and to see how each category was rated

by them and the group average, thus supporting a relative judgment model. Finally, the

Category Rating Discussion tool provides a “chat room” type facility for the group to

discuss the current rating values.

Relationship Identification, like Rank Categories, is supported by two related tools: a

Relationship Identification tool and a Relationship Discussion tool. Using the Relationship

Identification tool, the participants create their individual cause maps using the

agreed upon, group-defined categories as nodes in their maps. The tool is set up in a

manner that allows the participant to identify the origin category, the destination

category, the type of causal relationship (positive or negative), and the strength of the

causal relationship (strong:3, moderate:2, or slight:1). Once the participant has made

these choices, the tool updates the evolving cause map and redraws it on the screen in

a givens-means-end (GME) order. This always allows the participant to see the flow of

causality in a left-to-right manner. The discussion tool allows the participants to discuss

the current set of causal maps. To do this, the tool provides a chat-room type of facility

to allow comments about the maps to be shared in an anonymous manner. The tool also

provides a set of display options to allow the participants to see how their map compares

to their fellow team members. The display options include:

• Showing individual maps only, collective maps, and/or both.

• Filtering of the maps based on their “strength” levels (3, 2, 1).

• Filtering the collective maps based on the level of agreement reached on the

individual causal relationships identified, i.e., the percentage of participants

agreed that the relationship existed.

The participants can also use the three options in combination. For example, they can

choose to show both the collective and individual maps that portray relationships at least

a 50% level of agreement and that the relationships have a strength level of 3.

Data Analysis Tools

The data analysis tools currently supported in the GCMS are divided into three

categories: categorization analysis, importance rating and ranking analysis, and causal

map analysis. In this section, we describe the tools used in identifying and evaluating

cognitive factions. For the interested reader, we refer to Tegarden & Sheetz (2003) for a

more complete description of the analysis tools supported.

From a cognitive faction analysis perspective, we have not used any of the categorization

analysis tools supported by the GCMS. However, some of the unique tools supported

include maps of attention (Huff, 1990), statistical level of agreement on categorization of

concepts, parallel coordinate graph based analysis using concept generation order and

participant concept categorizations (Inselberg, 1997), and association maps based on the

participant’s concept categorizations. Most of the tools are implemented using the report

generator in Access and SQL. The parallel coordinate graphs are implemented using SQL

and Excel.

The GCMS currently supports three independent measures of category importance. In

the current study, we described two of them: explicit importance ratings and cognitive

centrality. The third that is supported is based on the concept categorizations. The more

concepts placed in a category, the more focus on the issues contained in that category.

To be able to compare across all three measures, each of the measures are converted to

a rank-order scale. SQL queries are used to generate the data necessary to feed SPSS to

perform the Kendall’s coefficient of concordance (W) computation to determine the level

of agreement reached across participants for each measure and at the individual and

group level, across the three measures.

Causal Map Analysis

There are many approaches used to analyze causal maps. In the context of cognitive

factions, the GCMS supports map complexity computations, givens-means-ends (GME)

analysis, the analysis of the level of agreement reached and the strength of relationships

contained in the maps, and map similarity. The majority of the techniques are implemented

as a set of SQL-based reports.

As described in the paper, there are different ways to compute the complexity of a causal

map. The GCMS supports the computation of number of nodes, number of relationships,

the ratio of relationships to nodes, and cognitive centrality. Also, using a graph theory

program implemented in C++, the GCMS computes many graph theoretic measures

(Harary, 1969). The GME, level of agreement, and relationship strength analyses is

essentially identical to that described with the Relationship Discussion tool above. In

addition to the tool, there are SQL-based reports that are available for the researcher to

further analyze the maps.

From a map similarity perspective, the GCMS computes a similarity matrix that can be fed

to SPSS for cluster analysis. At this point in time, communication between the GCMS and

SPSS is one way. As such, once the clusters have been identified, the collective maps

associated with each cluster must be manually identified in the GCMS before further

analysis is possible. This is implemented through a set of SQL queries that places the

participants into their relevant cluster (faction). Once this has been executed, the map

analysis can proceed in the normal manner, i.e., each faction map is simply treated as

either a collective map, for within faction analysis, or as an individual map, for between

faction analysis.