Coding Procedures

К оглавлению
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 
136 137 138 139 140 141 142 143 144 145 146 

In general, from this stage on, work was performed by one author and verified by the

second author. (It is noted that the reverse was the case for other analyses of the same

data). Rather than coding separately then comparing outcomes quantitatively as one

does with most communication research coding, we discussed differences to consensus.

This method resulted in many observations of differences based more on linguistics than

on substantially different understandings. However, we chose this approach because we

did not want to start with predefined coding schemes, but rather to generate the language

for describing elements based on the content of the interviews themselves.

Coding occurred in two stages. First, we went through each transcript and identified

causal statements (or inferred causal statements). These statements were copied and

pasted into a table (see Table 1).

The table structure was designed to serve as a tool for organizing further coding efforts.

The instantiated tables also provided all relevant information about each statement in

a handy and easy-to-use manner. In the end, the transcripts were transformed into a set

of tables — one for each transcript.

Second, for each statement we identified “cause” and “effect” labels. Because we

examined these at a semantic level — searching for the meaning of the statements rather

than keying of specific words or syntactic level structures — we did not develop specific

rules, but rather treated each statement as a unique case and applied judgment and

discussion to consensus where meanings might have had multiple interpretations. These

labels were subject to change in the mapping phase for either of two reasons. First, in

the context of the entire interview, the meaning of the particular statement sometimes

showed slightly different orientation and second, slight differences in phrasing might

be collapsed into a single descriptive term. This occurred often in wording of “effect”

variables such as “project success.” Slightly different ideas such as project thoroughness,

project efficiency, and project learning might all be collapsed into project success.

In the end, this process resulted in a total of 270 comments identified from the 11

transcripts (24.4 comments per interview). In addition, some 36 additional cause-effect

combinations were observed (where one comment yielded more than one relationship)

and another 47 comments contained valuable insights into definition of terms or other

matters, but did not yield causal links. In sum, we observed 259 casual links (23.6 links

per interview). The complete list of cause concepts and effect concepts are presented in

appendices C and D.

Developing Causal Maps

We hasten to say that even though we have accounted for all comments in our current

set of causal maps, we continue to find ways of recombining them for the purpose of

conducting additional analysis. Following from the verbal description of the content of

the statements, visual aids are prepared to help with developing understanding and

observation of deeper relationships. This also occurred in several stages. First, we

transformed each page of the tables into a set of causal segments — each statement

generated a pair (occasionally multiple causal relations) of causal relations which were

mapped onto PowerPoint slides. This mapping was done one page of table to one

PowerPoint slide. Even at this stage, references to a single entity in multiple states would

be represented as multiple arrows to or from that entity. In other words if training

influenced user skills and user skills influenced use of UML, the segment would be

presented as shown in Figure 1.

Each causal segment represents our smallest unit of result documentation. Each interview

would yield a number of causal segments. Some of these might be related and some

260, 261

260 Developer skills

Mentors

261

UML/OO use

adoption

Figure 1. Example of a causal segment

Note: The number in the bubble refers to the statement in the table. These numbers make it much

easier to return at a later time to the actual text during the writing stage of the project. Also note

that the arrow heads show directionality which is derived from the causal or inferred causal

statement. On occasion, arrowheads may go in both directions — for example, developers’ skills

could influence use of UML, but prolonged use could also change the composition of developers’

skills. The arrowheads, therefore, represent the observations of the respondents rather than all

relational possibilities.

190, 193

190 Project success

Use cases

190

Documentation

success

193

DFD

191

Class/object

diagrams

192

Cost

192

On-line CASE

tools

Figure 2. Consolidated causal map — Interview #8, documentation success

might not — the latter representing stand-alone findings. The mechanism chosen for

combining segments derived from one interview was to look for variables that appear in

more than one causal segment. A variable that appears across causal segments can take

on the role of being either independent or dependent and sometimes both. Based on these

principles causal segments from an interview can be combined into a holistic view of the

relationships uncovered in that interview (see Figure 2 for an example).

At this point, we had a choice. We could consider each interview a separate case and treat

the discussion of findings in terms of a set of 11 case studies following Yin (1994).

However, we opted for the second approach given our focus on theory development and

understanding the range rather than central tendencies of the data, which was to further

consolidate findings across interview participants. Again, we did this by focusing on

“target” labels and searching for the entities that influenced them. This work is in process

and will, hopefully, yield insights that are publishable in their own right.

Challenges in Using Causal Mapping

Even if convinced that the method has the potential to yield a quality outcome, ending

up with insufficient “revelation” of new or interesting information is possible. Ultimately,

one may come away with a study that just doesn’t show any consistent findings or even

suggest different findings consistently among different contingencies. In principle, this

would be an excellent scientific “disconfirmation” of prevailing thought, but practically

speaking it is often not a favorite observation in the eyes of reviewers.

However, this problem is often tractable where the researchers have become too close

to the project and are either not recognizing what is “interesting” about it or not

presenting what is “interesting” in a format highlighting what editors and reviewers will

also find interesting. This leads to the question: What constitutes an interesting finding?

We reflect on this question by discussing the abandoning of Rogers’ diffusion theory,

surprising elements, and complex relationships emerging from our causal maps.

Abandoning Rogers’ Diffusion Theory

The findings support our view that Rogers diffusion theory is invalid as a lens for

studying UML in an organizational context (see Table 2).

Our findings confirm that Rogers’ theory does not apply. Our conclusion is restricted to

the employment of this theory as the overarching platform for our study. However, our

analysis covers major prerequisites within Rogers’ theory. Still, other aspects of this

theory may be relevant, for example, the constructs of trialability and observability. The

proposition is that the higher the degree of trialability and observability, the more likely

that adoption occurs. However, UML is in its very nature a highly complex phenomenon

— in fact so complex that Rogers definition of these terms quite likely does not have

meaning.

We cannot suggest in a straightforward manner what a theory that would describe the

phenomenon of UML in an organizational context might be. Therefore, listening to the

data, we think, is extremely necessary in our effort to increase our understanding.

Surprising Elements

We look for variables that have not been mentioned in the literature. For example, we find

in our causal maps that the implementation of standard application packages is not

related to successful use of UML. We found this surprising because many organizations

today install an integrated standard application package for critical business functions

such as accounting, procurement, inventory control, customer order processing, and

distribution. The package must be integrated into the portfolio of other information

systems in the organization.

However, the structure and architecture of the standard application package generally

is not made available to the buyer organization. The reason for this is that the standard

application package vendor looks upon the structure and architecture as a competitive

element. The vendor does not want to run the risk of having other vendors learn the inner

workings of the standard application package — the risk of which would increase to an

unacceptable level if a buyer organization were to learn about these in depth.

Therefore, the buyer organization is left with descriptions of input and output data as

the basis for integration with other information systems. It does not matter whether the

standard application package is developed fully in UML or not. Because of denied

access, the buyer organization cannot (fully) integrate the information systems portfolio

according to UML principles. In fact, if installing standard application packages becomes

the dominant strategy among organizations, the concept of UML may be rendered

obsolete in companies other than those creating the packages. The challenge would

Aspect of Rogers’ diffusion

theory Findings in the present study

The innovation is clearly

defined and does not change

during the diffusion process

UML is highly complex in its own right and exists in parallel with other methods

for representation. UML consists of multiple elements. UML is differently

understood among actors. UML is molded according to need during the systems

development process. When standard application packages are employed, UML

may be an improbable approach.

Over time, the adoption of the

innovation is sigmoidal

Even when a corporate strategic unit promotes the employment of UML, projects

in the organization may not use it – rendering a smooth cumulative usage curve

quite unlikely. Even the concept of use (of UML) in its most narrow interpretation

is a difficult issue. We found that a project manager clearly stated that UML was

the standard being used. The systems analysts disconfirmed.

Users can be divided into the

categories of early adopters,

early majority, late majority,

and laggards

Awareness and/or belief in UML occur collectively and individually at differing

points in time. Adoption of UML is not necessarily a “good thing.” Being a

laggard might be highly appropriate.

Individual adopters make the

adopt/non-adopt decision

Actors on many levels make decisions about UML. More often than not decisions

made are not followed through. The issue of mandatory (because of organizational

policy) and voluntary use is highly complex.

Table 2. Rogers’ diffusion theory and present findings

move from a consistent development of information systems to the integration among

them.