Methodology

К оглавлению
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 
136 137 138 139 140 141 142 143 144 145 146 

The first subsection addresses sample definition issues. The thinking behind data

collection and lessons learned from it are presented in the subsections of interview guide

development, interview execution, and lessons learned from conducting the interviews.

The last subsection addresses issues related to coding procedures.

Sample Definition

Organizations likely to meet the qualifications below were asked to participate by inviting

volunteers to participate in structured interviews. To qualify as a research participant,

the systems development project participants must have had at least three years of work

experience and they must have worked on information systems development projects of

at least a minimum threshold of size. It was our judgment that very small projects would

not require formal documentation. Without knowing an exact threshold where such

documentation is needed (in fact one sub-goal of the study is to consider this issue) we

projected a minimum of having participated in at least one project involving three fulltime

project members lasting at least six months. Otherwise the probability of need for

formal methods such as UML was viewed as being below an acceptable threshold.

The strategy for selecting participants involved seeking a broad range of systems

analysts working on IT development projects in a wide variety of roles. The selection of

individuals playing a wide range of roles is based on the primary goal of developing a

range of possible states or values for various aspects of UML’s role in the development

process. Participation was solicited in two ways. First, individuals of personal acquaintance

to the researchers meeting the qualifications discussed next were invited to

participant. Second, we requested referrals to eligible individuals from organizations

likely to have these individuals in their employ. Direct contact with eligible individuals

as then initiated.

Although we had a clear description of desired interviewees, we found the need to adjust

as opportunities presented themselves. We sought systems developers but ended up

with individuals playing a wide array of roles pertaining to system development including

corporate strategic information systems planners, project managers, user security

officers, information systems consultants, and about 40% systems developers (see

Appendix A for a full exposition of the characteristics of the interviewees). This resulted

from our collaborators in participating organizations’ interpreting our research agenda

and the qualifications of likely subjects differently from us. In retrospect, this turned out

to enrich the range of perspectives represented in our sample.

Interview Guide Development

As we decided to collect our data using interviews, it was important to develop an

interview guide in order to structure the information gathering process. Given that our

goal was the solicitation of maximum ideas and insights, we did not expect that each

participant follow the identical sequence of questions. However, we did not want to

inadvertently miss areas of inquiry with any of the participants.

Questions in the interview guide were mainly derived from our own experience. Our first

step involved the development of a “rich picture” (Checkland & Scholes, 1990). It

included major relationships among organizational units that may determine strategic

issues related to representation, formal educational units, systems development projects

(their managers, systems analysts, and programmers), external to the organization units

(consultants and vendors), and representation documentation. Literature was also used,

for example Booch, Jacobson and Rumbaug (1999) for UML-related issues, Rogers (1995)

on diffusion issues, Van de Ven, Polley, Garud and Venkataramann (1999) on innovation

issues, but also published research articles on UML. Questions were not copied directly

from these or any other source. Rather, these sources were used as a means for checking

type and breadth of questions to be included in the interview guide.

In the end, the interview protocol went through nine iterations in its development as we

reviewed, combined, separated, and added new questions while tweaking wording of

those we intended to use. As part of the development process we read the questions

aloud to each other and simulated the sort of responses we expected — looking for

alternative interpretations and elements that needed further clarification or terms that

needed further definition. We also considered whether the simulated answers would lead

to the kind of information that we would want to analyze in later stages in the research

project.

In the final version of the interview protocol we had seven sections: background and

initiation (Appendix B); description of the most recent IT development project they had

worked on; methods and tools used in the most recent project; outcomes they observed

from the most recent project; observations about projects within the organization in

general; the IT environment in their firm; and the firm’s IT strategy. The logic for this

sequencing of questions was based on movement from the smallest to largest unit of

analysis. We believed that the respondents would be most familiar with their personal

experience and concrete observations. As the interview continued, more generalization

and global assessments would follow.