Elicitation

К оглавлению
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 
136 137 138 139 140 141 142 143 144 145 146 

The project manager and four team members were separately interviewed in on-site

meetings over two months. Framing questions were used to structure the interview. The

framework in Figure 2 was presented at the beginning of the interview as a means of

stimulating and triggering the discussion. Each interview, whose duration was on

average about two hours, was taped and transcribed.

Interviews were aimed at eliciting explanatory relations among concepts provided by

software developers such as concepts’ explication (e.g., “What does quality mean to

you?”), causal relationship: (e.g., “How do individual and team competencies influence

project development?”), actions and/or decisions justification (e.g., “How do you cope

with high risk when choosing priorities?”), and values and personal beliefs driving

actions and choice (“How important is it for you to ‘have the control?’”). Interviews

developed dynamically through interaction. The interviewer asked participants to

explain opinions and beliefs, and to develop arguments until a satisfying level of detail

was achieved.

Coding

Interview coding was completed in two steps according to the documentary coding

method (Wrightson, 1976), using a concepts dictionary to assemble and identify

explanatory relationships between concepts. This method was slightly modified and

generalized to code explanatory relationships, which do not express solely causal

influence but also justification and concept clarification.

In the first step, relevant concepts were identified and listed; a detailed description of

their meaning as emerging from the interview was provided for each concept. Examples

of concept descriptions appearing in the dictionary are reported in Table 1.

In the second step the interview text was carefully analyzed to identify and list

explanatory relationships linking two or more concepts. For example, in one case, the

interviewee underlined the importance of requirements understanding and specifications

as follows:

“Requirements are a fundamental variable: they represent 'what you need to do.'

[…] If requirements are ambiguous, as often it is the case, you need to create an

effective channel to communicate and interact with the client. This usually

means spending money and time. A possible outcome of a successful interaction

is the reduction of a requirement, that is a reformulation of the customer request

that is both able to satisfy his/her needs and is technically clear and feasible.

[…] A requirement reduction may imply success for a project. Summing up,

understanding a customer’s needs has a definite contribution on the quality

perception of the customer.”

It is easy to recognize in this quotation the definition of some concepts such as

“requirements,” “requirement reduction,” as well as several explanatory relationships

that were coded in the following format A -> ex -> B, where ex stands for “explain” and

can be read as A explains B, B because A, A is a reason for B to occur, A is (a better) way

to say B, A (may) influence B, A (may) cause B. Examples contained in the quotation

above are as follows:

Ambiguous requirements –ex–> Create a channel with the client

Create a channel with the client –ex-> Spending time and money

Successful interaction –ex-> requirement reduction

A list of explanatory relationships can be reported on the coding sheet where additional

information can be added concerning the location in the text, additional notes aimed at

further describing the relationships or reporting emphasis added by the interviewee, and

links to other items in the list, as shown in Table 2.

Results

The mapping step may be carried out in a direct way from the results of the coding step

whenever text analysis and synthesis have been extensively and carefully performed.

Given the coding results, mapping means essentially to assemble the several explanatory

relationships that emerged from the coding through the well-known graphical representation

of causal maps. It is important to remark that mapping concerns relationships such

as A causes B, A influences, B, A has an impact on B, and so on. Explanatory relationships

state influence between two concepts.

In Figure 3 an example of a causal map elicited from the interviews is presented. The minus

sign on the arches between two concepts represents negative influence.

Table 1. Part of a Concept Dictionary Drawn From One of the Interviews

Control: set of activities needed to manage project’s development.

Dimension: can be assessed in terms of project’s estimated duration in days/months/years.

Flexibility: capability to develop product that are easy to modify and adapt to customer’s requests that may arise during

project development.

Quality: perceived performance, usefulness, measure of “what must be done.”

Relationship with customers: needed to make ambiguous requirements expressed by customers. as clear as possible.

Customers can be more or less easy to manage … in any case it is necessary to establish a communication channel.

Requirements: represent “what you need to do,” what the company must deliver to the customer, and this may or not be

always made explicit by the customer itself.

Requirements reduction: is the possibility to reformulate customer’s requests that is both able to satisfy his/her needs

and technically clear and feasible. Sometimes reduction means requirements simplification, in other cases it may imply

requirements dropping.

Risk: risk may arise from an under-qualified team with little or no experience or from huge dimension of a project or

from scare familiarity with new technologies.

Team: work group and its competencies.

Technology: tools for development.

Time: it can be a requirement for the customer or time to market.

Visibility of change: represents the possibility to customize the product on the basis of customer’s request.

Observing the map in Figure 3, it is possible to recognize as input variables project

dimension, skilled team (competencies), degree of access to technology, standardization

(versus customization) and ambiguous requirements. These variables are perceived as

sort of basic ingredients for the project success and as constraints usually beyond the

scope of developers’ control. On the other hand, control procedures and relationships

with the customer are considered the variables developers can actually have a certain

degree of control over, to have a positive impact on final results such as delivery time,

costs, and product’s performance.

Through a software tool (Decision Explorer), a quantitative analysis of the concepts’

relevance contained in the map of Figure 3 was performed. Relevance was evaluated

through domain and centrality analysis.

The domain analysis gives an indication of the complexity of the linkages around

concepts. The centrality analysis gives an indication of the influence of a concept in the

wider context of the map. The rationale behind domain analysis is that concepts

representing “key issues” will be highly elaborated, and consequently the algorithm

assigns to key concept a high domain score equal to the number of incoming and

outgoing links.

Centrality analysis is complementary to domain analysis in that it looks beyond the

immediate environment around the concept and examines the complexity of links at a

Table 2. Example of coding sheet

Figure 3. Causal map example

Location Explanatory relationship Note Links

P1r3 Ambiguous requirements –ex–> Create a channel

with the client

Provide justification for action

Channel as a metaphor for finding a

possible way of interaction

P1r5

P1r5 Create a channel with the client –ex-> Spending time

and money

Customer may be “hard”, i.e., it may

be difficult to interact and work with

them, to speak their same language

P1r3

-

-

-

1 Ambiguous

requirements

2 Costs

3 Quality (as

perceived

performance)

4 Delivery time

5 Relationship with

the customer

7 Skilled team 6 Standardization

8 Risk

9 Technology out of

control

10 Control

11 Project's

dimension

12 Requirements

reduction

-

-

-

-

-

number of levels away from the center. The centrality analysis evaluates the capability

of a given concept “C” to influence other concepts belonging to the same map by

calculating a score increasing with the number of other concepts directly and indirectly

influenced by “C.” Table 3 contains the results of domain and centrality analysis.

Individual Map Comparison

The same procedure of coding and mapping was applied to each interview collected in

the field analysis. Centrality and domain analysis were performed for each map. The aim

of this step was to compare individual perceptions of the problem to identify the

consensus level among developers belonging to the same team. Consensus can be

assessed with respect to three main aspects: problem framing, meaning, and concept

relevance.

a) Problem framing: By examining the structure of the collected maps, a high degree

of homogeneity appeared, in terms of map overall structure, input and output

variables, and connections among concepts (see Figures 3 and 4). Dominant and

recurring issues are: ambiguity of the requirements, team competencies, concerns

about scheduling and delivery time, and project dimensions. Other concepts such

as the customer’s role and the need to manage customer relationships appeared

only in two maps out of five. The project manager’s map appeared to be slightly

more complex and rich in feedback and the relationships between concepts.

Customer-related concepts play a major role only in the project manager map,

whereas the developers’ maps appear to be more concerned with scheduling,

project dimensions, and delivery time.

b) Concept meaning: By analyzing the concept dictionary attached to each map it is

possible to identify convergence and discrepancies in the meaning attached to

each concept by different developers. In the considered case, though a high level

of consensus has been achieved in defining the overall framework of Figure 2 , there

were differences across the team members’ concept meanings in some cases.

Top 4 concepts importance

according to the priority declared by

the interviewee

Top 4 concepts obtained from the

domain analysis

Top 4 concepts obtained from the

centrality analysis

Requirements

Costs

Delivery time

Quality

Costs

(5 links around)

Delivery time

(5 links around)

Relationship with customer

(4 links around)

Quality

(as perceived performance)

Delivery time

(7 of 11 concepts)

Costs

(7 of 11 concepts)

Control

(6 of 11 concepts)

Relationship with customer

(6 of 9 concepts)

Table 3. Results of domain and centrality analysis of the causal map depicted in Figure

3

Table 4 contains a description of the meanings attributed by each interviewee to

the main concepts. Summing up, the main discrepancies in meaning that emerged

from the explanation analysis involve the following concepts: quality (for the

customers), relationships with the customers, and requirements.

c) Concept relevance. The main differences in the importance of variables, that

emerged from the centrality and domain analysis concern delivery time and, again,

quality. Table 5 reports a comparison between the five interviewees concerning

concept relevance as obtained from interviewee’s declaration, domain and centrality

analysis. Note that Table 5 shows a certain degree of homogeneity for concept

relevance.

Numeric weights were assigned to concepts through an importance indicator calculated

on the basis of the domain and centrality analysis according to the following rule: the

1 Delivery time 2 Company’s profit

5 Respect of budget

constraints

8 Project’s dimension

6 Risk

7 Skilled team

9 Ambiguous

requirements

4 Product’s flexibility

3 Perceived

Quality

10 Relationship

with the customer

-

-

-

- -

1 budget flexibility

2 Respect of

delivery time

3 Quality

5 Ambiguous 4 Risk

project’s

architecture

7 Project’s dimension

6 Skilled team

8 Product’s flexibility

9 Ambiguous

10 Company’s profit requirements

- -

-

-

-

Interview B

Interview C

Figure 4. Causal maps from interviews with two software developers

importance of a given concept increases with: i) the score in centrality analysis, ii) the

score in domain analysis, and iii) the number of interviewees that consider the given

concept relevant. By applying such a rule, the weights contained in Table 6 were

obtained. The procedure to calculate importance weights contained in Table 6 is the

following:

1) calculate domain and centrality scores, respectively dij and cij for the i-th concept

in the j-th interview;

2) calculate normalized domain and centrality scores d’ij and c’ij;

3) calculate the concept weights wi through the following equation

wi = 1/2 j=1,..,n d’ij + 1/2 j=1,..,nc’ij

where n is the number of interviews and c’ij, d’ij = 0 if concept i-th does not appear

in the interview j-th;

4) normalize the wi to obtain concept importance.

Table 4. Differences and similarities in concept meaning as emerged from interviews

analysis

Concepts Interview A Interview B Interview C Interview D Interview E

Requirement What has to be done to

accomplish customer’s

request

Client’s desire or

intention both

formally and

informally expressed

Customer’s

request

Customer’s

request

How the customer

formalizes the request

Quality - performance

- usefulness

- faultiness level

(quality measures,

imperfections)

- quality manual,

software

documentation

- quality of the

organization and of

the development

process

-easy to

maintain

software,

- easy to read

software

- few errors,

well

documented

- easy to use

product

- customer care

- delivery time

- software

robustness

- availability of all

product’s functions for the

customers, - well

documented software for

each development phase

- adequate level of

technological

sophistication

- optimization

- durable quality

Time - completion time as a

performance parameter for

the customer

-time to market

- delivery time - delivery time - delivery time - delivery time

Relationship

with the

customer

- level of interaction,

communication channel,

- keep into account of

interaction on the level of

trust in the company

perceived by the customer

NA NA NA - Requirements

identification and

refinement

Visibility of

changes

possibility to customize

the product on the basis of

customer’s request, also

customer care

NA NA NA NA

Team Competencies NA NA Team working Competencies

Conflict

Risk - inadequate skills

- tight scheduling

- available budget

- inexperienced team

- project dimensions

- new activities

- new technologies

- ambiguous

requirements

- project

dimensions

- low quality

product

- inadequate

skills

- tight budget

- underestimation of

resources and time