Critical Issues in Formal Methodology

Adoption for Software Development

Several studies have dealt with the issue of determining suitable methods for the

selection of a life-cycle model for software development applications (Boehm, 1988;

Bradac, Perry & Votta, 1994; Humphrey, 1989; McConnell, 1996; Putnam, 1992).

The life cycle of a software product begins with the idea formulation and the initial design

and ends when the product is no longer available for further use. The life-cycle model

of a software product is a formal description of how the product should be developed,

usually specifying the development phases, deliverables, guidelines, and evaluation of

intermediate and final results.

Examples of life cycle models are the following:

• Waterfall: in this model the development of software products is articulated into

a linear sequence of phases usually concerning problem analysis, requirements

analysis, development, integration, testing, installation and maintenance. This

model requires the definition of intermediate output and deliverables, minimum

overlapping between phases and feedback to previous phases in cases of unsatisfying

results. The waterfall model is very simple and it can be useful in stable

situations when the identification of the requirements is not problematic. The major

disadvantages concern the scarce interaction with the user (usually limited at the

beginning and at the end of the product development), and the lack of flexibility.

• Prototyping model: in this model the design is carried out to develop a prototype

of the product as soon as possible. The realization of the final product is seen as

successive refinements of the first prototype to achieve a satisfying degree of

convergence between the user’s needs and requirements’ identification and

implementation. This model is particularly useful when the user’s needs are

ambiguously defined.

• Incremental delivery: incremental models conceive the development of software

products as a set of stages, each one organized as a linear sequence of phases

(similar to the waterfall model). At the end of each development stage the product

presents new characteristics and improvements, i.e., it can be considered an

evolution of the previous stage. This model can be used in the case of big projects

when the available budget at the beginning of the project may be insufficient to

ensure the development of the entire project or when it is important to gain flexibility

and adaptation through incremental improvements.

The interest in the definition of suitable life-cycle models is clearly demonstrated by the

CMM (Capability Maturity Model) developed by the SEI (Software Engineering Institute,

http://www.sei.cmu.edu/) for the evaluation of the maturity level achieved by

software companies (Paulk, Curtis, Chrissis &Weber, 1993).

The CMM ranks software development organizations in a hierarchy of five levels, each

with a progressively greater capability of producing quality software (Gainer, 2003). Each

level is described as a level of maturity. At level one, the “Initial” level, there is little or

no formalization of processes. A project’s success depends on individual efforts and

capabilities, but if this expertise leaves the company there is no guarantee that such

knowledge has been incorporated into organizational routines and practice and that it

will be re-used effectively in the future.

At the second “repeatable” level, at least basic project management activities are in place,

such as configuration management and requirements management. Another step up the

scale is the “defined” process level, where basic quality assurance and quality control

activities are defined and practiced, such as defined standards, a defined process,

structured walkthroughs and formal testing.

To increase their own capability maturity from level two and level three, companies must

adequately define the life-cycle of all of their projects. Furthermore, according to the wellknown

international normative ISO IEC 12207 regarding software process management,

in software project planning the project manager must select activities and tasks of the

development process and map them onto the appropriate life-cycle model.

The availability in the literature and in professional practice of several life-cycle models

implies the problem of selecting the one that best fits the development of a specific

software application, given constraints such as requirements, level of perceived risks,

and scheduling (Bohem, 1981; Matson, Barret & Mellichamp, 1994; McConnell, 1996;

Pressman, 2000; Putnam, 1992).

To support the selection of the best life-cycle model, formal methodologies are often

employed to reduce risk, time to market and development costs. Despite these advantages,

however, recent literature on software development has investigated why software

developers often show resistance to using formal methodologies. Drawing on

previous research (Davis, 1989; Riemenshneider, Hardgrave & Davis, 2002; Thompson,

Higgins & Howell, 1991), we classify the determinants of resistance to the adoption of

formal methodologies in software development into three main categories:

• individual factors related to individual disposition and willingness, as well as

capability to employ formal methodologies and tools in software development (e.g.,

compatibility of the methodology with how developers perform their work, perceived

usefulness, ease of use);

• organizational factors related to organizational support and incentives to use

formal development methodologies (e.g., management commitment, facilitating

conditions and tools, training, career consequences);

• social factors related to the social acceptance of formal methodology adoption

(e.g., peer and supervisor opinions, social consensus, image and status).

What seems to emerge from such studies is that developers may perceive formal

methodologies as constraining, boring and time-consuming instead of as effective

support to software development.

In this chapter it is argued that the adoption of formal methodologies in software

development, and in particular in life-cycle selection, should be accompanied by a deep

analysis of the context structured into: knowledge elicitation, knowledge coding and

mapping through causal maps, and knowledge analysis.

The results of the analysis are obtained by analyzing developers’ experience and

knowledge embedded in their cognitive constructs (frames, patterns of action, cognitive

schemata, beliefs, etc.). Results can then be discussed with the developers to help them

achieve deeper knowledge and awareness of the development process. In this way, a

cognitive approach to the analysis of life cycle model selection can help to increase the

perceived usefulness of formal methodologies at the individual and at the organizational

level by transforming them from standard “constraining” tools to learning and knowledge

management procedures.

Traditional software development methodologies try to eliminate subjectivity through

standardization and usually neglect social and human factors in software engineering

(Pfleeger, 1999). In particular their use does not take into account how people frame

problems, select clues from the environment, attribute meaning to new events, and

leverage knowledge and expertise to deal with ambiguity and novelty rising from new,

poorly defined and unexpected situations.

A Cognitive Approach to Life Cycle

Model Selection through Causal Maps

In the following we characterize the life-cycle selection as a problem in which developers

are expected to choose the best model given information on the project and constraints

of time, cost and human resource availability. From a theoretical point of view, such a

problem can be investigated by drawing on the literature in organizational decisionmaking.

It is well known that classical approaches to decision-making are not able to deal with

incomplete information and ambiguous definitions in choice problems (March, 1988). In

particular, the theory of rational choice (Von Neumann & Morgestern, 1944) assumes that

human behavior in choice problems is driven by absolute rationality to the minimization

of the cost/benefit ratio. According to this perspective the decision-maker has complete

information, is able to process all available information, and expresses consistent

preferences and predictions.

Rational choice theory has been criticized by many studies both on theoretical and

empirical grounds because it fails in predicting and describing human behavior in most

of the choice situations in which the choice itself is not trivial. The main critics of rational

theory of choice are the following:

a) Bounded rationality: information is always incomplete and individuals have limited

capability to process huge amounts of data (Simon, 1961);

b) Interdependence between choice and evaluation (Cyert & March, 1963): individuals

do not evaluate a priori the best course of action among several possible ones,

but rather their final choice is influenced by intermediate results and by their

perception of success;

c) Bounded irrationality: in everyday life individuals solve choice problems through

cognitive strategies that do not respect theoretical assumptions and prescriptions

of the rational choice theory (Tversky & Kahneman, 1974; Nisbett & Ross, 1980);

d) Knowledge structures: individual tend to activate reasoning schemata and patterns

which proved to be effective in past similar situations, thus they do not

evaluate all of the possible courses of actions each time (Galambos, Abelson &

Black, 1986, Schank & Abelson, 1977);

e) Ex-post rationality (Weick, 1976): individual choices are the result of past actions

of which individuals make sense only afterward;

f) Decision making as a decomposable process (Mintzberg, Raisinghani & Theoret,

1976): organizational decision making is the result of a complex and long evaluation

processes involving many heterogeneous actors and is only partially analytically

investigable;

g) Decision making and power (Allison, 1971; Crozier, 1964): choices are influenced

by political variables.

According to many studies, in complex ill-defined situations decision-making problems

can be better framed in the “sense-making” perspective (Daft & Weick, 1984; Weick,

1976). In the sense-making perspective, choice is only the final act of an ongoing process

in which individuals make sense of the uncertain external environment by drawing on

previous knowledge accumulated through past experience, interaction with other people,

individual and organizational memory (Walsh & Ungson, 1991).

In the sense-making perspective individuals create knowledge in a three stage process

consisting of enactment, selection, and retention (Weick, 1976). In the enactment stage,

individuals, on the basis of their pre-existing knowledge, select clues and signals

belonging to the ongoing and uninterrupted data flow from the environment. Through

enactment people try to reduce the ambiguity of incoming information. In the selection

stage, people draw from their mental models of actions (e.g., recipes, scripts, theories,

etc.) constructed through experience and learning and that proved to be useful in the

past. In the retention phase successful models of action are stored for possible future

re-use.

Being influenced by and strictly interrelated with action, cognition is necessarily

situated and, as such, influenced by the particular organizational context in which it

develops (Blackler, 2002; Brown & Duguid, 1991). This means that enactment, selection

and retention may be strongly conditioned by the presence of shared values, roles,

organizational procedures, socially and organizationally accepted behaviors, rules, and

organizational culture.

By adopting a sense-making perspective to investigate choice and decision-making

problems with respect to a specific context of action, one needs to analyze how individual

cognition takes place in organizations. For instance, how people frame problems, which

values and beliefs influence or draw their actions, how existing models of action influence

current and future choices, and how people make and justify their choices. Starting from

these considerations, in the following the life-cycle selection, and more in general, choice

problems are formulated according to the sense-making approach.

According to this approach, a new fact is interpreted when an individual is able to link

the new fact to existing knowledge in a coherent way. Natural language is the most

immediate tool to express such knowledge, because it allows individuals to represent

nuances, ambiguities, uncertainties and conflicts usually neglected by formal methods

to achieve coherence, simplicity and certainty. It is possible to have an idea of the

complex contextual knowledge used by an individual when he/she explains the motivations

of his/her judgement to other people. Through an explanatory discourse, people

introduce hypotheses on the basis of their own background knowledge in an attempt to

explain some evidence by relating new facts to known ones. Explanatory discourse means

any spoken or written discourse through which an individual tries to make explicit the

reasons justifying a choice or an evaluative judgment.

The idea proposed in this chapter is that mapping evaluation through explanations could

shed light on how people make their choices in organizations in terms of how they select

and relate evaluation criteria. For example, in evaluating different life-cycle models,

explanation can be a way to elicit from evaluators’ subjective knowledge which project’s

characteristics play a crucial role in determining his/her preference for one model rather

than another, which attributes are evaluated, what is the meaning of those attributes in

a specific context, and if and how the attributes are related among them.

Starting from this theoretical background the main assumptions behind the methodology

proposed in this chapter and presented in the next section can be summarized as follows:

a) Individual knowledge is incorporated in mental schemata and organizational

procedure. Patterns of action, scripts, models of behavior, facts, shared values, and

stereotypes resulting from ongoing sense-making activities are stored in both

individual and organizational memory (Weick, 1976; Walsh & Ungson, 1991).

Organizational and individual memories are socially constructed thanks to an

ongoing activity of individual interpretation and collective interaction (Berger &

Luckmann, 1966; Nicolini & Meznar, 1995).

b) Ambiguity related to inputs from the environment (e.g., ambiguous requirements

in software development) is resolved through interpretation; interpretation occurs

when an individual is able to develop an explanatory discourse linking the new fact

to existing knowledge in a coherent way (Schanck, 1986; Thagard, 1992; Zollo,

1998). Through explanatory discourses, people relate new facts to known ones by

introducing or implicitly assuming hypotheses to explain (enacted) “evidence” on

the basis of their own background knowledge.

c) Natural language is the most direct tool to express such knowledge, because it

allows the representation of nuances, ambiguities, uncertainties and paradoxical

assertions (Quinn & Cameron, 1983).

Following this background one expects that sense-making is deeper and more intense in

knowledge-intensive organizations (e.g., software companies) working in an unstable

environment, facing new and unexpected situations, and performing non-routine tasks.

Consequently, explanations provided by software developers regarding the problem of

choosing the best life cycle model in the early stages of a new software product

development given information about situational constraints should provide considerable

and deep knowledge about how people perceive and frame problems, select and

attribute meanings and relevance to critical variables, and eventually make their decisions

regarding the development process.

In the following section we propose a methodology based on causal mapping to elicit

and represent cognitive constructs that software developers activate when requested

to choose a life cycle model given some ambiguous and incoherent information about a

product’s requirements, customer’s need, and constraints about available resources,

costs and development time. Following the above assumptions, causal maps represent

a suitable methodological approach because:

a) they allow researchers to obtain a formal representation of cognitive constructs

activated by developers as a set of causal relationships; this representation is a

practical way to elicit and capture, at least partially and in simplified form, the mental

schemata and shared beliefs driving individual actions and choice models;

b) explanatory discourse can be easily translated into causal maps; explanations are

very often attempts to provide a picture of cause/effect relationships among facts

occurring in the real world;

c) causal maps can be constructed from the analysis of natural language information

contained in explanatory discourses.

The Methodology: Eliciting and

Mapping Developers’ Knowlege in the

Life Cycle Selection

The idea proposed in this chapter is that causal mapping can be employed to map

individual knowledge contained in explanations provided by software developers in the

early stages of a new project when a life cycle model has to be selected. Causal maps can

be used to model concepts and the relationships between the concepts contained in

explanatory discourses. The proposed methodology is articulated according to the

following steps:

a) Sample Selection: one or more development teams made up of experienced practitioners

in software development are selected according to certain criteria: i) level

of expertise, as recognized by other experts or estimated from their position; ii)

variety, in terms of roles, organizational positions, background (Calori, 2000).

b) Problem framing: a set of general framing questions concerning the main decision

variables involved in the problem of choosing the “right” life cycle model is

designed on the basis of the literature analysis and through a first involvement of

the developers;

c) Elicitation: framing questions are employed to collect explanatory discourses

through interviews;

d) Coding: explanatory discourses are analyzed to elicit concepts and relationships

among them (Fletcher & Huff, 1990); relevant concepts are described in detail and

reported in an interview dictionary;

e) Mapping: individual causal maps are used to represent concepts and relationships

between concepts and analyzed to identify input and output variables, and

influential and relevant concepts;

f) Comparison: individual maps are compared to identify similarities and differences

in the framing of the life cycle selection problem across different individuals;

g) Data validation is performed through inter-coder reliability and feedback from

interviewees;

h) Implication for decision support: once validated, the results emerging from the

causal map analysis can be used to improve the design of Decision Support

Systems (DSS) for the selection of the “best” life cycle model.

The choice of interviewing people instead of documentation analysis can be justified

given the characteristics of the context. First, documentation containing developers’

opinions and perceptions about how they frame the problem and make their choices

during the development process does not exist.

Second, managing a software development project requires coping with ambiguous

requirements, facing unexpected situations and making continuous adjustments to the

customer’s needs. Developers try to exploit existing knowledge and experience to cope

with such novelty. Interviews and explanations try to “force” developers to make sense

of an ever changing and unstable environment and to actively reconstruct (ex post) the

rationale behind their past behaviors and decisions. Linguistic explanation may play a

fundamental role in this reconstructing action because the tacit knowledge of informal

organization is very often not known and opaque to its same constructors (Schanck,

1986; Weick, 1976). The taken-for-granted world is the world of the “obvious” and its

opacity lies in the uncritical and automatic ways which it is enacted and recalled to

memory. For these reasons, interviews, which permit direct involvement and interaction,

favor the capture of tacit knowledge more than explicit sources such as written speech

and documentation.

The proposed methodology was tested through a case study in an Italian software

company producing software for accounting, management, office automation, and

telecommunication systems. Other activities of the company concern outsourcing

management of data elaboration centers. The company belongs to an important group

with more than 1,000 employees, half of which are software developers, and a turnover

of 80 million euros in 2002.

The activities of a software development team belonging to a specific organizational subunit

and involved in the realization of a new product were investigated in depth through

the proposed methodology. In the following, a step-by step application of the methodology

is presented to provide the reader with practical examples and an overview of the

critical issues and results that emerged during and after the field analysis.

Sample Selection

The sample was selected according to the above-mentioned criteria of level of expertise

and variety. More specifically, level of expertise was estimated through years of

experience in software development, involvement in the development of large projects,

and peer and management indication. Through such criteria, it was not difficult to identify

an “experienced team.” A satisfying degree of variety was ensured by the way the

company selects development teams. The team is usually made up of several developers

with different roles (system analyst, programmers, network experts, etc.) and by one or

more project managers. Of course team composition and duration depend on the

characteristics of the project and may change over time. It is worthwhile to note that the

number of people involved in the field research may vary as well depending on the

research purpose. For example, if one wants to build a very comprehensive collective

knowledge base (Calori, 2000), it is necessary to interview a large number of experts to

integrate as many different points of view as possible into representative and reliable

descriptions. On the other hand, if the research purpose is to investigate in-depth a given

organizational aspect through an action research approach (Argyris & Schon, 1978), the

number of people involved may be lower than in the previous case. As far as this chapter

is concerned, our aim was to use the field analysis primarily to test the proposed

methodology and to provide the reader with as many details as possible. Consequently,

the presented results are mainly descriptive. However, in the last two sections of this

chapter, we outline how the same methodology can be used to develop prescriptive or

supportive tools for software project management.

Problem-Framing

The aim of this step was to identify a set of general framing questions needed to structure

the interviews performed in the next step. These questions concerned the main decision

variables involved in choosing the “right” life cycle model. The framing questions have

been designed starting from the literature analysis. On this basis a framework describing

variables considered relevant to the final choice of the life-cycle has been constructed.

This framework was then presented to some of the company’s project managers and

developers to be validated and integrated before being employed in the interview phase.

Project managers’ and developers’ suggestions were collected and used to refine the

framework, whose final version is depicted in Figure 2. Given the way it has been

constructed, one can look at this framework as a sort of collective representation of the

problem of choosing the right life-cycle model, though very general, unspecific and

approximate.

Framework variables have been grouped into three main clusters:

a) Organizational variables concerning resource availability for the project, investments,

manager and team commitment, leadership of the project managers, and

organizational culture;

b) Customer-related variables such as requirements, concerns for costs, quality, time,

and visibility of changes, i.e., customer perception that his/her requests have been

recognized and satisfied through actual changes in the product;

c) Process variables related to production such as team competencies, maintenance,

relationships with the customer, and the possibility of further development.

Clusters have been obtained through two steps. In the first step an initial classification

was proposed by the authors on the basis of meanings that the considered variables

usually assume in the literature. The proposed classification was then refined through

developers’ observations and suggestions. Moreover, it emerged that each cluster

represented a dominant point of view shaping the way each interviewee perceives and

looks at the problem. In other words, among the interviewees, some developers empha-

Figure 2. Variables influencing the choice of life cycle model according to team

perception

Life-cycle

choice

Costs

Requirements

Times Quality Visibility of

changes

CUSTOMER

Possibility of

further developments

Maintenance Relationship with

Customers

Competencies

of the team

PROCESS

Resouces availability

Investments

Committment

O

R

G

A

N

I

Z

A

T

I

O N

Organizational

culture

o Leadership

sized more the customer-related cluster, others seemed more concerned about process,

while a third group paid more attention to organizational constraints.