Critical Issues in Formal Methodology
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135
136 137 138 139 140 141 142 143 144 145 146
Adoption for Software Development
Several studies have dealt with the issue of determining suitable methods for the
selection of a life-cycle model for software development applications (Boehm, 1988;
Bradac, Perry & Votta, 1994; Humphrey, 1989; McConnell, 1996; Putnam, 1992).
The life cycle of a software product begins with the idea formulation and the initial design
and ends when the product is no longer available for further use. The life-cycle model
of a software product is a formal description of how the product should be developed,
usually specifying the development phases, deliverables, guidelines, and evaluation of
intermediate and final results.
Examples of life cycle models are the following:
• Waterfall: in this model the development of software products is articulated into
a linear sequence of phases usually concerning problem analysis, requirements
analysis, development, integration, testing, installation and maintenance. This
model requires the definition of intermediate output and deliverables, minimum
overlapping between phases and feedback to previous phases in cases of unsatisfying
results. The waterfall model is very simple and it can be useful in stable
situations when the identification of the requirements is not problematic. The major
disadvantages concern the scarce interaction with the user (usually limited at the
beginning and at the end of the product development), and the lack of flexibility.
• Prototyping model: in this model the design is carried out to develop a prototype
of the product as soon as possible. The realization of the final product is seen as
successive refinements of the first prototype to achieve a satisfying degree of
convergence between the user’s needs and requirements’ identification and
implementation. This model is particularly useful when the user’s needs are
ambiguously defined.
• Incremental delivery: incremental models conceive the development of software
products as a set of stages, each one organized as a linear sequence of phases
(similar to the waterfall model). At the end of each development stage the product
presents new characteristics and improvements, i.e., it can be considered an
evolution of the previous stage. This model can be used in the case of big projects
when the available budget at the beginning of the project may be insufficient to
ensure the development of the entire project or when it is important to gain flexibility
and adaptation through incremental improvements.
The interest in the definition of suitable life-cycle models is clearly demonstrated by the
CMM (Capability Maturity Model) developed by the SEI (Software Engineering Institute,
http://www.sei.cmu.edu/) for the evaluation of the maturity level achieved by
software companies (Paulk, Curtis, Chrissis &Weber, 1993).
The CMM ranks software development organizations in a hierarchy of five levels, each
with a progressively greater capability of producing quality software (Gainer, 2003). Each
level is described as a level of maturity. At level one, the “Initial” level, there is little or
no formalization of processes. A project’s success depends on individual efforts and
capabilities, but if this expertise leaves the company there is no guarantee that such
knowledge has been incorporated into organizational routines and practice and that it
will be re-used effectively in the future.
At the second “repeatable” level, at least basic project management activities are in place,
such as configuration management and requirements management. Another step up the
scale is the “defined” process level, where basic quality assurance and quality control
activities are defined and practiced, such as defined standards, a defined process,
structured walkthroughs and formal testing.
To increase their own capability maturity from level two and level three, companies must
adequately define the life-cycle of all of their projects. Furthermore, according to the wellknown
international normative ISO IEC 12207 regarding software process management,
in software project planning the project manager must select activities and tasks of the
development process and map them onto the appropriate life-cycle model.
The availability in the literature and in professional practice of several life-cycle models
implies the problem of selecting the one that best fits the development of a specific
software application, given constraints such as requirements, level of perceived risks,
and scheduling (Bohem, 1981; Matson, Barret & Mellichamp, 1994; McConnell, 1996;
Pressman, 2000; Putnam, 1992).
To support the selection of the best life-cycle model, formal methodologies are often
employed to reduce risk, time to market and development costs. Despite these advantages,
however, recent literature on software development has investigated why software
developers often show resistance to using formal methodologies. Drawing on
previous research (Davis, 1989; Riemenshneider, Hardgrave & Davis, 2002; Thompson,
Higgins & Howell, 1991), we classify the determinants of resistance to the adoption of
formal methodologies in software development into three main categories:
• individual factors related to individual disposition and willingness, as well as
capability to employ formal methodologies and tools in software development (e.g.,
compatibility of the methodology with how developers perform their work, perceived
usefulness, ease of use);
• organizational factors related to organizational support and incentives to use
formal development methodologies (e.g., management commitment, facilitating
conditions and tools, training, career consequences);
• social factors related to the social acceptance of formal methodology adoption
(e.g., peer and supervisor opinions, social consensus, image and status).
What seems to emerge from such studies is that developers may perceive formal
methodologies as constraining, boring and time-consuming instead of as effective
support to software development.
In this chapter it is argued that the adoption of formal methodologies in software
development, and in particular in life-cycle selection, should be accompanied by a deep
analysis of the context structured into: knowledge elicitation, knowledge coding and
mapping through causal maps, and knowledge analysis.
The results of the analysis are obtained by analyzing developers’ experience and
knowledge embedded in their cognitive constructs (frames, patterns of action, cognitive
schemata, beliefs, etc.). Results can then be discussed with the developers to help them
achieve deeper knowledge and awareness of the development process. In this way, a
cognitive approach to the analysis of life cycle model selection can help to increase the
perceived usefulness of formal methodologies at the individual and at the organizational
level by transforming them from standard “constraining” tools to learning and knowledge
management procedures.
Traditional software development methodologies try to eliminate subjectivity through
standardization and usually neglect social and human factors in software engineering
(Pfleeger, 1999). In particular their use does not take into account how people frame
problems, select clues from the environment, attribute meaning to new events, and
leverage knowledge and expertise to deal with ambiguity and novelty rising from new,
poorly defined and unexpected situations.
A Cognitive Approach to Life Cycle
Model Selection through Causal Maps
In the following we characterize the life-cycle selection as a problem in which developers
are expected to choose the best model given information on the project and constraints
of time, cost and human resource availability. From a theoretical point of view, such a
problem can be investigated by drawing on the literature in organizational decisionmaking.
It is well known that classical approaches to decision-making are not able to deal with
incomplete information and ambiguous definitions in choice problems (March, 1988). In
particular, the theory of rational choice (Von Neumann & Morgestern, 1944) assumes that
human behavior in choice problems is driven by absolute rationality to the minimization
of the cost/benefit ratio. According to this perspective the decision-maker has complete
information, is able to process all available information, and expresses consistent
preferences and predictions.
Rational choice theory has been criticized by many studies both on theoretical and
empirical grounds because it fails in predicting and describing human behavior in most
of the choice situations in which the choice itself is not trivial. The main critics of rational
theory of choice are the following:
a) Bounded rationality: information is always incomplete and individuals have limited
capability to process huge amounts of data (Simon, 1961);
b) Interdependence between choice and evaluation (Cyert & March, 1963): individuals
do not evaluate a priori the best course of action among several possible ones,
but rather their final choice is influenced by intermediate results and by their
perception of success;
c) Bounded irrationality: in everyday life individuals solve choice problems through
cognitive strategies that do not respect theoretical assumptions and prescriptions
of the rational choice theory (Tversky & Kahneman, 1974; Nisbett & Ross, 1980);
d) Knowledge structures: individual tend to activate reasoning schemata and patterns
which proved to be effective in past similar situations, thus they do not
evaluate all of the possible courses of actions each time (Galambos, Abelson &
Black, 1986, Schank & Abelson, 1977);
e) Ex-post rationality (Weick, 1976): individual choices are the result of past actions
of which individuals make sense only afterward;
f) Decision making as a decomposable process (Mintzberg, Raisinghani & Theoret,
1976): organizational decision making is the result of a complex and long evaluation
processes involving many heterogeneous actors and is only partially analytically
investigable;
g) Decision making and power (Allison, 1971; Crozier, 1964): choices are influenced
by political variables.
According to many studies, in complex ill-defined situations decision-making problems
can be better framed in the “sense-making” perspective (Daft & Weick, 1984; Weick,
1976). In the sense-making perspective, choice is only the final act of an ongoing process
in which individuals make sense of the uncertain external environment by drawing on
previous knowledge accumulated through past experience, interaction with other people,
individual and organizational memory (Walsh & Ungson, 1991).
In the sense-making perspective individuals create knowledge in a three stage process
consisting of enactment, selection, and retention (Weick, 1976). In the enactment stage,
individuals, on the basis of their pre-existing knowledge, select clues and signals
belonging to the ongoing and uninterrupted data flow from the environment. Through
enactment people try to reduce the ambiguity of incoming information. In the selection
stage, people draw from their mental models of actions (e.g., recipes, scripts, theories,
etc.) constructed through experience and learning and that proved to be useful in the
past. In the retention phase successful models of action are stored for possible future
re-use.
Being influenced by and strictly interrelated with action, cognition is necessarily
situated and, as such, influenced by the particular organizational context in which it
develops (Blackler, 2002; Brown & Duguid, 1991). This means that enactment, selection
and retention may be strongly conditioned by the presence of shared values, roles,
organizational procedures, socially and organizationally accepted behaviors, rules, and
organizational culture.
By adopting a sense-making perspective to investigate choice and decision-making
problems with respect to a specific context of action, one needs to analyze how individual
cognition takes place in organizations. For instance, how people frame problems, which
values and beliefs influence or draw their actions, how existing models of action influence
current and future choices, and how people make and justify their choices. Starting from
these considerations, in the following the life-cycle selection, and more in general, choice
problems are formulated according to the sense-making approach.
According to this approach, a new fact is interpreted when an individual is able to link
the new fact to existing knowledge in a coherent way. Natural language is the most
immediate tool to express such knowledge, because it allows individuals to represent
nuances, ambiguities, uncertainties and conflicts usually neglected by formal methods
to achieve coherence, simplicity and certainty. It is possible to have an idea of the
complex contextual knowledge used by an individual when he/she explains the motivations
of his/her judgement to other people. Through an explanatory discourse, people
introduce hypotheses on the basis of their own background knowledge in an attempt to
explain some evidence by relating new facts to known ones. Explanatory discourse means
any spoken or written discourse through which an individual tries to make explicit the
reasons justifying a choice or an evaluative judgment.
The idea proposed in this chapter is that mapping evaluation through explanations could
shed light on how people make their choices in organizations in terms of how they select
and relate evaluation criteria. For example, in evaluating different life-cycle models,
explanation can be a way to elicit from evaluators’ subjective knowledge which project’s
characteristics play a crucial role in determining his/her preference for one model rather
than another, which attributes are evaluated, what is the meaning of those attributes in
a specific context, and if and how the attributes are related among them.
Starting from this theoretical background the main assumptions behind the methodology
proposed in this chapter and presented in the next section can be summarized as follows:
a) Individual knowledge is incorporated in mental schemata and organizational
procedure. Patterns of action, scripts, models of behavior, facts, shared values, and
stereotypes resulting from ongoing sense-making activities are stored in both
individual and organizational memory (Weick, 1976; Walsh & Ungson, 1991).
Organizational and individual memories are socially constructed thanks to an
ongoing activity of individual interpretation and collective interaction (Berger &
Luckmann, 1966; Nicolini & Meznar, 1995).
b) Ambiguity related to inputs from the environment (e.g., ambiguous requirements
in software development) is resolved through interpretation; interpretation occurs
when an individual is able to develop an explanatory discourse linking the new fact
to existing knowledge in a coherent way (Schanck, 1986; Thagard, 1992; Zollo,
1998). Through explanatory discourses, people relate new facts to known ones by
introducing or implicitly assuming hypotheses to explain (enacted) “evidence” on
the basis of their own background knowledge.
c) Natural language is the most direct tool to express such knowledge, because it
allows the representation of nuances, ambiguities, uncertainties and paradoxical
assertions (Quinn & Cameron, 1983).
Following this background one expects that sense-making is deeper and more intense in
knowledge-intensive organizations (e.g., software companies) working in an unstable
environment, facing new and unexpected situations, and performing non-routine tasks.
Consequently, explanations provided by software developers regarding the problem of
choosing the best life cycle model in the early stages of a new software product
development given information about situational constraints should provide considerable
and deep knowledge about how people perceive and frame problems, select and
attribute meanings and relevance to critical variables, and eventually make their decisions
regarding the development process.
In the following section we propose a methodology based on causal mapping to elicit
and represent cognitive constructs that software developers activate when requested
to choose a life cycle model given some ambiguous and incoherent information about a
product’s requirements, customer’s need, and constraints about available resources,
costs and development time. Following the above assumptions, causal maps represent
a suitable methodological approach because:
a) they allow researchers to obtain a formal representation of cognitive constructs
activated by developers as a set of causal relationships; this representation is a
practical way to elicit and capture, at least partially and in simplified form, the mental
schemata and shared beliefs driving individual actions and choice models;
b) explanatory discourse can be easily translated into causal maps; explanations are
very often attempts to provide a picture of cause/effect relationships among facts
occurring in the real world;
c) causal maps can be constructed from the analysis of natural language information
contained in explanatory discourses.
The Methodology: Eliciting and
Mapping Developers’ Knowlege in the
Life Cycle Selection
The idea proposed in this chapter is that causal mapping can be employed to map
individual knowledge contained in explanations provided by software developers in the
early stages of a new project when a life cycle model has to be selected. Causal maps can
be used to model concepts and the relationships between the concepts contained in
explanatory discourses. The proposed methodology is articulated according to the
following steps:
a) Sample Selection: one or more development teams made up of experienced practitioners
in software development are selected according to certain criteria: i) level
of expertise, as recognized by other experts or estimated from their position; ii)
variety, in terms of roles, organizational positions, background (Calori, 2000).
b) Problem framing: a set of general framing questions concerning the main decision
variables involved in the problem of choosing the “right” life cycle model is
designed on the basis of the literature analysis and through a first involvement of
the developers;
c) Elicitation: framing questions are employed to collect explanatory discourses
through interviews;
d) Coding: explanatory discourses are analyzed to elicit concepts and relationships
among them (Fletcher & Huff, 1990); relevant concepts are described in detail and
reported in an interview dictionary;
e) Mapping: individual causal maps are used to represent concepts and relationships
between concepts and analyzed to identify input and output variables, and
influential and relevant concepts;
f) Comparison: individual maps are compared to identify similarities and differences
in the framing of the life cycle selection problem across different individuals;
g) Data validation is performed through inter-coder reliability and feedback from
interviewees;
h) Implication for decision support: once validated, the results emerging from the
causal map analysis can be used to improve the design of Decision Support
Systems (DSS) for the selection of the “best” life cycle model.
The choice of interviewing people instead of documentation analysis can be justified
given the characteristics of the context. First, documentation containing developers’
opinions and perceptions about how they frame the problem and make their choices
during the development process does not exist.
Second, managing a software development project requires coping with ambiguous
requirements, facing unexpected situations and making continuous adjustments to the
customer’s needs. Developers try to exploit existing knowledge and experience to cope
with such novelty. Interviews and explanations try to “force” developers to make sense
of an ever changing and unstable environment and to actively reconstruct (ex post) the
rationale behind their past behaviors and decisions. Linguistic explanation may play a
fundamental role in this reconstructing action because the tacit knowledge of informal
organization is very often not known and opaque to its same constructors (Schanck,
1986; Weick, 1976). The taken-for-granted world is the world of the “obvious” and its
opacity lies in the uncritical and automatic ways which it is enacted and recalled to
memory. For these reasons, interviews, which permit direct involvement and interaction,
favor the capture of tacit knowledge more than explicit sources such as written speech
and documentation.
The proposed methodology was tested through a case study in an Italian software
company producing software for accounting, management, office automation, and
telecommunication systems. Other activities of the company concern outsourcing
management of data elaboration centers. The company belongs to an important group
with more than 1,000 employees, half of which are software developers, and a turnover
of 80 million euros in 2002.
The activities of a software development team belonging to a specific organizational subunit
and involved in the realization of a new product were investigated in depth through
the proposed methodology. In the following, a step-by step application of the methodology
is presented to provide the reader with practical examples and an overview of the
critical issues and results that emerged during and after the field analysis.
Sample Selection
The sample was selected according to the above-mentioned criteria of level of expertise
and variety. More specifically, level of expertise was estimated through years of
experience in software development, involvement in the development of large projects,
and peer and management indication. Through such criteria, it was not difficult to identify
an “experienced team.” A satisfying degree of variety was ensured by the way the
company selects development teams. The team is usually made up of several developers
with different roles (system analyst, programmers, network experts, etc.) and by one or
more project managers. Of course team composition and duration depend on the
characteristics of the project and may change over time. It is worthwhile to note that the
number of people involved in the field research may vary as well depending on the
research purpose. For example, if one wants to build a very comprehensive collective
knowledge base (Calori, 2000), it is necessary to interview a large number of experts to
integrate as many different points of view as possible into representative and reliable
descriptions. On the other hand, if the research purpose is to investigate in-depth a given
organizational aspect through an action research approach (Argyris & Schon, 1978), the
number of people involved may be lower than in the previous case. As far as this chapter
is concerned, our aim was to use the field analysis primarily to test the proposed
methodology and to provide the reader with as many details as possible. Consequently,
the presented results are mainly descriptive. However, in the last two sections of this
chapter, we outline how the same methodology can be used to develop prescriptive or
supportive tools for software project management.
Problem-Framing
The aim of this step was to identify a set of general framing questions needed to structure
the interviews performed in the next step. These questions concerned the main decision
variables involved in choosing the “right” life cycle model. The framing questions have
been designed starting from the literature analysis. On this basis a framework describing
variables considered relevant to the final choice of the life-cycle has been constructed.
This framework was then presented to some of the company’s project managers and
developers to be validated and integrated before being employed in the interview phase.
Project managers’ and developers’ suggestions were collected and used to refine the
framework, whose final version is depicted in Figure 2. Given the way it has been
constructed, one can look at this framework as a sort of collective representation of the
problem of choosing the right life-cycle model, though very general, unspecific and
approximate.
Framework variables have been grouped into three main clusters:
a) Organizational variables concerning resource availability for the project, investments,
manager and team commitment, leadership of the project managers, and
organizational culture;
b) Customer-related variables such as requirements, concerns for costs, quality, time,
and visibility of changes, i.e., customer perception that his/her requests have been
recognized and satisfied through actual changes in the product;
c) Process variables related to production such as team competencies, maintenance,
relationships with the customer, and the possibility of further development.
Clusters have been obtained through two steps. In the first step an initial classification
was proposed by the authors on the basis of meanings that the considered variables
usually assume in the literature. The proposed classification was then refined through
developers’ observations and suggestions. Moreover, it emerged that each cluster
represented a dominant point of view shaping the way each interviewee perceives and
looks at the problem. In other words, among the interviewees, some developers empha-
Figure 2. Variables influencing the choice of life cycle model according to team
perception
Life-cycle
choice
Costs
Requirements
Times Quality Visibility of
changes
CUSTOMER
Possibility of
further developments
Maintenance Relationship with
Customers
Competencies
of the team
PROCESS
Resouces availability
Investments
Committment
O
R
G
A
N
I
Z
A
T
I
O N
Organizational
culture
o Leadership
sized more the customer-related cluster, others seemed more concerned about process,
while a third group paid more attention to organizational constraints.