2.3 Programmatic Explanations

К оглавлению
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 
119 120 121 122 123 

Each assertion listed in the Programmatic Performance Checklist, shown in

Table 2-1, is supported by a Cause Description. In the case of the Performance

Checklist, the support is to broaden and deepen the understanding of the assertion.

Following are explanations of the assertions found in the Programmatic

Performance Checklist.

1 STATEMENT OF WORK (SOW)

1a The SOW was properly defined.

An SOW is properly defined when it fully describes the products or services

to be delivered and states when and where they are to be delivered. Each product

or service (sometimes called Contract Line Item Numbers or CLINs) must be

separately listed. Additionally, the following documents should be referenced

but are usually not included:

Task Description

Deliverable Documents List (sometimes called Contract Data Requirements

List or CDRL)

Period of Performance

Schedule

Reference Documents (referenced but not included)

Modifying Factors (for example, the number of labor hours of specific

disciplines that must be provided)

Specification

Financial Information (usually referenced but not included in SOW)

Any item in or referenced by the SOW is a legal part of the SOW. Therefore,

each of these items must be understood. It is a good idea to search the entire

T a b l e 2 - 1 — P r o g r a m m a t i c P e r f o r m a n c e C h e c k l i s t

1 STATEMENT OF WORK (SOW) Explain Yes

1a The SOW was properly defined 10

1b The SOW is within our capabilties 12

1c The SOW was propertly interpreted 13

1d The SOW was properly negotiated 14

1e The SOW is properly monitored 15

1f The SOW is being properly performed 16

2 SPECIFICATION Explain Yes

2a The Specification was properly defined 17

2b The Specification is within our capabilities 18

2c The Specification was properly interpreted 20

2d The Specification was properly negotiated 20

2e The Specification was properly monitored 21

2f The Specification is being properly performed 22

3 POLICIES, PLANS, AND PROCESSES Explain Yes

3a There is a clear trail between standard policies and plans and the 23

Project/Program Plan and Technical Plan

3b There is a clear trail between customer policies and plans and the 23

Project/Program Plan and Technical Plan

3c There is a clear trail between enterprise policies and plans and the 24

Project/Program Plan and Technical Plan

4 ORGANIZATION Explain Yes

4a The numbers of personnel assigned to each task are correct 25

4b The mix of personnel to accomplish the task is appropriate 26

4c The personnel are acting and reacting as a team 26

5 TEAMING, ALLIANCES, AND SUBCONTRACTS Explain Yes

5a The subcontracts were properly defined 26

5b The subcontract tasks are within the capabilities of each team 28

member, partner, or subcontractor

5c The subcontracts were properly negotiated 28

5d The subcontracts are properly monitored 29

5e Team members, partners, and subcontractors are performing 29

properly

6 MATERIALS Explain Yes

6a Purchase Orders were properly written 30

(continues)

T a b l e 2 - 1 — ( C o n t i n u e d )

6b All vendors are competent to perform their tasks 31

6c Purchase Orders are properly monitored 31

6d Vendors are performing properly 33

7 PERSONNEL Explain Yes

7a Each person is competent to perform the tasks assigned 33

7b Each person is available when needed 34

7c Salaries/wages are equal to or less than those bid 34

7d Interpersonal conflicts do not exist 34

8 TRAINING Explain Yes

8a All personnel have been adequately trained 35

8b The training program is economical 35

9 DATA MANAGEMENT Explain Yes

9a The proper amount of data is being delivered on time 35

10 QUALITY Explain Yes

10a The Quality Plan is thorough, complete, and authorized 36

10b Specific quality characteristics were identified that are important to 36

the project

10c Quality is measured so that improvement or degradation is clear 36

11 FINAL DELIVERY Explain Yes

11a Final delivery was accepted by the customer without delay 36

11b Third-party or drop shipping is not involved 37

SOW and find all the requirements and the modifiers and group them together

for your own purposes.

A properly defined SOW will contain (either incorporated or appended) the

findings of the requirements discussions (negotiations). These findings are as

much a part of the requirements document (contract) as the initial document.

ADDITIONALRESOURCES:

MIL-STD-245 (see glossary)

1b The SOW is within our capabilities.

You can make a quick assessment of your ability to perform the task by using

the Experience Window in Table 2-2. Ask yourself the customer and product

questions and then compare your answers to the answers and capabilities shown

in the table.

T a b l e 2 - 2 — E x p e r i e n c e W i n d o w

Have Customer Have Product Capability to

Condition Experience Experience Perform

1 Yes Yes High

2 No Yes Moderate

3 Yes No Low

4 No No Unknown

That’s not the end of it, however. Just having customer experience and product

experience is not enough; it must be positive experience. If you have had

experience with this customer, but it was not positive experience, you must

neutralize the negative effects. If you do not do this, your ability to perform (or

win) is in doubt. The same is true of product experience. If you have negative

experience with a product, you are in the same boat. If either your customer

experience or your product experience is negative, it is likely you will move

downward at least two conditions on the chart. In other words, if you have

good product experience but bad customer experience, you no longer have a

high ability to perform. It is likely you now have a low to unknown ability to

perform. Generally speaking, negative experience is worse than no experience.

In addition to satisfying the conditions of the table and the extra conditions,

you must:

Provide the personnel required to perform the task.

Provide the facilities required by the task.

Provide the finances required by the payment schedule to support the

task.

Perform the requirements of the Specification (as evaluated under Cause

Description 2e).

1c The SOW was properly interpreted.

An SOW is properly interpreted when it is fully defined and when you fully

understand the products or services to be delivered and the conditions surrounding

the deliveries.

An SOW is properly defined when it fully describes the products or services

to be delivered and when and where they are to be delivered. Each product

or service (sometimes called Contract Line Item Numbers or CLINs) must be

separately listed. Additionally, the following documents should be referenced,

but are usually not included:

Task Description

Deliverable Documents List (sometimes called Contract Data Requirements

List or CDRL)

Period of Performance

Schedule

Reference Documents (referenced but not included)

Modifying Factors (for example, the number of labor hours of specific

disciplines that must be provided)

Specification

Financial Information (usually referenced but not included in the SOW)

Any item in or referenced by the SOW is a legal part of the SOW in a program

and an ethical part of the SOW in both a program and a project. Therefore,

each of these items must be understood. It is a good idea to search the entire

SOW and find all the requirements and the modifiers and group them together

for your own purposes.

To ensure you fully understand the SOW, you should:

Meet with the customer and use the content listing above as a guide for

your meeting. Ensure every item is covered.

Go through each paragraph of the SOW that is or might be in question.

Come to an understanding with the customer as to exactly what is wanted.

Come to an understanding with the customer on how recovery can be

made.

You should have the project manager and the technical manager on the proposal

team and the requirements definition (negotiation) team.

1d The SOW was properly negotiated.

A properly negotiated SOW is one that has a balance between all its elements,

is complete, and for which:

The amount of money to be paid is adequate to complete the task.

The time allowed is adequate to complete the task.

The requirements definition (negotiation) minutes are documented and

signed by both parties.

Again, follow the procedure outlined under Cause Description 1c above.

It is the responsibility of the requirements definition (negotiation) team to

ensure that this balance exists and that minutes are taken and confirmed. One

of the best ways to ensure balance is to require that the project manager be on

the requirements definition (negotiation) team. The project manager will ensure

there is a balance or will suffer the consequences.

And now, to be mugged by reality! Sometimes a strategic decision is made

by the company to accept a task at a price less than it will actually cost; this is

commonly called ‘‘buying in’’ (see glossary). In that event you must negotiate

your position with your management to understand who will take the ‘‘hit.’’

Get that understanding in writing!

1e The SOW is properly monitored.

The SOW is properly monitored when the work being performed is being

monitored by lead technical and program personnel using accepted monitoring

techniques such as:

Schedule Reviews

Budget Reviews

Design Reviews

Technical Interchange Meetings

Team Meetings

In-Process Reviews

Project Reviews

Customer Meetings

See the glossary for an expanded explanation of each of these meetings or

reviews. Due to the variability of projects, the content of these meetings and

reviews must be your own.

These interchanges must be conducted at frequent intervals. The lower the

position in the hierarchy, the more frequent the interchange needs to be. In

other words, Team Meetings should be held more often than Project Reviews,

and Project Reviews should be held more often than Customer Meetings, and

so forth.

Just because the SOW is being properly monitored does not necessarily mean

the program is running properly; it only means that it is being monitored properly.

The point is that if the program is not being monitored properly, you will

not know it until it is too late.

These meetings and reviews pervade the entire process, as Table 2-3 shows.

T a b l e 2 - 3 — M e e t i n g s a n d R e v i e w s

Review or Meeting Cause Description Appearance

Schedule Reviews 1f, 5e, 6d

Budget Reviews 1f, 5e

Design Reviews 11a, 51e, 52a, 53

Technical Interchange Meetings 1f, 5d, 5e, 6d

Subcontractor Meetings 5d, 5e

In-Process Reviews 5d

Customer Meetings 5d, 5e

The reviews must have metrics established to indicate if each event is in

tolerance or out-of-tolerance. The content of each of the reviews must be appropriate

for that review.

1f The SOW is being properly performed.

The SOW is being properly performed when the Design Reviews, the In-

Process Reviews, the status meetings, the schedule, and the actual production

of the product is on schedule, within budget, and is being produced in accordance

with the Specification. These factors should be evident in such reviews

as:

Schedule Reviews

Budget Reviews

Design Reviews

Technical Interchange Meetings

Team Meetings

In-Process Reviews

Project Reviews

See the glossary for an expanded explanation of each of these meetings or

reviews. Due to the variability of projects, the content of these meetings and

reviews must be your own.

The reviews must have metrics established to indicate whether each event is

in tolerance or out-of-tolerance.

2 SPECIFICATION

2a The Specification was properly defined.

The proof, as they say, is in the pudding. Can you understand exactly what

the customer wants? Is it testable and is it provable? If you can answer YES

to both those questions, the Specification is properly defined. A well-defined

Specification contains at least the following topics:

Scope of the Document

Applicable Documents

Requirements

Item Definition

Performance Characteristics

• The performance requirements related to manning, operating, maintaining,

and logistically supporting the prime item to the extent these

requirements define or constrain design of the prime item and include

response time, throughput rates, and exclusion times

Physical Characteristics

• The design constraints and standards necessary to assure compatibility

of prime item components

The electrical, mechanical, functional, and other interfaces between the

principal item being specified and other items with which it must be compatible

The major components of the principal item and the primary interfaces

between such major components

Qualification Requirements (for software) or Quality Assurance Provisions

(for hardware)

Process Requirements, if needed

Materials Requirements, if needed

There are several types of Specifications. MIL-STD-490 has established and

defined five different Specification (Spec) types as well as a number of subtypes.

The standard provides a great deal of good information regarding the content

and purpose of each Specification type. The Specification types are shown in

Table 2-4.

T a b l e 2 - 4 — S p e c i f i c a t i o n T y p e s

Type Specification

A System/Subsystem/Segment

B Development

B1 Prime Item

B2 Critical Item

B3 Noncomplex Item

B4 Facility of Ship

B5 Software

C Product

C1a Prime Item Function

C1b Prime Item Fabrication

C2a Critical Item Function

C2b Critical Item Fabrication

C3 Noncomplex Item Fabrication

C4 Inventory Item

C5 Software

D Process

E Material

2b The Specification is within our capabilities.

A quick assessment can be made of your capabilities to perform by using the

Experience Window in Table 2-5.

T a b l e 2 - 5 — E x p e r i e n c e W i n d o w

Have Customer Have Product Capability to

Condition Experience Experience Perform

1 Yes Yes High

2 No Yes Moderate

3 Yes No Low

4 No No Unknown

In addition to satisfying the conditions of the table above, you must:

Provide the personnel required to perform the task.

Provide the facilities required by the task.

Provide the finances required by the payment schedule to support the

task.

Perform the requirements of the Specification.

The Specification is within your capabilities if you have previously established

credentials in performing each requirement.

In order to fill in the ‘‘Have Product Experience’’ column in Table 2-5 properly,

you may need to construct a matrix similar to the one shown in Table

2-6. The matrix lists all the requirements or tasks along the side and the programs

(including Independent Research and Development (IR&D) programs) that the

company has performed across the top. Every requirement or task should have

an ‘‘X’’ at the intersection between the task and at least one program.

T a b l e 2 - 6 — T a s k Q u a l i f i c a t i o n

Project A Project B Project C Project D Project E

Task 1 X

Task 2 X X

Task 3 X

Task 4 X

Task 5 X X

Task 6

If you do not have the requisite qualifications and thus cannot enter an ‘‘X’’

into the intersect, continue with the process to bring your capabilities up to

the requirements of the Specification. Refer to Cause Description 2b (NO) for

recovery.

2c The Specification was properly interpreted.

A Specification is properly interpreted when you fully understand the products

or services to be delivered. Each product or service must be fully described

in the Specification.

At a minimum, the Specification should contain:

Scope of the Document

Applicable Documents

Requirements

Item Definition

Performance Characteristics

• The performance requirements related to manning, operating, maintaining,

and logistically supporting the prime item to the extent these

requirements define or constrain design of the prime item and include

response time, throughput rates, and exclusion times

Physical Characteristics

• The design constraints and standards necessary to assure compatibility

of prime item components

The electrical, mechanical, functional, and other interfaces between the

principal item being specified and other items with which it must be compatible

The major components of the principal item and the primary interfaces

between such major components

Qualification Requirements (for software) or Quality Assurance Provisions

(for hardware)

Process Requirements, if needed

Materials Requirements, if needed

2d The Specification was properly negotiated.

The Specification was properly negotiated if there is a thorough understanding

and agreement on the part of both parties as to what constitutes the scope,

the schedule, and the budget. A well-defined Specification contains at least the

following topics:

Scope of the Document

Applicable Documents

Requirements

Item Definition

Performance Characteristics

• The performance requirements related to manning, operating, maintaining,

and logistically supporting the prime item to the extent these

requirements define or constrain design of the prime item and include

response time, throughput rates, and exclusion times

Physical Characteristics

• The design constraints and standards necessary to assure compatibility

of prime item components

The electrical, mechanical, functional, and other interfaces between the

principal item being specified and other items with which it must be compatible.

The major components of the principal item and the primary interfaces

between such major components

Qualification Requirements (for software) or Quality Assurance Provisions

(for hardware)

Process Requirements, if needed

Materials Requirements, if needed

The negotiator must keep thorough and complete minutes regarding all

changes to the Specification and these changes must be covered by both schedule

and budget considerations. The minutes will have been signed by both parties

of the negotiation. Later these minutes will be incorporated into the Specification

as changes.

2e The Specification was properly monitored.

A Specification that is properly monitored is one that is under constant and

complete control and has a change process that controls all changes made to

the baseline.

One of the first things to be done in the Planning Phase is to develop a

Requirements Traceability Matrix (RTM). You should have one RTM for the

programmatics (SOW) and one for the product (Specification). Once you know

where the requirement is being satisfied, it should be reasonably easy to assign

a person to monitor the performance. The content of the RTM should follow a

requirement from beginning to end. An example of such a document is shown

in Table 2-7.

T a b l e 2 - 7 — R e q u i r e m e n t s T r a c e a b i l i t y M a t r i x ( R T M )

Unit System

SOW/ WBS S/C SOW/ Test Test

Spec Para Requirement Number Spec Para Number Para Monitor

SOW

4.3.1 Security 06-03-02 N/A T-0304 4.4.1 Smith

Spec

3.2.1 System weight 02-04-03 3.4.6 T-0045 3.4.1 Jones

shall be less

than 10,000

pounds

In this case, an additional column should be added for the name of the

monitor.

For additional information, see Attachment 7.

2f The Specification is being properly performed.

The Specification is being properly performed when the Design Reviews or

Milestone Reviews are properly passed and accepted by the customer and the

product is fabricated or produced in accordance with the design and accepted

by the customer.

You can only answer YES to this assertion if you answered YES to assertions

2a, 2b, 2c, 2d, and 2e. If you answered NO to any of those assertions, you must

rectify the situation before proceeding.

Ensure that every major milestone, such as the Preliminary Design Review

(PDR) has inch stones leading up to it. Performance must be evaluated at every

inch stone. Evaluate performance using the requirements of the major mile-

stone and develop a ‘‘percent complete’’ chart for each inch stone and for the

milestone.

3 POLICIES, PLANS, AND PROCESSES

3a There is a clear trail between standard policies and plans

and the Project/Program Plan and Technical Plan.

The Project/Program and Technical Plans link to standard policies and plans

through two avenues. One avenue is through enterprise policies and processes;

the other is through the requirements document (contract). The requirements

document (contract) references those standards through the Statement OfWork

(SOW) and the Specification.

You should have a Standards Traceability Matrix (STM) similar to Table

2-8.

T a b l e 2 - 8 — S t a n d a r d s T r a c e a b i l i t y M a t r i x ( S T M )

STANDARDS APPEARANCE

Industry Customer Enterprise Project Plan Technical Plan

ISO-9001 ISO-9001 Enterprise Quality Para 4.6.8 Part I, Para

Policy 09350 4.5.6

MIL-STD-100 Enterprise Engineering N/A Part II, Para

Standards 06050 1.2.3

The STM shown here is a multipurpose table in that the requirement source

such as the contract paragraph, the company standard, and the standards documents

are all included in one chart. You can use this technique to divide the

three requirements into three separate charts. The STM is further explained in

Attachment 13.

3b There is a clear trail between customer policies and plans

and the Project/Program Plan and Technical Plan.

Customer policies and plans are linked to the Project/Program Plan through

the requirements document (contract). The Statement Of Work (SOW) and the

Specification should clearly spell out those customer policies, processes, and

plans that are invoked as a part of the contract.

TEAMFLY

You should have an STM similar to Table 2-9.

T a b l e 2 - 9 — S t a n d a r d s T r a c e a b i l i t y M a t r i x ( S T M )

STANDARDS APPEARANCE

Industry Customer Enterprise Project Plan Technical Plan

ISO-9001 ISO-9001 Enterprise Quality Para 4.6.8 Part I, Para

Policy 09350 4.5.6

MIL-STD-100 Enterprise Engineering N/A Part II, Para

Standards 06050 1.2.3

The STM shown here is a multipurpose table in that the requirement source

such as the contract paragraph, the company standard, and the standards documents

are all included in one chart. You can use this technique or separate the

three requirements into three separate charts. The STM is further explained in

Attachment 13.

3c There is a clear trail between enterprise policies and plans

and the Project/Program Plan and Technical Plan.

A clear trail between enterprise policies and plans and the Project/Program

Plan and Technical Plan exists when the Vision drives the Mission Statement,

which, in turn, drives the Policies. The Policies are the foundation that sets

standards for the Processes and Plans. The Project Plan and Technical Plan are

a fundamental part of all the plans and should reflect the parts, sections, chapters,

and paragraph upon which they are based.

The documentation of a corporation, company, or enterprise is near the top

of the management ‘‘to do’’ list on how to run a company. The only things at

higher rungs are the Strategic Plan, the Mission Statement, and the Vision. If

the entity is ongoing and the documentation is at fault, it is not a good sign. If

the documentation does not exist, it’s even worse. If it is a new start, the entity

should look upon the situation as a learning experience and fix the documentation.

If the project documentation is lacking and the enterprise documentation

is in order, it is an indication that there is a disconnect between the enterprise

and the programs it runs. This can be overcome by using an ‘‘Executive Summary’’

that is agreed to by enterprise management and program management

before the program is kicked off. The continuity is maintained by a Project

Advisory Council, a group of senior executives assigned to follow and advise

each project.

You should have an STM similar to Table 2-10.

T a b l e 2 - 1 0 — S t a n d a r d s T r a c e a b i l i t y M a t r i x

STANDARDS APPEARANCE

Industry Customer Enterprise Project Plan Technical Plan

ISO-9001 ISO-9001 Enterprise Quality Para 4.6.8 Part I, Para

Policy 09350 4.5.6

MIL-STD-100 Enterprise Engineering N/A Part II, Para

Standards 06050 1.2.3

The Standards Traceability Matrix shown here is a multipurpose table in that

the requirement source such as the contract paragraph, the company standard,

and the standards documents are all included in one chart. You can use this

technique or separate the three requirements into three separate charts. The

STM is further explained in Attachment 13.

4 ORGANIZATION

4a The numbers of personnel assigned to each task are correct.

When you first start your project, the organization chart and staffing table

from the proposal will probably be your guides. As the project progresses, performance

will be the rule. You must constantly ask yourself: ‘‘Is the job getting

done?’’ Then, follow up with these questions: ‘‘Is the job getting done without

working overtime?’’ ‘‘Is the morale of the team high?’’ If the answer to all those

questions is YES, you’re probably in good shape. You must however also ask

yourself: ‘‘Do I have too many people?’’ The job could be getting done, you are

not working overtime, and morale is high but you have too many people. Yes,

that actually does happen on projects! Optimizing manpower is a constant task.

If your project is a large one that will last over a period of time, it is possible

that the organization chart and manpower table will change. It is normal to

have one organization and manpower table for design and another for test,

production, and so forth. That’s another reason you must ask yourself the opening

question frequently.

4b The mix of personnel to accomplish the task is appropriate.

The mix of personnel to accomplish the task is appropriate if the job is

getting done and the mix of personnel matches the mix shown on the organization

chart and the staffing plan.

Mix, of course, means the different types of persons assigned, not just the

numbers. The starting point is usually the proposal or the document that was

the basis for providing the correct technical manpower and for costing the manpower

for the project. The correct mix is absolutely essential. It is the basis for

correctly accomplishing the task technically and for maintaining the budget.

It is important to keep the organization chart and the staffing plan up-todate.

Keep all staffing plans from the proposal on to show the transitions of

personnel from time to time in the project. This will help when accounting for

personnel changes and will provide an operational history for the next bid.

4c The personnel are acting and reacting as a team.

Just because a group of individuals are assigned to a project does not mean

they are a team. In order to be a team, the individuals must act and react with

regard to the team’s goals. The group acts and reacts as a team when the responses

to team goals are greater than the responses to individual goals.

While actions and reactions are the most important factor, there are other

considerations that will help the group to continue to act as a team. First, and

most important, is to have had team training using a facilitator qualified to

conduct team training and an acceptable team training format. Second, which

is usually a result of team training, is that the team has a Vision (see glossary)

and a Mission Statement (see glossary).

5 TEAMING, ALLIANCES, AND SUBCONTRACTS

5a The subcontracts were properly defined.

The tasks of subcontractors, which include Teaming (see glossary) and Alliances

(see glossary), are properly defined if there is a clear trail between the

customer’s requirement documents (SOW and Specification) to the subcontractor’s

requirements document (subcontract SOW and subcontract Specification)

using a Requirements Flow-down Matrix (RFM) and from the subcontractor’s

requirements documents through the subcontractor’s process using a Subcontract

Requirements Traceability Matrix (SRTM).

An RFM with the characteristics shown in Table 2-11 should be used.

Additional information can be found in Attachment 8.

Additionally, the subcontractor should have an SRTM, such as that shown

in Table 2-12.

The current industry standard is a family of products titled DOORS (for

large and enterprise wide projects) and DOORSrequireIT (for smaller projects).

T a b l e 2 - 1 1 — R e q u i r e m e n t s F l o w - D o w n M a t r i x ( R F M )

Company Design S/C Plan S/C A S/C B

Spec Para Reqt WBS Plan Para Para Para Para

1.3.2 02-03-01 5.3.2 5.3.2 1.3.2 1.3.2

1.3.3 02-03-02 5.3.3 5.3.3 1.3.3 N/A

1.3.4 02-03-03 5.3.4 5.3.4 1.3.4 1.3.4

QA Plan 04-01-01 8.2.6 8.2.6 4.3.6 4.3.6

CM Plan 05-01-01 9.3.1 9.3.1 5.6.2 5.6.2

T a b l e 2 - 1 2 — S u b c o n t r a c t s R e q u i r e m e n t s

T r a c e a b i l i t y M a t r i x ( S R T M )

Unit System

SOW/ WBS S/C SOW/ Test Test

Spec Para Requirement Number Spec Para Number Para Monitor

SOW

4.3.1 Security 06-03-02 N/A T-0304 4.4.1 Smith

Spec

3.2.1 System weight 02-04-03 3.4.6 T-0045 3.4.1 Jones

shall be less

than 10,000

pounds

Both are commonly referred to as ‘‘Doors.’’ Additional information on Doors

can be found in Attachment 7.

5b The subcontract tasks are within the capabilities of each

team member, partner, or subcontractor.

The subcontract tasks are within the capabilities of each team member, partner,

or subcontractor if each has performed the same or a similar task before

and no substantial changes have since occurred (i.e., critical personnel are still

in place, and critical facilities are still available) or the subcontractor has some

unique capability or capacity to perform the task. Such data should be maintained

by the enterprise as a part of the Vendor/Subcontractor Database. If your

enterprise does not keep such a database, a ‘‘quick fix,’’ can be reached by

constructing a matrix with the tasks along the side and a place for program

entries across the top. The potential subcontractor then identifies the program

where the same or a similar task has been performed.

If anything has changed (i.e., critical personnel are no longer available, critical

facilities are not available, etc.) you should go to Chapter 4, Cause Description

5b (NO), Recovery section. If the subcontractor has a unique capability or

capacity to perform that has not been tried before, this is the time to create a

Risk Mitigation Plan (see Attachment 3).

As project manager, it is a good idea (in my opinion absolutely necessary)

that you have written confirmation of this fact. Do not simply accept the statements

of Marketing (who usually makes Teaming Agreements) or Management

(who usually makes Alliances) that the company with whom you are aligned is

qualified to perform the task. Teaming Agreements and Alliances are frequently

made for political purposes. If you find such a situation exists, refer to Chapter

4, Cause Description 5b (NO) for recovery.

5c The subcontracts were properly negotiated.

The subcontracts were properly negotiated if they are fully understood by

both parties and contain a balance between all the elements, and if:

The amount of money to be paid is adequate to complete the task.

The time allowed is adequate to complete the task.

Both you and the subcontractor understand what is to be done and when.

It is the responsibility of the requirements definition (negotiation) team to

ensure that this balance exists and that minutes are documented and signed.

One of the best ways to ensure balance is to require that the project manager

be on the requirements definition (negotiation) team. The project manager will

ensure there is a balance or will suffer the consequences.

5d The subcontracts are properly monitored.

The subcontract is properly monitored when the work being performed is

being monitored by lead technical and project personnel using accepted monitoring

techniques such as:

Subcontract Progress Reviews—Subcontractor presents technical progress,

budget status, schedule status, deliverables status, and data status

Subcontractor Meetings—Special, single subject meetings as required

Technical Interchange Meetings (TIMs)—Informal reviews of technical

subjects

Design Reviews—Formal reviews of designs. Subcontractor presents and

defends the design and its support

In-Process Reviews—Usually, informal reviews between milestones

Pretest Meetings—Briefings to establish the basis for a test

Posttest Reviews—Review of test data and issuance and formalization of

action items and, if appropriate, sign-off

These reviews must be conducted at frequent and consistent intervals. The

lower in the hierarchy (e.g., the project is lower in the hierarchy than company,

etc.), the more frequent the review.

Simply conducting these meetings and reviews does not mean the subcontract

is performing properly; it only means that the subcontract is being monitored

properly. But, if the subcontract is not being monitored properly, you will

not know if it is performing properly.

Within each of these must be monitoring points or metrics that indicate that

an event is in tolerance or out-of-tolerance.

5e Team members, partners, and subcontractors are

performing properly.

Team members, partners, and subcontractors are performing properly when

all monitored events are being performed on schedule, within budget, and in a

technically competent way. The method you use in determining this status is to

conduct regular and frequent reviews at strategic points in the process to ensure

that performance is proper. Such reviews are presented in detail in Cause Description

5d.

Monitored events are those events that are typical for a particular review.

Usually, Schedule Reviews, Budget Reviews and Progress Reviews are held concurrently.

Within each review there must be monitoring values and metrics to

determine if the project is performing in tolerance. While projects vary infinitely

in subject matter there are some values that must be monitored on all projects.

Such meetings are frequently called Plans, Progress, and Problems Meetings.

Such values and metrics are, at a minimum:

Actual cost to date versus planned cost to date

Actual performance to date versus planned performance to date

Cost at completion

Completion date

Performed activities versus planned activities for last period

Problems and recommended resolution

Planned activities for next period

6 MATERIALS

6a Purchase Orders were properly written.

Each Purchase Order is complete and properly written when it contains:

Reference Number, Order Date, Vendor, Contact Information, Name of Item,

Stock (Catalog) Number, Number of Units, Price, Delivery Schedule, Delivery

Location, Purchaser, and Authorizing Signature.

It is beneficial that the information contained in the Purchase Order be complete

and properly written for a few reasons. For example, it conveys to the

vendor exactly what is expected. Also, you must know exactly the status of each

Purchase Order because of the impact it has on your schedule and your budget.

Most companies have preprinted Purchase Order forms. If yours does not,

create your own. Even if your Purchase Order form is nothing more than a

memo, at least it is documentation of what has been ordered and provides a

basis for the schedule and for financial accountability. If your preprinted form

does not contain all the information above, I suggest you add the information

within the body of the Purchase Order.

6b All vendors are competent to perform their tasks.

All vendors are competent to perform their tasks if they have passed the

criteria set forth in your enterprise standards. If you do not have enterprise

standards, the following should be established as the criteria:

Technical Performance

Cost Performance

Delivery Performance

Management Performance

Procurement Policies and Plans

Quality Assurance Program (see Attachment 6 for Quality Assurance

Plan)

Cost of Quality Position (see glossary)

Figure 2-1 on the following page provides a framework for collecting this

data.

6c Purchase Orders are properly monitored.

The Purchase Order is properly monitored when the work being performed

is being monitored by the Materials Manager calling upon technical and program

personnel as required and using accepted monitoring techniques such as:

Vendor Progress Reports

Vendor Meetings

In-Process Reviews

The above include schedule and budget, if proper.

These reviews must be conducted at regular, frequent, and strategic intervals.

Simply conducting these meetings and reviews does not mean the vendor is

performing properly; it only means that the Purchase Order is being monitored

properly. But, if the Purchase Order is not being monitored properly, you will

not know if the vendor is performing properly.

F i g u r e 2 - 1 — V e n d o r E v a l u a t i o n S h e e t

VENDOR EVALUATION

Date 4-Jul-02

Program High-Flyer

Subcontractor/Vendor National Software

Equipment/Software Analog Selction Algorithm

Evaluator G. Smith

Scale Factor 0-5

Item Consideration Rating*

1 Organization 3

2 Management 4

3 Manpower 5

4 Access to Management 5

5 Processes 3

6 Procedures 2

7

8

9

10

Subtotal** 22

No. of items rated** 6

Average of ratings (Subtotal/No of items)** 3.7

*An evaluated number within the Scale Factor.

**Calculated number.

M-M Form

Within each of these reviews there must be monitoring points or metrics

that indicate that an event is in tolerance or out-of-tolerance.

If schedule is critical, the Purchase Order should include an incentive or

liquidated damages clause that is invoked in the event the delivery time is not

kept.

6d Vendors are performing properly.

Vendors are performing properly when all monitored events are being performed

on schedule, within budget, and in a technically competent way. The

method you use in determining this status is to conduct regular and frequent

reviews at strategic points in the process to ensure that performance is proper.

Such monitored events typically are:

Vendor Meetings, including Schedule Reviews and Budget Reviews1

Technical Interchange Meetings2

In-Process Reviews

7 PERSONNEL

7a Each person is competent to perform the tasks assigned.

It should be apparent whether or not each person is competent to perform

the tasks to which he or she is assigned. It is not usual to have job descriptions

for project positions, so the personnel must learn what is expected of them in

other ways. The best way is through team training, where each individual is

apprised of the expectations of his position and the input the individual needs

from others.

Knowing what is expected is one thing. Competency is quite another. The

best way to establish competency is to interview each person before assigning

individuals to the project. In a small company, you can usually rely on reputation

and personal contact. In a large company, you may need to interview the

individuals and read their background re´sume´s. When the project is running,

you judge competency by your observation, by MBWA (Management By Walking

Around), and by asking questions of other team members.

Competency (a long-term characteristic) is not the same as reaction (a shortterm

characteristic). In other words, don’t characterize a person based on a

TEAMFLY

single sample of work or on his or her response on one day (the person may be

having a bad day). Conversely, don’t expect the ‘‘leopard to change its spots’’

based on a single day of responses.

Refer to Cause Description 7a (NO) for recovery.

7b Each person is available when needed.

If each person is available when needed, it will be self-evident. As project

manager you will constantly be scanning the personnel and you can tell if anyone

is missing. Watch the wording here carefully. As project manager, you must

control the personnel. If you operate in a matrix organization, you must have

an understanding with the functional manager that, once assigned, the personnel

report to you. You must be a party to any planned absences.

No matter if you operate in a matrix or a projectized organization, you must

keep up with your people and what they are doing from day to day.

7c Salaries/wages are equal to or less than those bid.

It should be clear by simply looking at the details supporting your budget

that the individual salaries are equal to or less than those that were bid. You

must use your judgment in this case. Use the bottom line of the budget as the

guide—it is easier to achieve the bottom line than to achieve each and every

line item. It may be to your benefit to have an individual of higher pay in a

certain position and several of lower pay in other positions. You must make

that judgment.

Take care if your program is longer than a few months and you use the

matrix form of management. You may find that salaries fit the bid profile at the

start of the program but some of your people get raises during the conduct of

the program. People should be rewarded for good work and should have raises

when they are due. Just make sure those raises are built into your budget.

7d Interpersonal conflicts do not exist.

Interpersonal conflicts do not exist when there is mutual respect between the

members of the team.

If you have no interpersonal conflicts at all, one of two things is happening:

You are the luckiest project manager alive, or you are not very close to the

people part of your project. Even if you answered YES to this assertion, it might

be a good idea to look at the Recovery association in Chapter 4, Cause Description

7d (NO) just to give the question a little more depth.

8 TRAINING

8a All personnel have been adequately trained.

All personnel have been adequately trained when they know their basic jobs

thoroughly, when they know the mission of the team, when they know the

product(s) to be produced, and when they know what part they play on the

team and in developing the product(s).

It is advantageous to the project and the team and frequently to the individual

that they be cross-trained. If it is not in your plan, it should be—unless it is

against union rules or against some other rule over which you have no control.

8b The training program is economical.

The training program is economical when the dollar cost of the training

program is equal to or less than the value derived from the training program.

Further, in order to be economical, the value imparted to the individuals attending

the training class must be worth the time employed in attending the program,

for every person in attendance.

9 DATA MANAGEMENT

9a The proper amount of data is being delivered on time.

The proper amount of data is being delivered on time when the data deliveries

match the data required in the Data Plan, and the Data Plan matches the

requirements. Some documents are delivered once (i.e., System Test Results),

and some require multiple deliveries (i.e., Monthly Status Reports).

Each line item of deliverable documentation in the requirement should contain

a delivery date or schedule of dates, a format, a content requirement, and

the name of the person responsible for generating the data. If it does not, you

should create these requirements. All requirements are then included in the

Data Plan.

Your deliveries may be formalized as they are in a government contract. In

this case, you will have a Contract Data Requirements List (CDRL) that spells

out what is to be delivered and numerous Data Item Descriptions (DIDs) that

spell out the frequency of submission and the format of the submission.

Your Data Plan should specify in what form the data is to be delivered and

to whom. Many projects have engineering deliver raw or refined reports to the

Data Manager, who adds the boiler plate, coordinates the review, reproduces

the number of copies necessary, and handles distribution. These same techniques,

except for reproduction, can be used for electronic transmissions as

well.

10 QUALITY

10a The Quality Plan is thorough, complete, and authorized.

The Quality Plan is thorough, complete, and authorized when it completely

addresses and fills the requirements of the Quality Standards imposed by standard,

customer, or your internal requirements document and it is authorized

by the highest quality official of the enterprise. (See Attachment 6 for a Quality

Assurance Plan outline complete with explanations and details.)

10b Specific quality characteristics were identified that are

important to the project.

Specific quality characteristics were identified that are important to the project

when these characteristics are documented and used as a checklist.

10c Quality is measured so that improvement or degradation is

clear.

Quality is measured so that improvement or degradation is clear when each

quality characteristic is measured and tracked via metrics.

Quality is measured so that improvement or degradation is clear when each

quality characteristic shows clear improvement or degradation between the current

reading and some standard or the last reading.

11 FINAL DELIVERY

11a Final delivery was accepted by the customer without delay.

It is expected that final delivery will be accepted by the customer without

delay because it is one of the most important steps in the closure process. In

order to ensure that this happens, the following must have been completed:

Design Reviews or milestone reviews signed by customer

All In-Process Documentation completed

All Deliverable Documentation delivered

All In-Process Tests accepted by the customer

The final System Test accepted by the customer

Product shipped to the point of delivery and in deliverable condition

11b Third-party or drop shipping is not involved.

Third-party or drop shipping is not involved whenever the product is

shipped directly from your facilities to the customer’s facilities.

If drop shipping is involved, you must have absolute control over shipping

and receiving of the product. See 11b (NO).

Notes

1. Usually not for an FFP contract.

2. Unless the product is a commercially available commodity (i.e., a catalog item).