К оглавлению
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 
102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 
119 120 121 122 123 

Return, once again, to the Requirements Traceability Matrix (RTM) and follow

the columns to the right. There should be entries in the testing columns

showing where the requirement is tested and proved. If this does not ring true

or if you do not have an RTM, this is the time to build one. Refer to Attachment

7, Requirements Traceability Matrix.

In general, your RTM should look similar to Table 5-13.

Once the RTM is complete and the test reference data is appropriately entered,

it should expose the holes in the test string.

T a b l e 5 - 1 3 — R e q u i r e m e n t s T r a c e a b i l i t y M a t r i x ( R T M )

Unit System

SOW/ WBS S/C SOW/ Test Test

Spec Para Requirement Number Spec Para Number Para Monitor


4.3.1 Security 06-03-02 N/A T-0304 4.4.1 Smith


3.2.1 System weight 02-04-03 3.4.6 T-0045 3.4.1 Jones

shall be less

than 10,000


59c (NO) Unit Test findings were not reviewed for completeness and

not forwarded to be incorporated into Subsystem Tests and

the System Test.

If this is the condition with which you are faced at the subsystem or system

level, you have a lot of work to do. The initial requirements must be proved

somewhere in the test process.


The quick and dirty method of recovery is to lay out the initial requirements

as stipulated in the specification and then identify where the requirement was

proved in the System Test. If you can prove that the requirements were incorporated

and your customer accepts the process, you are in luck. If not, you will be

relegated to the ‘‘complete documentation’’ method.

The complete documentation method requires starting with the Requirements

Traceability Matrix (RTM) and flowing each requirement into the unit

and subsystem that will be built as a part of the system. The requirements must

first be flowed down from the requirements document (contract) to the specification

for each unit and subsystem and then flowed up in the Unit Test Plan,

the Subsystem Test Plan and the System Test Plan. This methodology is especially

critical in the development of a secure system where security qualification

must be proved and certified at every level of development. That philosophy

really should ensure whether the system is secure or not; then, there is no question.

If you do not have an RTM, refer to Cause Description ‘‘51a All Critical

Success Factors (CSFs) such as MTTR, MTBF, etc., have been documented and

understood’’ and Attachment 7 for suggestions for how to develop your RTM.

If you do not have a Unit Test Plan (UTP), refer to Cause Description ‘‘59a

Each Unit Test correctly reflects the requirement’’ for suggestions for how to

develop your UTP.

If you do not have a System Test Plan (STP) refer to Cause Description ‘‘60c

(NO) The System Test has not tested all elements of the system concurrently’’

for suggestions for how to develop your STP.

59d (NO) All Problem Test Reports (PTRs) were not captured,

dispositioned, or worked off.

All (PTRs) were not captured, dispositioned, or worked off when there is not

complete accountability for every error that occurred during test conduct.

Those errors were not assigned to responsible individuals for correction and

the results were not worked into the system. The System Test as written was

subsequently not run without error.