Late Inspection

In normal development, programmers create a number of modules. Each of these modules is tested with data testing, code inspection, or some combination of these and other tools.

After the modules have been tested, the entire program in assembled and there is another stage of testing, namely late inspection.

In a real sense, late inspection error rates indicate how well module testing was done. In general, module error rates are about 5%, as discussed on the Normal Development page. Late inspection error rates typically are about 0.3%. This suggests that module testing is about 90% effective. However some errors in modules are caught after module testing but before late inspection, so module testing per se probably is less than 90% effective.

In another field, Pollock and Zamora [1984] looked for errors in a machine-readable text database. They found errors in 0.2% of all words.
 
Study Details
Error Rate
Beizer [1990] Late inspection. 6.9 million lines of code. Faults per line of code at final inspection, after unit testing of individual pieces. 
0.24%
Endress [1975] Late inspection, DOS/VS mainframe operating system. 96 KLOC.
0.2%
Grady [1992] Hewlett-Packard, 5 systems.
0.2% - 2.3%
Grady [1992] Simpler program / More complex programs
0.4% / 0.8%
Nikora & Ryu [1996] Systems testing, 5 systems at Jet Propulsion Laboratory.
0.4% - 1.0%
Putnam & Myers [1992] Late inspection. Sample of 1486 projects, 117 million lines of code, 78 languages. Faults per line of code at final inspection, after unit testing of individual pieces. ~1,000 Lines of code.
0.3%
Putnam & Myers [1992] Late inspection. Sample of 1486 projects, 117 million lines of code, 78 languages. Faults per line of code at final inspection, after unit testing of individual pieces. ~10,000 Lines of code.
0.3%
Putnam & Myers [1992] Late inspection. Sample of 1486 projects, 117 million lines of code, 78 languages. Faults per line of code at final inspection, after unit testing of individual pieces. ~100,000 Lines of code.
0.2%
Putnam & Myers [1992] Late inspection. Sample of 1486 projects, 117 million lines of code, 78 languages. Faults per line of code at final inspection, after unit testing of individual pieces. ~1,000,000 Lines of code.
0.1%

Copyright 1997-2008 Panko.