After the modules have been tested, the entire program in assembled and there is another stage of testing, namely late inspection.
In a real sense, late inspection error rates indicate how well module testing was done. In general, module error rates are about 5%, as discussed on the Normal Development page. Late inspection error rates typically are about 0.3%. This suggests that module testing is about 90% effective. However some errors in modules are caught after module testing but before late inspection, so module testing per se probably is less than 90% effective.
In another field, Pollock and Zamora [1984] looked for errors in a machine-readable
text database. They found errors in 0.2% of all words.
| Study | Details |
Error Rate
|
| Beizer [1990] | Late inspection. 6.9 million lines of code. Faults per line of code at final inspection, after unit testing of individual pieces. |
0.24%
|
| Endress [1975] | Late inspection, DOS/VS mainframe operating system. 96 KLOC. |
0.2%
|
| Grady [1992] | Hewlett-Packard, 5 systems. |
0.2% - 2.3%
|
| Grady [1992] | Simpler program / More complex programs |
0.4% / 0.8%
|
| Nikora & Ryu [1996] | Systems testing, 5 systems at Jet Propulsion Laboratory. |
0.4% - 1.0%
|
| Putnam & Myers [1992] | Late inspection. Sample of 1486 projects, 117 million lines of code, 78 languages. Faults per line of code at final inspection, after unit testing of individual pieces. ~1,000 Lines of code. |
0.3%
|
| Putnam & Myers [1992] | Late inspection. Sample of 1486 projects, 117 million lines of code, 78 languages. Faults per line of code at final inspection, after unit testing of individual pieces. ~10,000 Lines of code. |
0.3%
|
| Putnam & Myers [1992] | Late inspection. Sample of 1486 projects, 117 million lines of code, 78 languages. Faults per line of code at final inspection, after unit testing of individual pieces. ~100,000 Lines of code. |
0.2%
|
| Putnam & Myers [1992] | Late inspection. Sample of 1486 projects, 117 million lines of code, 78 languages. Faults per line of code at final inspection, after unit testing of individual pieces. ~1,000,000 Lines of code. |
0.1%
|