General Error Detection

Proofreading catches about 90% of all nonword spelling errors and about 70% of all word spelling errors. The table below shows that error detection varies widely by the type of task being done.

In general, our error detection rate only approaches 90% for simple mechanical errors, such as mistyping a number.

For logic errors, error detection is far worse, often 50% or less.

For omission errors, where we have left something out, correction rates are very low.
 

Source

Event

Initial Error Rate

Detection Rate

Residual Error Rate

Allwood [1984]

Students solving statistics problems.

 

64%

 

All Errors

 

 

64%

 

Simple Mechanical Errors

 

 

83%

 

Simpler Logic Errors

 

 

48%

 

Complex Logic Errors

 

 

25%

 

Omission Errors

 

 

0%

 

Baddeley & Longman [1973]

Postal workers typing mail codes.

1%

50%

0.5%

Barnara et al. [1987]

8 experienced operators of hot strip steel mill, simulated production planning exercise. 95 errors, detected 74.

 

78%

 

Basili, Daskalantonakis, & Yacobelis [1994]

System functional specifications

 

72%-85%

 

Basili, Daskalantonakis, & Yacobelis [1994]

Software Functional Specification

 

67%-80%

 

Basili, Daskalantonakis, & Yacobelis [1994]

Detailed Design

 

49%-81%

 

Basili, Daskalantonakis, & Yacobelis [1994]

Code Inspection

 

25%-44%

 

Boehm and Basili (2001)

Many code inspections. Range / Median.

 

31%-93%

 

60%

Cohen, A. [1980]

Shadowed (repeated) spoken speech; against instructions, corrected phonological errors

 

60%

 

Cohen, A. [1980]

Shadowed (repeated) spoken speech; against instructions, corrected word transposition errors

 

15%

 

Cohen, G. [1980]

Shown all but last word in a sentence. Then shown last word. Asked it it is correct. Endings really correct. Percent declared to be correct.

 

90%

 

Eick et al. [1992]

Team inspection of 13 documents with teams.

 

80%

 

Eick et al. [1992]

Team inspection. Inspection of document with team of 8. Found 18 faults alone before meeting (12 only by 1 person), another 6 in meeting. Mathematically, probably missed another 4. So missed 4/28.

 

85.7%

 

Fagan [1976]

Percentage of programming errors found in team code inspection. 3.8% of lines found with errors in inspection. Errors found in another 0.8% after delivery.

 

82%

 

Hall, A. [1996]

Errors per LOC. Total, discovered after delivery.

1.1%

68%

0.75%

Hotopf [1980]

Daily life sample speech errors, S ample (oral speech errors)

 

71%-83%

 

Hotopf [1980]

Written grammatical errors, written undergraduate sample, per word

 

17%-65%

 

Johnson & Tjahjono, [1997]

Code inspection of program with seeded errors, errors found by individuals working alone. Best subject found 38%.

 

23%

 

Johnson & Tjahjono, [1997]

Code inspection of program with seeded errors, errors found by interacting teams of 3. Best group found 63%

 

44%

 

Johnson & Tjahjono, [1997]

Code inspection of program with seeded errors, errors found by nominal groups of 3 drawn from individuals working alone

 

43%

 

Jones [1986]

Team inspection. Errors found in formal technical review; summary from knowledge of several studies

 

75%

 

Jones [1998]

Defect removal rate suggested by Jones

 

 

 

Each software test (p. 198)

30%

Each formal design inspections (p. 199)

65%

Each formal code inspection (p. 199)

60%

Kolkhorst & Macina [1988]

Space shuttle. Percentage of errors caught in design and code inspections, compared to total errors found before release. Initially / late in project.

 

~50%

 

85%

Lackner [1980]

Shadowed (repeated) spoken speech; against instructions, corrected grammatical errors

 

28%

 

Lackner [1980]

Shadowed (repeated) spoken speech; against instructions, corrected semantic errors

 

14%

 

McCormick [1983]

Code inspection of AT&T system. 1/10 delivered errors of previous releases (6% of all LOC in previous). Total / Found after delivery.

4.4%

86%

0.6%

Noteboom [1980]

Meringer's corpus of linguistic slips, phonological corrections

 

75%

 

Noteboom [1980]

Meringer's corpus of linguistic slips, lexical corrections

 

57%

 

Rabbit [1990]

Flash one of two letters on display screen. Subject hits one of two keys in response. Allowed to correct.

5.8%

90%

0.6%

Rabbit [1990]

Flash one of two letters on display screen. Subject hits 1 of 1 keys in response. Cannot correct but signals error by hitting button. Corrected more errors than recognized: error correction partially automatic, unconscious. Percent of errors signaled.

 

64%

 

Rizzo, et al. [1986]

16 naļve subjects trained to use a database system. Trainined in tocking aloud methodology. 4 experimental sessions each. No time limits. 924 errors. Corrected 780.

 

 

84%

 

 

80%

Ruby, Dana & Biche [1975]

Errors during coding versus errors at validation stage (late). Faults per line of code.

10%

87%

2%

Schoonard & Boies [1975]

Line-oriented text editor. Error rate per word. 3.4% without correction, 0.52% with error correction.

3.92%

67%

0.52%

Wing & Baddeley [1980]

Grammatical errors per word, Cambridge examination, per word

 

1.24%

56%

 

0.54%

38%

Woods [1984]

23 nuclear plant crews in simulations. 8 failure events each. Detected errors overall.

 

~2/3

 

Woods [1984]

24 nuclear plant crews in simulations. 8 failure events each. Detected execution errors (slips and lapses) overall.

 

~Half

 

Woods [1984]

25 nuclear plant crews in simulations. 8 failure events each. Detected errors involving a misdiagnosis of the situation, overall.

 

0%

 

 

Copyright 1997-2008 Panko.