Experts versus Novices

It is sometimes believed that there are great gulfs in error rates between experts and novices. For tasks such as brain surgery, this is probably the case. At the same time, in many other activities, while experts have lower error rates than novices, differences tend to be less than an order of magnitude.
 
Study Detail
Error Rate
Galletta, et al. [1993] Percentage of errors not found during code inspections. 12 errors total. Expert spreadsheet developers with 250 hours or more of spreadsheet development. Mean 1,548 hours (median 925). No better than novices but worked faster.
57%
Galletta, et al. [1993] Percentage of errors not found during code inspections. 12 errors total. Novice spreadsheet developers with 100 hours or less of spreadsheet development. Mean 54 hours (median 44)
55%
Grudin [1983] Typing errors per character transcribing magazine text subjects told not to detect errors, although some did.
Six expert typists (0.4% to 1.9%%)
1%
Eight novice typists
3.2%
Hayes, et al. [1985] Errors detected in problem-ridden text. Ratio of number of errors detected by expert writers to the number of errors detected by novices.
1.6
Ledgard [1980] Percent of erroneous commands with text editor.
Inexperienced (N=8) English-like editor / Notational editor
11%, 19%
Familiar (N=8) English-like editor / Notational editor
6.9%, 18%
Experienced (N=8) English-like editor / Notational editor
5.6%, 9.9%
Total sample
7.8%, 16%
Lesgold, et al. [1988] Diagnosing X-Ray pictures. Number of findings
Residents: 1st and 2nd year
6.56 
Residents: 3d and 4th year
6.63 
Experts
9.09 
Panko & Sprague [1998] Subjects developed a spreadsheet model based on simple task requiring almost no domain knowledge. Cell error rate (CER). The percentage of cells containing errors.
102 Undergraduate MIS majors
2.2%
26 MBA students with little or no spreadsheet development experience
2.3%
17 MBA students with more than 250 hours of spreadsheet development experience
1.1%
Reisner [1975] Percentage of query specifications with errors on final exam. Looking at specific types of errors, experts had much lower error rates for some types, only slighlty lower error rates for other types.
SEQUEL language, Programmers
22%
SEQUEL language, Novices
35%
SQUARE language, Programmers
22%
SQUARE language, Novices
45%
 

Copyright 1997-2008 Raymond R. Panko.