Experts versus Novices

It is sometimes believed that there are great gulfs in error rates between experts and novices. For tasks such as brain surgery, this is probably the case. At the same time, in many other activities, while experts have lower error rates than novices, differences tend to be less than an order of magnitude.
Study Detail
Error Rate
Galletta, et al. [1993] Percentage of errors not found during code inspections. 12 errors total. Expert spreadsheet developers with 250 hours or more of spreadsheet development. Mean 1,548 hours (median 925). No better than novices but worked faster.
Galletta, et al. [1993] Percentage of errors not found during code inspections. 12 errors total. Novice spreadsheet developers with 100 hours or less of spreadsheet development. Mean 54 hours (median 44)
Grudin [1983] Typing errors per character transcribing magazine text subjects told not to detect errors, although some did.
Six expert typists (0.4% to 1.9%%)
Eight novice typists
Hayes, et al. [1985] Errors detected in problem-ridden text. Ratio of number of errors detected by expert writers to the number of errors detected by novices.
Ledgard [1980] Percent of erroneous commands with text editor.
Inexperienced (N=8) English-like editor / Notational editor
11%, 19%
Familiar (N=8) English-like editor / Notational editor
6.9%, 18%
Experienced (N=8) English-like editor / Notational editor
5.6%, 9.9%
Total sample
7.8%, 16%
Lesgold, et al. [1988] Diagnosing X-Ray pictures. Number of findings
Residents: 1st and 2nd year
Residents: 3d and 4th year
Panko & Sprague [1998] Subjects developed a spreadsheet model based on simple task requiring almost no domain knowledge. Cell error rate (CER). The percentage of cells containing errors.
102 Undergraduate MIS majors
26 MBA students with little or no spreadsheet development experience
17 MBA students with more than 250 hours of spreadsheet development experience
Reisner [1975] Percentage of query specifications with errors on final exam. Looking at specific types of errors, experts had much lower error rates for some types, only slighlty lower error rates for other types.
SEQUEL language, Programmers
SEQUEL language, Novices
SQUARE language, Programmers
SQUARE language, Novices

Copyright 1997-2008 Raymond R. Panko.