Surveys of Developers

The following studies surveyed spreadsheet developers. This table is adapted from Panko and Halverson, "Spreadsheets on Trial," 1996. In these studies, the subject often described a single spreadsheet.
 
Study Method/Sample Selected Findings
Cragg & King [1993] Audit of 20 SSs from 10 firms. 150 to 10,000 cells. Averaged 3 days to develop. Median use 6 months. Mean number of updates 7, usually because of lack of initial specification. 8 used by others. 6 had experienced problems. 5 had errors; but audit only lasted 2 hrs; "probably more." Informal iterative development. 10 built without prior design or planning; 8 had prior design, usually rudimentary. 10 used poor layout. 1/3 used cell protection. Only 2 had good comments. All but one tested with data, but only 1 tested by another person; 1 by an auditing package. 7 used historical data; 5 used cross-checks. Only ½ had documentation of any kind; usually only sample input & output. One commented had troubles understanding old SSs.

 

Davies & Ikin [1987] Audit of 19 SSs from 10 people in 10 firms. 4 had actual errors. General belief in no errors. 1/2 used no cell protection. Only 2 used audit packages. Manual audits "rare." 74% had structural problems; 68% had inadequate documentation.

 

Floyd; Walls, & Marr [1995] 72 end users in 4 corporations Average size 6,000 cells. Material value (economic impact of decision) not correlated with controls. High level of confidence. Mostly informally trained in SS.

 

Hall [1996] 106 SS developers Averaged 218 KB. 36% had links to other SSs; 21% had links to databases; 55% used macros; 47% used If function. Only 7% of low importance. 27% modified existing corporate data; 49% created new corporate data. Only 17% run once or a few times. Only 18% prepared for others to use. Examined 55 controls for 82 non-trivial SSs. In most cases, experts used more; respondents felt should be used more than was. Use of controls: Half modular; 49% cell protection. 71% used test data; only 50% knew of expected results prior to testing; only 33% used test data at limits of normal range; only 42% used test data with errors. Only 23% documented formulas; 23% assumptions & known limits. Checked by another: 17% another developer; 5% internal auditor; 6% external auditor. Only 40% formally trained.

 

Schultheis & Sumner [1994] 32 end user SS developers at authors’ university. Each subject was instructed to select his or her most recent spreadsheet. The study looked at 11 risks, 9 controls. Controls had multiple items. Respondent could select several items in a control; each counted. 54% of developers formally trained. 31% of SSs had design reviewed by other. 16% audited by auditor or consultant. 16% tested with data with known outputs; 62% checked with calculator. Very little documentation: 25% none, 25% detailed enough to list purpose, most just sample audit. Higher-risk SSs used more controls (6.7/11) than did lower-risk SSs (4.6/11). Higher used more verification of logic, training, and documentation. But use of controls was low in both cases.