Some Things about Surveys They Forgot to Teach Us in Social Science 301: The Performance Audit Perspective
Persaud, Narayan, DiSalvo, David R., The Government Accountants Journal
Performance auditor's use surveys as methods of gathering program information is not yet commonplace, but the practice is growing. In conducting performance audit surveys, many performance auditors have come to realize that even though program personnel (auditees) are statutorily required to provide auditors with information, due care still needs to be exercised if one is to collect reliable and valid data on sensitive program issues(1), In general, we argue that given the sensitivity inherent in audit concerns, coupled with the necessity to conduct efficient audits, performance auditors cannot afford to overlook some common methodological principles when undertaking a survey.
In particular, we focus on questionnaire development and show, by way of examples, that performance auditors can effectively apply social science principles -- tailoring these to write good audit questions within the constraints of the sensitive audit environment. Since performance auditing is often compared to program evaluation, we begin our analysis by examining these two professions. The purpose is to show that performance auditing and program evaluation typically differ in focus on analysis, methodological principles, sponsorship and audience, all of which impact on the level of sensitivity within these analytical fields.
According to Brown, Gallagher and Williams (1982), performance auditing is a "custom crafted analysis of specific aspects of program efficiency and effectiveness... it! deals with a combination of financial and nonfinancial measures and usually must define a unique set of measurements and standards for each audit that is undertaken," (see Schwandt and Halpern, 1988:21).
Implicit in the above definition, and also noted by Davis (1990) and Chelimsky (1985), is performance auditing's roots in the accounting and financial auditing professions. Emerging out of these professions, performance auditing brought with it certain standard practices and procedures for evidence gathering. Published as Standards for Audit of Governmental Organizations, Programs, Activities and Functions, and known by performance auditors as "Yellow Book Standards," these practices and procedures by and large guide performance auditing (see U.S. General Accounting Office 1988; Davis, 1990). As summarized in the words of Wheat (1991:386), "A performance audit is conducted according to generally accepted auditing standards that are promulgated by the federal General Accounting Office (GAO)."
PERFORMANCE AUDITING AND PROGRAM EVALUATION
Although grounded in accounting and financial auditing, performance auditing shares some similarities with program evaluation. Both are expost-facto in approach, seek to address stakeholders' concerns and abide by systematic analytic procedures in assessing program results (see Chelimsky, 1985 and 1990).
Despite these commonalities, significant differences exist between the two professions. The very definition of program evaluation sets it apart from performance auditing. According to Rossi and Freeman (1989:18), program evaluation, or evaluations research(2), is "the systematic application of social research procedures for assessing the conceptualization, design, implementation and utility of social intervention programs." Thus program evaluation grew out of the social sciences. And, by way of this grounding, program evaluators became wedded to social science research procedures (see Berk and Rossi 1990). Performance auditors, whose profession emerged out of accounting and governed by auditing standards, are cautious, or experience a modicum of success in adopting social science procedures.
Performance audits and program evaluation also typically differ in focus. Performance audits centers on assessing program efficiency and effectiveness within the framework of compliance with applicable laws and regulations (GAO, 1988). Program evaluation focuses analysis on program designs, functions," the assessment of program impact, and the analysis of the program benefits relative to their cost," (Berk and Rossi, 1990:8). …