Magazine article The CPA Journal

How Reliable Is Haphazard Sampling?

Magazine article The CPA Journal

How Reliable Is Haphazard Sampling?

Article excerpt

Current audit standards sanction the use of both statistical and nonstatistical sample-selection methods, and suggest that the method chosen should depend on relative cost and effectiveness. Anecdotal evidence and results of a recent survey of practicing auditors indicate that the majority of audit samples are nonstatistical, with haphazard sampling being the method of choice in most circumstances [see "Sampling Practices of Auditors in Public Accounting, Industry, and Government," Accounting Horizons, 16 (2) 2002].

Regardless of which method is used, Statement on Auditing Standards (SAS) 39 indicates that the selection method should be expected to yield a sample that is representative of the population. Similarly, recent guidance on implementing section 404 of the Sarbanes-Oxley Act (SOA) notes that while statistical sampling is not required in these audits, samples testing internal controls should be selected in an unbiased manner (see A Framework for Evaluating Control Exceptions and Deficiencies, version 3, AICPA, December 2004).

To select a sample that satisfies SAS 39's requirement for representativeness, current standards and related guidance indicate that all population elements must have a chance of selection and that due care must be exercised to avoid selection bias. In circumstances where nonstatistical selection methods are used, auditors must select sample items without regard to their size, shape, location, or other physical features. Also, auditors are warned to avoid distorting samples by selecting only unusual or physically small items, or omitting the first or last items in a population (see AICPA Technical Practice Aids, 2002). Auditors are presumed to be exercising appropriate due care and to be capable of selecting representative samples using nonstatistical selection methods.

In the case of haphazard sampling, the selection process is intended to emulate equal probability sampling, with the effect that all population elements have the same chance of selection. More than 40 years ago, however, noted business sampling experts W. Edwards Deming and Herbert Arkin expressed concerns that nonstatistical methods, including haphazard sampling, are susceptible to unintended selection biases (differences between desired and actual selection probabilities). Two recent research studies confirmed that haphazard sampling is susceptible to selection bias and therefore may not yield representative samples.

Research

The first research study to document selection bias in haphazard samples appeared in 2000 in Behavioral Research In Accounting (vol. 12). In this study, individuals selected samples of vouchers and inventory bins using haphazard selection. Analyses of these samples disclosed selection biases in favor of population elements that were larger, conveniently located, brightly colored, or that had fewer adjacent neighbors. Furthermore, the magnitudes of these selection biases were significant, with some elements selected 57% more often than appropriate for equalprobability sampling.

A second study, which appeared in 2001 in Auditing: A Journal of Practice & Theory (vol. 20), tested whether doubling the size of an audit sample would reliably eliminate the bias inherent in haphazard sampling. The study used populations of vouchers and inventory bins similar to those employed in the BRIA study. Typically, only about 12% of the bias was eliminated, making this approach ineffective as a method for eliminating selection bias in haphazard samples.

Why Haphazard Sampling Is Bias-Prone

The tendency of haphazard sampling to yield biased selections appears to result from subconscious human behavior in the areas of 1) visual perception and 2) the performance of tasks requiring physical effort. Regarding visual perception, research in psychology has long established that individuals see what they consciously direct their attention to, and they subconsciously see other objects that fall into their field of view. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.