Academic journal article North American Journal of Psychology

Interviewers' Ratings of Data Quality in Household Telephone Surveys

Academic journal article North American Journal of Psychology

Interviewers' Ratings of Data Quality in Household Telephone Surveys

Article excerpt

Data quality is assessed in many ways, including level of error, effects of the interviewer, how the instrument itself impacts the type and quality of data collected, and how respondents' cognitive and motivational abilities affect their answers. One aspect of data quality that has not been examined in detail is the interviewer's perception of the quality of the interview process itself and the subsequent data collected. Interviewers provide a unique glimpse into the data collection process. We had interviewers systematically evaluate the quality of the data collected from each interview (N=1,355) and examined the differences in respondent answers based on interviewers' ratings. While most interviews were rated as excellent by these data collectors, those interviews that were rated as less than excellent show differential responding patterns. Instead of focusing solely on minimizing interviewer related error, we should also capitalize on the fact that the interviewer is an accessible informant into the data collection process. By having interviewers assess the quality of the data provided in the interviews they conduct, we get another piece of information with which to rate overall data quality.

One aspect of data quality that is under addressed in the survey research literature is interviewer ratings of the interview process or subsequent data quality. This article examines interviewers' ratings of data quality from a computer assisted telephone interview on alcohol and drug use.

Data quality is assessed in many ways, including sampling and measurement error, effects of the interviewer, how the instrument itself impacts the type and quality of data collected, and how respondents' cognitive and motivational abilities affect their answers. This emphasis has produced a significant amount of research aimed at examining the effects of a wide variety of variables on data quality, including interviewer demographic characteristics, expectations, and training (Fowler & Mangione, 1990; Singer, Frankel, & Glassman, 1983), the content and the validity of the data collection instrument itself (Belson, 1981; Bradburn & Sudman, 1979; Fowler, 1995; 1998; Schuman & Presser, 1981), as well as the effect of the respondents' demographic status (Schuman & Converse, 1971) and cognitive abilities (Linton, 1982; Krosnick & Alwin, 1987; Sudman & Bradburn, 1973).

The Interviewer as a Source of Information

Another aspect of data quality is the interviewer's perception of the quality of the interview process and the resulting data. Typically, interviewer ratings have been used in personnel research (e.g., job interviews, evaluations of employees in training situations) (Baron, 1989; Fletcher, 1990; Pulakos, Schmitt, Whitney, & Smith, 1996), and clinical settings (e.g., evaluations of patient behavior and therapeutic progress) (Franklin, et al., 1998; Harris, et al., 1970; Zimmerman, et al., 1986). In these and other areas interviewers' opinions about the quality of an interaction are often assessed.

Despite their use in other areas, interviewer ratings are not widely used in survey research houses. Most research on interviewer ratings of data quality are decades old (Brown, 1955; Cannell, Fowler, & Marquis, 1968; Nuckols, 1953) and therefore not necessarily indicative of the computer assisted method of data collection common in today's survey research industry. One more recent study (Tran, 1996) examined congruency between the ratings given by the interviewer about the respondent on a variety of psychosocial and health factors as compared to respondent self-report on these same issues. Tran (1996) found that interviewer ratings were often predictive of the respondent self-report.

Thumbnail sketches, brief synopses by the interviewer clarifying possible ambiguities and providing a brief overview of the interaction, are common in face-to-face interviews, but are used less often in telephone surveys. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.