Academic journal article Italian Sociological Review

Cognitive Interviewing to Pretest Attitude Questions

Academic journal article Italian Sociological Review

Cognitive Interviewing to Pretest Attitude Questions

Article excerpt

1. Background

In order to collect high quality data, that are data corresponding to the actual status that they represent in the data matrix, we should be able to trust the questionnaire before it is finally adopted for the survey (Marradi, 1990). We will achieve this if we monitor its operation during a pretesting stage focused on checking the reliability of its operational definitions (Pitrone, 2009: 146). Only by pretesting the questionnaire, we can «evaluate in advance whether it causes problems for interviewers or respondents» (Presser et al., 2004: 109) and understand how to improve it. For these reasons, the pretest is universally agreed as an indispensable stage in the questionnaire developmental process and the scholars have proposed different pretesting methods to evaluate survey questions. Among them, in the last thirty years the cognitive interview has emerged as one of the more widespread method to detect problems with the questions that may compromise the quality of response .

The cognitive interview is based on the theoretical and methodological assumptions of the Cognitive Aspects of Survey Methodology (CASM), a movement born in the first eighties in order to bridge the communication gaps between survey research and the cognitive sciences (Jabine et al. 1984). The exponents of the movement have developed a pretesting method aimed at obtaining the information needed to reconstruct the cognitive processes of respondents, supposing that this reconstruction may help the detection of biases and suggest solutions. The object of the cognitive interview is to provide a view of the response process elicited by survey questions which is assumed to be divided into four steps: comprehension of the question, retrieval of relevant information, use of the information retrieved to make a judgment and formatting of an answer (Tourangeau, 1984). Because problems may occur in each of these steps, the goal of the cognitive interviewer is to learn how respondents perform them in order to gain insight into whether the results of their performance are likely to produce sufficiently accurate answers consistent with the researcher's intent (Blair and Brick, 2009: 5691). The way to learn how responses are formulated and reported is to administer draft questions while collecting additional information about the response, useful for evaluating the quality of the response or helping to determine whether it is coherent with the author's intention (Beatty and Willis 2007: 288). Cognitive interview thus allows the researcher to control those sources of response error not immediately identifiable during the data collection.

To explore the processes by which respondents answer survey questions, it is possible to choose between two technique: the think-aloud and the verbal probing. The first one consists in encouraging respondents to verbalize their thoughts as they answer survey questions. After being given the necessary instructions, subjects are asked to verbalize their thoughts as they were talking to themselves. This strategy is not based on conversation, but it reduces to the minimum the role of the interviewer, limiting his interventions only to encouragements to think aloud for those who keep silent. The second technique consists in asking probes after the respondent's answer or at the end of the interview. The cognitive interviewer assumes an active role in the interaction and probes the emerging problems during the interview. The assumption underlying this procedure is that direct and focused questions help reconstruct the response process and determine whether the question is generating the information that its author intends.

The think-aloud has deep roots in psychology that are too often neglected by the cognitive interview practitioners. The first attempts to value the verbalization of thoughts as instrument of research and analysis can be traced in the introspection technique proposed by Wundt and enhanced by Küple and Buhler in the School of Wurzburg, in the Freud's clinical analysis of thought and in the clinical speech used by Piaget to check his hypotheses on the functioning of psychological systems . …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.