Academic journal article Italian Sociological Review

Cognitive Interviewing to Pretest Attitude Questions

Academic journal article Italian Sociological Review

Cognitive Interviewing to Pretest Attitude Questions

Article excerpt

Abstract

The purpose of this paper is to assess the potentials and the limits of the cognitive interview to pretest attitude questions. The paper begins with a preface describing what cognitive interview is and what are its theoretical roots. Then we take part in the debate on the cognitive interview and we discuss the strengths and weakness of its strategies. Lastly, we present the design and the findings of a methodological research on the effectiveness of the two strategies typical of the cognitive interview - the think-aloud and the verbal probing - in order to pretest attitude questions.

Keywords: Pretest, Cognitive interview, Attitude questions

1. Background

In order to collect high quality data, that are data corresponding to the actual status that they represent in the data matrix, we should be able to trust the questionnaire before it is finally adopted for the survey (Marradi, 1990). We will achieve this if we monitor its operation during a pretesting stage focused on checking the reliability of its operational definitions (Pitrone, 2009: 146). Only by pretesting the questionnaire, we can «evaluate in advance whether it causes problems for interviewers or respondents» (Presser et al., 2004: 109) and understand how to improve it. For these reasons, the pretest is universally agreed as an indispensable stage in the questionnaire developmental process and the scholars have proposed different pretesting methods to evaluate survey questions. Among them, in the last thirty years the cognitive interview has emerged as one of the more widespread method to detect problems with the questions that may compromise the quality of response .

The cognitive interview is based on the theoretical and methodological assumptions of the Cognitive Aspects of Survey Methodology (CASM), a movement born in the first eighties in order to bridge the communication gaps between survey research and the cognitive sciences (Jabine et al. 1984). The exponents of the movement have developed a pretesting method aimed at obtaining the information needed to reconstruct the cognitive processes of respondents, supposing that this reconstruction may help the detection of biases and suggest solutions. The object of the cognitive interview is to provide a view of the response process elicited by survey questions which is assumed to be divided into four steps: comprehension of the question, retrieval of relevant information, use of the information retrieved to make a judgment and formatting of an answer (Tourangeau, 1984). Because problems may occur in each of these steps, the goal of the cognitive interviewer is to learn how respondents perform them in order to gain insight into whether the results of their performance are likely to produce sufficiently accurate answers consistent with the researcher's intent (Blair and Brick, 2009: 5691). The way to learn how responses are formulated and reported is to administer draft questions while collecting additional information about the response, useful for evaluating the quality of the response or helping to determine whether it is coherent with the author's intention (Beatty and Willis 2007: 288). Cognitive interview thus allows the researcher to control those sources of response error not immediately identifiable during the data collection.

To explore the processes by which respondents answer survey questions, it is possible to choose between two technique: the think-aloud and the verbal probing. The first one consists in encouraging respondents to verbalize their thoughts as they answer survey questions. After being given the necessary instructions, subjects are asked to verbalize their thoughts as they were talking to themselves. This strategy is not based on conversation, but it reduces to the minimum the role of the interviewer, limiting his interventions only to encouragements to think aloud for those who keep silent. The second technique consists in asking probes after the respondent's answer or at the end of the interview. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.