Academic journal article Monthly Labor Review

Cognitive Testing of Racial and Ethnic Questions for the CPS Supplement

Academic journal article Monthly Labor Review

Cognitive Testing of Racial and Ethnic Questions for the CPS Supplement

Article excerpt

Extensive cognitive testing aided the development of a new questionnaire that is readily understood, no longer evokes negative emotional responses, and allows analysts to pinpoint the circumstances associated with multiracial reporting

"Race" and "ethnic origin" are two of the most emotionally charged and conceptually ambiguous terms in U.S. contemporary social discourse. Surveys with racial and ethnic questions that have been thoroughly researched, sensitively tested, and carefully structured should enable researchers to collect reliable data and make sound conclusions. Thus, a study was conducted on the May 1995 CPS supplement to provide questions that would allow respondents to identify themselves most appropriately by race, ethnic origin, and multiracial groups.

This article describes the development of the Current Population Survey Supplement on Race and Ethnic Origin that could be easily understood and would no longer evoke negative emotional responses. (See the article by Clyde Tucker and Brian Kojetin in this issue for a description of the supplement.)

An interagency team of behavioral scientists representing anthropology, psychology, and sociology was closely involved in the development and testing of the questions for the May 1995 CPS supplement.(1) The team continued to be involved during and after collection of the supplement, analyzing live and taped interviews, and contributing insights from the analysis to help interpret findings from the statistical analysis of the data.

Pretesting by cognitive research

To test successive versions of the supplement, cognitive research interviews were conducted by a multiracial and multi-ethnic team of behavioral scientists from several Federal agencies, with additional support from survey specialists in academia. The research protocol called for individual, face-to-face interviews in which the respondent answered all of the questions in one of the four panel questionnaires to be tested.(2) After responding to each question, the respondent was asked to paraphrase the question, that is, to tell the interviewer what the question meant in his or her own words. For questions containing terms of special interest to the research, for example, race, ethnicity, Latino, the respondent was also asked to provide a definition of these terms.(3)

The research plan called for matching the race and Hispanic origin of the respondent and researcher. This was achieved for all groups except for the American Indian respondents, who were interviewed by an Asian-American researcher. All of the materials used in the cognitive research, including the four panel questionnaires of the supplement and the research interview protocols, were translated into Spanish for respondents who were monolingual in Spanish. The cognitive interviews were carried out in three phases between November 1994 and January 1995. Each successive phase tested a version of the questionnaire that had been revised in light of problems found in earlier rounds of cognitive research.

A total of 82 cognitive interviews were carried out in the following locations with the groups indicated: Albuquerque (American Indians); Chicago (blacks); Houston (Hispanics, whites); New Orleans (Creoles); New York (Hispanics, whites); Rural California (Hispanics); Rural Mississippi (blacks); Rural West Virginia (whites); San Francisco (Asians and Pacific Islanders, Hispanics, multiracials); and Washington, DC (Asians and Pacific Islanders, blacks, Hispanics, multiracials, whites). For each of the racial groups, it was possible to interview respondents who had a high school education or less, as well as respondents with some years of college.

The phase one interviews revealed a multitude of problems in the four panel questionnaires of the supplement. These problems included vague or imprecise questions, sensitive questions, abstract questions, vocabulary problems, and order effects. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.