The Problem of Respondent Attrition: Survey Methodology Is Key; Longitudinal Surveys Will Suffer from Attrition and Nothing Will Change That; However, Years of Lessons Learned in the Field Show That Straightforward Survey Methodology Can Minimize the Impact of Losing Respondents

Olsen, Randall J., Monthly Labor Review

The central problem of longitudinal surveys is attrition. The National Longitudinal Survey of Youth in 1979 (NLSY79), which this issue of the Monthly Labor Review features, is the gold standard for sample retention against which longitudinal surveys are usually measured. However, we cannot understand how the NLSY79 has done so well without considering what was done differently in the other cohorts of the NLS and what we have learned by formal evaluations of attrition aversion measures that evolved over a quarter century of field work. The lessons here are hard-won and, to some, unconventional.

Background

The NLS began in 1965 at the urging of an Assistant Secretary of Labor, Daniel Patrick Moynihan. He believed that although the Current Population Survey provided crucial snapshots of the Nation's labor force and labor market, the Nation needed a data source that was more dynamic and capable of tracking the long-run evolution of careers. The task of starting the study went to Howard Rosen at the Department of Labor, who enlisted Herb Parnes from Ohio State University, to assemble a team, design the surveys, and analyze the data. This team comprised representatives from the Census Bureau, Ohio State University, and the Department of Labor.

The original plan was to follow the cohorts for 5 years to study some of the pressing questions of the time--the shrinking labor force participation rate of older men, the problem of youth unemployment and the transition from school to work, and the growing labor force participation of women whose children were entering school, leading to steady growth in the number of working mothers. Childcare was an important issue along with the problem of how the family would pay for a college education for the children of the baby boom.

Over time, the project has expanded. (Table 1 shows the various cohorts of the NLS, their start and stop dates, sizes, and age ranges covered.) Because the project began with a 5-year horizon, neither the Census Bureau, Ohio State, nor the Department of Labor had a plan for sample retention over the long run; after all, longitudinal surveys were still quite rare. The studies shortly proved their worth and the project became open-ended in terms of duration. However, the original limitations on intended duration led to some problems with attrition that conflicted with a revised plan to follow the respondents over the balance of their lives. In particular, the "following-rule" (1) that the Census Bureau used specified that when a respondent missed two consecutive interviews, the Census Bureau would drop that respondent from the study.

The following-rule and the original 5-year horizon struck with the greatest force on the Young Men's cohort. In 1981, about two-thirds of the cohort responded to the survey. Some analysts believed that the rate of attrition reflected veterans' refusing to participate in a Government survey. Although the rate of attrition among black veterans was a few percentage points higher than that for nonveterans, for whites, the differential for veterans as a whole was essentially zero. Within the Young Men's cohort, blacks had the highest attrition rates. For whites, attrition in the Young Men's cohort was a bit higher than that for two women's cohorts, but male respondents have always had higher attrition than females.

The two-and-out-following-rule that the Census Bureau employed had serious ramifications, given the attrition pattern for the young men, and high attrition among blacks. By 1981, the Census Bureau had stopped tracking 11 percent of the young men because they had, at some point, missed two consecutive interviews. Blacks make up 28 percent of the young men's sample, but 57 percent of the cases dropped because of the following-rule were black. Our current rule-of-thumb is that in the next round, one can obtain an interview on about 25 percent of the respondents who have missed two interviews in a row. When interviewing began for the NLSY79, performance specifications did not allow respondents to be dropped simply based on consecutive missed interviews.

The original design for the surveys alternated in-person interviews with telephone and mail-out surveys, with the in-person version conducted every 5 years. (2) As a result, the content of the interview was more comprehensive every 5 years, with smaller updates in between. The NLS approach to the Mature Women's cohort is emblematic of the general approach of the survey. The 1967 interview of the women ages 30-44 focused on the longest held jobs between schooling and marriage, between marriage and the birth of her first child, and after the birth of the first child. The survey sought the most important (that is, longest held) job holdings, probed for significant periods not working, and ascertained why the woman did not work. The respondent answered CPS questions about the previous week; these questions accounted for a significant part of the interview.

This approach to collecting labor force behavior data left unanswered questions about work …

Questia, a part of Gale, Cengage Learning. www.questia.com
Publication information: Article title: The Problem of Respondent Attrition: Survey Methodology Is Key; Longitudinal Surveys Will Suffer from Attrition and Nothing Will Change That; However, Years of Lessons Learned in the Field Show That Straightforward Survey Methodology Can Minimize the Impact of Losing Respondents. Contributors: Olsen, Randall J. - Author. Journal title: Monthly Labor Review. Volume: 128. Issue: 2 Publication date: February 2005. Page number: 63+. © 1999 U.S. Bureau of Labor Statistics. COPYRIGHT 2005 Gale Group.

This material is protected by copyright and, with the exception of fair use, may not be further copied, distributed or transmitted in any form or by any means.