Academic journal article Journal of Rural Social Sciences

Moving Survey Methodology Forward in Our Rapidly Changing World: A Commentary

Academic journal article Journal of Rural Social Sciences

Moving Survey Methodology Forward in Our Rapidly Changing World: A Commentary

Article excerpt

Survey methodology now faces an unprecedented challenge for how to collect information from samples of people that will provide scientifically defensible estimates of the characteristics of the population they represent. Many decades of research have shown that in order for such estimates to be made with known precision four major sources of error-coverage, sampling, measurement and nonresponse-must be controlled (Groves, 1989). Subsequent research has produced a great amount of knowledge on how those sample estimates are affected by different survey modes, sample sources, sample sizes, the failure of certain types of people to respond to survey requests, and how questions are structured and worded.

Today's challenge stems from many considerations. Response rates for some survey modes, especially voice telephone in national surveys, have fallen precipitously and are not expected to recover (Dillman, Forthcoming). In addition, RDD landline surveys miss nearly half of all households, and adding cell phone numbers for public surveys requires asking questions about household member access to telephones. Thus, survey "adjustment" questions compete with substantive questions for the few minutes allowed for asking questions in most telephone surveys. In addition, internet surveys that rely only on emailed response requests, now used by many potential surveyors as a cost cutting measure, produce poor response rates. Requests for web responses also miss households without internet access and potential respondents who will not click on web links to surveys from unknown sources. In contrast to the past, U.S. Postal Service residential addresses now provide the best household sampling frame for providing the opportunity to be included in a scientifically defensible survey sample (Harter et al. 2016).

Surveyors interested in analyzing specific community, county, regional or agricultural populations, as is especially common among rural social scientists, face additional survey challenges. Telephone area codes are now portable so they often do not correspond to where one's residence is located. Internet coverage, although now about 85 percent, is significantly lower in rural areas, and especially those places with low incomes. Although the American Community Survey provides good estimates for states, as well as medium to large cities, the sample sizes are too small to provide information for smaller counties and towns. Thus, essential comparative data for analyzing nonresponse error in specific rural localities is not available, as it was until 2000 when for the last time the Decennial Census collected "long-form" demographic information from one in six residences in every community. Thus the survey design challenge facing those who wish to do surveys of rural populations includes not only the many challenges facing those who do national surveys, but additional concerns as well.

Three major trends in how survey designers are responding to changes in the technological and social aspects of the society in which we now live shape the context for needed research on how to improve survey quality. These macrochanges include, 1) the change from interviews to self-administration for most survey data collection, 2) greater reliance on mixed-mode rather than single mode surveys to improve quality while reducing costs, and 3) tailored survey design decisions that take into account population characteristics, survey topic, and survey burden to better target particular survey designs to the need.


In-person and telephone interviews dominated survey data collection until the end of the 20th century. They were deemed the only acceptable modes of data collection because nearly all households could be accessed that way. These survey modes also suffered from measurement problems, i.e., eliciting socially desirable answers (e.g., "no, I have not smoked marijuana") that would presumably place the respondent in a more favorable light to the interviewer and sometimes the survey sponsor. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.