Regression Modeling and Meta-Analysis for Decision Making: A Cost-Benefit Analysis of Incentives in Telephone Surveys

By Gelman, Andrew; Stevens, Matt et al. | Journal of Business & Economic Statistics, April 2003 | Go to article overview

Regression Modeling and Meta-Analysis for Decision Making: A Cost-Benefit Analysis of Incentives in Telephone Surveys


Gelman, Andrew, Stevens, Matt, Chan, Valerie, Journal of Business & Economic Statistics


Regression models are often used, explicitly or implicitly, for decision making. However, the choices made in setting up the models (e.g., inclusion of predictors based on statistical significance) do not map directly into decision procedures. Bayesian inference works more naturally with decision analysis but presents problems in practice when noninformative prior distributions are used with sparse data. We do not attempt to provide a general solution to this problem, but rather present an application of a decision problem in which inferences from a regression model are used to estimate costs and benefits. Our example is a reanalysis of a recent meta-analysis of incentives for reducing survey nonresponse. We then apply the results of our fitted model to the New York City Social Indicators Survey, a biennial telephone survey with a high nonresponse rate. We consider the balance of estimated costs, cost savings, and response rate for different choices of incentives. The explicit analysis of the decision problem reveals the importance of interactions in the fitted regression model.

KEY WORDS: Decision analysis; Hierarchical linear regression; Meta-analysis; Survey nonresponse.

1. INTRODUCTION

Regression models are often used, explicitly or implicitly, for decision making. However, the choices made in setting up the models (e.g., stepwise variable selection, inclusion of predictors based on statistical significance, and "conservative" standard error estimation) do not map directly into decision procedures. We illustrate these concerns with an application of Bayesian regression modeling for the purpose of determining the level of incentive for a telephone survey.

Common sense and evidence (in the form of randomized experiments within surveys) both suggest that giving incentives to survey participants tends to increase response rates. From a survey designer's point of view, the relevant questions are as follows:

* Do the benefits of incentives outweigh the costs?

* If an incentive is given, how and when should it be offered, whom should it be offered to, what form should it take, and how large should its value be?

The answers to these questions necessarily depend on the goals of the study, costs of interviewing, and rates of non-response, nonavailability, and screening in the survey. Singer (2001) reviewed incentives for household surveys, and Cantor and Cunningham (1999) considered incentives along with other methods for increasing response rates in telephone surveys. The ultimate goal of increasing response rates is to make the sample more representative of the population and to reduce nonresponse bias.

In this article we attempt to provide an approach to quantifying the costs and benefits of incentives, as a means of assisting in the decision of whether and how to apply an incentive in a particular telephone survey. We proceed in two steps. In Section 2 we reanalyze the data from the comprehensive meta-analysis of Singer, Van Hoewyk, Gebler, Raghunathan, and McGonagle (1999) of incentives in face-to-face and telephone surveys and model the effect of incentives on response rates as a function of timing and amount of incentive and descriptors of the survey. In Section 3 we apply the model estimates to the cost structure of the New York City Social Indicators Survey, a biennial study with a nonresponse rate in the 50% range. In Section 4 we consider how these ideas can be applied generally and discuss limitations of our approach.

The general problem of decision analysis using regression inferences is beyond the scope of this article. By working out the details for a particular example, we intend to illustrate the challenges that arise in going from parameter estimates to inferences about costs and benefits that can be used in decision making.

By reanalyzing the data of Singer et al. (1999), we are not criticizing their models for their original inferential purposes; rather, we enter slightly uncharted inferential territory for poorly identified interaction parameters to best answer the decision questions that are important for our application. …

The rest of this article is only available to active members of Questia

Already a member? Log in now.

Notes for this article

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
One moment ...
Default project is now your active project.
Project items

Items saved from this article

This article has been saved
Highlights (0)
Some of your highlights are legacy items.

Highlights saved before July 30, 2012 will not be displayed on their respective source pages.

You can easily re-create the highlights by opening the book page or article, selecting the text, and clicking “Highlight.”

Citations (0)
Some of your citations are legacy items.

Any citation created before July 30, 2012 will labeled as a “Cited page.” New citations will be saved as cited passages, pages or articles.

We also added the ability to view new citations from your projects or the book or article where you created them.

Notes (0)
Bookmarks (0)

You have no saved items from this article

Project items include:
  • Saved book/article
  • Highlights
  • Quotes/citations
  • Notes
  • Bookmarks
Notes
Cite this article

Cited article

Style
Citations are available only to our active members.
Buy instant access to cite pages or passages in MLA, APA and Chicago citation styles.

(Einhorn, 1992, p. 25)

(Einhorn 25)

1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

Cited article

Regression Modeling and Meta-Analysis for Decision Making: A Cost-Benefit Analysis of Incentives in Telephone Surveys
Settings

Settings

Typeface
Text size Smaller Larger Reset View mode
Search within

Search within this article

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

Help
Full screen

matching results for page

    Questia reader help

    How to highlight and cite specific passages

    1. Click or tap the first word you want to select.
    2. Click or tap the last word you want to select, and you’ll see everything in between get selected.
    3. You’ll then get a menu of options like creating a highlight or a citation from that passage of text.

    OK, got it!

    Cited passage

    Style
    Citations are available only to our active members.
    Buy instant access to cite pages or passages in MLA, APA and Chicago citation styles.

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn, 1992, p. 25).

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn 25)

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences."1

    1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

    Cited passage

    Thanks for trying Questia!

    Please continue trying out our research tools, but please note, full functionality is available only to our active members.

    Your work will be lost once you leave this Web page.

    Buy instant access to save your work.

    Already a member? Log in now.

    Author Advanced search

    Oops!

    An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.