Replicability, Real-Time Data, and the Science of Economic Research: FRED, ALFRED, and VDC

By Anderson, Richard G. | Review - Federal Reserve Bank of St. Louis, January/February 2006 | Go to article overview

Replicability, Real-Time Data, and the Science of Economic Research: FRED, ALFRED, and VDC


Anderson, Richard G., Review - Federal Reserve Bank of St. Louis


This article discusses the linkages between two recent themes in economic research: "real time" data and replication. These two themes share many of the same ideas, specifically, that scientific research itself has a time dimension. In research using real-time data, this time dimension is the date on which particular observations, or pieces of data, became available. In work with replication, it is the date on which a study (and its results) became available to other researchers and/or was published. Recognition of both dimensions of scientific research is important. A project at the Federal Reserve Bank of St. Louis to place large amounts of historical data on the Internet holds promise to unify these two themes.

Federal Reserve Bank of St. Louis Review, January/February 2006, 88(1), pp. 81-93.

REPLICATION AND REAL-TIME ECONOMETRICS

During the past 25 years, two themes have flowed steadily, albeit often quietly, through economic research: "real time" data and replication. In replication studies, the issue is determining which data were used and whether the author performed the calculations as described; in real-time data studies, the issue is determining the robustness of the study's findings to data revisions. These themes share the same core idea: that scientific research has an inherent time dimension. In both real-time data and replication studies, the time dimension is the date on which particular observations, or pieces of data, became available to researchers. Projects at Harvard University and at the Federal Reserve Bank of St. Louis promise to improve the quality of empirical economic research by unifying these themes.1

Although replication studies focus on the correctness of results and real-time studies on their robustness, economic theory suggests that these are related-the likelihood that an author's error will become visible to other researchers is an inverse function of the cost of conducting tests for replicability and robustness. Yet, for the profession, excessive emphasis on the criminaldetection aspects of replication (Did the author fake the results? Or did the author cease experimenting prematurely when a favorable result appeared?) has tended to increase the reluctance of researchers to share data and program code. That is, to the extent that the profession overemphasizes the manhunt of David Dodge's 1952 To Catch a Thief, it risks foregoing the benefits of Sir Isaac Newton's 1676 dictum, "If I have seen further it is by standing on the shoulders of giants."

The incentives and disincentives for a researcher to share data have been discussed by numerous authors (e.g., Fienberg, Martin, and Starf, 1985; Boruch and Cordray, 1985; Dewald, Thursby, and Anderson, 1986; Feigenbaum and Levy, 1993; Anderson and Dewald, 1994; Bornstein, 1991; Bailar, 2003).2 Researchers receive a stream of rewards for the new knowledge contained in a published article, which begins with publication and eventually tapers to near zero. Furnishing the data to other researchers invites the risk that a replication will demonstrate the article's results to be false, an event which immediately ends the reward stream. If the replication further uncovers malicious or unprofessional behavior (such as fraud or other unethical conduct), "negative rewards" flow to the researcher.

Creating original research manuscripts for professional journals is craft work. Although often referred to as "knowledge workers," researchers might equally well be regarded as artisans, with creative tasks that include collecting data, writing code for statistical analysis or model simulation, and authoring the final manuscript.3 Similar to the work of other craftsmen, researchers' output contains intellectual property-not only the final manuscript, but also the data and programs developed during its creation. Yet, for academic-type researchers, some of the intellectual property must be relinquished so the work can be published in peer-reviewed journals. …

The rest of this article is only available to active members of Questia

Sign up now for a free, 1-day trial and receive full access to:

  • Questia's entire collection
  • Automatic bibliography creation
  • More helpful research tools like notes, citations, and highlights
  • Ad-free environment

Already a member? Log in now.

Notes for this article

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
One moment ...
Default project is now your active project.
Project items

Items saved from this article

This article has been saved
Highlights (0)
Some of your highlights are legacy items.

Highlights saved before July 30, 2012 will not be displayed on their respective source pages.

You can easily re-create the highlights by opening the book page or article, selecting the text, and clicking “Highlight.”

Citations (0)
Some of your citations are legacy items.

Any citation created before July 30, 2012 will labeled as a “Cited page.” New citations will be saved as cited passages, pages or articles.

We also added the ability to view new citations from your projects or the book or article where you created them.

Notes (0)
Bookmarks (0)

You have no saved items from this article

Project items include:
  • Saved book/article
  • Highlights
  • Quotes/citations
  • Notes
  • Bookmarks
Notes
Cite this article

Cited article

Style
Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

(Einhorn, 1992, p. 25)

(Einhorn 25)

1

1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

Cited article

Replicability, Real-Time Data, and the Science of Economic Research: FRED, ALFRED, and VDC
Settings

Settings

Typeface
Text size Smaller Larger Reset View mode
Search within

Search within this article

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

Full screen

matching results for page

Cited passage

Style
Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

"Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn, 1992, p. 25).

"Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn 25)

"Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences."1

1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

Cited passage

Thanks for trying Questia!

Please continue trying out our research tools, but please note, full functionality is available only to our active members.

Your work will be lost once you leave this Web page.

For full access in an ad-free environment, sign up now for a FREE, 1-day trial.

Already a member? Log in now.