Assessing the Value of Economics Research: The Case of the Bias in the Consumer Price Index
Schimmelpfennig, David E., Norton, George W., Contemporary Economic Policy
Few economists would doubt that their research and advice has value to society. But does the value change when the results of similar work are in circulation? Does it make a difference if the other work draws similar or different conclusions? Economists provide knowledge of how economic systems work and the consequences of economic actions, often directly or indirectly contributing to policy design. How does one value this type of knowledge and assistance, particularly when several sets of results or interpretations of empirical data are available? Clearly, not all economists--or the value of their research programs--are equal. If research results from more prestigious sources get greater weight in policy considerations, does that influence the value of the research? Partly driven by accountability concerns, economists have at times evaluated impacts of technological innovations (e.g., Evenson 1967; Griliches 1958; Mansfield et al. 1977; Scherer 1999), but for the most part have steered clear of attempts to assess impacts of social science research, including economics.
Reluctance to try to quantify the value of economics is understandable. Economics research (ER) produces a diverse set of difficult-to-measure outputs that are embedded in theories, recommendations, institutions, or quantitative methods. These outputs are often not valued in the market and may be aimed at one or more objectives associated with efficiency, risk, or distribution. These diverse outputs and multiple objectives complicate the assessment of aggregate research programs, forcing the evaluator to consider specific research products and objectives. Establishing causality between ER and specific policies is a challenge because ER is usually only one of many inputs in a decision-making process that involves political and other factors.
This article presents a relatively simple political-economic model for assessing the contributions of ER. The model allows for multiple groups of researchers with different levels of influence over a policy center. It is tested using a Bayesian decision theory (BDT) approach to value research information. An example is provided that concerns research by economists on the degree of bias in the U.S. Consumer Price Index (CPI). Conclusions are drawn about the usefulness of the BDT approach for assessing social science research that may be focused on one topic but is completed by multiple sources.
Few attempts have been made to evaluate ER, and these are primarily related to agricultural economics. Just et al. (2002), Freebairn (1976), Hayami and Peterson (1972), and others have assessed the impacts of commodity outlook research. Gardner (2004), Norton and Alwang (1997, 2004), and Ryan (2004) have evaluated the benefits of agricultural research directed at policy or institutional change. These studies and others have concluded that the output of ER is information. Therefore, approaches for valuing information, such as BDT, may be fruitful for valuing ER (Lindner 1987; Schimmelpfennig and Norton 2003).
ER can help policy makers by providing information that improves their ability to predict outcomes of their actions. When different information on the same topic reaches policy makers from multiple sources, no one source may be definitive. Even if this information leads to greater uncertainty, the information may be better than what was available before and have positive value if strongly held prior but erroneous beliefs are called into question (Hirshleifer 1971). If the information is actually worse, the possibility does exist that it may have no value or negative value. Economic efficiency is a convenient metric for valuing this information, which substitutes knowledge and analytical skill for the more expensive process of learning by trial and error (Ruttan 1984).
These characteristics of ER highlight the need for a theoretical framework that captures (1) the impacts of ER on policy makers' and private decision makers' subjective beliefs about the consequences of actions, (2) the economic efficiency or welfare effects of those actions, and (3) the political-economic interactions between policy makers and the information produced by multiple ER groups. …