Moving Forward by Looking Back: Comparing Laboratory Results with Ex Ante Market Data
Chermak, Janie M., Krause, Kate, Brookshire, David S., Burness, H. Stu, Economic Inquiry
In this research, we address the central question of whether data obtained in an experiment with participants from the population of interest parallels market data from the same group. Utilizing information provided by our participants we construct salient payoffs, based on participants' revealed preferences to elicit the experimental responses. Combining these experimental data with naturally occurring market data for the identical set of consumers, we test for consistency between the two data sources. Unique to this research, the market data, unlike field experiment data, are not obtained through manipulation by the researchers. Instead these data are the actual expenditures and consumption choices of the participants reflected in billing records for the years preceding lab participation. Under these conditions, we find that experiment responses parallel ex ante market behavior.
Experiment data, if consistent with market data, could be used to fill data gaps. This is increasingly important as policy makers find themselves having to respond to situations for which data do not exist, making it difficult to assess the efficiency of proposed policies. Examples include consumer response to changes in depth and frequency of energy shortages, market responses to rapid technological advancements such as the Internet, and structural changes in markets where current and future conditions are fundamentally different from the historic record. This is the case of water in the arid western United States. Increases in demand associated with urbanization have stressed the limited supply of water, and the price of water--invariably set by a public utility and not the market--will surely rise. How can a policy maker design an effective, efficient policy, when the characteristics of the market and the potential range of data are outside anything seen before? The policy relevance of experiments depends on parallelism between the lab and the field (Smith 1982). This research provides evidence for that parallelism.
Because we focus on parallelism between the laboratory and the market we deviate from traditional economics experiments protocol. Traditional experiments that seek to test theory normally provide participants with explicit information concerning payoffs to induce preferences. In our case, as in any actual market, individual payoffs are endogenous and based on individual preferences. We could not directly link experiment payoffs to water consumption because no water was actually consumed as part of the experiment protocol. Instead, we linked experiment payoffs to estimated household water consumption based on information provided by participants prior to the experiment.
We offer an inquiry into residential water markets in Albuquerque, New Mexico, a midsize city in the desert southwestern United States. We compare participant response in an experiment to the actual water bill data generated by those same participants in the years preceding the experiment. The water bill data are unfettered by the experiment participation. It predates the experiments and the participants could not have predicted that their household consumption choice data would one day be used in an experimental investigation. Prior to the experiment participants completed a survey describing their household's water-using features (e.g., type of landscape), which we used to categorize their preferences and predict their water use. Participants' water bills and survey responses provide evidence of the underlying utility function for each participant, which were used to calculate payoffs as described in the following text.
For both the experiment data and the water bill data we econometrically estimate consumer water demand, using comparable explanatory variables. We find approximately 75% of the independent variables exhibit the same significance across the two data sets, over 90% of the statistically significant variables exhibit the same direction of impact, and for almost 70% of the participants, there is no statistical difference between the water demand elasticities estimated from the water bills and the experiment data. …