Well-Being Analysis vs. Cost-Benefit Analysis
Bronsteen, John, Buccafusco, Christopher, Masur, Jonathan S., Duke Law Journal
Cost-benefit analysis (CBA) is the primary tool used by policymakers to inform administrative decisionmaking. Yet its methodology of converting preferences (often hypothetical ones) into dollar figures, then using those dollar figures as proxies for quality of life, creates significant systemic errors. These problems have been lamented by many scholars, and recent calls have gone out from world leaders and prominent economists to find an alternative analytical device that would measure quality of life more directly. This Article proposes well-being analysis (WBA) as that alternative. Relying on data from studies in the field of hedonic psychology that track people's actual experience of life--data that have consistently been round reliable and valid--WBA is able to provide the same policy guidance as CBA without CBA's distortionary reliance upon predictions and dollar figures. We show how WBA can be implemented, and we catalog its advantages over CBA. In light of this comparison, we conclude that WBA should assume CBA's role as the decisionmaking tool of choice for administrative regulation.
TABLE OF CONTENTS Introduction I. How Cost-Benefit Analysis Works, and Its Core Limitation. A. CBA and Welfare B. The Core Advantage of WBA over CBA II. Well-Being Analysis A. WBA: The Basic Framework B. The Data of WBA 1. Life Satisfaction Surveys 2. Experience Sampling Methods 3. The Quality of Well-Being Data 4. Criticisms of Well-Being Data 5. Deliberate Manipulation of Well-Being Data C. Well-Being Analysis: An Example 1. EPA Regulation of Pulp and Paper Production: A Cost-Benefit Analysis 2. The EPA's Cost-Benefit Analysis as a Well-Being Analysis III. Willingness To Pay and Well-Being A. Revealed Preferences 1. Informational and Computational Problems 2. Wealth Effects 3. Affective Forecasting Errors B. Contingent Valuations 1. Hypothetical Questions 2. Wealth Effects C. Willingness-To-Pay Measures and WBA: A Summary D. Wealth and Welfare IV. WBA and the Value of Lives A. Not All Types of Death Are Equivalent 1. Different Types of Threats to Life 2. Different Types of Death 3. How One Person's Death Affects Another Person's Welfare B. CBA's Attempted Improvements 1. Statistical Life and Life Years 2. Quality-Adjusted Life Years 3. Well-Being Units V. Discounting in CBA and WBA Conclusion
Virtually every law makes people's lives better in some ways but worse in others. For example, a clean-air law could make people healthier, but it could also force them to pay more money for the products they buy. (1) Every proposed law thus raises the question: Would its benefits outweigh its costs? (2)
To answer that question, there needs to be a way of comparing seemingly incommensurable things like health and buying power. The most common method is to ask how much money people are willing to pay for benefits like improved health (or how much money they are willing to accept for negatives like increased risks to their health). Suppose, for example, it could be determined that people are willing to pay $100 more per year in return for the health benefits of cleaner air. Those benefits could then be compared, by this first approach, to increased consumer costs.
This approach is called cost-benefit analysis (CBA), and it has long been the dominant method of systematic analysis for evaluating government policy. (3) Every economically significant regulation from executive-branch agencies must, by law, be evaluated via CBA (4) (or in some cases via cost-effectiveness analysis). (5) This has been the case since 1981, when President Reagan mandated it by executive order. (6) That order has been reaffirmed by every president since, including Presidents Clinton (7) and Obama. (8)
Despite CBA's prominence, however, it has been criticized harshly from the moment it was first required by executive order (9) to the present day,'" and countless times in between. (11) More often than not, the criticisms are scathing. (12) Indeed, even CBA's most prominent defenders have written entire books and major articles prompted by their own acknowledgments of CBA's flaws. (13)
Along these lines, an important if subsidiary contribution of this Article is to combine our own new criticisms of CBA with those of others to make the case that CBA suffers from limitations inherent to its methodology. (14) The only method ever used to compare laws' pluses and minuses--the method that has been mandated for the past three decades--is flawed.
Yet it survives. A primary reason for its survival is evident and voiced often: no comparably rigorous, quantitative, and workable alternative exists for commensurating a law's positive and negative consequences. (15) Since virtually any law will both help people and hurt them, an important element of deciding whether to enact it will typically be to weigh the good against the bad. (16) Asking how much people are willing to pay for the good--and thereby converting all consequences into dollar figures--is viewed by many as the best option for rigorously attempting to commensurate the effects. (17)
In this Article, we propose an alternative method for comparing the positive and negative consequences of a law. This method, which we label "well-being analysis" (WBA), would analyze directly the effect of costs and benefits on people's quality of life. For example, clean-air laws would be assessed by comparing how much more people would enjoy their lives if they became healthier with how much less they would enjoy their lives if their buying power were reduced. (18) This is the most natural and direct way to put seemingly incommensurable things on the same scale. And it yields the specific answer that is needed: whether a law will make people's actual experience of life better or worse on the whole.
Until now, this sort of direct assessment has been assumed to be impossible. But it has been made feasible by the emergence of a new field within social science known as hedonic psychology. Hedonics is the study of how people experience their lives, and in particular the measurement of how much any factor improves or worsens that experience. Originally, some critics questioned whether hedonic studies could credibly measure the quality of people's experiences. (19) But over the past fifteen years, these critics have been quieted by the success of such studies in producing replicable results that pass social science's rigorous tests of validity. (20)
Accordingly, there have been widespread calls for the findings of hedonic psychology to be used to inform government policy. The United Nations General Assembly recently passed a resolution urging countries "to pursue the elaboration of additional measures that better capture the importance of the pursuit of happiness and well-being ... with a view to guiding their public policies." (21) This view has also been endorsed by Great Britain's Prime Minister David Cameron, (22) France's then-President Nicolas Sarkozy, (23) three widely divergent winners of the Nobel Prize in Economics, (24) and a recent president of Harvard University. (25) The U.S. government, as well as several states and localities, has begun exploring the possibility of using hedonic data to formulate policy. (26)
To make this a reality, however, a methodology must be created for using the data from hedonic psychology to evaluate prospective laws. (27) We create such a methodology in this Article, and we show how it can be used to analyze the same regulations currently assessed by CBA. (28) We then explain how many of the flaws of CBA, some of which have long been recognized and others of which we expose here, would be corrected by WBA. (29)
Policymaking and social science are not like mathematics, and thus any of their tools will have imperfections. WBA is no exception, as we acknowledge in the ensuing Parts. However, WBA cures many of the largest problems of CBA. It is capable of immediate implementation, and even in its infancy, it may be able to produce analyses more accurate than the ones CBA now produces after three decades of refinement. (30) We demonstrate this point directly by using WBA to reengineer an actual CBA that was used to assess a clean-water regulation. (31)
In Part I, we provide an overview of CBA and its methodology. In Part II, we explain how WBA would work in practice and the data upon which it would rely. In doing so, we contrast an actual CBA with a prototype of a WBA for the same regulation. The following Parts address the major problems with CBA that undermine its reliability and validity, and they suggest how WBA solves these problems. Part III addresses the shortcomings of CBA's use of stated and revealed preferences as proxies for well-being, Part IV focuses on limitations in the way that CBA defines the value of life, and Part V addresses issues associated with discounting the value of future money. At each step, we explain the ways in which WBA would overcome many of CBA's shortcomings and potentially provide a more accurate accounting of a prospective policy's effects on the quality of life.
I. HOW COST-BENEFIT ANALYSIS WORKS, AND ITS CORE LIMITATION
How do elected officials and regulators decide which policies to enact? They are surely influenced by political considerations, (32) and they may also have ideological commitments. But at least in some cases, they simply want to make good policy. And even when politics or ideology constrains a choice, a range of acceptable options typically remains. (33) Accordingly, regulators and elected officials and their staffs devote substantial time to identifying which policies are worth undertaking. (34)
Before they even begin, they must define what makes a policy worthwhile. A metaphysically correct definition of worth, if such a thing exists, may be beyond humanity's current grasp. However, there is widespread agreement that improving the quality of human life is at least an important component. Because virtually everyone deems it desirable to make people's lives better, at least when all else is equal, that has become the primary focus of policy analysis. (35) What it means to make someone's life better is, in turn, a potentially difficult question.
In a previous article, we argued that a person's quality of life-or, as it is more commonly labeled in economics, "welfare" or "wellbeing"--is simply the sum of the positive and negative feelings she experiences throughout her lifetime. (36) This view differs from those held by some economists (who view welfare as preference satisfaction--that is, getting what one wants) and some philosophers (who view welfare as the attainment of certain objective qualities or capabilities). (37) Importantly, however, the different conceptions of welfare overlap in practice far more than they diverge. (38) The question, then, is not what it means to make life better, but rather how to decide which policy would do so.
A. CBA and Welfare
Understanding whether a regulation does, in fact, improve quality of life is often difficult. At least theoretically, a new policy may improve the lives of a group of people without negatively impacting anyone. (39) In almost every case, however, the benefits of a regulation to one group of people will come at the expense of costs borne by either the same or another group of people. (40) Policymakers thus need a tool that can tell them whether a proposed law or regulation would improve the overall quality of human life. That is, would the policy help those who benefit more than it would hurt those who are harmed? (41)
Suppose a regulation would reduce the amount of chemical pollution emitted into the waterways and thereby reduce the number of people who die of cancer from the chemical. In so doing, however, it would increase the cost of manufacturing some good, forcing the millions of consumers who purchase it to pay more per person for the good. Whether the benefit of reducing cancer rates is greater than the cost of increasing the prices that consumers must pay depends, in part, on the respective effects of health and consumer purchases on human welfare.
CBA provides a method for comparing such seemingly incommensurable values. Its solution is to convert all costs and benefits into a uniform metric, monetary value, by figuring out how much money people would be willing to pay for the positives that regulations can give them. Via this method, an agency can monetize the value of health and compare it to the monetary value of consuming goods.
Imagine that the clean-water regulation would save ten lives (42) per year, but that it would also drive up manufacturing costs substantially. Each of the 1 million consumers who purchase the affected good would have to pay $50 more per year to acquire that product. CBA asks whether it is worth spending $50 million ($50/person x 1 million people) to save 10 lives. To answer this question, CBA must place a price on the lives being saved.
To find out the cost people would be willing to pay for any type of regulatory benefit, such as avoiding the loss of life from cancer, CBA has two methods available. The first is "revealed preferences" (43) and the second is "stated preferences," the latter of which is most commonly determined by contingent valuation surveys that ask people how much they would be willing to pay for a benefit. (44) Revealed preferences are available when people have been faced with an opportunity to choose between some regulatory benefit and some amount of money in their actual lives, such that CBA can simply observe which option they chose. Their decision is said to reveal whether they prefer, for example, having more money or reducing their risk of death. Identifying that preference enables regulators to place a value on something like increased water quality, because it shows how much money people are willing to spend in order to minimize or eliminate a risk to their life. When they are available, revealed preferences are typically preferred to stated preferences, although this is not an absolute: a high-quality stated-preference study may be chosen over a lower-quality revealed preference study.
When analyzing actual regulations with trade-offs like those of the clean-water regulation mentioned above, economists performing CBA would typically use the revealed preference method. (45) They would look for a real-life situation in which people have chosen between having more money and avoiding a low-probability risk of death. Such a situation is said to arise when people choose their jobs, because one thing that differentiates jobs is the degree of mortality risk that they entail. Being a firefighter, for example, is more dangerous than being an accountant. CBA's idea is as follows. First, it uses statistical analysis to try to identify two jobs that are the same in every way except two: Job A is riskier than Job B, and to compensate for that risk, Job A pays more than Job B. People who choose Job A rather than Job B are said to have willingly accepted a somewhat higher risk of death (one that is low probability in absolute terms, but still higher than the risk in other jobs) in return for the benefit of higher wages. The amount of extra money that they make is the revealed market value of risk avoidance. If a job with a 1-in-10,000 annual risk of death pays $600 more annually than an otherwise comparable job with no risk (the hypothetical no-risk job is used here for simplicity of explication), then the value of avoiding such a risk is pegged at $600. Accordingly, society would collectively be willing to spend $6 million ($600 multiplied by 10,000) for each life saved. (46) Indeed, this is close to the actual number that economists employing CBA have produced. (47) A regulation that will save 10 lives is thus deemed to increase overall well-being if and only if it costs consumers a collective total of $60 million or less.
If no revealed preference were available, then CBA would call for the use of a contingent valuation study. This would entail giving people surveys that ask how much money they would be willing to spend in return for avoiding a 1-in-10,000 risk of death. These surveys have also been used, for example, to learn people's willingness to pay for things like preserving the lives of endangered species. (48)
B. The Cote Advantage of WBA over CBA
CBA is based on this idea: how much money a person is willing to pay for a thing shows how much the thing increases her welfare. But that is not true. When someone buys a thing in the hope of improving her welfare, she has made a prediction--a guess--about how the thing will affect her. That prediction may well be wrong, and indeed it usually is. Daniel Gilbert and Timothy Wilson's pioneering work has demonstrated that people are not good at predicting how their choices will affect how they feel in the future. (49)
By contrast, people are good at reporting how they feel right now. In-the-moment self-reports pass the same tests of reliability and validity that are failed by affective predictions. (50) This should not be surprising; guessing how you will feel in the future is of course more error-prone than saying how you feel now. And the reasons for this are apparent: "[The mind's] simulations are deficient because they are based on a small number of memories, they omit large numbers of features, they do not sustain themselves over time, and they lack context. Compared to sensory perceptions, mental simulations are mere cardboard cut-outs of reality." (51)
Thus, a decision tool will be better at approximating welfare if it is based on self-assessments of how people feel in the moment than if it is based on predictions of how people will feel in the future. This is the central insight behind well-being analysis and its primary advantage over cost-benefit analysis. (52)
II. WELL-BEING ANALYSIS
Defenders of CBA have long argued that, despite its flaws, cost-benefit analysis is the best available means for determining the welfare effects of a project or regulation. (53) That may no longer be the case. We propose here an alternative method for analyzing regulatory policy: well-being analysis (WBA). WBA shares the basic framework of CBA, that of comparing costs and benefits, but it differs in the data and analytical tools it employs to make such comparisons.
Instead of monetizing the effects of regulation, WBA "hedonizes" them. That is, it measures how much a regulation raises or lowers people's enjoyment of life. For example, if a regulation would result in improved health but higher prices of products, then WBA would compare how much more people enjoy their lives when they are made healthier with how much less they enjoy their lives when their buying power decreases.
Like CBA, WBA is a tool for analyzing the welfare effects of policies--not a panacea meant to be the last word on what should be done. Policy analysis often proceeds by analyzing welfare effects and then weighing those effects against whatever other considerations are deemed relevant by regulators, legislators, and the citizenry they serve, (54) including fairness, justice, and human dignity. (55) Our contribution is to try to improve upon the first step of the process, the step in which welfare effects are measured. This would influence policy, but it in no way implies that we think the first step is the only step. Like proponents of CBA, we acknowledge the role that other considerations may play. (56)
Subsequent Parts of this Article argue that WBA solves many of the conceptual and methodological problems facing CBA. This Part introduces WBA and explains the sources, validity, and reliability of its data.
A. WBA: The Basic Framework
WBA directly analyzes the effects of regulations on people's quality of life. To do that, it relies on hedonic-psychology data that measure how different factors affect people's enjoyment of their lives. In theory, such measures could perhaps be purely neurological--taken by a machine that reports how good someone feels at all times. But unless and until that sort of technology is created, psychologists must rely instead on individuals' personal assessments of how their lives are going for them at a particular moment in time. Fortunately, these self-assessments can be taken in ways that yield highly reliable results, as we explain in detail in the following Section.
Individuals' self-assessments indicate their level of subjective well-being (SWB), or "happiness." Recently, psychologists and economists have developed increasingly sophisticated surveying and statistical methods that enable the collection and analysis of well-being data on a large scale. (57) WBA uses these data to evaluate the welfare consequences of regulations by comparing the well-being gains and losses of affected parties. This Section explains the conceptual framework behind WBA, whereas the following Section discusses the data upon which WBA relies. The final Section of Part II explains how the data would be used in the actual performance of WBA.
WBA relies on the same basic cost-benefit-weighing principle that undergirds CBA: all else equal, regulations whose benefits exceed their costs are valuable because they enhance overall welfare. The main difference between the two techniques involves the way in which costs and benefits are calculated and compared. Regulations involve both market and nonmarket costs and benefits. For CBA, market effects are relatively easy to handle, because computing their monetary value is straightforward. Nonmarket effects, however, are more difficult for CBA. As we will describe in greater detail, CBA must apply a variety of problematic tools to monetize the value of health, lives, and the environment. WBA avoids many of these difficulties by looking directly to a regulation's effects on people's experiences and lives. In WBA, all effects of a regulation are hedonized, which is to say that they are converted into units directly measuring their impact on the subjective well-being of the affected parties. The positive and negative hedonic impacts can then be compared with one another. They are the relevant costs and benefits.
Instead of converting regulatory effects into monetary values, WBA converts them into well-being units (WBUs). WBUs are intended to be subjective, hedonic, cardinal, and interpersonally comparable units that indicate the degree of a person's happiness for a given period of time. They are, in some respects, similar to the quality-adjusted life years (QALYs) that are increasingly popular in health economics. (58)
WBA maps a person's SWB onto a scale that would ideally run from -10 to 10, in which 10 indicates perfect happiness (subjectively defined), -10 indicates perfect misery, and 0 indicates neutrality or the absence of experience. This type of scale would allow individuals to register experiences that are worse than nonexperience (undergoing a root canal, for instance) and would simplify the comparison between experience and nonexperience. Most of the well-being data that have been collected to date have employed a scale from 0 to 10. (59) Accordingly, in the WBA that we conduct below, we utilize a scale running from 0 to 10. As the science of WBA evolves, we would envision transitioning to the preferred -10 to 10 scale. (60)
Each decile of the scale is equivalent and indicates a 10 percent change in the person's SWB. (61) Moreover, we treat the scale as identical across individuals, although, of course, the kinds of things that affect different individuals' SWB may not be. (62) One WBU is equivalent to 1.0 on the scale for a period of one year. Thus, if a person lives to the age of 100 and has an SWB of 7.0 for each year, that person has experienced 700 WBUs (7.0 WBU/year x 100 years). If an event such as illness causes a person's SWB to drop from 7.0 to 5.5 for a period of ten years, that person loses 15 WBUs (1.5 WBU/year x 10 years) over her lifetime.
This type of scale has significant benefits for any type of decision analysis, particularly regulatory analysis, because it enables the direct comparison of the hedonic impact of proposed policy changes. Imagine, for example, that the Occupational Safety and Health Administration (OSHA) is contemplating a simple regulation of workplace safety that will prevent 100 workers from each losing an arm while on the job. Implementing such a measure, however, will increase the costs of production and force factories to tire 300 workers in the affected industry.
CBA would attempt to calculate the value of the regulation by monetizing the costs and benefits it generates. With respect to the costs, CBA would in theory be able to estimate the lost wages of the 300 unemployed people. (63) The benefits, however, are trickier. Establishing a market price for the value of an arm is a fraught enterprise. (64) Given these shortcomings, the value CBA applies to the loss of an arm will be beset by a number of systematic errors associated with wealth effects, labor-market effects, and people's poor ability to predict how events like losing an arm will affect them. Accordingly, CBA may substantially and systematically misstate the benefits of the regulation.
WBA would approach the measure in the same general fashion but with different analytical data. Like CBA, WBA would attempt to quantify the cost of unemployment. But instead of looking solely to the workers' lost wages, it would calculate the hedonic cost of being unemployed. (65) Some data suggest that unemployment has a significant effect on well-being. (66) Thus, the welfare costs of unemployment may be much greater than CBA predicts. On the other side of the ledger, WBA is well positioned to hedonize the benefits of the regulation. Studies of people who have lost limbs provide fairly accurate information on the hedonic loss associated with losing an arm (and thus the benefits of avoiding these losses). (67) Again, the results are likely to be different from those determined by CBA. Studies show that individuals who lose limbs often adapt substantially to their new condition, recovering most of their lost happiness within a few years. (68) This result is contrary to the predictions of healthy people, who typically assume that such disabilities will be devastating and discount the possibility that they will adapt to the loss. (69) Accordingly, the welfare benefits of the regulation may be overstated by CBA if contingent valuation or revealed preference surveys rely on mispredictions about hedonic-adaptation." (70)
Although this example suggests that the hypothetical OSHA regulation may be less valuable than CBA implies, in many other circumstances WBA will point in the direction of more stringent regulation than CBA would suggest. For many regulations, the chief benefits will involve extending human lives, and the major costs will come in the form of higher consumer prices. In the context of WBA, loss of life constitutes an enormous hedonic cost, whereas many studies indicate that money has a relatively small effect on well-being. (71) When money is traded off against life, therefore, WBA is likely to favor health and safety regulations more than does CBA.
B. The Data of WBA
Social scientists have been attracted to the idea of measuring human welfare directly for a long time, but they have had difficulty securing valid and reliable data. (72) WBA is now feasible because of the availability of relevant data about the effects of different circumstances on individual well-being. Over the last decade or so, new social science techniques have emerged that enable researchers to study subjective well-being from a variety of different perspectives with a number of different tools. (73) These techniques allow for a more or less direct measurement of people's happiness levels, overcoming the problem that had initially driven economists to seek monetary proxies for welfare. (74) Importantly, they enable the measurement of what Daniel Kahneman has termed "experienced utility" (how good people feel), in contrast to the "decision utility" that is typically studied in CBA. (75) Decision utility measures only whether people get what they want, on the assumption that getting it will make them better off. But because that assumption has been shown to be deeply imperfect, (76) Kahneman and others have turned toward measuring directly the quality of people's experience of life. This Section will briefly discuss a few of the most promising techniques for collecting such experiential data and their relative strengths and weaknesses.
1. Life Satisfaction Surveys. The oldest method of measuring SWB is the life satisfaction survey. These types of surveys ask individuals to respond to a question such as, "All things considered, how satisfied with your life are you these days?" (77) Respondents answer on a scale that ranges from "not very happy" to "very happy." Lire satisfaction surveys have been included in the U.S. General Social Survey since the 1970s; as a result, we now have substantial quantities of longitudinal data on thousands of individuals. (78) The principal value in such surveys is the ability to correlate SWB data with a variety of other facts about people's lives. Using multivariate regression analyses that control for different circumstances, researchers are able to estimate the strength of the correlations between SWB and factors such as income, divorce, unemployment, disability, and the death of family members. (79) For example, on average, the death of a father will yield the loss of 0.25 life satisfaction points on a scale of 1 to 7 for a period of time, whereas the loss of a spouse will typically yield the loss of 0.89 points. (80)
Life satisfaction surveys are relatively inexpensive to administer and can be easily included in a variety of larger survey instruments. Accordingly, they are most valuable as sources of large-scale data about many subjects and of longitudinal data about changes in SWB over time. The latter use is especially valuable in assessing the causal effects of life events (such as marriage, disability, or unemployment) on SWB, because the same individual can be surveyed both before and after the event. This eliminates the need for between-subjects comparisons. (81) Life satisfaction surveys are less helpful, however, for assessing particularly granular changes in circumstances. More importantly, they rely on global judgments about how people's lives are going, rather than on those individuals' moment-by-moment hedonic experiences. Because hedonic experiences are often poorly remembered, such judgments can be biased because of a person's momentary mood (82) or the order in which questions are posed, among other errors. (83)
2. Experience Sampling Methods. Researchers sought to overcome the limitations of life satisfaction surveys by developing techniques that enabled them to more directly measure people's emotions while they were being experienced. The gold standard of such measures is the experience sampling method (ESM), which uses handheld computers and iPhones to survey people about their experiences. (84) Subjects are beeped randomly throughout the day and asked to record what they are doing and how they feel about it. The data that emerge from these studies provide a much more detailed picture of how people spend their time and how their experiences affect them. (85)
Despite their considerable value, ESM studies can be expensive to run. (86) This is why researchers have sought other methods that produce most of the advantages of ESM but at a lower price. One such technique is the day reconstruction method (DRM) pioneered by Daniel Kahneman and his colleagues. The DRM uses daily diary entries about each day's experiences to reconstruct an account of subjects' emotional lives. DRM studies correlate strongly with ESM studies and can be run at lower cost. (87) Similarly, the Princeton Affect and Time Survey (PATS) asks subjects to report and evaluate their experiences from the previous day. (88) It can be distributed via telephone and incorporated into other survey devices, enabling it to reach a larger population. (89)
3. The Quality of Well-Being Data. The ability to generate data is not the same as the ability to actually measure the thing that one seeks to measure. Nor is it the ability to measure it well. Data are only useful if they are reliable and valid. Much of the remainder of this Article analyzes the reliability and validity of the valuation measures used by CBA. As a means of comparison, we now discuss the quality of the data upon which WBA will rely.
Reliability is an indication of the consistency of a measurement instrument. (90) For example, a scale that reported very similar numbers every time the same weight was placed on it would be judged highly reliable. In the context of well-being measures, reliability can be assessed by examining correlations between tests and retests of the same question at separate times, as well as correlations between different questions that ask about similar concepts. (91) Meta-analyses of different well-being tools have round high levels of reliability for both life satisfaction and experience sampling methods. (92) This is especially true of more advanced multi-item measures. (93)
Just because a measure reliably provides consistent data does not mean that it is measuring what you want it to measure. (94) The ability to actually measure the thing sought to be measured is called validity. (95) Although a full review of the validity of well-being measures is unnecessary here, (96) it is worth noting a number of findings that support the conclusion that a person's well-being can be validly measured by the tools discussed in the previous Subsection. First, despite the rather different techniques used to collect data, the various measures of well-being tend to correlate with one another. (97) One's overall life satisfaction is correlated both with the amount of positive and negative affect that one feels (98) and with one's satisfaction with the domains of one's life. (99) Not only are subjective reports of well-being correlated with one another, they are also correlated with external measures, such as third-party informant reports, (100) facial expressions, (101) and neurological data. (102) Well-being measures also tend to be fairly stable over time and exhibit high test-retest reliability. (103) But despite their overall stability, (104) they are also sensitive to changes in life circumstances: people who experience apparently negative events do indeed report lower levels of well-being--at least for a time, before they adapt. (105) Moreover, well-being scales can detect the relative magnitude of life events. For example, people who are more seriously injured predictably report lower happiness ratings than do people who are less seriously injured. (106) This suggests both that people are capable of consistently reporting how experiences make them feel and that their emotional responses generally exhibit credible and predictable patterns following specific events.
Just as CBA alternately relies upon revealed preference and contingent valuation studies, WBA would draw upon each of the data sources mentioned in the preceding Section. In some cases, longitudinal studies of overall well-being may provide the best data available for tracking people after events with potentially long-term effects. (107) These studies have been used, for example, by researchers to understand the hedonic impact of no-fault divorce laws on women in different states. (108) In other circumstances, the availability of ESM studies will enable more fine-grained analyses of regulations' effects on people's lives.
4. Criticisms of Well-Being Data. Economists and other defenders of CBA have raised a number of objections to well-being data, and before we proceed further it is worth addressing those objections. The first, and most important, is that well-being data lack interpersonal cardinality because different individuals may interpret the scales differently. (109) For example, a 5.0 on one person's scale may not be the same as a 5.0 on another person's scale. If people interpret the hedonic scales differently, it becomes impossible to know whether one person's reported change from an SWB of 5.0 to 6.0 was equivalent to another person's reported change from 5.0 to 6.0.
Although some limited evidence for concern about cardinality exists in certain contexts, methodological solutions to this problem are almost certainly available. First, differential use of the scale will only be a problem when that differential use is related to the populations being compared. For instance, imagine an agency using WBA to evaluate a project that will reduce traffic and commuting times on a highway. To determine the hedonic cost of commuting in traffic, the agency would compare the well-being of people while they are commuting with the well-being of people who are not commuting. Unless people who commute in traffic systematically use the hedonic scale differently from people who do not, different uses of the scale will simply show up as random noise. Variations among individuals in how they rate their own happiness--what they mean when they rate themselves a 5 or a 6, for instance--are likely to be random, not biased. (110) This randomness should wash out across large numbers of people. (111) In many of the situations most relevant to WBA, this is virtually certain to be the case. (112)
Cost-benefit analysis is equally subject to concerns about cardinality. Because of the diminishing marginal value of money, two individuals with differing levels of personal wealth can obtain vastly different amounts of welfare from the same gain (or loss) of income. (113) Adjusting CBA in accordance with variations in marginal values of money is quite technically complex, and the proper solution is frequently unclear or highly context dependent. (114) And the problems for CBA do not end there. Even two equivalently wealthy individuals may have vastly divergent welfare functions--additional wealth might benefit one far more than the other. Individuals' welfare functions are unobservable; (1150 economists know (or assume) that marginal values of money are positive and diminish with increasing wealth, but they can be sure of little else. (116) Economists typically respond to this problem by simply ignoring it (117) or by assuming that its effects dissipate across large populations (118)--in precisely the same way that it will for WBA. It is thus hard to imagine that interpersonal comparisons will present greater difficulty for WBA than they do for classical CBA.
A second possible obstacle for WBA lies in the ambiguities involved in aggregating interpersonal welfare states. For instance, if Person A's welfare decreases from 6.0 to 5.0 and the welfare of Persons B through Z increases from 6.0 to 6.1, it is difficult to know whether this net gain of 1.5 WBUs (119) actually indicates that overall welfare has increased, decreased, or remained constant. (120)
This objection has two components. The first is simply a repetition of the interpersonal comparison problem discussed above: it is impossible to know whether a hedonic improvement for Person B from 6.0 to 6.1 is of equivalent magnitude to a hedonic regression for Person A from 6.0 to 5.9. We have already addressed this question. The second component is the argument that, when a project leaves some people better off and others worse off, a weak welfarist (121) cannot conclude that it is worth pursuing merely because overall welfare has increased. This claim is certainly correct, but it is again identical to the problems faced by CBA or any other wealth-based decision procedure. The simple fact that a project will result in Person A receiving $100 and Person B losing $50 is not sufficient reason to undertake the project in light of distributional issues and other considerations beyond aggregate welfare. (122) This is merely another way of stating that there is no independent moral or normative significance to Kaldor-Hicks efficiency. (123) The fact that Kaldor-Hicks efficiency is not morally decisive is by now a well-accepted conclusion among even CBA's most sophisticated defenders. (124)
The final important objection to WBA focuses on hedonic compensations for prior events--when someone is compensated during Period 2 for a decrease in welfare that occurred during Period 1. Imagine that an individual has been injured in a car accident, causing her average moment-by-moment well-being to fall from 6.0 to 5.0 for a period of one year (after which time it returns to 6.0). (125) Imagine that there were two potential methods of compensating her for her injury: Plan A would raise her well-being from 6.0 to 7.0 for one year, and Plan B would raise her well-being from 6.0 to 6.5 for two years. A critic might argue that it is unclear whether either of these plans would compensate her appropriately. Depending on the relationship between her survey responses and her actual well-being, and on how she values the well-being of each of her various temporal selves, either Plan A or Plan B might over- or undercompensate her.
Upon examination it becomes evident that this objection again reduces to a combination of two arguments, one of which we have already addressed. The issue of whether a decline from 6.0 to 5.0 is of equivalent magnitude to an improvement from 6.0 to 7.0 (or twice that of an improvement from 6.0 to 6.5) is merely an intrapersonal variant on the quandary regarding interpersonal comparisons and the shape of hedonic curves. (126) We have already dealt with this question and shown that it is, if anything, more easily handled than the parallel problems surrounding CBA. On the other hand, the intertemporal problem--whether a gain in Period 2 effectively counterbalances a loss in Period 1--is simply an intrapersonal variant of a broader question of interpersonal aggregation. That is, if a project increases overall welfare, is that a sufficient condition for it to be worth pursuing, even if it decreases the welfare for some individuals? This is a difficult moral question, and one that we do not attempt to answer here. There may be many instances in which a project is welfare increasing but, for distributional reasons, should not be undertaken. Well-being analysis is not meant as an answer to distributional concerns, though of course it could be used to provide information relevant to those concerns.
To facilitate the comparison with cost-benefit analysis, we will proceed here as if the primary governmental objective were to increase aggregate well-being. (127) This parallels the principal goal underlying cost-benefit analysis. Accordingly, in the Sections that follow we describe a regulation or project as "well-being justified" or "welfare justified" if it would increase the overall aggregate welfare of the population.
5. Deliberate Manipulation of Well-Being Data. Technical and theoretical problems with well-being data aside, it is also possible that individuals or groups would seek to manipulate well-being data in order to accomplish various policy objectives. After all, it is nearly costless for an individual to answer untruthfully in response to a well-being survey. An individual who hoped to affect future policy decisions could shade her response in order to make similar policy choices appear more or less beneficial. For instance, suppose that social conservatives in Washington State, where same-sex marriage became legal in December 2012, (128) wished to prevent it from being legalized in other states as well. They might begin registering extremely low levels of subjective well-being in the wake of the legalization in order to make it appear to policymakers as if the law has harmed overall well-being in the state.
This is a serious concern, but there are a number of potential policy correctives. First, policymakers would ideally be collecting well-being data on an ongoing (longitudinal) basis in order to facilitate analysis of policy changes. This means that an individual in Washington would be completing the same well-being survey after the legalization of same-sex marriage that she was completing before same-sex marriage was ever placed on the agenda. This would reduce the salience of any given policy issue to survey respondents.
In addition, respondents would not know what policy issue their responses would be used to analyze. Policymakers might use a given set of responses to gauge the effects of same-sex marriage, or they might use them to estimate the effects of a park being built across the street or the installation of a new light-rail line. An individual who reported artificially low (or high) well-being in an effort to hamper (or promote) one type of project or regulation might well end up influencing another instead.
Finally, policymakers could employ the same types of algorithms that online reputation regimes (such as Zagat or eBay) use to detect deliberately malicious feedback. (129) These algorithms typically screen for outliers--reports that are highly inconsistent with the vast majority of other feedback on the same firm or individual. (130) Here, policymakers could conceivably use algorithms that screen out data that are inconsistent with an individual's other self-reports with no discernible basis for the inconsistency. In some cases this might mean throwing out useful information, but such screening algorithms have nevertheless proven to be accuracy enhancing in other contexts. (131) More generally, online reputation regimes have remained fairly reliable despite the strong incentives of particular individuals and firms to spread misinformation. (132) It is unlikely that well-being surveys will rare worse.
Moreover, CBA is hardly immune from this type of problem. An individual who responds to a contingent valuation survey has no incentive to provide an accurate response. (133) Thus, for instance, the same social conservative might offer an artificially high answer when asked how much she would be willing to pay to keep same-sex marriage illegal. Similarly, an environmentally conscious individual might provide an artificially high answer when asked how much she would pay for cleaner skies. Sophisticated social scientists have attempted to devise correctives to this issue, but it is impossible to eliminate the problem entirely. (134)
These types of problems are, if anything, more significant for contingent valuation surveys than they are for well-being surveys. The reason is that a contingent valuation survey necessarily highlights and makes salient the policy choice in question--the individual is asked how much she would pay for some policy outcome--which makes it easier for an individual to provide a deliberately misleading answer. The question at issue is not obscured, as it is within well-being surveys. We will discuss contingent valuation surveys in much greater depth in Part III. For the moment it suffices to note that the types of highly charged political issues that might cause individuals to manipulate well-being surveys would also cause them to manipulate contingent valuation surveys, possibly to greater effect.
C. Well-Being Analysis: An Example
How feasible is well-being analysis, and how would it differ from cost-benefit analysis? To answer those questions, in this Section we take an actual cost-benefit analysis conducted as part of an EPA regulation and recalculate the costs and benefits of the regulation using WBA.
This exercise actually stacks the deck overwhelmingly in favor of CBA and against WBA. The actual CBA used here was the product of decades of opportunities to refine CBA, and countless millions of dollars spent on studying these phenomena and performing these analyses. (135) By contrast, this Section constitutes the first WBA that has ever been conducted. There has never been any systematic collection of well-being data related to any government project, much less the regulation we analyze here.
For that reason, our analysis falls far short of the level of accuracy that could be achieved were WBA to be adopted in practice. Nonetheless, and strikingly, the WBA sketch we provide yields results that are likely to be no less reliable than those of the cost-benefit analysis that the Environmental Protection Agency (EPA) itself conducted. This demonstrates the inherent advantages of WBA, the ease with which it could immediately be implemented, and the potential for truly impressive results if it were conducted with the resources currently available to CBA.
1. EPA Regulation of Pulp and Paper Production: A Cost-Benefit Analysis. (136) The regulation we examine was promulgated by the EPA under the Clean Water Act (137) in 1998 to curb toxic effluents from pulp and paper mills. (138) Prior to 1998, pulp, paper, and paperboard mills used a number of chlorine-based chemicals in the normal manufacturing process. Dioxin and furan, two carcinogens, are among the byproducts that result from producing paper and paperboard with these chlorine-based chemicals. (139) Pulp and paper mills then released those chemicals into the waterways in quantities great enough to sicken and kill fish and cause a number of diseases, including cancer, in humans who ate the fish. (140)
The EPA considered three regulatory options. "Option A" required the mills to substitute chlorine dioxide for elemental chlorine in the production process, which reduces but does not eliminate the discharge of dioxin and furan. (141) "Option B" was a stricter rule, combining the Option A limits and a requirement that the mills eliminate lignin (a material in wood pulp), along with several other restrictions on the manufacturing process. (142) Option B would have resulted in even lower emissions of dioxin and furan than Option A. Finally, "Option TCF" ("totally chlorine free"), required that pulp and paper mills eliminate all chlorine from the production process, thereby also eliminating the discharge of furan and dioxin. (143)
The EPA estimated that this regulation would produce several different types of benefits. First, there would be fewer cancer deaths among recreational and subsistence anglers who consume fish that have swum near pulp and paper mills. (144) The EPA refused to specify a single monetary value of life, instead announcing that each life saved was worth between $2.5 and $9 million. (145) However, it is worth noting that these figures refer only to the value of the lives lost. The EPA did not possess and did not employ data on the cost of being stricken with cancer, above and apart from eventual mortality. (146) Second, reducing the quantity of dioxin released into fisheries would reduce the number of "fish consumption advisories," during which fishing must cease, and thus increase the number of days that fishing could take place. (147) Third and finally, pulp and paper mills produce sludge, which must be disposed of. Reducing the amount of dioxin and furan in the sludge would allow the mills to dispose of the sludge via cheaper means. (148)
At the same time, the regulation also imposed significant costs. Mills were forced to switch from chlorine-based chemicals to more expensive alternatives and to treat their effluents before they were released into the waterways. (149) Table 1 lists the annual costs and benefits, as calculated by the EPA, of all three options the agency considered in its regulation of pulp and paper.
As Table 1 makes clear, none of the options is cost-benefit justified according to standard CBA methodologies. The EPA selected Option A, which appears to do the least harm, yet even under that option the costs exceed the benefits by more than $228 million per year. (151)
In addition, and importantly for our analysis, the EPA calculated that the regulation would lead to the loss of significant numbers of jobs. The increased regulatory costs would increase pulp and paper prices, reducing consumer demand for pulp and paper products. (152)
This reduction in demand would force mills to lay off workers. (153) As pulp and paper production declined, suppliers and affiliated industries would also surfer and be forced to lay off workers. However, the EPA did not include these lost jobs in its cost-benefit analysis. We suspect that this stemmed from a belief, which continues to hold sway throughout the regulatory state, that workers will soon find alternative employment and the net costs of unemployment will be zero. (154) This assumption is almost certainly false, and one of us has separately criticized the EPA and other regulatory agencies for refusing to include the costs of unemployment in their cost-benefit analyses. (155)
We calculate, in Table 2, a revised cost-benefit analysis that includes unemployment costs. (The welfare costs of unemployment will also figure prominently in the WBA that follows.) For ease of explication, we list the compliance costs from Table 1 separately but combine the median figures for the three types of benefits (cheaper sludge disposal, elimination of fishing advisories, and lives saved) into one row, which we label "Median total benefits." It is worth noting that the EPA did not estimate the total unemployment that would result under Option TCF, though it did estimate the number of jobs that would be eliminated under that Option due to pulp and paper mill closures alone. (156) Based upon those numbers, which we provide below, the job loss from Option TCF would have likely been quite substantial.
What should be immediately evident from Table 2 is that regulatory-compliance costs--principally the costs of shifting to nonchlorinated chemicals--dominate even this revised cost-benefit analysis. Even for Option A, the least costly regulatory option, these compliance costs are nearly ten times greater than the total estimated benefits and more than twenty times greater than the costs related to unemployment. It is not atypical for compliance costs to dominate the cost side of the ledger in cost-benefit analysis. Industrial costs can be very steep and easily monetized, and so they can dwarf other inputs to the CBA. In addition, a glance back at Table 1 reveals that the monetized benefits of reducing deaths from cancer are quite modest when compared with the other benefits that the regulation will provide. The monetized benefits from cheaper sludge removal and fewer fishing advisories, in combination, exceed the benefits from reducing the number of deaths from cancer. These are both remarkable findings, and they shed light on the (possibly distorting) effects of monetizing costs and benefits. What remains to be seen is whether they are indicative of the true welfare effects of the regulation. That is a question we address in the following Subsection.
2. The EPA's Cost-Benefit Analysis as a Well-Being Analysis. In this Subsection we reengineer the EPA's cost-benefit analysis as a well-being analysis. To do so, we convert the costs and benefits of the regulation into well-being units. Wherever possible, we make this conversion directly. That is, we translate the benefits of reduced cancer deaths directly to WBUs, rather than adopting the EPA's pricing of those lives and then converting the dollars into WBUs. (159) All calculations are based on a well-being scale that runs from 0.0 to 10.0. What follows is a summary of the conversion of each of the costs and benefits involved.
a. Compliance Costs, Sludge Disposal, and Fewer Fishing Advisories. Compliance costs and the benefits of cheaper sludge disposal are both entirely monetary. Ideally we would measure the welfare value of fewer fishing advisories by estimating the hedonic value of fishing and multiplying it by the additional hours that anglers will be able to spend engaged in that activity. However, to our knowledge hedonic data on fishing does not yet exist. Accordingly, we use the EPA's monetary estimate of this benefit. We sum these three quantities to determine the aggregate monetary cost of the regulation.
The next question is how to translate that monetary cost into WBUs. These expenditures will have an effect on well-being only to the extent that they are paid for and felt by individuals. Some of the benefits will accrue to the anglers who are able to fish with fewer interruptions. Compliance costs and sludge-related benefits will be borne by some combination of consumers of pulp and paper and shareholders in pulp and paper companies. (The exact division depends on the extent to which pulp and paper firms are able to pass their costs along to consumers.)
It is impossible to know precisely how many households will share these costs, though nearly every household consumes paper to some degree. For purposes of this analysis we assume that the monetary costs and benefits will be equally borne by one million Americans. (160) Each individual will bear several hundred dollars in net monetary costs, depending upon the regulatory option. We also assume that each individual earns the median household income, which in 1998 was $38,885. (161)
What effect will these monetary costs have on welfare? Studies have round that life satisfaction increases logarithmically with income. We use the results of one of the largest and most recent of these studies, which found that an approximately threefold increase in income was associated with a 0.11 increase in WBUs. (162) (Similarly, a two-thirds decrease was associated with a 0.11 decrease in WBUs.) (163) That is, an individual whose income increased from $100,000 per year to $272,000 per year would gain 0.11 WBUs per year. If that same individual's income decreased from $100,000 to $36,700, she would lose 0.11 WBUs. The total gain or loss is given by the following formula:
(1) Welfare loss due to income decline = 0.11 WBUs x (ln (new income) - In (old income))
We apply this formula to the income loss caused by the net costs of EPA's regulation in Table 3, below.
b. Cancer Cases Avoided. The EPA provided a range of estimates for the number of cases of cancer that will be avoided under each regulatory option. In the interest of simplicity, we base our calculations on the median number. There are limited available data on the welfare loss that an individual experiences when she is sick with cancer, but one study calculated the welfare loss from "stomach/liver/kidneys or digestive problems," which we believe is the closest analog. (164) That welfare loss is 0.238 WBUs per year while the person is sick. (165) We assume that the typical individual who dies from cancer caused by dioxin and furan effluents is sick with cancer for two years and then dies thirty years before she normally would. (166) This is obviously a rough assumption, but it is no rougher than the EPA's assumption that all lives are equivalently valuable and have a median value of $5.75 million. (167) The average American has a life satisfaction of 7.4 (again, on a scale of 0.0 to 10.0). (168) When an individual dies, she loses all of the welfare that she might otherwise have experienced throughout the remaining years of her life. (169) Thus, we calculate the welfare benefit from avoiding one fatal case of cancer by the following equation:
(2) Welfare benefit from avoided fatal cancer = 2 x (0.238 WBUs) + 30 x (7.4 WBUs) = 222.48 WBUs
c. Unemployment. Unemployment is one condition about which there exists substantial hedonic data. Studies indicate that unemployment has a significant impact on well-being. (170) Unemployed individuals surfer a loss of 0.83 WBUs per year during the time that they remain unemployed. (171) Even after finding new employment, these same individuals lose an average of 0.34 WBUs per year during the next seven years after they begin working again. (172)
For purposes of this WBA, we assume that the average person who becomes unemployed as a result of this regulation is out of work for six months. This corresponds roughly to the median duration of unemployment in the years 2011 and 2012. (173) Each unemployed individual thus loses 0.83 x 0.5 = 0.415 WBUs during the period of unemployment. In addition, she loses 0.34 WBUs per year for the seven years following reemployment, for a total of 0.34 x 7.0 = 2.38 WBUs.
The EPA's CBA presents only yearly costs and benefits, not total costs and benefits. The agency annualized all costs over a 30-year period. (174) However, the agency calculated total (as opposed to yearly) unemployment. Accordingly, we divide the hedonic costs of being unemployed by 30 to obtain the yearly costs, similarly annualized over a 30-year period. The hedonic effect of the unemployment caused by the EPA's pulp and paper regulation is given by the following equation:
(3) Welfare cost of unemployment per job lost = (-0.83 x 0.5 - 0.34 x 7.0)/30 = -0.093 WBUs
We are now prepared to aggregate the welfare effects of the various costs and benefits. Table 3 presents the WBA of the EPA's regulation.
This WBA diverges from the EPA's CBA in two particularly notable respects. First, Option A now appears welfare justified: it will increase overall well-being in the net. Option B is still not welfare justified, but it appears less egregiously harmful than it did through the lens of cost-benefit analysis. The EPA may well have been correct to choose Option A (rather than not regulating at all), contrary to what CBA would indicate. Second, and perhaps more importantly, the monetary costs of the regulation, which dominated the CBA, are nearly irrelevant here. Instead, the benefits of saving lives and the costs of unemployment produce the dominant welfare effects. This may appear surprising to scholars steeped in cost-benefit analysis, but it is entirely consistent with reams of evidence demonstrating that changes in wealth and income have extremely small impacts on individual well-being. (176)
This is not to say that policymakers should begin ignoring the effects of their regulations on wealth. As we explain in Part III.D, regulations that increase welfare at the expense of vast amounts of wealth might eventually become self-defeating and eliminate future opportunities for welfare gains. This is why we would not rule out preserving CBA as a complement to WBA. But the WBA we perform here makes clear the distortions introduced by CBA's focus on wealth and monetization. Regulations that do not appear cost-benefit justified might in fact be found to greatly enhance welfare once that welfare is measured more directly.
Of course, we present here only a back-of-the-envelope sketch of a WBA. Our conclusion that the EPA's pulp and paper regulation was welfare-enhancing is necessarily tentative and dependent upon our assumptions, which may be incorrect. But this exercise should demonstrate the feasibility of WBA as a workable decision tool. It is possible to conduct a full-scale WBA of a major regulation using only the scattered data currently available. With sustained effort and attention on the part of the regulatory state, WBA could revolutionize the accuracy with which prospective laws are evaluated.…
Questia, a part of Gale, Cengage Learning. www.questia.com
Publication information: Article title: Well-Being Analysis vs. Cost-Benefit Analysis. Contributors: Bronsteen, John - Author, Buccafusco, Christopher - Author, Masur, Jonathan S. - Author. Journal title: Duke Law Journal. Volume: 62. Issue: 8 Publication date: May 2013. Page number: 1603+. © 2009 Duke University, School of Law. COPYRIGHT 2013 Gale Group.
This material is protected by copyright and, with the exception of fair use, may not be further copied, distributed or transmitted in any form or by any means.