Introduction: The Empirical Revolution in Law
Ho, Daniel E., Kramer, Larry, Stanford Law Review
I. IMPETUS II. SKETCH III. THEMES
In November 2012, Stanford Law School hosted the Seventh Annual Conference on Empirical Legal Studies (CELS), purportedly "the largest annual refereed academic legal conference in the world." (1) Of the forty-some members of Stanford's law faculty, nearly thirty became integrally involved with the program, reflecting an enormous shift in interest over the years. We see this same shift in the proportion of articles in the Stanford Law Review invoking empirical work, which (as Figure 1 shows) has skyrocketed over the decades.
[FIGURE 1 OMITTED]
How did this come to be? What role does empirical work play in law'? Where is that work leading? And why might (or might not) one be skeptical? Rather than answer these questions abstractly and top-down--a near-impossible task--this issue of the Stanford Law Review addresses these questions bottom-up, through the lens of particular substantive fields. It selects some of the best papers presented at CELS as a springboard to contextualize the intellectual history of questions in the field, to provide a sense of why empirical work has become so important in legal scholarship, and to develop a more affirmative vision for empirical work in law.
The Essays in this volume cover six different fields--civil procedure, contracts, constitutional law (specifically, election law), criminal law, corporate governance, and law and philosophy--and demonstrate the unique, field-specific, and vibrant role of empiricism across different areas of law. While some areas have a long history of empirical work (criminal law being the obvious example), in others this kind of work is relatively new, raising unique questions about its interplay with different disciplines (e.g., civil procedure or experimental philosophy). The choice of fields is not meant to be comprehensive, but rather to illustrate empiricism's engagement across diverse areas. (3)
Such analysis is in many ways natural for Stanford, which has a long tradition of engagement with legal empiricism. Lawrence Friedman, one of the early proponents of law and society and its "law in action" approach, joined the faculty in the 1960s. (4) Bob Rabin conducted one of the first quantitative studies of prosecutorial discretion in the 1970s. (5) Deborah Rhode deployed surveys to assess enforcement of the unauthorized practice of law and the legal profession in the 1980s. (6) And the addition of scholars like John Donohue and Deborah Hensler in the 1990s, and Rob Daines, Dan Kessler, Alison Morantz, Joan Petersilia, and a number of others in more recent years, has strengthened the empirical turn. But Stanford's receptivity to empirical work stems not just from faculty members who specialize in such work, but from the engagement by and with the faculty at large. The contributors to this volume, though not all principally empiricists, represent this broad level of empirical engagement. Each of them brings a long-term synoptic vision of his or her field to consider how the empirical movement has unfolded within it, in spite of its many fits and starts. (7)
In The Twiqbal Puzzle and Empirical Study of Civil Procedure, David Engstrom describes how "a hundred empirical flowers have bloomed" (8) around the question of the effects of Bell Atlantic Corp. v. Twombly (9) and Ashcroft v. Iqbal, (10) or "Twiqbal," which heightened the pleading standard in civil litigation. Engstrom attributes this blossoming of interest to a combination of easily available evidence (most notably in the form of docket sheet data) and advances in statistical methods. But he also questions how well those methods have been deployed. Roughly ten years have passed since Lee Epstein and Gary King excoriated law reviews for publishing so much empirical work that violated basic rules of inference, (11) yet Engstrom documents how much this pathology persists in the context of Twiqbal. Researchers commonly introduce sampling bias, omit key confounding covariates, and fail to adjust for potential selection effects. Moreover, many of these design errors hinge on substantive questions: focusing on claims as opposed to parties as the unit of analysis, for example, may exaggerate the impact of Twiqbal, as some plaintiffs will still remain in court. In an informal recta-analysis of studies purporting to assess the effect of Twiqbal, Engstrom shows that the size of any purported effects decreases dramatically as better design principles are taken into account. Of course, methodological challenges exist in any field of observational studies. But to better understand civil procedure, Engstrom concludes, both producers and consumers of empirical work must grapple more with research design.
In But Seriously, Folks, What Do People Want?, (12) Barbara Fried looks at recent contract law scholarship examining broad differences between the "heuristics and biases" and "rational choice" schools. Fried challenges an assumption both schools share--that "consumers' 'true' preference is to optimize on [commodity] bundles"---and asks what difference it would make if consumers instead "minimize[d] the time and mental energy spent on trades." (13) She illustrates how subjects in experimental studies on the endowment effect and contract precautions may be interpreted to be "Friedian" satisficers, and how policy prescriptions driven by concerns for bias may fundamentally misfire if consumers simply want to move on. Fried then connects these findings to a larger literature on hedonics, which suggests that once basic needs are met, increased wealth has little effect on individuals' sense of well-being. At core, Fried complicates the world for contract empiricists: challenging them in future work to recognize the possibility of, and account for the difference between, Friedian satisficers and more conventional contracting agents.
In Answering Questions, Questioning Answers, and the Roles of Empiricism in the Law of Democracy, Pam Karlan discusses the complex and longstanding relationship between empirical social science and election law. (14) In the area of representation, legal doctrine spurred innovations in social science, and expert witness testimony has become critical to resolving legal disputes about elections. When it comes to campaign finance, on the other hand, courts commonly eschew social science while making strong empirical assumptions--assumptions that may well be challenged by or at odds with social science. Karlan shows that institutional insight ("situation sense") is critical to evaluating the credibility of inferences. For instance, Karlan argues, one cannot assess the effect of Missouri's tightening of political spending disclosure on attack ads without knowing why Missouri reformed its campaign finance laws (in econometrics parlance, without understanding the possibility for "exogeneity"). For if something else is afoot in Missouri politics, any changes in attack ads that accompany the new disclosure rules may not be attributable to the rules. Karlan concludes by questioning the proper allocation of work between law schools and social science departments, given that the disciplinary boundaries have been porous in election law from the start.
In Intuitions, (15) Mark Kelman critically examines the use of experimental surveys about moral intuitions in law. Experimental philosophy is a rapidly growing field using surveys about moral intuitions to interrogate moral principles. (16) Numerous CELS papers employ such techniques in a variety of settings, as varied as legal ethics, family law, and the law of war. Kelman concedes that there might be a valuable role for such work: "all else equal, legal rules should track commonplace moral beliefs." (17) But he also expresses deep skepticism about whether such surveys can answer "what legal rules are truly normatively desirable." (18) Even assuming philosophers ought to care about such intuitions--a point Kelman questions--it's not at all clear that such surveys properly capture them. If the respondents in a study of views about the laws of war were all trained in the army, for example, their answers may reflect learned principles as opposed to genuine intuitions. Kelman posits that, to the extent moral intuitions matter, philosophers have most to learn from unreflective intuitions that are culturally immutable.
In Fact and Fiction in Corporate Law and Governance, (19) Mike Klausner argues that empirical work has uprooted widely held theoretical propositions about corporate law and governance. The view of the corporation as a "nexus of contracts," which, absent transaction costs, should lead to governance mechanisms that minimize agency costs (especially at the initial public offering stage) long influenced the intellectual agenda of corporate law scholars. Klausner argues that three facts have caused some to question that view. First, firms rarely innovate or customize corporate governance provisions at the initial public offering stage, "contracting" instead into default rules. Second, state competition in the market for corporate law does not appear particularly robust. Third, only after thirty years of managerial dominance and other legal interventions has shareholder activism begun to enhance corporate governance. In short, Klausner shows how theory can go astray when not well grounded in institutional facts. He further argues that a similar pathology of failing to understand legal institutions can plague empirical work. Commonly used numerical indices, which purport to measure the "strength" of corporate governance arrangements, reflect misunderstandings of governance elements. Corporate governance empiricists lacking legal training should, Klausner admonishes, "take a lawyer to lunch." (20)
Finally, in Empirical Criminal Law Scholarship and the Shift to Institutions, Bob Weisberg focuses on a shift in empirical criminal scholarship towards microanalysis of institutions. (21) Part of the so-called "credibility revolution" in the social sciences, (22) such work tends to prioritize research design (often approximating a natural experiment) to credibly estimate the causal effect of some intervention of interest. One common criticism of such approaches is that credible natural experiments are inherently limited in scope. (23) When it comes to designing penal systems, however, Weisberg argues that small-scale evaluations--precisely because they focus on direct interventions--may be of more use in formulating legal policy than large-scale evaluations of, for example, whether incarceration per se meets a cost-benefit test. While Weisberg exhibits great optimism about natural experiments, he warns that studies of criminal case processing implicitly operate under the existential shadow of McCleskey v. Kemp's discomfort with empirical evidence. (24) In clarifying processes and consequences, their chief effect may thus ultimately be to expose and lay bare a confrontation with sharp, normatively laden decisions.
Each Essay highlights a distinct role that empirical legal studies plays in a particular field. In civil procedure and voting rights, new legal developments gave prominence to empirical questions. In contracts and corporate governance, empirical research unsettled widely accepted theoretical paradigms. In criminal law and the law of war, empirical research helped expose or frame ineluctable normative desiderata. Perhaps not surprisingly, given these differences, the Essays also illustrate how receptivity to the empirical turn varies--from Kelman's skepticism about empirically ascertained intuitions to Weisberg's general enthusiasm about the empirical study of criminal justice. But while the Essays present rich and nuanced views that are in many ways specific to the substantive area, two themes emerge vividly across them all.
First, each Essay shows how empiricism cannot be divorced from substance. The design of credible empirical work must be informed by substantive, institutional knowledge. Consider the well-known natural experiment using "Maimonides' Rule," a practice in Israel's public schools that requires capping class enrollment at forty students. (25) As it's plausibly random within this system whether enrollment ends up at thirty-nine students or forty-one students, the splitting of classes greater than forty into two classrooms ingeniously allows researchers to study the causal effect of class size. But the same natural experiment may fail in a country with a thriving private school system, as parents may withdraw students upon discovering a classroom of thirty-nine students. (26) Context and the factors that shape it matter. (27)
Each contributor couches this fundamental point in somewhat different terms. But when Engstrom focuses on research design, when Fried reinterprets how consumers may have experienced experiments, when Karlan champions "situation sense," when Weisberg cautions about extrapolating beyond natural experiments, when Klausner urges greater understanding of legal mechanisms, and when Kelman questions intuitions that may in fact be learned views, each is underscoring the critical role of substantive, institutional knowledge in empirical inquiry. One of us has argued that law students must not merely learn to "think like a lawyer," but should also learn to "think like a client." (28) And just as lawyers cannot effectively help clients without grappling with domain knowledge, empiricists cannot effectively deploy statistics without grappling with substantive law.
Second, the Essays underscore the extent to which empirical legal studies have become part of mainstream legal analysis. In election law, engagement with sophisticated social science has long been the norm. In civil procedure, Engstrom documents how droves of civil procedure scholars are diving in to assess Twiqbal. Kelman discusses the unique role of empiricism in law and philosophy and, despite his skepticism, values and draws on empirical work on moral intuitions. (29) And Fried herself draws on empirical work in hedonics to question shared assumptions across rational choice and heuristics and biases schools.
Although the precise questions differ by field, the empirical turn is perhaps natural with increasing data availability. Theories and conjectures abound, but they often involve assumptions and predictions about empirical facts and effects (whether implicitly or explicitly). How do pleading standards affect plaintiffs? Will consumers take optimal precautions in contracting? Do increases in campaign contributions affect public perceptions of democracy? Does prosecutorial discretion affect racial and gender disparities in sentencing? Do states care about incorporation franchise fees? Do laws of war track ordinary moral intuitions? Empirical work grounds and refines such conjectures.
The fact that not just specialists and dedicated proponents, but also broad swaths of scholars, courts, and decisionmakers, are grappling with data is a sign of the empirical revolution's vitality. This trend may be somewhat more pronounced at Stanford than other law schools, but the movement is everywhere.
To be sure, the precise role--and synergy with theoretical and doctrinal scholarship--may differ by field, as the Essays spell out. But if legal realists were the pioneers, we are all empiricists now. (30)
* The title is drawn--with apologies but to make a point--from JONATHAN KLICK, THE EMPIRICAL REVOLUTION IN LAW AND ECONOMICS (2011).
Thanks to David Engstrom, Barbara Fried, Pam Karlan, Mark Kelman, Mike Klausner, and Bob Weisberg for contributing Essays to this issue, to Liz Magill for her generous continuing support of CELS, to Ted Eisenberg, Chris Hu, Mark Kelman, Gary King, Mike Klausner, and Matt Waring for helpful comments and conversations, to the Stanford faculty for making CELS happen, and to an extraordinary team of editors of the Stanford Law Review, in particular Kendall Turner.
(1.) Theodore Eisenberg, The Origins, Nature, and Promise of Empirical Legal Studies and a Response to Concerns, 2011 U. ILL. L. REV. 1713, 1713.
(2.) For a more comprehensive content analysis, see Shari Seidman Diamond & Pare Mueller, Empirical Legal Scholarship in Law Reviews, 6 ANN. REV. L. & SOC. SCI. 581 (2010).
(3.) Though empirical work encompasses both qualitative and quantitative approaches, see Lee Epstein & Gary King, The Rules" of Inference, 69 U. CHI. L. REV. 1, 2 (2002), CELS conference papers tend to center around the latter--an emphasis that is to some extent reflected in the Essays that follow. But see Pamela S. Karlan, Answering Questions, Questioning Answers', and the Roles of Empiricism in the Law of Democracy, 65 STAN. L. REV. 1269 (2013) (reflecting on the role of qualitative empirical work). It bears noting, however, that the underlying conference papers exhibited a wide range of approaches, employing everything from descriptive inference based on in-person interviews, to observational studies with public records, to causal inference with natural experiments.
(4.) For Friedman's take on the law and society movement and its focus on empirical work, see Lawrence M. Friedman, The Law and Society Movement, 38 STAN. L. REV. 763 (1986).
(5.) See Robert L. Rabin, Agency Criminal Referrals' in the Federal System: An Empirical Study of Prosecutorial Discretion, 24 STAN. L. REV. 1036 (1972).
(6.) See Deborah L. Rhode, Policing the Professional Monopoly." A Constitutional and Empirical Analysis of Unauthorized Practice Prohibitions, 34 STAN. L. REV. 1 (1981).
(7.) See JOHN HENRY SCHLEGEL, AMERICAN LEGAL REALISM AND EMPIRICAL SOCIAL SCIENCE (1995); Michael Heise, The Past, Present, and Future of Empirical Legal Scholarship: Judicial Decision Making and the New Empiricism, 2002 U. ILL. L. REV. 819, 822-24.
(8.) David Freeman Engstrom, The Twiqbal Puzzle and Empirical Study of Civil Procedure, 65 STAN. L. REV. 1203, 1204 (2013).
(9.) 550 U.S. 544 (2007).
(10.) 556 U.S. 662 (2009).
(11.) See Epstein & King, supra note 3.
(12.) Barbara Fried, But Seriously, Folks, What Do People Want?, 65 STAN. L. REV. 1249 (2013).
(13.) Id. at 1250.
(14.) Karlan, supra note 3.
(15.) Mark Kelman, Intuitions, 65 STAN. L. REV. 1291 (2013).
(16.) See EXPERIMENTAL PHILOSOPHY (Joshua Knobe & Shaun Nichols eds., 2008).
(17.) Kelman, supra note 15 at 1292.
(19.) Michael Klausner, Fact and Fiction in Corporate Law and Governance, 65 STAN. L. REV. 1325 (2013).
(20.) Id. at 1268.
(21.) Robert Weisberg, Empirical Criminal Law Scholarship and the Shift to Institutions, 65 STAN. L. REV. 1371 (2013).
(22.) See Joshua D. Angrist & Jorn-Steffen Pischke, The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con Out of Econometrics, J. ECON. PERSP., Spring 2010, at 3; Daniel E. Ho & Donald B. Rubin, Credible Causal Inference for Empirical Legal Studies, 7 ANN. REV. L. & SOC. SCI. 17, 20-21 (2011).
(23.) See, e.g., Nancy Cartwright, Are RCTs the Gold Standard?, 2 BIOSOCIETIES 11 (2007).
(24.) See 481 U.S. 279, 314-17 (1987).
(25.) See Joshua D. Angrist & Victor Lavy, Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement, 114 Q.J. ECON. 533 (1999).
(26.) See Miguel Urquiola & Eric Verhoogen, Class-Size Caps, Sorting, and the Regression-Discontinuity Design, 99 AM. ECON. REV. 179 (2009).
(27.) See Ho & Rubin, supra note 22, at 27-28.
(28.) See Anna Caraveli, Learning to Think Like Your Clients, U. Bus. (Jan. 1, 2008), http://www.universitybusiness.com/article/learning-think-your-clients (quoting Larry Kramer).
(29.) See, e.g., Mark Kelman & Tamar Admati Kreps, Playing with Trolleys (1): Intuitions About Aggregation (Stanford Pub. Law Working Paper No. 2138475, 2012), available at http://ssrn.com/abstract=2138475.
(30.) Cf. Michael Steven Green, Legal Realism as Theory of Law, 46 WM. & Mary L. REV. 1915, 1917 (2005).
Daniel E. Ho ([dagger]) & Larry Kramer ([double dagger])
([dagger]) Professor of Law, Stanford Law School; President, Society for Empirical Legal Studies (2011-2012).
([double dagger]) President, The William and Flora Hewlett Foundation; Professor of Law and Former Dean, Stanford Law School.…
Questia, a part of Gale, Cengage Learning. www.questia.com
Publication information: Article title: Introduction: The Empirical Revolution in Law. Contributors: Ho, Daniel E. - Author, Kramer, Larry - Author. Journal title: Stanford Law Review. Volume: 65. Issue: 6 Publication date: June 2013. Page number: 1195+. © 1999 Stanford Law School. COPYRIGHT 2013 Gale Group.
This material is protected by copyright and, with the exception of fair use, may not be further copied, distributed or transmitted in any form or by any means.