Randomized Evaluation in Legal Assistance: What Difference Does Representation (Offer and Actual Use) Make?
Greiner, D. James, Pattanayak, Cassandra Wolos, The Yale Law Journal
INTRODUCTION A. The Research Program B. Offers Versus Actual Use I. OUR STUDY A. The Unemployment System and First-Level Appeals in Massachusetts B. The Status of Research on Representation in First-Level Appeals Prior to Our Study C. The Harvard Legal Aid Bureau D. Study Methodology E. Results II. THE CURRENT STATE OF OUR KNOWLEDGE A. Three Methodological Problems 1. Failure to Specify the Intervention: When Is Representation Assigned? 2. Selection Effects: How Is Representation Assigned? a. Judge-Induced Selection Effects: Outcome-Driven Counsel Appointments? b. Client-Induced Selection Effects, Part I: Only the Strong Want To (and Do) Survive? c. Client-Induced Selection Effects, Part II: Different Circumstances? d. Lawyer-Induced Selection Effects: Different Ways of Culling Clients? e. Selection Effects: Are Plausible Guesses Possible? 3. Accounting for Uncertainty B. How Well-Run Randomized Experiments Solve These Problems III. WHERE DO WE GO FROM HERE? A. The Limits of Randomized Studies 1. Systemic Change 2. Fielding Studies 3. Provider Objections B. Maximizing Information 1. Nonpecuniary Interests 2. Outreach, Intake, and Client Choice 3. System Accessibility and Accuracy CONCLUSION APPENDIX: "POWER CALCULATIONS"
A. The Research Program
Particularly with respect to low-income clients in civil cases, how much of a difference does legal representation make? (1) The question is fundamental to the legal profession and to legal pedagogy, and it implicates overlapping areas of inquiry: from access to justice and the civil Gideon movement, (2) to the design of adjudicatory systems; from legal ethics and the unauthorized practice of law, to efficiency in the delivery of legal services (a subject addressed in the organic statute of the Legal Services Corporation). (3) In 2007, we initiated discussions with various legal services providers to generate interest in a series of randomized trials (4) that we hoped would provide gold-standard answers to the question of how much of a difference both an offer of and actual use of legal representation make. We sought practice areas in which demand for legal services outstripped a provider's capacity, and in those areas, we suggested that the provider allow us to randomize which of several eligible potential clients would receive an offer of representation. (5) The randomization would create "treated" (offered representation from this service provider) and "control" (no such offer) groups identical (up to random variation) in all ways except for the offer of representation. Examination of official records for the treated and control groups would allow rigorous measurement of the difference the offer of representation made with respect to the set of outcomes recorded in those records (an important but not exhaustive set of possible outcomes of interest). Fancier statistical techniques might allow inferences about the effect of actual use of representation for the set of persons who requested help from the particular provider.
As we explain in Part III, this provider-centered framework is not the only kind of randomized trial one can use to measure representation effects. Thus, one of our hopes was to use these early efforts as a springboard for additional randomized studies that were court-centered as well as spread across various other dimensions of interest, such as legal area (e.g., housing, family law, immigration, bankruptcy), level of representation (e.g., full attorney-client relationships, lawyer-for-the-day, advice), type of adjudicator (e.g., judge, administrative law judge (ALJ)), and style of adjudication (e. …