Estimating Empirical Blackstone Ratios in Two Settings: Murder Cases and Hiring

Bushway, Shawn D., Albany Law Review

ABSTRACT

There is a growing awareness in the legal literature of the need to estimate the prevalence of errors that exist within the criminal justice system. A majority of the time, the focus is on the false positive, or wrongful conviction, rate. Yet, a complete picture of the decision process requires estimates of both false positives and false negatives. In this paper, I generate an estimate of the false negative rate for a representative sample of murders in Chicago. I also estimate the cost ratio of false negatives to false positives that would be needed to justify using records of incarceration to identify people at risk in the Chicago metropolitan area. Both estimates should shed meaningful light on the growing debate about what rules should be set to achieve more socially optimal decisions in both the criminal justice system and the labor market. Future work should focus on replicating and extending these preliminary estimates.

I. INTRODUCTION

Some decisions involve a choice between two options. In the case of the trial, the jury is trying to decide if a person is guilty or innocent, starting from the null hypothesis that the person is innocent. In the case of employment, an employer is concerned about hiring a risky employee who will harm fellow employees or clients. In this simplest kind of decision framework, there are two kinds of errors. False positives are innocent or not risky people who are identified as guilty/risky. (1) False negatives are guilty/risky people who were not identified as guilty/risky. (2)

The Blackstone ratio on which this special issue is based makes it clear that policymakers can specify the nature of the tradeoff between these error rates. Specifically, in the context of conviction, Blackstone hypothesizes that it would be ideal to have a justice system that generates ten false negatives for every false positive. (3) There has been considerable subsequent debate about the relative desirability of a conviction of an innocent man versus allowing a guilty man to go free, or whether such a tradeoff is even morally acceptable. (4) A detailed review of the literature by Alexander Volokh found that the most commonly accepted standard in the U.S. is that ten guilty men should go free before one innocent man should be found guilty. (5) Volokh also found states that advocate for a one-to-one standard, as well as one state, Oklahoma, with a standard of one false positive for every one hundred false negatives. (6)

I am not aware of a systematic effort to describe the false negative rate in the U.S. criminal justice system using archival data, but Shawn Bushway and Brian Forst provide a back-of-the-envelope estimate based on aggregate data of 1500 to 3000 false negatives to each false positive in the U.S. criminal justice system. (7) In this paper, I use existing data to determine a rudimentary estimate of the number of false negatives in murder investigations in Chicago in 1979. I find an empirical Blackstone ratio of sixty-one false negatives for every false positive, with a lower bound of thirty-five-to-one.

Employers and others who use criminal history records are more concerned about false negatives than false positives; that is, they want to avoid hiring risky people who will harm someone while working. As a result, employers are plausibly willing to tolerate a certain number of false positives for every false negative. In the extreme case, employers would be worried only about false negatives. Most of the costs of false positives, like increased crime due to frustration or lack of legitimate income, are born by agents other than the employer. However, society can make employers feel some of those costs (for example, through the threat of Title VII litigation). And employers can also have direct costs if they cannot find enough qualified employees. Indeed, there is evidence that employers do willingly hire ex-offenders. (8)

I am aware of no literature that tries to quantify the acceptable tradeoff between false positives and false negatives by employers. In the second half of the paper, I conduct an exercise to back out the implied "acceptable" ratio of false negatives to false positives that is implied by an employer which uses a prison record as a predictor for homicide. I find that employers who decide not to hire people with prison records would have to argue, at minimum, that 930 false positives have the same cost as one false negative.

II. ESTIMATING THE IMPLIED "BLACKSTONE RATIO" IN MURDER CASES

There are two good outcomes for any criminal case--the guilty are convicted, and the innocent are not convicted. But there are two errors that can and will be committed in any system of justice--false negatives and false positives. Part of the value of the Blackstone ratio, in my view, is the explicit recognition that wrongful convictions are not the only errors that are committed in any system. The only way to assure no false positives would be to fail to convict anyone.

The Blackstone ratio also implies that the ratio in any given system of justice is open to manipulation. Conceivably, policymakers could set certain rules and standards that would change the "justice" outcome to achieve the desired "Blackstone Ratio," holding constant the kinds of technological innovations that could conceivably reduce both forms of error simultaneously.

However, this idea is hypothetical at this point because there are few good estimates of the amount of errors made in the system. Through the advent of DNA testing, there has been a recent burst of identifying (and correcting) wrongful convictions. Although some are skeptical that good estimates are even possible, others believe that good strategies exist for generating reliable estimates of wrongful convictions in the system. A recent detailed effort found that 3.3% of those convicted in capital death cases were factually innocent, (9) and there is some consensus from recent reviews that one to five percent of all felony convictions are of factually innocent people. (10)

Within this literature, there is a consistent effort to differentiate the "factually innocent" from those who are acquitted and might actually be guilty. However, I found no explicit attempt to estimate the number or rate of false negatives. (11) But these two errors are linked systematically and unavoidably. Identifying one without the other captures only half of the error story, and could lead to changes in policy that dramatically increase the number of guilty people who are not convicted.

In the following, I make use of a little known study of police investigative techniques, based on crimes committed in 1979, to conduct a simple exploration of false negatives for murder in Chicago. …

Questia, a part of Gale, Cengage Learning. www.questia.com
Publication information: Article title: Estimating Empirical Blackstone Ratios in Two Settings: Murder Cases and Hiring. Contributors: Bushway, Shawn D. - Author. Journal title: Albany Law Review. Volume: 74. Issue: 3 Publication date: Spring 2011. Page number: 1087+. © 1999 Albany Law School. COPYRIGHT 2011 Gale Group.

This material is protected by copyright and, with the exception of fair use, may not be further copied, distributed or transmitted in any form or by any means.