Estimating Empirical Blackstone Ratios in Two Settings: Murder Cases and Hiring
Bushway, Shawn D., Albany Law Review
There is a growing awareness in the legal literature of the need to estimate the prevalence of errors that exist within the criminal justice system. A majority of the time, the focus is on the false positive, or wrongful conviction, rate. Yet, a complete picture of the decision process requires estimates of both false positives and false negatives. In this paper, I generate an estimate of the false negative rate for a representative sample of murders in Chicago. I also estimate the cost ratio of false negatives to false positives that would be needed to justify using records of incarceration to identify people at risk in the Chicago metropolitan area. Both estimates should shed meaningful light on the growing debate about what rules should be set to achieve more socially optimal decisions in both the criminal justice system and the labor market. Future work should focus on replicating and extending these preliminary estimates.
Some decisions involve a choice between two options. In the case of the trial, the jury is trying to decide if a person is guilty or innocent, starting from the null hypothesis that the person is innocent. In the case of employment, an employer is concerned about hiring a risky employee who will harm fellow employees or clients. In this simplest kind of decision framework, there are two kinds of errors. False positives are innocent or not risky people who are identified as guilty/risky. (1) False negatives are guilty/risky people who were not identified as guilty/risky. (2)
The Blackstone ratio on which this special issue is based makes it clear that policymakers can specify the nature of the tradeoff between these error rates. Specifically, in the context of conviction, Blackstone hypothesizes that it would be ideal to have a justice system that generates ten false negatives for every false positive. (3) There has been considerable subsequent debate about the relative desirability of a conviction of an innocent man versus allowing a guilty man to go free, or whether such a tradeoff is even morally acceptable. (4) A detailed review of the literature by Alexander Volokh found that the most commonly accepted standard in the U.S. is that ten guilty men should go free before one innocent man should be found guilty. (5) Volokh also found states that advocate for a one-to-one standard, as well as one state, Oklahoma, with a standard of one false positive for every one hundred false negatives. (6)
I am not aware of a systematic effort to describe the false negative rate in the U.S. criminal justice system using archival data, but Shawn Bushway and Brian Forst provide a back-of-the-envelope estimate based on aggregate data of 1500 to 3000 false negatives to each false positive in the U.S. criminal justice system. (7) In this paper, I use existing data to determine a rudimentary estimate of the number of false negatives in murder investigations in Chicago in 1979. I find an empirical Blackstone ratio of sixty-one false negatives for every false positive, with a lower bound of thirty-five-to-one.
Employers and others who use criminal history records are more concerned about false negatives than false positives; that is, they want to avoid hiring risky people who will harm someone while working. As a result, employers are plausibly willing to tolerate a certain number of false positives for every false negative. In the extreme case, employers would be worried only about false negatives. Most of the costs of false positives, like increased crime due to frustration or lack of legitimate income, are born by agents other than the employer. However, society can make employers feel some of those costs (for example, through the threat of Title VII litigation). And employers can also have direct costs if they cannot find enough qualified employees. Indeed, there is evidence that employers do willingly hire ex-offenders. (8)
I am aware of no literature that tries to quantify the acceptable tradeoff between false positives and false negatives by employers. …