CHANGES IN RELIABILITY STANDARDS SINCE DAUBERT
This section examines trends in how often federal district court judges address reliability in written opinions and how frequently evidence is found unreliable. We use the findings to draw inferences about whether judges have more actively screened evidence for reliability and about the consequences of any increased scrutiny. The evidence suggests that since Daubert, judges have examined the reliability of expert evidence more closely and have found more evidence unreliable as a result. Our analysis, however, does not allow us to conclude whether this increased scrutiny resulted in better outcomes.
We begin this section by examining tends over time in the frequency with which reliability is addressed in the written opinions and the proportion of challenged evidence that is found unreliable. We then explore how broadly Daubert has been applied in order to determine whether judges have restricted their attention to the “hard” sciences addressed by the decision or have more actively examined reliability in other substantive areas as well.
EVIDENCE IS FOUND UNRELIABLE
For each of the 601 elements of evidence in our sample, we recorded whether the judge addressed reliability. We considered reliability addressed if the judge evaluated the theory, methods, procedures, and logic underlying the findings or the validity of the findings themselves.1 Reliability pertains to the trustworthiness of the evidence, not whether it is relevant to the particular case.2 The proportion of evidence in which reliability is addressed in large part reflects the proportion of evidence in which reliability is questioned by challengers. Parties challenging evidence likely raise the issue of reliability only when questioning it, and judges by and large address reliability only when challengers question it.
In addition, we recorded whether the judge found the evidence unreliable. The proportion of challenged evidence that is found unreliable reflects both the proportion of challenged evidence in which reliability is questioned and the standards applied by judges to evaluate reliability. In the discussion below, we report changes in the proportion found unreliable given that reliability was addressed. This conditional probability can be thought of as the success rate of challenges to reliability. It reflects the standards that judges apply in determining reliability (as____________________