Valuable Patents Redux: On the Enduring Merit of Using Patent Characteristics to Identify Valuable Patents

Article excerpt

In 2004, John Allison, Mark Lemley, Kimberly Moore, and Derek Trunkey published an article, Valuable Patents, reporting the results of the most comprehensive study ever done comparing various characteristics of patents that ended up in infringement litigation with patents that had not been litigated.1 Allison et al. compared a very large population of unlitigated patents-approximately three million-with a set of more than 6,800 litigated patents for which the infringement litigation terminated in 1999 or 2000.2 Appropriate adjustments were made for the different ages of the patents.3 Because of the size of the data set, the large study was not finely graded.4 Thus, Allison et al. also performed a much more finely graded study comparing characteristics of a random sample of 300 of the litigated patents that were issued during 1996-1998 (25% of the litigated patents issued during that time period) with a random sample of 1,000 unlitigated patents issued during the same time period.5 Based on the consensus that litigated patents are on average more valuable than unlitigated ones and thus represent a subset of all valuable patents, Allison et al. found among other things that litigated patents have significantly more total claims, independent claims, prior art references, and forward citations (citations to the patents by later patents) than unlitigated patents.6 Litigated patents are also relatively young; that is, litigation tends to occur during the first few years after patents are issued.7 The authors concluded not only that many valuable patents can be identified retrospectively but also that reasonable predictions can be made about patents that are likely to be valuable enough to be litigated.8

David Adelman and Kathryn DeAngelis challenge the use of patent characteristics to identify valuable patents either retrospectively or prospectively.9 Adelman and DeAngelis 's primary empirical contentions regarding the unsuitability of patent characteristics for identifying valuable patents are: (1) the distribution of the value of patents is highly skewed, with most having little or no value and only a relatively small portion having any value at all;10 (2) the distributions of the several patent characteristics (called "patent metrics" by Adelman and DeAngelis) often viewed as indicators of value are skewed, thus rendering them unreliable as relevant value metrics;11 (3) Valuable Patents' findings of differences in the characteristics of litigated and unlitigated patents that are statistically significant does not mean that the differences are of a practically significant magnitude;12 and (4) the "base-rate" problem, which may occur when attempting to predictively identify a small subset of a population, prevents Valuable Patents' results from having any predictive power.13

Among other, less important criticisms, Adelman and DeAngelis observe that the sample study yields some results at variance with the large population study for a few technology areas.14 Allison et al. stated in Valuable Patents that their small study is not comparable with the large population study15-a position that we reaffirm here. In the large population study, Allison et al. were forced to use the same National Bureau of Economic Research (NBER) database of patents16 that Adelman and DeAngelis use in their article because there is no other available database containing such a large number of patents with some usable patent characteristics. Although useful, that database is deficient in a number of ways, largely because of its sheer size. The six technology or industry fields reported in that database are extremely coarse and are based on the highly flawed classification system of the U.S. Patent and Trademark Office (PTO). As a result, the identified fields are characterized by huge data-comparability problems; a category like Computers and Communications (CompComm) necessarily includes patents covering many disparate technology areas. This is the exact reason why Allison et al. …