Performance Evaluation in Financial Economics. (Research Summaries)

Article excerpt

Andrew Metrick *

A mutual-fund manager earns annualized returns of 20 percent per year for a five-year period. Over the same period, the stock market as a whole earns 10 percent per year. Was this manager smart, or just lucky?

Some companies engage in a lot of merger activity. Other companies do not. A researcher finds that the former group performs less well than the latter group in the stock market. Is this difference related to the merger activity, or does it simply reflect underlying differences between the two groups of firms?

While the questions just raised may seem quite different, they can be answered using similar methods. In both cases, it is necessary to define some appropriate "benchmark" return. This benchmark return then can be compared to the actual return earned by the mutual fund manager, group of merged firms, or group of non-merged firms. The difference between the actual and benchmark returns then can be defined as an "abnormal" return. Abnormal returns then can be tested for statistical and economic significance.

These are the key steps in performance evaluation (PE), a methodology central to the investigation of many questions in financial economics. The seminal PE study, Jensen (1968), uses the classic Capital Asset Pricing Model (CAPM) as its benchmark and analyzes mutual funds (1); for the next 25 years, most PE studies followed this same strategy In the last ten years, though, researchers have developed many new models of benchmark returns and demonstrated their usefulness in PE studies of both investor performance and corporate finance. In this article, I illustrate some of these diverse applications with recent examples from my own work and with studies of investment newsletters, insider trading, and corporate governance. I then discuss a new approach to PE that allows fresh insights into the canonical mutual-fund topic. I conclude with a discussion of future directions for PE-based research.

Applications

Investment newsletters have been around since the early 1900s, and the current industry of over 500 active letters has about 2 million subscribers. The typical newsletter is produced by a small staff and provides a wide range of advice targeted at the retail investor. Is any of this advice useful? Using PE methodology, I analyze the performance of newsletters' equity recommendations using a dataset of 153 newsletters' that spans 17 years. (2) In contrast to most PE studies, this study's data contain information about every transaction, rather than just the periodic returns earned by these transactions. Thus, I can address two questions: First, do investment newsletters have stock-selection ability? Second, can transactions data be used to improve the precision of PE?

In response to the first question, I find that newsletters do not demonstrate significant abnormal performance: average abnormal returns are close to zero; the best performing newsletter does not seem unusual given the sample size; and the number of extreme performers is not surprising Taken together these results imply that the average subscriber is not getting useful stock-selection advice.

To address the second question, I compare several methods. Most PE refinements involve adding additional benchmarks and forming multifactor extensions to the regression framework of the CAPM. These methods require only periodic return data. When transactions data are available, portfolios can be compared on a day-to-day basis, with each stock matched to an appropriate benchmark. (3) Using a measure of precision defined in the paper, I find that the transactions-based approach yields a median improvement of 10 percent over an analogous multifactor model, with the former approach providing more precise estimates of abnormal performance for over 80 percent of the newsletters. This compares with a median improvement of less than one percent achieved by adding factors to the CAPM. …