Towards Building Ranking Models with Annual Reports

Article excerpt

1. Introduction

Corporate disclosures play an important role in supporti ng market efficiency and integrity. There are in general three types of disclosures: mandatory (or regulated) disclosures such as annual and quarterly reports, voluntary disclosures by management such as earnings press releases, and reports from information intermediaries such as financial news stories. Annual reports, a major mandatory disclosure, are regulated by Securities and Exchange Commission (SEC) to minimize the information asymmetry between investors and management. These reports are publicly available and contain quantitative data on firms' current financial performance, as well as qualitative analysis, narrative discussion on changes of strategies and operations, and forward-looking information. Investors and analysts depend on these financial and non-financial disclosures to assess firm values and make investment decisions such as choosing a portfolio of securities.

In accounting and finance domains, researchers have studied how the quality of mandatory disclosures is related to the forecast of company performance. For example, Barron et al.[3] studied the relationship between the SEC's ratings of Management Discussion & Analysis section of annual reports and analysts' earnings forecasts. They found that higher report ratings and better disclosure quality were associated with more accurate earnings forecasts. Gelb and Zarowin[12] empirically confirmed that high disclosure firms provided greater stock price informativeness to the investors. These studies verified how the disclosure level and quality affect company performance forecast and stock market efficiency. However, these studies relied on the subjective ratings from analysts and SEC which are no longer available and a smaller sample size due to the labor-intensive document analysis process.

Another focus in the study of the textual disclosure is on the narrative features and their roles in prediction. Li[20] found that the annual reports of firms with lower earnings were harder to read, while firms with more persistent positive earnings provided easier to read annual reports. Davis et al.[8] showed that the positive or negative tone in earnings press releases is associated with firm's future performance, and captured in market returns. Most of the studies on the textual features tend to focus on a few linguistic features or writing styles that are specified ex ante such as the risk sentiment[19, 9], the tone[10], or readability[20, 30].

Core[6] suggested that the study of disclosure could substantially benefit from the techniques in natural language processing, computer science and artificial intelligence. The machine learning and text mining techniques, though widely applied in information retrieval, biomedicine, and web domains, has rarely been used in the study of textual disclosures, whether mandatory, voluntary, or through intermediary. Some exceptions include the following. Henry[15] applied the data mining algorithm (classification and regression tree) to study the market reaction to firms' earnings announcement. She found that the verbal features and writing style of earnings press releases can improve the prediction of abnormal market returns. The predictive models, though shed light on the predictive potential of textual feature, are not implementable in practice. Magnusson et. al.[22] studied the changes in the textual content as well as the quantitative data in quarterly reports with clustering methods. They found that the changes in textual content seemed to proceed the changes in financial performance.

With the above review, we see that applying data mining and text processing technologies has advantages in processing larger sample set, learning models from data, analyzing higher-dimensional feature space rather than few pre-specified parameters. Our research goal in general is to evaluate the effectiveness of applying machine learning and text mining techniques to building predictive models with annual reports. …