The Predictive Power of Personal Statements in Admissions: A Meta-Analysis and Cautionary Tale
Murphy, Sara C., Klieger, David M., Borneman, Matthew J., Kuncel, Nathan R., College and University
Personal statements are a widely used and popular predictor in academic admissions; however, relatively little is known about their effectiveness as a predictor of student performance. This study involved a meta-analysis of the relationships of personal statements to measures of student performance (e.g., GPA) and other predictors. Results suggest that while they have little overlap with other predictors, personal statements also have small predictive relationships with grades and faculty performance ratings. In addition, once standardized test scores and prior grades are taken into account, they provide no incremental validity.
Beyond standardized tests and prior grades, admissions committees often gather non-cognitive data about their applicants. Many of these data are collected on the basis of convenience, habit, history, and what other institutions are doing; rarely does the collection of such data involve discussion of validity or incremental validity. One of the most common (presumed) predictors of student performance is the personal or biographical statement (Anderson and Shackleton 1993; Gibbs 1994).
Programs collect personal statements and essays for a variety of purposes. The purpose generally depends on the school and its goals, although personal statements usually allow the applicant to provide information that cannot be found elsewhere in the application packet. Currendy, litde is known about either the predictive power of the personal statement or its usefulness alongside other predictors. Evidence suggests that personal statements, personal essays, and biographical statements are positively related to student performance; however, the magnitude of this effect varies greatly from study to study.
A sense of the effectiveness of personal statements is blurred by the numerous and sometimes ill-defined role they are supposed to fill. Personal statements are used to measure an applicant's general drive and motivation; to match a student with an advisor; to assess an applicant's goals for the future; and to understand an applicant's past experiences with the goal of measuring skills that will be useful in the future (Powers and Fowles 199e). In addition, it has long been held that personal statements can provide insight into an applicant's interest in and dedication to a particular field (Freun 1980; Willingham 1974). Personal statements also give applicants an opportunity to explain any weaknesses and to highlight strengths not described elsewhere in the application package. Finally, personal statements are used to evaluate applicants' writing abilities.
Some research has sought to determine how effective personal statements are at accomplishing these various purposes. Indeed, some scholars have found that personal statements can be modest predictors of college success (Ra 1989; Smith and Pratt 1966; Willingham 1985). For example, Shahani, Dipboye, and Gehrlein (1991) report that students' admissions essay score correlated 0.21 with their grade point average after the first year of college; this is similar to results found in other studies.
Researchers have studied the use of personal statements as a type of biographical data instrument e.g., Ferguson et al. 2000). B io data measures have exhibited a positive relationship with future performance across domains, including both academic (Oswald et al. 2004) and work settings (Hunter and Hunter 1984). It is reasonable to hypothesize that personal statements that measure the same characteristics could achieve the same results by providing useful information about past experiences and behavioral tendencies that would yield success in school.
However, bio data measures use structured response formats to collect biographical data about the applicant and are generally based on items chosen through empirical or rational keying (Hogan 1994; Hough and Paullin 1994). Personal statements, as currendy used, are largely unstructured measures that are not likely to yield consistent information across applicants. …