Academic journal article Education Next

Testing College Readiness: Massachusetts Compares the Validity of Two Standardized Tests

Academic journal article Education Next

Testing College Readiness: Massachusetts Compares the Validity of Two Standardized Tests

Article excerpt

THE STATE OF MASSACHUSETTS introduced a system of standardized testing in its public schools three years before the federal No Child Left Behind Act of 2001 mandated such practices for all 50 states. Although the tests have evolved over time, the Massachusetts Comprehensive Assessment System (MCAS) has been in place ever since. But after Massachusetts adopted the Common Core State Standards in 2010, its education leaders faced a decision: whether to stick with MCAS, which it had already revised to align with the Common Core, or switch to a "next-generation" test that was specifically designed for the Common Core--and to assess students' readiness for college. More than 40 other states have signed on to Common Core, and many face similar decisions about their student assessment systems.

As a member of the multistate Partnership for Assessment of Readiness for College and Careers (PARCC) consortium, Massachusetts had a ready alternative in the new PARCC assessments. As of 2010, 45 states had joined either PARCC or the Smarter Balanced Assessment Consortium that was likewise developing new assessments seeking to better gauge students' higher-level thinking skills, but the number of states participating in both consortia has since fallen.

The stated goal of the PARCC exam is to measure whether students are on track to succeed in college, while the MCAS test aims to measure students' proficiency relative to statewide curriculum standards. But whether the PARCC test actually does a better job of measuring college preparedness was an open question prior to the fall of 2015. The Massachusetts Executive Office of Education commissioned this study in hopes of uncovering timely, rigorous evidence on how accurately the two tests assess college readiness.

This is the first study of its kind. Prior to its authorization, there was no reliable evidence that could demonstrate whether the new Common Core-aligned assessments (PARCC or Smarter Balanced) provide accurate information about which students are prepared for success in college.

Ultimately, we found that the PARCC and MCAS 10th-grade exams do equally well at predicting students' college success, as measured by first-year grades and by the probability that a student needs remediation after entering college. Scores on both tests, in both math and English language arts (ELA), are positively correlated with students' college outcomes, and the differences between the predictive validity of PARCC and MCAS scores are modest. However, we found one important difference between the two exams: PARCC's cutoff scores for college-and career-readiness in math are set at a higher level than the MCAS proficiency cutoff and are better aligned with what it takes to earn "B" grades in college math. That is, while more students fail to meet the PARCC cutoff, those who do meet PARCC's college-readiness standard have better college grades than students who meet the MCAS proficiency standard.

These results likely played a role in the November 2015 decision of the Massachusetts Board of Elementary and Secondary Education to adopt neither MCAS nor PARCC, but rather to develop a hybrid assessment that will aim to draw on the best of both tests. Our analysis cannot speak to the wisdom of that choice, which will become clear only with time. Nor should one assume that our study's results are applicable to other states facing similar decisions: Massachusetts has been a national leader in establishing high-quality learning standards for its students, and MCAS is widely regarded as one of the country's more sophisticated assessment systems. We do not have evidence on whether PARCC outperforms the assessment systems used in other states.

By examining rigorous evidence about the validity of both of these tests, however, Massachusetts provides a model for other states facing difficult choices about whether and how to upgrade their assessment systems.

An Experimental Test

Whether the PARCC test succeeds in measuring college preparedness better than the MCAS is an empirical question; answering it requires a rigorous, independent analysis of which test better predicts college outcomes. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.