Academic journal article Phi Delta Kappan

Value-Added Models: What the Experts Say: Researchers Share the Opinions of Scholars Who Have Analyzed and Written Extensively about Value-Added Models

Academic journal article Phi Delta Kappan

Value-Added Models: What the Experts Say: Researchers Share the Opinions of Scholars Who Have Analyzed and Written Extensively about Value-Added Models

Article excerpt

When a collection of experts, people with special knowledge and skills pertaining to a problem, are all skeptical about a proposed solution to that problem, it's a good idea to hear what they say--and, perhaps, find a better solution.

The problem in question? Value-added models (VAMs). For the past decade, expert statisticians and econometricians have been exploring alternative methods to this approach to documenting teacher performance. Its use is now widespread. Before the passage of the Every Student Succeeds Act (ESSA) in 2015, 44 states and the District of Columbia had implemented high-stakes policies to evaluate teacher effectiveness based on VAMs. Although ESSA has since curbed the extent to which states are adopting and implementing VAMs (as in Alabama, Georgia, and Louisiana), in other states (such as New Mexico, New York, and Texas), VAMs are still playing substantial roles in their teacher-level accountability efforts.

Practice versus theory

Value-added models are designed to measure how much value a teacher purportedly adds to (or detracts from) students' growth as evidenced on large-scale standardized achievement tests over each school year. The models statistically control for students' prior testing histories, with some also controlling for student-level variables (for example, demographics, English language status, or special education status) and school-level variables (such as class size or school demographics). In theory, measuring teachers' value-added allows for richer analyses of standardized test score data because groups of students are followed to assess their learning trajectories from the time they enter a teacher's classroom to the time they leave. That measured growth, so it's argued, can be used to quantify and determine a teacher's purported effect on student growth in achievement over time.

Five sticky issues

In practice, however, whether these models are working as intended is under debate. Five issues are at the core of the disputes surrounding VAMs:

* Reliability. Teachers classified as effective one year might be classified as ineffective the next, or vice versa, and often to the extremes. Given the stability of teachers' levels of effectiveness otherwise, these swings should not occur.

* Validity. There's no evidence that indicators of teacher value-added are adequately correlated with at least one other concurrent measure of teacher effectiveness, such as supervisors' observational assessment of teachers or students' survey-based assessment.

* Bias. Evidence suggests that teacher value-added estimates systematically differ, given the varying demographic characteristics of students nonrandomly assigned to their classrooms.

This occurs despite the statistical controls put in place to block bias.

* Transparency. Teachers and administrators don't seem to understand the models being used to evaluate them, which simultaneously thwarts the extent to which they might use their value-added estimates to improve instruction or initiate reforms.

* Fairness. Current research suggests that predominantly math teachers and teachers of reading and language arts are being held accountable using these systems, leaving about 70% of all public school teachers value-added ineligible. The ineligible teachers typically teach children in early childhood and high school grades and in noncore subject areas such as social studies, science, art, music, and physical education.

VAMS and high-stakes decision making

Consequential use of VAMs to make high-stakes decisions--such as promotions, tenure, merit pay, or termination--is also a major area of concern. For example, although research suggests that, ideally, three years of teacher-level data are needed to make the most accurate value-added estimates, some states' tenure and due process laws have provisions that allow districts to terminate or untenure teachers using only one or two consecutively unsatisfactory value-added scores (for example, Delaware, Florida, Indiana, and Pennsylvania). …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.