Continuous Improvement: It Takes More Than Test Scores: Analyzing State Assessment Results Is Only the Beginning of Effective Data-Driven Decision Making
Bernhardt, Victoria L., Leadership
There is no question that the passage of No Child Left Behind in 2001 has impacted schools in at least two ways: First and foremost, NCLB has made the use of data to improve student achievement imperative; and second, NCLB has increased the need for continuous improvement processes within schools.
Summative data just the beginning
Schools in our country hear that data makes the difference in improving student achievement. Not all schools, however, have felt the positive impact from what they believe is data-driven decision making. The most common reason: Most school districts in this country believe they are being data-driven when they have analyzed the dickens out of their state assessment results.
Some school districts feel they are being data-driven when they analyze the dickens out of their state assessment results and use some formative assessments to help students prepare for the statewide test. Unfortunately, this is only the beginning of data-driven decision making.
Most states' assessment scores can speak volumes for what is going on in their schools and districts, and with student learning. Following assessment scores of the same groups of students (or the same students) over time can indicate the presence or absence of a continuum of learning that makes sense for the students. It can give information about student learning as well as the degree to which standards are being implemented at each grade level (provided the state assessment truly measures the state standards).
Looking at state assessment results by grade level over time can give information about the impact of the programs being implemented. Disaggregating state assessment results can tell us if our schools are meeting the needs of the students the schools are attempting to serve.
Test score analyses are important. In fact, in a perfect world, schools would use both formative and summative assessments to ensure that all students are learning. If only summative assessment data are studied, however, solutions for improving the scores can come out half-baked.
For example, when I started working with Lemon Middle School, the staff had determined that their students' scores in English/language arts and mathematics were lower than the previous years' scores. Since the math scores were the lowest, they decided to "focus" on math that year.
In their focus, they set up several strategies: remediation for the students not meeting proficiency standards, an afterschool program to assist students with their math homework, and a required math summer school program for any student not passing the state math assessment at the proficient or advanced levels.
Unfortunately, their gallant efforts did not lead to the test score improvements they had hoped for. Both math and English/language arts scores went down. They were devastated. As the staff and I reviewed their data and their solutions, we talked about establishing a continuous improvement plan. We set out to gather a bit more data to see if we could figure out how to work smarter, not harder, and get better results.
We had the student achievement data. We determined that other types of data, including demographic, perceptional and school process data, needed to be gathered and analyzed. What those data are and what we found in the data analysis at Lemon Middle School are discussed below.
Demographics can tell schools all about who they have as students, who they have as teachers, and how teachers are aligned to the students. Demographics begin to tell us about school processes and how the school is preparing to meet the needs of students. Demographics are important for setting the context of the school, and they are critical for understanding all other numbers.
Lemon Middle School's demographics showed that while they had a fairly diverse student population--60 percent Caucasian, 30 percent Hispanic/Latino and 10 percent African American--their teaching staff was 100 percent Caucasian and 80 percent female. …