Data Analysis Demystified: Standardized Tests Don't Do What Matters Most -- Meeting the Immediate Needs of Individual Students. That's Why Schools Must Design, Deliver and Analyze Additional Sources of Meaningful Student Data
Ackley, Dave, Leadership
When we design curriculum, choose instructional strategies and prepare for program implementation, we should also plan for data analysis. There are two essential perspectives for data analysis: summative analysis and formative analysis. Summative analysis is collecting and presenting information that is necessary to make final statements and judgments about the value of an activity, usually at the end of the activity's implementation. Formative analysis is the continuous monitoring of short-term results and procedures to provide ongoing information to improve student achievement.
In other words, summative analysis is using student achievement results to determine how well something worked after an activity is completed. Formative analysis is using today's student achievement results to help us determine what we will do tomorrow.
With state-mandated tests, our vision is often skewed because we only see the summative aspect of assessment and data analysis. To counteract this situation, teachers, schools and school districts must actively collect and analyze other formative student achievement data to help them improve instruction and student learning.
Comprehensive data analysis efforts should certainly include scores from all the state testing programs; however due to the nature of the tests, state test scores do not reveal enough detail about individual students' mastery of the specific content of the assessment.
Calkins, Montgomery & Santman (1998) stated that "standardized tests don't measure what matters most." State test scores do not reveal how the test content relates to student mastery of specific state standards. Therefore, their use as formative tools is limited.
The nature of the state tests may actually impede our ability to make formative decisions about instruction. State tests are not administered frequently enough to help school sites drive program implementation decisions or determine program effectiveness. State test scores do not assess enough of the standards to allow districts to conduct a comprehensive summative analysis of their local curriculum. Therefore, local school districts should work with teachers and site administrators to design, deliver and analyze additional sources of meaningful student performance data that are aligned to state standards.
According to Douglas Reeves (1998), "Standards without standards-based assessments are merely a very expensive and time consuming pep talk."
It is a mistake to think of data simply in terms of test scores. According to Webster's Unabridged Dictionary, data are premises upon which things can be argued or inferred. In education, these premises are meaningful bits of information upon which educators can base their decisions about the effectiveness of instructional strategies, the relevance of the curriculum content, and the subsequent impact on student achievement. While this may sound simple, producing bits of meaningful information regarding student performance is a major challenge for most school districts.
The first step in using data to improve student learning is to help school staff members become proficient at assessment and data analysis. Michael Fullan (1991) said, "The crux of the matter is getting the right people together with the right information." Even if districts successfully produce meaningful formative student achievement data, placing the data into coherent formats that allow for simple analysis is still a major challenge.
Once local assessments are developed, teachers need time and training. Districts must train their teachers to properly analyze multiple local assessment results and apply their findings to guide their instruction. Since constructive data analysis incorporates more than merely looking at test scores, teachers need training on different methods to assess their students' progress toward mastering standards. …