In the April issue of Technology & Learning, part one of this article described the first two stages that schools move through as they learn to link data to higher student achievement (www.techlearning.com/story/showArticle.jhtml?articleID=160400818). In stage one, schools make initial efforts to contextualize the many data sources available, but their data analysis is mostly for its own sake. Few of the findings make their way into classroom practice, and the connection of data analysis to student achievement remains tenuous.
As individuals and stakeholder groups develop experience and sophistication in understanding the limitations, context, and implications of various data sources, they move into stage two--using data to improve educational efficiency. In this phase, the school does everything it can to maximize the performance of students who are on the verge of moving to the next level and thereby get as many as possible over the bar. Schools in stage two that are persistent in gathering formative assessment data and adjusting teaching to fill the identified gaps in student performance are able to achieve significant gains in school performance for a few years as they remove the slack in the system and push up the "bubble" students to the next level.
In the long run, however, schools that stay in stage two--dedicated to improving efficiency by focusing on the students at the margin--will run headlong into the ever rising NCLB standards. Data analysis that focuses on improving efficiency works mostly at the edges of the problem, and eventually the school will pull all of the slack out of the system.
Imagine an electronics company that has a strong history in the VCR business. A few years ago, this company noticed that their profits were beginning to slip. To address the problem, they reduced costs by improving the efficiency of VCR production. They modernized their plants, streamlined their efforts, and consequently were able to generate better margins for each VCR sold. For a few quarters, the company's profits rose. Soon, however, the pattern reversed itself and no matter how much effort was made to improve the efficiency of the production process, the company could not meet its profit targets.
The company had responded aggressively to dwindling profits, but its efforts to improve efficiency could only take it so far. The market had changed and the product they produced was no longer in demand, so the company's only hope would be to fundamentally change their work. They would need to develop new products--such as DVD players or DVRs--to meet the needs of the changed market.
Like our fictional electronics company, American schools are operating in a radically changing marketplace. Schools are being required to produce greater numbers of higher performing students. Improving educational efficiency is an absolutely necessary response to these changes, but schools that focus solely on improving efficiency will ultimately stall in their efforts to sustain growth. For schools to reach the ever increasing standards of NCLB, they must do more than push marginal students over the next NCLB threshold. Moreover, to reach a plane of sustained growth and improvement, they must develop transformational ways of working and produce fundamentally different kinds of results.
The move to stage three--analysis for sustained achievement--requires that a school adopt a core belief that all its students can perform to higher standards. While most educators will agree with this concept in principle, the challenge is establishing structures and systems that fundamentally change the practices of the school to support this philosophy. In stage three, schools combine data analysis and organizational reform to create an environment driven by an abiding belief that all students can and will meet the standards. This is a major shift in the way schools operate, and it requires careful collection and analysis of data to sustain its implementation and maintain its progress. Following are a number of strategies used by stage-three schools to deliver targeted instructional efforts for students at all levels, including the advanced and those thought incapable of reaching proficiency.
1. Focus on Individuals
Stage-two schools center their data analysis efforts and instructional interventions on fairly broad groups of students: classes, measurable sub-groups, "bubble" students, or whole tested grade levels. Typically, only those students who are performing at the margin of proficiency (and therefore can contribute most readily to improving the school's overall performance) get significant individual attention. The real, but less than comfortable, corollary of this approach is that stage-two schools give only modest attention to other groups, such as those students who are securely proficient performers and those students who are performing well below the proficiency threshold. Individualized attention is even less available for students in these groups, unless a student is eligible for Special Education services.
Stage-three schools shift the focus from groups to individual students--every single, individual student. At Jeremiah Gray and Rosa Parks elementary schools in Perry Township outside Indianapolis, principals Ann Puckett-Harpold and Gary Robinson have created structures to ensure that all students are seen and treated as individuals and not just members of a larger group. Classroom performance information is collected via an online grade book and combined with data from online formative assessments to create individualized Student Learning Profiles. Teachers, children, and parents meet quarterly to review this information and develop individualized Student Learning Agreements. These agreements--which include specific goals and actions for the student, parents, and teachers--are stored electronically and reviewed against student progress in subsequent meetings. As a result of such targeted efforts, these two schools have risen to having more than 90 percent of their students achieving above the state standard in reading.
2. Personalized Learning
Stage-three schools understand that traditional school structures related to time and class organization limit their ability to address the individual needs of the students. While students are still organized into classes, schools committed to sustained improvement combine regular data collection and analysis with flexible grouping structures to personalize learning opportunities for students. At Thomas A. Edison Charter School in Wilmington, Delaware, students have a dedicated 90-minute reading period every morning. Students get additional individual attention because reading groups are smaller than normal classes and all teachers and many administrators are assigned to them. The groups are multi-age and levelled according to the learning needs of the students. Most importantly, the groups are flexible, and membership is adjusted based on the results of frequent assessments. The school uses an electronic version of the Scholastic Reading Inventory to measure reading performance at least every eight weeks. The results are analyzed by teachers, and students are regrouped to maximize the personalization of their learning experience.
3. Look at "Added Value"
NCLB's targets are based on a percentage of students in a particular cohort reaching a defined level of proficiency. Through 2014, incremental improvement is required in this percentage. Interestingly, these requirements do not require individual students to demonstrate improved performance over time. As long as a student scores above the proficiency threshold, her performance counts toward AYP. In fact, a student could score significantly lower in the 8th grade compared to the 5th grade, but not hurt her school's overall performance as long as she stayed above the proficiency level. Conversely, a student could make significant gains in performance from 3rd grade to 5th grade, but still not contribute to her school's AYP goal because her 5th-grade performance was still slightly proficient. This situation leads many stage-two schools to maintain their focus mostly on the students near the proficiency threshold and give little individual attention to those well above or below the cut-off point.
Stage-three schools look beyond the requirements of NCLB and track individual student performance over time with a goal of getting improvement from all students over the full time they are in the school. Some American schools have taken an important lesson from the United Kingdom, where the Department for Education and Skills includes "added value" data when reporting high-stakes assessments scores. Added value is reported in terms of the average gain in performance for same students from one high-stakes assessment to the next. This gain is then indexed against expected performance improvement targets. Schools are judged on same student improvement rather than increasing the number of students over a particular threshold. Consequently, the steady, continuous improvement of all students is required for the school to meet its performance targets.
4. Systems of Accountability
In stage-three schools, school leaders establish structures of accountability that ensure teachers regularly analyze student performance data, talk about it in functional units, and enact specific action plans at the classroom, team, grade, and school level. Sharif El-Mekki, principal of Anna T. Shaw Middle School in Philadelphia, conducts monthly ABCs or Administrative Benchmarks Conferences with his teachers. At these conferences, each teacher presents an analysis of the month's formative assessment data, a summary of their findings, and an action plan for addressing the student learning needs identified. By asking guiding questions about general trends and specific students, El-Mekki coaches his teachers through the difficult task of customizing teaching to meet the learning needs of all students. Following the ABCs, assistant principals and team leaders meet with groups of teachers to tackle thorny problems, share best practices, and collaborate to address general findings. The result is a school where every teacher is highly conversant about specific performance details of their students.
Moving a school from stage two to stage three requires transformational change. Stage-three schools must not only adopt new core beliefs, they must be willing to radically restructure time and teams based on frequent data collection and analysis. This endeavor creates extraordinary organizational challenges. How can teachers hope to provide customized instruction to dozens or even hundreds of students? Where do you find the time to do additional assessments, analyze the data, develop action plans, and implement changes? While there are no instant solutions to these challenges and getting to stage three requires sustained hard work, the following suggestions can help to build the capacity to support this radical change.
1. All Hands on Deck
Stage-three schools use all available resources to personalize learning. All qualified employees run reading groups. Students provide peer tutoring in structured sessions. Creative arts and physical education teachers are trained in data analysis and support the school's goals through integrated learning units. Pastoral advisors monitor and report on Student Learning Agreements. In short, no resource is wasted and all are focused on the same objective: helping all students achieve higher standards.
2. Integrated Technology Systems
Select student information systems, formative assessment systems, and student performance tracking tools that easily integrate. Teachers lose focus and efficiency when they have to learn and juggle several data systems simultaneously.
3. Students Take Ownership
The ultimate key in sustained improvement is getting the students to share in the responsibility for their own learning. When students have the skills to analyze their own formative assessment data, for example, they have the tools to take ownership of their own learning. This means training students to analyze data on their own performance. Specifically, they need to learn how to look beyond top-level scores to strand and skill-level details of their performance. Then, they must be given opportunities to practice setting goals and creating action plans to guide their own learning. When this is accomplished, the capacity of the organization to grow is nearly unlimited, and stage three of data utilization is fully embedded.
Indiana Monthly Edison Benchmark Review
Strand/Skill Analysis--Patterns and Anomalies
1) Which strands were overall weaknesses for your class? (Strands where the majority of the questions are 600% and lower)
2) Of these strands, which ones did you focus on during the previous month of instruction? (Look back at lesson plans)
3) Of these strands, which ones will you focus on during the next month of instruction? (Look forward at scope and sequence)
4) Were there particular skills within weak strands in which students did perform well? What were they and what about those questions allowed students to do well?
5) Which strands were overall strengths for your class? (Strands where the majority of the questions are 65% or higher)
6) Of these strands, which ones did you focus on during the previous month of instruction? (Look back at lesson plans)
7) Of these strands, which ones will you focus on during the next month of instruction? (Look forward at your scope and sequence)
8) Were there particular skills within strong strands in which students did not perform well? What were they and what about those questions caused students to do poorly?
Planning for Instruction
1) Looking forward to your next month of instruction, which skills will you emphasize with your students based on your data analysis and scope and sequence?
2) What particular vocabulary words or phrasing of questions will you now use in your teaching to ensure student success the next time they encounter those words/ questions on the Benchmarks?
Asking the right questions about data gleaned from high stakes tests is central to spotting the performance trends, strengths, and weaknesses that will help teachers map a customized instructional course.
Monthly Benchmark Review
Question Review Q# Skill Tested Score Answer Breakdown A B C D 18 Context Clues 67 25 66 3 3
25% of the students picked "another country" (answer A) as a definition for "foreign". Although this is a correct meaning, it's not the correct meaning in this context.
Next Steps: Students
Students overall scored very high. Class average was 77.
Zachary, Crystal, Kelsey, Peter and James all scored 900%. They will need more challenging material. James and Emanuel scored 50 and 45 respectively. I don't believe this reflects their ability so I would have to question their attention to the test. Prior to the next test, we will have a conversation about how important it is for them to take the benchmarks seriously.
Next Steps: Curriculum
Review context clues and the need for students to go back and find the word in context before choosing an answer, even if they feel they know what the word means.
Stage-three schools' accountability systems include regular teacher analysis of performance data and specific action plans at both the individual and classroom levels.
Streamline Your Data Reporting By Steven Guttentag
Here are four tips for setting up a data collection and reporting system.
Integration If your assessment engine doesn't talk to your grade book, the grade book doesn't talk to your report card, and your report card doesn't talk to your transcript, the system won't work. Look for an enterprise solution that allows for the seamless sharing of data among systems for IT, users, administration, reporting, and accountability, as well as time, communications, and content/lesson management.
Local Control and Flexibility Districts need to report data across all their schools and generate data for state and federal reports. However, individual schools may need certain information reported differently or other types of data to support local initiatives. Be sure your system allows for customization of both reporting and collection of data elements at all levels.
Decision Support Data by itself is often of little value to teachers and principals in terms of making daily instructional and administrative decisions. Synthesize data in such a way that it can directly support their decisions at a quick read. For example, our teacher homepage shows each student along with three key metrics: participation, attendance, and performance.
It Takes a School Community Coordinating meetings can be a real challenge as there are often up to 15 adults who come in contact with a student during a given week. Consider using a system that combines online reporting, a student contact log, and "tasking" to be sure all stakeholders are involved and that contact with students is recorded, tracked, and analyzed.
Steven Guttentag (firstname.lastname@example.org) is chief education officer of Connections Academy.
Todd McIntire is a vice president at Edison Schools, Inc. He is currently involved in developing Edison's new division in the United Kingdom.
AYP Report--Schoolwide Longitudinal, Rosa Parks-Edison, 2004-2005 Reading Oct Nov Dec Jan Feb Mar 108 * 107 * 105 * 105 * 105 * 105 * Goal 74.6% Last Year 69% Thresholds 54% 40% Note: Table made from bar graph. Stage-three schools that have built-in systems for examining data from a variety of sources and perspectives on a regular basis are seeing tremendous growth in student achievement.…