Measuring Outcomes in K-12 Online Education Programs: The Need for Common Metrics
Pape, Liz, Revenaugh, Mickey, Watson, John, Wicks, Matthew, Distance Learning
Online learning at the K-12 level is growing rapidly, as educators, parents, and students discover the benefits of learning unconstrained by time and place. States and school districts are offering full-time and supplemental programs to students across all grade levels in order to provide a greater number of courses to students in rural and urban schools; to meet the "highly qualified teacher" requirements under the No Child Left Behind Act; and more generally to strive toward public education goals of equity, access, and a high-quality education for all students.
This rapid growth, however, has challenged policymakers responsible for over seeing public education. One of the challenges that policymakers face is the lack of common measures of outcomes and quality in online programs. Although most programs track student outcomes and other measures of quality, these measures are not consistent across programs, and a metric used by two programs with the same name (e.g., course completion rate) may not in fact measure the same thing. This lack of consistency makes measuring outcomes across programs difficult and hinders development of appropriate policies.
This article begins with a brief overview of the scope of K-12 online education in the United States. It then explores outcomes measures in general before discussing how three programs-Illinois Virtual High School, Virtual High School, and Connections Academy-track student outcomes. It uses these examples to discuss the need for consistency in outcome measures, and calls for common, agreed-upon metrics to be developed and used voluntarily by online programs in order to facilitate development of suitable policies for guiding the sustainable growth of online education.
THE SCOPE OF K-12 ONLINE LEARNING
K-12 online learning in the United States is rapidly growing as new virtual programs are continually emerging. These include both online supplemental programs that provide virtual courses to students who are otherwise enrolled in brick-and-mortar schools, and full-time virtual programs sometimes known as "cyberschools." The distinction between the two is beginning to blur as students opt for full course loads from supplemental programs, and cyberschools offer part-time enrollment. Adding further complexity is the American tradition of local control, which in an age of readily available courses and course-building tools essentially means that any local school district can launch a virtual program regardless of official state policy.
Bearing those caveats in mind, the current tally of states with significant virtual learning activity is:
* Ten states with both statewide supplemental and full-time cyberschools, including Missouri, Georgia, and Mississippi, which passed legislation in 2006 to allow both.
* An additional 17 states with statewide supplementary online programs, including North Carolina, which expects to launch in fall 2006.
* An additional 12 states with full-time cyberschools, including Indiana, which recently amended its charter law to allow virtual charters.
In the 2005-2006 school year, an estimated 600,000 students took supplementary courses (Smith, Clark, & Blomeyer, 2005) while 65,000 enrolled in cyberschools (Rotherham, 2006). While still less than 1% of the total U.S. student population, this number is growing rapidly, having doubled since 2003.
WHAT AND HOW TO MEASURE
What outcomes a program should measure is a complex issue that should be strongly influenced by the mission of the program. A program created to make advanced placement courses more accessible to students will have different metrics than a program created to increase high school graduation rates. The mission determines measures of success, and program measurement ultimately needs to indicate if a program accomplishes its goals.
Another factor to keep in mind is how program evaluation data will be disseminated. …