A New Direction for CTE Accountability and Evaluation
Kotamraju, Pradeep, Techniques
MANY OF THOSE WORKING IN THE FIELD OF CAREER AND TECHNICAL EDUCATION (CTE) have continuously grappled with the need for a uniformly global set of in formation--a national-level database or, at minimum, a common, standardized set of definitions and measures to meet CTE's multiple needs, including accountability and evaluation, career guidance and program improvement. This article primarily focuses on data and CTE accountability and why these matter in the current policy context. It also describes the work currently being undertaken by the National Research Center for Career and Technical Education (NRCCTE) and others toward establishing common data standards and a new direction for CTE accountability and evaluation that anticipates changes in federal policy.
A group consisting of researchers, data experts and policymakers met in January of" 2010 at the University of Louisville to continue work on the NRC CTE's Perkins Crosswalk Validation Project--an evolving multi-state, multi-institutional collaborative effort seeking (1) greater consistency and clarity in Perkins secondary and postsecondary data collection and reporting, (2) a common data crosswalk that links occupations to educational programs, career clusters, and career pathways, and (3) a foundation for more standardized accountability requirements in later iterations of the Perkins legislation. Participants also discussed the pressing need to use CTE data and accountability systems to shape current and future education and workforce development policy, as I will describe here.
Why Don't We Have a National CTE Database?
At present, the United Stales has no national, comprehensive database that meets the accountability requirements prescribed in the Carl D. Perkins Career and Technical Education Act of 2006 (otherwise known as Perkins IV). Individual state-level databases that collect CTE accountability information do exist, but these generally have been built to serve state-specific purposes, needs and requirements. Most have no connection either to other databases within the state--most states keep their secondary and postsecondary CTE data separate--or to other states, let alone to a national system. As I will note below, many have sought to achieve the ideal of a uniform, global database for CTE accountability and evaluation, and much effort at both the national and state levels has gone into standardizing variable definitions and measurement approaches with the goal of developing a common set of CTE indicators.
Many factors make both a national database and a common framework difficult goals to achieve. At present, CTE data are obtained from three disparate and generally unconnected sources: (1) the aforementioned state-based CTE data systems. (2) national data systems such as the Education Data Exchange Network (EDEN) or the Integrated Postsecondary Education Database System, and (3) the National Center for Education Statistics Sample Survey Data. Each system requires considerable technical expertise to operate; each is also limited in the extent to which it could serve as a common CTE accountability and evaluation system. As an example, the United States currently has at least 51 different state-based CTE data systems--in reality, it has at least 102, given states' separation of their secondary and postsecondary CTE data.
In these systems, definitions of and measurement approaches for arriving at CTE indicators are all different; such conditions are not optimal for a common accountability framework. If these disparate data could be combined in a single database, they would need to be housed in one location. State data privacy and Family Educational Rights and Privacy Act (FERPA) regulations make such a central database problematic for most state and local education agencies. For these reasons, large, longitudinal national and state educational databases have not been without controversy; nevertheless, the U.S. Department of Education recently funded the Statewide Longitudinal Data Systems (SLDS) in an effort to help states better manage and use student educational data for accountability and evaluation purposes. How much involvement the CTE community will have in the development of SLDS is an open question.
To What Policy Sphere Does CTE Belong?
Confusion has always existed regarding where CTE fits in and to which policy sphere it belongs: education, workforce development, or both. At the secondary level, CTE is usually discussed within the context of college and career readiness. In the past, both vocational education (the old term for CTE) and workforce development were seen as the options available to non-college going secondary students. As secondary-level CTE continues to evolve under Perkins IV toward greater academic rigor and an emphasis on progressing to postsecondary education, some see secondary CTE as being more about education and less about workforce development. The prevailing view is that as long as CTE students take an appropriate mix of academic and technical courses in high school, they will graduate both college- and career-ready.
However, many question the notion that college and career readiness are one and the same, particularly as CTE course-taking in high school is decreasing and academic course-taking is increasing. Moreover, most students' educational pathways no longer represent a direct progression from high school to college. Whether because of need, inclination or ability, most young people do a stint in the workforce before moving on to some form of postsecondary education; most see themselves as college-ready, only after they have experienced the workplace conditions for becoming career-ready. How to track students' non-linear movement from secondary education, into and out of the workforce, and into and through postsecondary education will affect how future accountability systems in CTE are structured.
Defining and Measuring CTE Student Success Current Perkins IV Beyond Perkins IV Definition Engagement Completing high school Attending, focusing and and/or postsecondary specializing in coursework programs and work-based learning within programmatic career pathways or programs of study Achievement Both technical and Academic performance; academic achievement completing (graduating) outcomes; the acquisition school or college; skill of industry credentials development, including technical competencies Transition From high school to Accessing and acquiring a continued formal learning postsecondary education; at the postsecondary level managing the swirl that without the need for takes place between remediation; movement from education and the education to the workplace workplace
At the postsecondary level, gaps and disconnects exist between CTE and workforce development accountability systems that are not being addressed either within Perkins (as managed by the U.S. Department of Education) or the Workforce Investment Act (WIA; as managed by the U.S. Department of Labor). Growing numbers of new and currently employed workers are using Adult Basic Education programs (ABE; also managed by the U.S. Department of Education but considered distinct from CTE) to address employ ability deficiencies before entering college occupational programs. Connections between these ABE programs and Perkins are currently hampered by Perkins regulations explicitly barring expenditures on prevocational, ABE, or remedial education programs.
Despite the growing importance of CTE to other federal programs like ABE and WIA, and vice versa, CTE, ABE, and WIA continue to employ separate accountability systems. This disconnect underscores the need to clearly align not only federal education and workforce development policies, but also CTE, WTA, and ABE data systems. Any national-level CTE postsecondary accountability system will need to include data elements from the WIA and ABE accountability systems in order to provide a more robust picture of CTE student success, particularly when many more students are coming to post-secondary CTE from ABE and WTA than directly from high schools.
Broadening the Definition of CTE Student Success
Within this policy context, how to define CTE student success--at any level--remains a key, unresolved issue. As shown in the above table, the NRCCTE has conceptualized CTE student success as occurring across the three domains: engagement, achievement and transition. How these are measured by states and local education agencies under Perkins IV and how they may be measured in future legislation are also shown.
In measuring CTE student success, accountability systems need to take into consideration the different "stop-out" points CTE students may encounter in their movement from secondary education to the workforce and postsecondary education. Postsecondary systems generally use a cohort (entering or exiting) of students and extract information on performance indicators such as retention (engagement), graduation (engagement, achievement), and transfer (engagement, achievement, transition). Adding these values together determines the rate of student success. To this, one might add CTE students' employment placements (transition) in order to extend the CTE student success measure beyond mere educational attainment.
Toward Forward-looking CTE Accountability Systems
In order for it to be seen as a vehicle for secondary and postsecondary student success, CTE should build on previous efforts to provide greater consistency and clarity on current CTE secondary and postsecondary data collection and reporting requirements under Perkins IV. Such efforts include the extensive work of multiple national Data Quality Institutes (DQIs) organized by the U.S. Department of Education, Office of Vocational and Adult Education (OVAE). The DQI's efforts to define terms and establish crosswalks between systems could potentially lead to a common accountability and evaluation framework.
The Next Steps Working Group, an OVAE-sponsored monthly electronic town hall meeting on Perkins and CTE accountability, is studying and comparing the alignment of secondary CTE measures to those available within EDEN. This group is beginning to create appropriate linkages and connections within and between CTE accountability data systems. The Postsecondary Common Data Dictionary project, jointly conducted by researchers at MPR, and the NRC-CTE. also sought to address the extent to which the development, formulation and usage of a common data dictionary will result in more consistent, valid and reliable state and national postsecondary accountability measures.
Perkins IV envisioned CTE as transcending secondary and postsecondary systems through the implementation of programs of study, which combine career-focused CTE content with rigorous academics in a seamless pathway to post-secondary education, industry-recognized credentials and employment. If programs of study become the new model of CTE--or education more generally--the field will need to better prepare itself for new and modified accountability requirements.
To do so, it must begin by eliminating disconnections between policymakers, secondary and postsecondary administrators, and technical experts; building a common knowledge base about accountability system specifications; and gaining administrative and management support. By matching future legislative requirements and regulations to the opportunities and constraints posed within current and yet-to-be built accountability systems, CTE can begin advocating for a forward-looking accountability system that is able to capitalize on CTE as a means of moving the United States out of its current economic crisis.
Pradeep Kotamraju, Ph.D.,
is deputy director of the Notional Research Center for Career and Technical Education. He can be contacted of pradeep. email@example.com. For mote information on this topic and a podcast featuring Kotamraju end members of the crosswalk validation project team, visit nrccte.org.…
Questia, a part of Gale, Cengage Learning. www.questia.com
Publication information: Article title: A New Direction for CTE Accountability and Evaluation. Contributors: Kotamraju, Pradeep - Author. Magazine title: Techniques. Volume: 85. Issue: 3 Publication date: March 2010. Page number: 50+. © 2007 Association for Career and Technical Education. COPYRIGHT 2010 Gale Group.