Comparing Computer-Adaptive and Curriculum-Based Measurement Methods of Assessment
Shapiro, Edward S., Gebhardt, Sarah N., School Psychology Review
Response to intervention (RTI) is a multitiered system of supports designed to provide high-quality instruction to all students along with early identification and intervention of students likely to develop academic and behavioral difficulties (Batsche et al., 2005). At the center of the RTI process is the use of universal screening of students' academic performance in order to identify students who, despite a strong core instructional program, are still showing potential difficulties in academic performance (Johnson, Jenkins, & Petscher, 2010). Currently, there is relatively little research on universal screening in mathematics.
Among the available methods, curriculum-based measurement (CBM) has been found to meet the criteria needed for effective universal screening (Fuchs & Deno, 1991). Two types of CBM measures are used for math, one that examines the acquisition of computational skills and the other that assesses mathematical concepts and applications across grade level curriculum objectives. The score obtained on these measures embeds within it a student's performance on skills related to grade level computational or concepts/application development.
Research examining CBM math found computational and concepts/applications CBM have sufficient psychometric properties as screening measures of mathematical constructs (e.g., Christ, Scullin, Tolbize, & Jiban, 2008), and a growing body of research supports the use of CBM math as school-wide, universal screening measures to guide mathematics instruction for elementary students (e.g., VanDerHeyden & Burns, 2005). Research has further found that both computation and concepts/applications CBM have acceptable levels of diagnostic accuracy in predicting student performance on state tests of mathematics performance, although this evidence is less robust than that of reading CBM to state tests of reading performance (Keller-Margulis, Shapiro, & Hintze, 2008).
Computer-adaptive testing (CAT) has emerged as a viable option for universal screening. CAT is a measure designed to adapt to a student's ability level. The test refines the selection of items on the basis of a student's response and provides a mechanism for identifying the particular abilities and potential problem areas within the domain of assessment. This particular aspect of the CAT measure appeals greatly to school practitioners, as the measure provides more diagnostic links to instruction than CBM (Shapiro, 2012). CAT measurement systems are based on the accuracy of student responding and examine the key skills that lead to effective mastery of competencies within an academic domain. The basic concepts on which CAT measures are built are the recognition that there is a progression of skills underlying the academic domain being assessed. Using item response theory to build the measure, a student's knowledge of specific skills demonstrating proficiency in the domain can be ascertained. The overall scores obtained on CAT reflect the overall performance of student's skills in the academic domain of evaluation. CAT measurement systems are taken by students on computers and are capable of assessing a broad range of student skills in periods as short as 10-15 min.
Currently, there has been no published research on the use of CAT as a universal screening measure of mathematics. STARMath (Renaissance Learning, 2011) is a CAT that has been used in several studies that examined Accelerated Math, which is a technology-enhanced process for systematic and targeted practice of mathematics skills (Renaissance Learning, 1998). Although STAR-Math did reflect gains in student outcomes as an effect of implementation of the instructional program similar to other measures such as the Terra-Nova (Ysseldyke & Bolt, 2007), the purpose of those studies was not primarily comparison across measures. In the only published study directly evaluating the reliability and concurrent validity of CAT measures, McBride, Ysseldyke, Milone, and Stickney (2010) compared STAR Early Literacy (STAR EL) (www. …