Cognitive Assessment: An Introduction to the Rule Space Method

Cognitive Assessment: An Introduction to the Rule Space Method

Cognitive Assessment: An Introduction to the Rule Space Method

Cognitive Assessment: An Introduction to the Rule Space Method

Synopsis

This book introduces a new methodology for the analysis of test results. Free from ambiguous interpretations, the results truly demonstrate an individual's progress. The methodology is ideal for highlighting patterns derived from test scores used in evaluating progress. Dr. Tatsuoka introduces readers to the Rule Space Method (RSM), a technique that transforms unobservable knowledge and skill variables into observable and measurable attributes. RSM converts item response patterns into attribute mastery probabilities. RSM is the only up-to-date methodology that can handle large scale assessment for tests such as the SAT and PSAT. PSAT used the results from this methodology to create cognitively diagnostic scoring reports. In this capacity, RSM helps teachers understand what scores mean by helping them ascertain an individual's cognitive strengths and weaknesses. For example, two students may have the exact same score, but for different reasons. One student might excel at processing grammatically complex texts but miss the main idea of the prose, while another excels at understanding the global message. Such knowledge helps teachers customize a student's education to his or her cognitive abilities. RSM is also used for medical diagnoses, genetics research, and to help classify music into various states of emotions for treating mental problems.

The book opens with an overview of cognitive assessment research and nonparametric and parametric person-fit statistics. The Q-matrix theory is then introduced followed by the Rule Space method. Various properties of attribute mastery probabilities are then introduced along with the reliability theory of attributes and its connection to classical and item response theory. The book concludes with a discussion of how the construct validity of a test can be clarified with the Rule Space method.

Intended for researchers and graduate students in quantitative, educational, and cognitive psychology, this book also appeals to those in computer science, neuroscience, medicine, and mathematics. The book is appropriate for advanced courses on cognometrics, latent class structures, and advanced psychometrics as well as statistical pattern recognition and classification courses taught in statistics and/or math departments.

Excerpt

The methodology portrayed in this book is central to current missions in education in the United States. the “No Child Left Behind” federal government initiative requires that each state develop diagnostic tests of basic skills that not only assess overall educational progress but also identify areas of individual student weakness that will lead to tailored educational experiences. Valuable diagnostic profiles include information about how well students perform on the underlying knowledge and cognitive processing skills required for answering problems. Measuring the underlying knowledge and cognitive skills is not an easy task because it is impossible to directly observe them; therefore, they are named latent variables. Statisticians and psychometricians have developed the latent trait theory, now called item response theory (IRT), to measure an underlying ability. irt is a statistical model that tries to explain students’ responses on test items by using a mathematical function on a latent variable, called irt ability. However, the latent variables useful in cognitive diagnosis must be in the hundreds and not just one variable. For example, when we closely examined students’ performances on the Scholastic Aptitude Test (SAT) Verbal test, we have frequently observed that although students may have exactly the same scores, they could have entirely different profiles of strengths and weaknesses. in the case of two students, Jason and Juliana, they had very different individual profiles, but both attained 500 on the verbal scaled score. Jason was not good at bringing together material from two passages and processing grammatically complex texts. He also did not understand the main idea when it was not explicitly stated. On the other hand, Juliana was not good at synthesizing scattered information and applying general background knowledge; however, her global understanding was passable, and she had the same irt ability value as Jason had.

Factor analysis, cluster analysis, and traditional latent class models produce factors, clusters, and classes, but they are exploratory methods that merely group observed responses into similar classes or patterns. For this reason, they may produce solutions with no clear interpretation of the resulting groups of items or respondents. Ideally, diagnostic analyses of test results should be descriptive, objective, free from ambiguous interpretations, and uniquely express an individual’s true state of knowledge. To achieve these goals, we need a new methodology that will transform many unobservable knowledge and skills variables (defined as attributes . . .

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.