Academic journal article School Psychology Review

Generalizability and Dependability of Behavior Assessment Methods to Estimate Academic Engagement: A Comparison of Systematic Direct Observation and Direct Behavior Rating

Academic journal article School Psychology Review

Generalizability and Dependability of Behavior Assessment Methods to Estimate Academic Engagement: A Comparison of Systematic Direct Observation and Direct Behavior Rating

Article excerpt

Measurement refers to the act of assigning numbers to behavioral samples in order to draw inferences about an individual (Crocker & Algina, 1986). It is crucial that the numbers assigned be trustworthy, given potential uses that can seriously affect the lives of children (e.g., special education eligibility, suitability for postsecondary education). One of the central requirements specified within the Individuals with Disabilities Education Act (2004) is that the tools used to gather information about student behavior must be technically sound, meaning that the resultant data are reliable and valid. However, as highlighted by Hintze (2005), although the psychometric properties of some instruments (e.g., intelligence, achievement) have been heavily scrutinized, behavior assessment methods as a whole have not received comparable attention. That is, the wealth of supporting evidence has been limited to either specific methods (e.g., behavior rating scales) or psychometric indices (e.g., interobserver agreement). Furthermore, although an increased focus on essential assessment practices in problem-solving models has accompanied recent legislation (i.e., Individuals with Disabilities Education Act, 2004), few suggestions have been provided with regard to measures best suited with regard to relevant indices of behavior.

Problem-solving assessment to initially identify and then monitor response to an intervention is not a novel measurement consideration; however, it does take on new significance in the context of a response to intervention framework of service delivery related to behavioral domains. A fundamental requirement of response to intervention implementation is that problem behavior be efficiently and proactively identified (i.e., screening), and then measured frequently to determine whether intervention efforts are successful or alternate strategies are warranted (i.e., progress monitoring). Although certainly desirable, two potential problems do emerge related to behavior assessment in general, and progress monitoring in particular. First, although several defensible measures are available for assessing student behavior, some (e.g., traditional behavior rating scales) were simply not designed for frequent administration to large numbers of students given both their length and frame of reference (e.g., rate behavior over past 6 months). Second, even when appropriate measures do exist, feasibility concerns related to time and resources may prevent practitioners from collecting data as often as needed to make timely decisions. A pressing need therefore exists to identify measures that are both technically adequate and feasible for applied use in problem-solving assessment. Two methods that have demonstrated potential, particularly as related to repeated (i.e., formative) assessment of social behavior, include systematic direct observation (SDO) and direct behavior rating (DBR).

Overview of SDO and DBR Methods

Given that SDO is capable of providing a very direct estimate of behavior, it has been regarded within a behaviorist paradigm as the most accepted and appropriate means of measuring student behavior. Although this may hold true in the context of highly controlled investigations, there are several practical limitations of SDO that potentially limit utility in applied settings. Perhaps the most frequently cited limitations are those related to the required resources (e.g., Briesch & Volpe, 2007). In addition, a less frequently discussed limitation is that SDO affords a very narrow, discrete snapshot of actual events. Although an isolated 15- or 20-min time block may be of interest in some cases (e.g., during transition times), more often the goal is to assess a student's behavior over a longer period of the school day. The feasibility of SDO therefore becomes significantly affected by the type and scope of inferences (e.g., specific, general) that one desires to make. As a result, SDO tends to be employed less often in schools than would be suggested by its prevalence in the research literature. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.