Academic journal article Education & Treatment of Children

Reliability of Ratings across Studies of the BASC

Academic journal article Education & Treatment of Children

Reliability of Ratings across Studies of the BASC

Article excerpt

Abstract

Reliability estimates of behavioral rating scale ratings are influenced by sample composition and variability. This study describes and documents reliability reporting practices in dissertation studies that have used the Behavior Assessment System for Children (BASC) since its inception until 2001. Only 9 of 106 studies examined reported reliability for the subpopulation at hand. Most cited reliability scores from the BASC Manual. The lack of reliability score estimates for subpopulations in studies of the BASC has implications for the use of the BASC to help identify culturally diverse students with emotional disorders. The lack of reliability data for behavioral ratings suggests that studies using rating scales as the primary dependent variable may be inherently flawed.

**********

Given the diversity of today's school populations, it is imperative that researchers investigate to what extent the ratings from behavioral rating scales used to help determine student eligibility for placement into special education services as defined by the Individuals with Disabilities Act (IDEA) for "seriously emotionally disturbed" (SED) demonstrate reliability for the diverse subpopulations on which they are being utilized. This consideration is important since African Americans, Hispanics, Native Americans, and students with low socioeconomic status are overrepresented in special education programs for students with SED. That is, given the percentages of these students in a normal school population, they are more likely than Whites and students with middle and high socioeconomic status to be referred for, qualify for, and be placed into special education programs for SED than their counterparts from the dominant culture (Artiles & Trent, 1994; Dooley & Voltz, 1999; Grossman, 1995).

The authors' initial purpose for conducting this study was to conduct a reliability generalization study (Thompson & Vacha-Haase, 2000; Vacha Haase, 1998; Vacha-Haase, Henson, & Caruso, 2002). Vacha-Haase (1998) listed three different uses or purposes of a reliability generalization study: (a) to describe the typical degree of reliability obtained from using a particular instrument, (b) to describe the extent to which reliability estimates obtained from using that instrument vary from study to study, and (c) to identify factors in the different assessment procedures that explain variation in the reliability of the scores or ratings from one measurement situation to another. Our intent was to describe the degree of reliability typically associated with BASC ratings used in doctoral dissertations, to describe how much the reported reliability estimates varied across studies, and then to identify factors that explained this variability. However, after identifying the many dissertations that used the BASC, we discovered that it was impossible to conduct a full- scale reliability generalization study for two reasons: (a) most of the dissertations reported reliability estimates from the BASC standardization sample instead of computing reliability estimates for their own data, and (b) the few studies that did compute and report the reliability estimates for the data in the study used different BASC scales or composites. Consequently, instead of conducting a full reliability generalization study, we conducted a descriptive study that describes and documents reliability reporting practices in dissertation studies that have used the BASC since its inception until the end of 2001. Similar studies have been conducted by Hogan, Benjamin, and Brezinski (2000), Vacha-Haase, Ness, Nilsson, and Reetz (1999), and Whittington (1998). The unique contribution of this study is the focus on doctoral dissertations that all used the BASC in one way or another. Since doctoral dissertations are reviewed by examining committees consisting of professors, we expected that most of the dissertation authors would report reliability coefficients for their own data rather than presuming that the obtained ratings were at the same degree of reliability as the ratings used in the BASC standardization sample and reported in the BASC manual. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.