Academic journal article Research Quarterly for Exercise and Sport

A Confirmatory Study of Rating Scale Category Effectiveness for the Coaching Efficacy Scale

Academic journal article Research Quarterly for Exercise and Sport

A Confirmatory Study of Rating Scale Category Effectiveness for the Coaching Efficacy Scale

Article excerpt

This study extended validity evidence for measures of coaching efficacy derived from the Coaching Efficacy Scale (CES) by testing the rating scale categorizations suggested in previous research. Previous research provided evidence for the effectiveness of a four-category (4-CAT) structure for high school and collegiate sports coaches; it also suggested that a five-category (5-CAT) structure may be effective for youth sports coaches, because they may be more likely to endorse categories on the lower end of the scale. Coaches of youth sports (N = 492) responded to the CES items with a 5-CAT structure. Across rating scale category effectiveness guidelines, 32 of 34 evidences (94 %) provided support for this structure. Data were condensed to a 4-CAT structure by collapsing responses in Category 1 (CAT-1) and Category 2 (CA T-2). A cross rating scale category effectiveness guidelines, 25 of 26 evidences (96 %) provided support for this structure. Findings provided confirmatory, cross-validation evidence for both the 5-CAT and 4-CAT structures. For empirical, theoretical, and practical reasons, the authors concluded that the 4-CAT structure was preferable to the 5-CAT when CES items are used to measure coaching efficacy. This conclusion is based on the findings of this confirmatory study and the more exploratory findings of Myers, Wolfe, and Feltz (2005).

Key words, coaches of youth sport, Rasch model, validity


Validity refers to the degree to which evidence and theory support the interpretations of scores entailed by proposed uses of tests (quote from the American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999, p. 9). Or, as Messick (1989) more fully explained in his treatise on a unified conceptualization of construct validity, "Validity is an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores or other modes of assessment" (p. 13). Within both conceptualizations, validity is a unified, although multifaceted, concept for which evidences need to be collected and evaluated across time and across purposes. An important facet of validity evidence is how effectively an instrument's rating scale structure represents a construct, what Messick (1995) referred to as a substantive aspect of validity evidence.

Respondents do not always use a rating scale in the manner intended by the constructor. Determining the function of a rating scale is important, because an effective structure increases the accuracy and precision of the resulting measures, the likelihood of measure stability, and related inferences for future samples (Linacre, 2002). "Rasch analysis provides an effective framework within which to verify, and perhaps improve, the functioning of rating scale categorization" (Linacre, 2004, p. 259). Data are fitted to a Rasch (1960) model, because select diagnostic statistics have proven useful in both determining an effective post hoc rating structure (Zhu & Kang, 1998; Zhu, Timm, & Ainsworth, 2001; Zhu, Updyke, & Lewandowski, 1997) and in cross-validating it in a confirmatory application (Zhu, 2002). A four-step conceptual summary of this procedure is worthwhile, because application of the methodology in sport and exercise sciences is relatively novel. This summary consolidates the work of Linacre (1999, 2002, 2004) and should not be viewed as new methodological work.

A Conceptual Four-Step Summary

Step 1. Items should be oriented in a consistent manner with the latent variable of interest. Simply, the rating scale should maintain its meaning across items that are influenced by the latent variable (e.g., 1 = the lowest amount, and 5 = the highest amount of the latent across items). For example, if an item was negatively oriented in which 5 indicated the lowest amount, and all of the other items were positively oriented, responses to this negatively oriented item should be reoriented to reflect the shared positive orientation of the other items (i. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.