Academic journal article Research Quarterly for Exercise and Sport

Selecting and Training Judges for Setting Performance Standards. (Peer-Reviewed Symposia)

Academic journal article Research Quarterly for Exercise and Sport

Selecting and Training Judges for Setting Performance Standards. (Peer-Reviewed Symposia)

Article excerpt

Setting performance standards requires a great deal of subjective input from judges. While no method appears to be superior they all rely on the subjective decisions. Identifying and selecting quality judges is extremely important. Judges must be qualified to make judgments on the test (AERA, APA, NCME, 1999). The purpose of this review is to examine the criteria for selecting judges and the common training methods used to produce correct performance standards. Selecting the best judges impacts the amount of training needed to set a successful standard. The quality of the judges is inversely proportional to the amount of training required (e.g., better judges, less training). Raymond and Reid (2001) propose six criteria or characteristics of qualified judges. The judges should be: (a) content matter experts; (b) knowledgeable about the examinee population; (c) able to estimate item difficulties; (d) aware of the opportunity to learn (environment); (e) appreciative of the ramifications of setting the standards ; and (f) sure all relevant interests are represented in the establishment of the standards. The number of judges should be large enough (n = 10-15) to ensure accurate results. The following questions have been used to evaluate the judges' performance: Do the judges' ratings remain stable over time? Are they consistent with the tasks of the method employed, such as rating the minimally acceptable candidate's performance? …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.