Validating a Standardized Patient Assessment Tool Using Published Professional Standards

Article excerpt

Background and Purpose. Standardized patient (SP) encounters are used to teach and assess various professional skills. Although well established in medical education, the validity of assessing student performance during SP encounters has not been adequately addressed in physical therapist education. This paper presents a method for validating an SP assessment tool for use in a high-stakes exam within a Doctor of Physical Therapy program using published professional standards.

Method Description and Evaluation. Faculty in a physical therapist education program developed an assessment tool for the first of 4 SP encounters students must pass to progress in the curriculum. Criteria were matched to specific statements in the Guide to Physical Therapist Practice, A Normative Model of Physical Therapist Professional Education, and Minimum Required Skills of Physical Therapists Graduates at Entry-Level. A survey was developed listing each assessment criteria with its corresponding statements from the 3 professional documents. Respondents' levels of agreement were assessed using a 4-point Likert scale. The chi-square (X^sup 2^) goodness of fit was used to determine the level of agreement between the assessment criteria and the practice expectation statements. Additional comments were analyzed using qualitative methods.

Outcomes. Thirty academic and clinical educators completed the survey. Ninety-six percent (22/23) of the SP assessment criteria reached statistically significant levels of agreement, with matching statements from at least one of the professional documents. Respondents indicated that the SP assessment tool was either more detailed or specific than the published professional documents; however, it was generally less comprehensive.

Discussion. Faculty concluded that the SP assessment criteria were generally consistent with practice expectations and thus a valid form of measurement. Differences can be attributed to the purposes of professional publications and the specific SP encounter.

Conclusion. SP encounters can help to ensure the safety of aspiring clinicians prior to clinical practice. The method presented uses published professional expectations to validate a high-stake assessment of student clinical competence during SP encounters. Using published professional documents to validate classroom assessment tools has the added benefit of connecting classroom teaching and assessment to practice.

Key Words: Standardized patient, Assessment, Validity, Content validity (lecture, lab, and clinical).

BACKGROUND AND PURPOSE

Assessment of clinical performance is an integral part of a health care professional education program. Developing valid assessment tools based on established professional expectations are especially critical to the safe transition of aspiring clinicians from the classroom to the clinical environment. Standardized patients (SP) or simulated patients, developed by Barrows1,2 in the 1960s, have been used as a method to teach and assess undergraduate medical student clinical skills and performance. Since the early 1980s, SP methodology also has been used in high- stakes clinical examinations, which determine students' advancement in their undergraduate medical education or residency training programs. An SP examination involves the use of actors (ie, SPs) coached by faculty to accurately and reliably portray signs and symptoms associated with a particular diagnosis or dysfunction. Students examine the patient (SP) and may be required to diagnose and/ or outline a plan of care based on their findings. The SP also has been used in objective, structured clinical examinations (OSCEs); limited performance assessments with multiple timed stations (510 minutes) in which the student performs specific tasks;35 and in more comprehensive assessments (clinical practice examinations) in which the student interacts with the SP in a less- structured environment.1'6

Trained by the faculty, the SP typically evaluates student performance with a checklist and also may provide qualitative feedback in the form of written comments or direct SPstudent feedback. …