An Outcome Assessment for Evaluating Occupational Therapy Students' Proficiency and Skills Required for Passing the NBCOT Computer-Based Certification Test
Avi-Itzhak, Tamara, Krauss, Andrea, Journal of Allied Health
The objective the current study was to perform an outcome assessment of occupational therapy students participating in the first offering of a graduate seminar. This seminar was designed to increase their proficiency and skills required for passing the computer-based test (CBT) NBCOT certification exam. Four outcome measures were assessed: pre- and post-seminar NBCOT CBT practice tests score and preand post-seminar NBCOT CBT numbers of areas of weakness delineated by NBCOT. Our hypotheses expected that the student score and number of identified areas of weakness at the post-seminar NBCOT practice testing to demonstrate: H^sub 1^-a score increase over pre-seminar practice testing; and H^sub 2^-a decrease in the number of areas of weakness delineated by NBCOT over pre-seminar practice testing. All 13 graduating occupational therapy students participated in the seminar and were the subjects of this assessment. The Wilcoxon test (1-tailed) was employed to test the research hypotheses. A significant (Z = 3.11, p = 0.002) post- over pre-seminar practice test score increase was found. No significant post- over pre-seminar practice testing decrease in number of areas of weakness was found. J Allied Health 2010; 39:81-87.
IN RECENT YEARS, a number of practice professions have moved from the pencil-and-paper test (PPT) to the use of computer-based tests (CBT) for certification. Occupational therapy (OT) is among these professions. Indeed, in 2003 the format of the National Board for Certification in Occupational Therapy (NBCOT) changed from the PPT to the CBT format.
As the use of computerized assessments has increased significantly over the last decade,1'2 concerns about the impact of the test mode on performance on CBTs have been raised. These concerns gave rise to comparative studies of CBT and PPT performance, generating variable results. A number of studies found test scores to be independent of the test mode.2 5 Yet, a considerable body of research found test scores to be dependent on the mode of assessment administration and reported that identical PPT and CBT tests did not generate similar results. Leesom6 performed a comprehensive literature review of human and technological issues related to "test mode effect." These studies identified content attainment/familiarity, computer familiarity/anxiety, race/ethnicity, and gender as participant issues most often associated with the mode of assessment administration.6-8
The mission of an OT program is to graduate effective and competent entry-level practitioners. In order to achieve this mission, OT programs need to successfully develop and instill entry-level competency that prepares its graduates for entry-level practice.9 Upon the successful fulfillment of all didactic and clinical requirements of an OT program, students are required to sit for the National Board (NBCOT) in order to become registered occupational therapists (OTR) and become eligible to practice as OTs. The NBCOT exam is considered an important measure for assessing student competence and OT program outcomes. Passing the NBCOT exam, or in fact any national board certification examination, is regarded as one of the significant benchmarks for assessing the effectiveness of a program's curriculum.9·10 The OT department is housed in a public urban institution, whose mission is to make higher education more accessible to more people. Public urban institutions of higher education tend to serve working-class students, who may have less than adequate high school preparation, are more likely to be first-generation college students, and may be English- language learners." In addition, financial hardship faced by many students requires that students work, which can interfere with learning.
Moving to the CBT mode has been reported to be more efficient for examiners, as it has the potential to save time and money.1,12-15 Results of previous research on "test mode effect" has raised OT faculty concerns that the CBT format of the NBCOT may further disadvantage potentially at-risk students. …