It is important for a nursing program to determine which variables are good predictors of student success or failure on the National Council Licensure Examination for Registered Nurses (NCLEX-RN) (1). Significant predictors for success may include such variables as National League for Nursing (NLN) Achievement Test scores, grade point averages (GPA), American College Test (ACT) and Standardized Assessment Test (SAT) scores, and grades in various nursing courses.
The purpose of this study was threefold: to determine which variables have the greatest predictive value of success on the NCLEX-RN; to investigate the efficacy of the continued use of standardized NLN Achievement Tests; and to develop and evaluate a new assessment instrument, the NCLEX-RN Risk Appraisal Instrument, which allows nursing educators to quickly assess a student's risk of failure by examining school records. Data were obtained from undergraduate nursing students at a large university in the Southeast. The potential predictors used included nursing theory course grades, nursing clinical course grades, and NLN Achievement Test scores.
It was hypothesized that students' scores on standardized tests can predict performance on the NCLEX-RN. To test this hypothesis, the relationships between students' scores on the Psychiatric, Obstetric, Pediatric, and Adult NLN Achievement Tests were investigated.
It was also hypothesized that achievement in particular nursing courses could predict performance on the NCLEX-RN. To investigate this hypothesis, the relationships between letter grades in six courses and NCLEX-RN performance were investigated. The six courses were Adult Health Nursing I and II, Psychiatric Mental Health Nursing, Obstetric Nursing, Pediatric Nursing, and Nursing Care of the Critically Ill. These specific courses were selected because their theoretical content was most consistent with information tested on the NCLEX-RN. It was also predicted that significant relationships could be found between failing the NCLEX-RN and earning Cs in theory courses and earning a C in any clinical course.
Review of the Literature A 1993 study by Wall, Miller, and Widerquist, which explored computerized NCLEX-RN success, examined variables at three time points: prior to admission, during the course of a nursing program, and at graduation (2). Of variables available prior to admission, sophomore GPA, GPA for science courses, and high school rank were found to be significant predictors of performance on the NCLEX-RN. High school rank was less significant in its ability to predict NCLEX-RN performance than the other two prenursing variables. These results suggest that early identification of "at risk" students is possible. Among academic variables obtained during the course of a nursing program, scores obtained on NLN Achievement Tests, the nursing GPA, the Mosby Assess Test, and the NLN Diagnostic Readiness Test appeared promising as predictors of success on the NCLEX-RN.
Similarly, Waterhouse, Carrol, and Beeman (1993) investigated the usefulness of certain variables in predicting NCLEX-RN performance at three different time periods: and of the junior year, between semesters of the senior years, and at graduation (3). They found that grades in physiology and pathophysiology, the second junior nursing course grade, the first senior course grade, the sophomore grade point index (GPI), and GPI at the time of graduation were all significant predictors of success on the NCLEX-RN.
Overall, success rate prediction is often reported as a multifaceted phenomenon. Nevertheless, by examining the scholastic profiles of students, various trends may emerge to assist in classifying students who are most likely to pass or fail. Wall and colleagues found that the success rate for predicting students who are most likely to pass or fail. Wall and colleagues found that the success rate for predicting students who passed the NCLEX-RN was 94 percent, compared with 53.3 percent for those who failed (2). This finding suggests that prediction accuracy is high for students who will pass the test, but much lower for students who are likely to fail.
Of particular relevance was Campbell and Dickson's analysis of the results of many recent NCLEX-RN studies (4). Grade point averages for various subjects, including nursing theory courses, nursing clinical courses, and chemistry, were significantly correlated with NCLEX-RN scores. SAT and ACT often predicted NCLEX-RN success in past studies, with the ACT significantly predicting NCLEX-RN success more often than the SAT. The NLN Pre-Admission Examination-RN was also found to be a predictor of success. Test anxiety, self-concept/self-esteem, and a support group intervention strategy were also significantly correlated with NCLEX-RN success.
Studies have also explored many nonacademic predictors of NCLEX-RN performance, such as age, race, sex, self-esteem, primary language, and test anxiety (1). However, the literature provides mixed results in attempts to categorize students according to potential performance. Overall, findings point out the particular difficulty in quantitatively identifying those students who are likely to fail the exam.
Method Participants The study was conducted using data obtained from College of Nursing records of 81 students in the Baccalaureate of Science in Nursing program. This group of students ranged in age from 21 to 26, with a mean age of 28.33 years (SD = 6.35); 62 were female and 19 were male. Fifty-one percent of the students were single; 39 percent were married; and 2 percent were divorced. In terms of ethnicity, 89.0 percent were white, 7.3 percent were African American, 2.4 percent were Asian, and 1.2 percent were Hispanic. Ten of the students were in the process of pursuing a second degree.
Procedure Data from the College of Nursing records were first coded to facilitate computer entry. For example, the grade data from courses the students had taken were converted into a standard four-point scale: A = 4 (90% to 100%), B = 3 (80% to 89%), C = 2 (70% to 79%), D = 1 (60% to 69%), and F = 0 (59% and below). Dichotomous information, such as whether or not the student had passed the NCLEX-RN, was coded on a scale of 0 or 1; 1 indicated that the student passed the NCLEX-RN, and 0 indicated that the student did not pass.
Data were entered into the Statistical Package for Social Sciences for Windows. Data analysis was conducted using descriptive statistics, the Man-Whitney U technique, the chi-square technique, and Pearson Product-Moment correlations.
Results Table 1 summarizes significant positive correlations found in this study. Strong correlations were found between performance on the NCLEX-RN and performance in the Pediatric Nursing course, the Adult NLN Achievement Test, and the Psychiatric Health nursing course. Moderate correlations were found between performance on the NCLEX-RN and the Psychiatric NLN Achievement Test, the Obstetrics NLN Achievement Test, the Adult Health Nursing I course, the Adult Health Nursing II course, the Nursing Care of the Critically Ill course, the Pediatric NLN Achievement Test, and the Obstetric Nursing course.
Table 1. Variables That Positively Correlate with Performance on the NCLEX-RN
Variable Pearson N Probability Correlation (r) (p) Pediatric Nursing Course .5873 81 .000 Adult NLN Test .5851 81 .000 Psychiatric Mental Health Nursing Course .5825 81 .000 Psychiatric NLN Test .5221 81 .000 Obstetrical NLN Test .5173 81 .000 Adult Health Nursing I Course .4851 81 .000 Adult Health Nursing II Course .4728 81 .000 Nursing Care of the Critically Ill Course .4537 44 .000 Pediatric NLN Test .4238 81 .000 Obstetrical Nursing Course .3780 81 .001 Variable Coefficient of Determination ([r.sup.2]) Pediatric Nursing Course .3449 Adult NLN Test .3423 Psychiatric Mental Health Nursing Course .3393 Psychiatric NLN Test .2726 Obstetrical NLN Test .2639 Adult Health Nursing I Course .2353 Adult Health Nursing II Course .2235 Nursing Care of the Critically Ill Course .2058 Pediatric NLN Test .1813 Obstetrical Nursing Course .1429
Note. The variables listed in the first column of this table are displayed in a rank-ordered fashion according to how well each variable correlated with NCLEX-RN performance. The Nursing Care of the Critically Ill course (n = 44) is a newly developed course.
In addition, statistical relationship were found that suggest that certain variables are related to increasing the probability of failing the NCLEX-RN. Probability of failure increase when a student earns a C in any clinical course, [chi-square] (4, N = 81) = 21.77, p = .000 or nursing theory course, [chi-square] (4, N = 81) = 15.87, p = .000. It increases to an even higher level when a student earns two Cs in nursing theory courses, [chi-square] (4, N = 81) = 27.76, p = .000, and is still greater when a student earns three Cs, [chi-square] (4, N = 81) = 29.18, p = 1.000. Finally, the probability of failing the NCLEX-RN is remarkably high when a student earns more than three Cs in nursing theory courses, [chi-square] (4, N = 81) = 44.01, p = .000.
As Table 2 illustrates, students who passed the NCLEX-RN and students who failed differed with respect to certain variables. Those who passed earned higher grades in the Psychiatric Mental Health Nursing course, the Pediatric Nursing course, the Adult Health Nursing I course, the Adult Health Nursing II course, and the Nursing Care of the Critically Ill course. Furthermore, they scored higher on the Adult NLN, Psychiatric NLN, the Obstetric NLN, and the Pediatric NLN achievement tests than students who failed the licensure exam.
Table 2. Differences Between Students Who Pass and Students Who Fail the NCLEX-RN
Variable Mean grade/score Mean grade/score for students for students passing the failing the NCLEX-RN (n=68) NCLEX-RN (n=13) Psychiatric Mental Health Nursing Course 3.06 2.21 Pediatric Nursing Course 3.19 2.14 Adult NLN Test 77.39 37.85 Adult Health Nursing II Course 2.82 2.07 Adult Health Nursing I Course 3.00 2.14 Psychiatric NLN Test 69.81 30.08 Nursing Care of the Critically Ill Course 2.56 1.75 Obstetrics NLN Test 84.97 59.15 Pediatric NLN Test 76.19 51.36 Variable Man-Whitney 2-tailed U value probably (p) Psychiatric Mental Health Nursing Course 123.5 .0000 Pediatric Nursing Course 91.0 .0000 Adult NLN Test 82.5 .0000 Adult Health Nursing II Course 157.0 .0000 Adult Health Nursing I Course 162.0 .0000 Psychiatric NLN Test 107.5 .0000 Nursing Care of the Critically Ill Course 53.0 .0023 Obstetrics NLN Test 132.0 .0001 Pediatric NLN Test 174.5 .0002
Support was found for the use of the NCLEX-RN Risk Appraisal Instrument (NCLEX-RN RAI). A high score on the NCLEX-RN RAI has a very strong correlation with failure on the NCLEX-RN, r(81) = -.7827, p = .001. The variance of scores in NCLEX-RN RAI accounts for 61.26 percent of the variance in NCLEX-RN performance ([r.sup.2] = .6126). Overall, the instrument correctly classified 95.59 percent of those students who passed the NCLEX-RN and 76.92 percent of those who failed the exam.
A significant relationship was found between passing the NCLEX-RN and students' scores on the Psychiatric, Obstetric, Pediatric, and the Adult NLN Achievement Tests. Therefore, the hypothesis that students' scores on standardized tests can predict performance on the NCLEX-RN was supported.
Furthermore, significant relationships were found between passing the NCLEX-RN and students' grades in Psychiatric Mental Health Nursing, Obstetric Nursing, Pediatric Nursing, Adult Health Nursing I, Adult Health Nursing II, and Nursing Care of the Critically Ill. These relationships support the second hypothesis, that achievement in particular nursing courses can predict performance on the NCLEX-RN.
Predictions that significant negative relationships could be found between performance on the NCLEX-RN and earning Cs in theory courses or earning a C in any clinical course where also supported. It should be noted that risk of failing the NCLEX-RN increased with the number of C (or lower) grades obtained in nursing theory courses. In other words, any student with three Cs in nursing theory courses is more at risk than a student with only one C in nursing theory courses.
From the significant relationships that have been found between certain variables and NCLEX-RN performance, it is possible to make predictions regarding which students are more at risk for failing the NCLEX-RN. Table 3 depicts the summary profile of students who failed the NCLEX-RN. Students who fail are more likely to score low on the Adult, Obstetric, and Psychiatric NLN Achievement Tests. Furthermore, they generally score lower in many nursing courses.
Table 3. Summarized Profile of Student Failure and Success on the NCLEX-RN
Variable Characteristics of of students who pass the NCLEX-RN (n = 68) Making Cs in nursing theory Less likely to make Cs in nursing courses theory courses Making a C in a clinical Typically do not earn any Cs in nursing course clinical nursing courses Psychiatric Mental Health Typically earn a B (3.06) Nursing Course Pediatric Nursing Course Average a B (3.19) Adult NLN Test Obtain an average score of 77.39 Adult Health Nursing II Course Have an average grade close to a B (2.82) Adult Health Nursing I Course Typically obtain a low B, on the average (3.00) Psychiatric NLN Test Obtain an average score of 69.81 Nursing Care of the Critically Have an average grade in the C Ill Course range (2.56) Obstetrics NLN Test Have an average score of 84.97 Pediatric NLN Test Have an average score of 76.09 Variable Characteristics of students who fail the NCLEX-RX (n = 13) Making Cs in nursing theory More likely to earn Cs in nursing courses theory courses. Often earn multiple Cs in nursing theory courses Making a C in a clinical More likely to earn a C in a nursing course clinical nursing course Psychiatric Mental Health Average a C (2.21) Nursing Course Pediatric Nursing Course Average a C (2.14) Adult NLN Test Obtain a much lower average score (37.85) Adult Health Nursing II Course Have an average grade of a low C (2.07) Adult Health Nursing I Course Typically average in the C range (2.14) Psychiatric NLN Test Have a much lower average score (30.08) Nursing Care of the Critically Have an average score in the D Ill Course range (1.75) Obstetrics NLN Test Have a lower average score (59.15) Pediatric NLN Test Have a lower average score (51.36)
In contrast, students who pass the NCLEX-RN ar unlikely to earn a C in either clinical courses or nursing theory courses. In addition, these students are more likely to score well on the Adult, Psychiatric, Obstetric, and Psychiatric NLN Achievement Tests. Students who pass the NCLEX-RN generally earn higher grades in many nursing courses. In summary, earning a C or lower grade in certain key courses, combined with low scores on NLN Achievement Tests, are signs that can indicate which students are most at risk for failing the NCLEX-RN.
The results of the current study were used to develop the NCLEX-RN Risk Appraisal Instrument, Figure 1. The data collected in this study were also used to construct a table, Figure 2, which nursing educators may use in conjunction with the NCLEX-RN RAI to quickly assess a students' risk of failure. Using a trial-and-error technique, weights for each item were determined following repeated analyses of the data. For example, it was identified that the NCLEX-RN RAI correlated most strongly with NCLEX performance when item 9 was given a weight of 8. That is, a weight for item 9 that was either less than or greater than 9 resulted in a lower significant correlation between the NCLEX-RN RAI and NCLEX-RN performance. THis process was repeated for each item on the NCLEX-RN RAI to ensure that the highest correlations were obtained between the two variables.
Figure 1. NCLEX-RN Risk Appraisal Instrument (RAI)
Directions: Answer each question by circling the appropriate response. A number is depicted to the left of each response in parentheses. After answering all of the questions, total the numbers in parentheses. Compare this score to the Risk Appraisal Table to identify the student's risk of failing the NCLEX-RN.
1. Did the student received a C (or lower) in the Psychiatric Mental Health Nursing course?
Yes (2) No (0)
2. Did the student receive a C (or lower) in the Pediatric Nursing course?
Yes (2) No (0)
3. Did the student obtain a score of 54 or lower on t the Adult NLN?
Yes (1) No (0)
4. Did the student obtain a C (or lower) in the Adult Health Nursing II course?
Yes (1) No (0)
5. Did the student obtain a C (or lower) in the Adult Health Nursing I course?
Yes (1) No (0)
6. Did the student obtain a score of 26 or lower on the Psychiatric NLN?
Yes (1) No (0)
7. Did the student obtain a C in any clinical course?
Yes (1) No (0)
8. How many Cs did the student obtain in nursing theory courses?
Two (1) Three (2) Four (or more (3)
9. Did the student obtain a final grade of less than 74 in Nursing of the Critically Ill course?
Yes (8) No (0)
Total points (in parentheses):--
Figure 2. Risk Appraisal Table
0 - virtually no risk of failing
1 to 7 - very minimal risk of failing
8 to 10 - this minimal risk of failing
11 or greater - this student will almost certainly fail
The inclusion or exlusion of predictors was implemented in two steps. First, only variables that correlated most highly with NCLEX-RN performance were considered. Second, variables being considered for inclusion on the risk appraisal instrument were investigated through the use of contingency tables in the attempt to identify a clear dividing line between students who passed and those who failed the NCLEX-RN. For example, all students in the Nursing Care of the Critically Ill course who had a final grade of 73.4 percent or lower failed the nCLEx-RN, while all students who scored higher than 73.4 percent passed the licensure exam. Thus, 74 percent was identified as the dividing line for separating students who are likely to pass the NCLEX-RN from those students who are likely to fail.
While a more sophisticated statistical technique could have been employed to assign weights to each NCLEX-RN RAI item, a deliberate attempt was made to design the instrument in a manner that does not require complicated statistical methods. Therefore, nursing educators may easily develop similar instruments or adaptations of the NCLEX-RN RAI that are more appropriate to their schools' program of studies.
The large correlation (r = .7827, p = .0001; [r.sup.2] = .6126) that was found between scoring high on the RAI and failing the NCLEX-RN provided support for the criterion validity of the instrument. However, the assessment instrument needs to be cross-validated on different populations of baccalaureate nursing students.
We recommend using the NCLEX-RN RAI the quarter before graduation. Early identification allows adequate time for remediation prior to completing the nursing program. At our university, an individualized plan is developed for each student who scores in the moderate to high-risk categories. The plan varies according to the individual needs of the student but generally consists of reviewing several computerized programs that are made available through the college of nursing. In addition, students are placed in a clinical practicum site most consistent with their deficient areas of performance. Biweekly meetings with faculty facilitators are conducted with each students to review their progress and revise the remediation plan as until the student scores in the 75th percentile or above.
The identification of potentialy at risk students appears to be advantageous for the student and for the institution. Responses from students regarding their individualized plans for remediation have been positive. They have stated that focused attention relative to the NCLEX has helped them in productive study and preparation for the exam. They have also communicated greater-self-confidence about the exam following remediation activities.
During biweekly meetings with faculty facilitators, students have expressed appreciation for specific directions outlined in their individual plans of study. Similarly, faculty have noted that the institution benefits, not only because of the high NCLEX-RN pass rates, but because of the positive impression made in the educational community by providing students with the best opportunity for initial career success.
(1.) Graham, M.E. (1994). New York state baccalaureate performance on NCLEX: Implications for Faculty. Journal of the New York State Nurses Association, 25(1), 12-15.
(2.) Wall, B. Miller, D. E., & Widerquist, J. G. (1993). Predictors of success on the newest NCLEX-RN. Western Journal of Nursing Research, 15, 628-643.
(3.) Waterhouse, J. K., Caroll, M. C., & Beeman, P.B. (1993). National Council Licensure Examination success: Accurate prediction of student performance on the post-1988 examination. Journal of Professional Nursing, 9, 278-283.
(4.) Campbell, A. R., & Dickson, C. J., (1996). Predicting student success: A 10-year review using integrative review and meta-analysis. Journal of Professional Nursing, 12(1), 47-59.
Thomas W. Barkley, JR., DSN, RN, CS, ACNP, is an assistant professor; Rosemary S. Rhodes, DNS, RN, is associate dean, professor, and Director of Undergraduate Studies; and Charles A. Dufour, MS, is a research assistant, University of South Alabama, College of Nursing Mobile, Alabama.…