Factorial Validity, Reliability, and Measurement Equivalence of the Noctcaelador Inventory across Three Ethnic Groups
Kelly, William E., Journal of Instructional Psychology
This study examined the reliability, factorial validity, and measurement equivalence of the Noctcaelador Inventory (NI) among three ethnic groups of college students. Participants included 200 Whites, 200 African Americans, and 200 Latino/Hispanics. The results indicated that although the African American sample scored slightly lower than the White and Hispanic samples, the internal consistencies were identical for all three samples. Factor structures were consistently unidimensional and accounted for a similar percentage of variance for all groups as well. It was concluded that the NI is suitably valid, reliable, and equivalent for White, African American, and Latino/Hispanic college students.
A recent survey found that 17.8% of college students endorsed viewing the night-sky at least once a night (Kelly, Kelly, & Batey, 2006). Research has also found that students with an interest in night-sky watching were apt to make certain life-decisions and allocate their time and energy to increase opportunities to view the night-sky (Kelly, 2004a; Kelly et al., 2006). Given that a substantial number of students obviously place some importance on night-sky watching, it might be beneficial for educational researchers to better understand individual differences of this segment of the student population.
Noctcaelador and its Measurement
Factor analytic studies have found that a single factor, or construct, accounted for the majority of the variance in students' night-sky watching attitudes, interests, and self-reported behaviors (Kelly, 2003, 2006a; Kelly & Kelly, 2003). Using Latin word parts, Kelly (2003) termed this construct noctcaelador, tentatively defined as an "emotional attachment to, or adoration of, the night-sky" (p. 196).
The primary instrument used to measure noctcaelador has been the Noctcaelador Inventory (NI; Kelly, 2004a). The NI appears to tap several aspects of night-sky related psychological phenomena, such as behavior, attitudes, attachment, and coping (Kelly, 2006a; Kelly & Daughtry, 2007). Previous studies examining noctcaelador have found the NI to correlate with two primary clusters of variables in student samples: 1) a curious, rational cognitive style (i.e., Kelly, 2004b, 2005a, 2005b) and 2) openness to unusual, novel experiences and thought processes (Kelly & Daughtry, 2005; Kelly, 2006b, 2007).
Assessment and Ethnodiversity
An important element of understanding a construct is its measurement (Anastasi & Urbina, 1997). In a multicultural environment it is especially important to ensure that measurement devices be reliable, valid, and demonstrate measurement equivalence across diverse ethnic groups (American Psychological Association, 2002). Despite the need for culturally equivalent measures, several instruments have been developed using relatively homogeneous samples (i.e., a measure of forgiveness by McCullough et al., 1998), or with no indication of the ethnic makeup of the samples (i.e., the Perceived Stress Scale by Cohen, Kamarck, & Mermelstein, 1983; the Multidimensional Sense of Humor Scale by Thorson & Powell, 1993).
The NI is another such instrument. The ethnic distribution of the sample was not reported in the article that described the development of the measure (Kelly, 2004a). An examination of the original data, however, revealed that 80% (119) of the 150 participant sample used to develop the NI was White, 17% (25) were African American, and less than 1% (1) Hispanic. These are three of the largest, current and projected for the next 50 years, ethnic groups in the United States (U.S. Census Bureau, 2004). However, they are not well represented in the sample used to develop the NI.
The Current Study
As previously noted, when using an assessment instrument with diverse populations, it is important to ensure that the measure is equally valid for different ethnic groups. Such equity can assist in generalizing the results beyond a particular sample. Without such equity, however, one cannot be confident of the reliability and validity of the scale or its scores across different ethnic groups (Anastasi & Urbina, 1997). Therefore, because the NI has not yet been examined across ethnically diverse populations, it is difficult to generalize the results it has yielded beyond the rather homogeneous samples with which it has been utilized.
The purpose of this study was to assess the factorial validity, reliability, and measurement equivalence of the NI for three ethnic groups. It was predicted that the reliability, factor structure, and average scores of the NI would be relatively consistent across ethnic groups.
Participants and Procedure
Six-hundred participants were randomly drawn from a normative database for the NI. Of these 600 participants, 200 identified themselves as White [148 (74%) females; M age = 23.2 years (SD = 5.9)], 200 as African American/Black [168 (84%) females; M age = 27.5 years (SD = 7.4)], and 200 as Hispanic/Latino [140 (70%) females; M age = 24.9 (SD = 6.4)].
The NI (Kelly, 2004a) is a 10-item self-report scale which measures interest in, and attachment to, the night-sky. Participants responded to items using a 5-point Likert scale (1 = "strongly disagree," 5 = "strongly agree"). Responses were summed to produce a total score; higher scores indicate more noctcaelador. Kelly (2004a) reported the scale to have suitable internal consistency (alpha = .92), and test-retest reliability (.88, one month). Validity of the NI has been found to be satisfactory (Batey & Kelly, 2005; Kelly, 2004a; Kelly et al., 2006).
An Analysis of Variance (ANOVA) revealed a significant difference in age between the ethnic groups, F (2,597) = 21.3, p < .001. Post-hoc tests found significant differences between all ethnic groups (average ages are noted above). An additional ANOVA was calculated to compare differences in sex (dummy coded, 1 = male, 2 = female) among the samples. A significant result was obtained, F (2,596) = 6.2, p < .01. The African American group had significantly more females than the White or Hispanic groups, which were not significantly different. There were no significant correlations for age or sex with NI scores within each ethnic group. Therefore, age and sex were not considered suitable candidates for co-variates (Baron & Kenny, 1986). Ergo, data was collapsed across age and sex for further analyses.
Means, standard deviations, and internal consistencies of the NI, respectively, for each sample were as follows: White--28.3,8.8, .94; African American--26.3,9.6, .94; Hispanic--28.3, 9.0, .94. An ANOVA was calculated to examine differences in NI scores among the ethnic groups. An overall significant result was obtained, F (2,597) = 3.2, p < .05. The African American sample scored significantly lower on the NI than the White or Hispanic samples, t (398) = 2.2,p < .05, and t (398) = 2.2, p < .05, respectively. The White and Hispanic groups were not significantly different, t (398) = .01, p = .99.
Principal components factor analyses were calculated to examine the factor structure of the NI for each ethnicity. For the African American sample, one factor was extracted accounting for 66.1% of the variance in responses (eigenvalue = 6.6). Factor loadings ranged from .76 to .85. The average factor loading was .81. For the Hispanic sample, one factor was identified accounting for 64.1% of the variance (eigenvalue = 6.4). Factor loadings ranged from .75 to .86. The average factor loading was .80. For the White sample, one factor was extracted accounting for 65.1% of the variance (eigenvalue = 6.5). Factor loadings ranged from .76 to .85. The average factor loading was .81.
The results were consistent with the hypothesis. The NI demonstrated consistent reliability and factorial validity across samples of White, African American, and Hispanic ethnic groups. These findings are consistent with previous research indicating that the NI has consistent internal consistency reliability across several samples (i.e., Kelly, 2004a, 2004b, 2005a, 2005b, 2006b, 2007; Kelly & Daughtry, 2005, 2007). Although previous research did not consider ethnicity, the results are also consistent with findings that the NI possesses a unidimensional factor structure across several samples (i.e., Kelly 2004a, 2006a; Kelly & Daughtry, 2007).
One interesting finding was the significantly lower NI scores of the African American group compared to the White and Hispanic groups. This result was consistent with previous research when considering that noctcaelador appears to exist along the broad personality domain openness to experience (Kelly, 2004b). Heuchert, Parker, Stumpf, and Myburgh (2000) found that White South Africans scored significantly higher than Black South Africans on openness. Albeit, White and Black South Africans and White and Black Americans should not necessarily be compared, this provides some precedent for ethnic differences of openness-related variables. Heuchert et al. suggested that socio-political differences may have accounted for the differences. The same may be the case in the American sample regarding NI scores. Additional research is needed to further examine possible reasons for this ethnic disparity in NI scores. Nevertheless, because the difference was so small, far less than one standard deviation in any of the samples, it seems safe to conclude that the difference, although statistically significant, likely has little practical relevance in score interpretation.
There are limitations of the present study which should be noted when interpreting the results. For example, although the samples were drawn as to be equal in number, the database from which they were drawn included far more Whites than Hispanic or African Americans. Therefore, the White sample represents more of a random sample. The African American and Hispanic samples were drawn from a smaller subset of data, perhaps limiting the representativeness of those samples. Also, other than factor analyses and internal consistencies, no other validity or reliability indicators were included. Therefore, the validity and reliability findings here are somewhat limited. In conclusion, despite the limitations of the present study, there appears to be some equality in the reliability, validity, and average scores of the NI for the three ethnic groups included herein. Hence, the NI appears to be suitably valid, reliable, and an equivalent measure for White, African American, and Latino/Hispanic college students.
American Psychological Association (2002). Guidelines on multicultural education, training, research, practice, and organizational change for psychologists. Washington, DC: Author.
Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed). Upper Saddle River, NJ: Prentice Hall.
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182.
Batey, J., & Kelly, W. E. (2005). Criterion-group validity of the Noctcaelador Inventory: Differences between astronomical society members and controls. Individual Differences Research, 3, 200-203.
Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24, 386-396.
Heuchert, J.W.P., Parker, W.D., Stumpf, H., & Myburgh, C.P.H. (2000). The Five-Factor Model of Personality in South African college students. American Behavioral Scientist, 44, 112-125.
Kelly, W. E. (2003). Night-sky watching attitudes among college students: A preliminary investigation. College Student Journal, 37, 194-196.
Kelly, W. E. (2004a). Development of an instrument to measure noctcaelador: Psychological attachment to the night-sky. College Student Journal, 38, 100-102.
Kelly, W. E. (2004b). The AOCEAN@ and the night-sky: Relations between the five-factor model of personality and noctcaelador. College Student Journal, 38, 406-408.
Kelly, W. E. (2005a). An investigation of vocational interests and noctcaelador. Journal of Instructional Psychology, 32, 164-166.
Kelly, W. E. (2005b). Some cognitive characteristics of night-sky watchers: Correlations between social problem-solving, need for cognition, and noctcaelador. Education, 126, 328-333.
Kelly, W. E. (2006a). Evidence of the existence of noctcaelador across three measures: A factor analytic study. Journal of Instructional Psychology, 33, 261-262.
Kelly, W. E. (2006b).Watching the night-sky and seeing things that aren't there: Relationships among noctcaelador, dissociative experiences, and hallucination-proneness in a non-clinical sample. Counseling and Clinical Psychology Journal, 3, 148-159.
Kelly, W. E. (2007). Getting a thrill from the night-sky: The relationship between sensation-seeking and noctcaelador. Psychology Journal, 4, 40-46.
Kelly, W.E., & Daughtry, D. (2005). Relationship between magical ideation and noctcaelador. Perceptual and Motor Skills, 101, 373-374.
Kelly, W. E., & Daughtry, D. (2007). Sleep-length, noctcaelador, and watching the night-sky to cope. Individual Differences Research, 5, 150-157.
Kelly, W. E., & Kelly, K. E. (2003). Further identification of noctcaelador: An underlying factor influencing night-sky watching behaviors. Psychology and Education: An Interdisciplinary Journal, 40, 26-27.
Kelly, W. E., Kelly, K. E., & Batey, J. (2006). Frequency of college students' night-sky watching behaviors. College Student Journal, 40, 166-168.
McCullough, M.E., Rachal. K.C., Sandage, S. J., Worthington, E. L., Brown, S. W., & Hight, T. L. (1998). Interpersonal forgiving in close relationships: II. Theoretical elaboration and measurement. Journal of Personality and Social Psychology, 75, 1586-1603.
Thorson, J. A., & Powell, F. C. (1993). Development and validation of a multidimensional sense of humor scale. Journal of Clinical Psychology, 49, 13-23.
U.S. Census Bureau (2004). U.S. interim projections by age, sex, race, and Hispanic origins. Retrieved from http://www.census.gov/ipc/www/usinterimproj/.
William E. Kelly, Department of Social Sciences, Robert Morris University.
Correspondence concerning this article should be addressed to William Kelly at email@example.com.…
Questia, a part of Gale, Cengage Learning. www.questia.com
Publication information: Article title: Factorial Validity, Reliability, and Measurement Equivalence of the Noctcaelador Inventory across Three Ethnic Groups. Contributors: Kelly, William E. - Author. Journal title: Journal of Instructional Psychology. Volume: 35. Issue: 3 Publication date: September 2008. Page number: 271+. © 2009 George Uhlig Publisher. COPYRIGHT 2008 Gale Group.
This material is protected by copyright and, with the exception of fair use, may not be further copied, distributed or transmitted in any form or by any means.