Psychometrics and Clinimetrics in Assessing Environments: A Comment Suggested by Mackenzie et Al., 2002 / Replies
Dijkers, Marcel P. J. M., Diamond, James J., Marion, Rodger, Journal of Allied Health
Psychometrics is the name commonly used for the principles and methods of developing valid and reliable measures of intelligence, attitudes, skills, and other characteristics. One focus of psychometrics is the homogeneity of the items selected to measure the (unidimensional) latent construct of interest. Clinical scientists often use operationalizations of constructs that incorporate multiple dimensions, which may be quantified using only a single indicator. The difference between the two approaches is significant enough that Feinstein proposed a new science, clinimetrics. Homogeneity of items is of limited importance in clinimetrics, and construct indicators may be "causal" rather than "effectual." In measuring environments of individuals, the clinimetric approach seems more appropriate than the psychometric one. An article by Mackenzie et al. (J Allied Health 2002; 31:222-228) is used to show how adhering to psychometric models may suggest analytical procedures that are misleading. Some principles of the clinimetric method are set forth. J Allied Health. 2003; 32:38-45.
THE SCIENCE OF PROPERLY quantifying constructs such as intelligence, attitudes, and other "unobservables" (latent traits) of interest to psychologists is psychometrics, and getting a degree in psychology or the social and behavioral sciences is impossible without exposure to the concepts of reliability, predictive and construct validity, and many other issues related to consistent and trustworthy measurement of whatever traits researchers and practitioners need to determine quantitatively. A crucial aspect of the reliability of instruments is internal consistency: Homogeneity of the items used to measure a latent trait is a necessary condition for reliability because all items supposedly tap the same construct and are correlated with one another. Similarly, assessment of certain aspects of validity is based on interitem correlations. The construct validity of a multidimensional scale may be assessed on the basis of the relatively strong correlations between items that tap the same dimension and weak correlations between items that represent different dimensions. "Classic test theory" has developed many techniques and formulas, such as Cronbach's ot, Kuder-Richardson, and Spearman-Brown, which make it possible to express the qualities of an instrument or test quantitatively, and many of these are available in common statistical computing packages.
The power of the psychometric techniques developed as part of classic test theory sometimes is seductive, and the instrumentarium (e.g., multitrait multimethod matrix, coefficient or) has been applied to entities that do not satisfy the assumptions underlying psychometrics. Take for instance the measurement of environments. For many years, occupational therapists and others working with or researching the elderly have attempted to assess the hazards persons with physical or cognitive deficits face in daily life in their homes, with the ultimate purpose of predicting and preventing falls and other injuries. Most of them have taken a familiar road to quantification: develop a bank of items that each constitutes a risk (e.g., loose rugs, dangling power cords, lack of grab bars), assign points for presence or absence, and add up to obtain a residence hazard score.1-3 Some, heeding their instruction in psychometrics, have calculated interrater reliability to establish that presence or absence of dangerous aspects of the residence can be determined consistently and objectively.1,4-6 A few have calculated interitem correlations or coefficient alpha to establish that their items are tapping the construct "riskiness."2,7,8
It is difficult, however, to make a claim that there is a latent construct riskiness that characterizes houses, and that drives them to exhibit consistently absence or presence of dangers to their inhabitants. There is no a priori reason why the presence of grab bars is correlated with the presence of safety strips on the stairs or why the absence of multiple extension cords plugged into one another is correlated with clutter in the walkways. …