The Reliability and Validity of the STEP Measurement Systems
Discussion of the reliability of a measurement system often precedes the presentation of validity evidence, since the reliability of a measure represents the upper bound of its validity. Theoretically, the validity coefficient should not exceed the reliability estimate since the latter represents all but the error variance associated with the measure.
In the STEP program the concern is not so much with the reliabilities of the individual dimension scores, which are given for the sixteen job function dimensions in Chapter 2 and for the predictive measures of the test battery scores in Chapter 4, but rather, the concern is with the derived composite Job Skill Assessments (JSAs) and estimates of Potential for Successful Performance (PSP) presented in the final STEP report for the individual. These composite scores will be defined in Chapter 7, which describes the applications of the STEP program. Suffice it to say here that the JSA reported for any of the twelve key positions in the four, three-rung managerial hierarchies is the average of the individual's MP- JFI--Ability scores for those functions that have been identified as important for overall successful performance in the position. Similarly, the PSP for any level of functioning in any managerial hierarchy is the weighted average of the individual's scores on the tests that have been identified as valid predictors of performance in the position.
The formula used in the calculations was one developed by Mosier ( 1943) for determining the reliability of a weighted composite. The formula is derived from the standard deviations, reliabilities, and intercor