Meta-Analysis and Validity Generalization
as Research Tools: Issues of Sample Bias and Degrees of Mis-Specification
Philip Bobko Gettysburg College
Philip L. Roth
The notion of validity generalization is one of the more useful methodologies available to both practitioners and researchers in human resource selection. Such a methodology is very practical because it provides organizations with the possibility of implementing selection systems without the prerequisite time and effort involved in designing and conducting a local, empirical validation study. On the other hand, validity generalization is also a researchbased procedure because its invocation requires the user to compile known literature, classify studies accordingly, and analyze the situation at hand. The technique also helps separate variation in study outcomes due to artifacts from variation due to “true” states of affairs. In sum, validity generalization can be considered as a remarkable blend of good science with organizational needs and practicality.
In this chapter, the focus is on meta-analysis as a more general case of validity generalization. The latter is usually quite focused on the transportability or generalization of the validity of selection systems, and the effect size of interest is most often the Pearson product moment correlation coefficient between a selection test and a measure of job performance (the validity coefficient). In contrast, meta-analysis focuses on cumulating knowledge about any organizational interventions, which might include not only selection systems, but training systems, motivational interventions, types of re