Validity Generalization From
a Bayesian Perspective
Michael T. Brannick University of South Florida
Steven M. Hall
Embry-Rlddle Aeronautical University
Early papers on validity generalization referred to the SchmidtHunter calculations as Bayesian (e.g., Algera, Jansen, Roe, & Vijn, 1984; Schmidt & Hunter, 1977; Schmidt, Hunter, Pearlman, & Shane, 1979). Subsequent papers dropped the Bayesian label (e.g., Hunter & Schmidt, 1990), instead describing the calculations as a method of meta-analysis. Although methods labeled as Bayesian were not applied subsequently to validity generalization problems, researchers continued to develop such methods. Currently available techniques include empirical Bayes meta-analysis (Brannick, 2001; Raudenbush & Bryk, 1985) and some applications of hierarchical linear models (e.g., Bryk & Raudenbush, 1992; Selzer, Wong, & Bryk, 1996).
This chapter attempts to re-establish connections between Bayesian aspects of meta-analysis and validity generalization. The first half of the chapter emphasizes statistics and calculations. The chapter begins by describing empirical Bayes meta-analysis and linking it to validity generalization. We then extend previous work by showing how empirical Bayes meta-analysis might be based on r rather than the Fisher transformed z. This is important for two reasons. First, it allows the incorporation of previous meta-analyses using the Schmidt-Hunter methods into current Bayesian calculations without having to locate the original data and recompute the analysis in z. Second, it allows us to provide a means for a Bayesian com