Academic journal article New Zealand Journal of Psychology

Task-Based Assessment Centre Scores and Their Relationships with Work Outcomes

Academic journal article New Zealand Journal of Psychology

Task-Based Assessment Centre Scores and Their Relationships with Work Outcomes

Article excerpt

Assessment Centres (ACs) represent an approach to behavioural evaluation in the workplace that has garnered international popularity (Thornton & Krause, 2009). ACs have been used in New Zealand for evaluation purposes (Taylor, Keelty, & McDonnell, 2002) and have been appraised with regard to their psychometric properties in New Zealand contexts (Jackson, Atkins, Fletcher, & Stillman, 2005; Jackson, Barney, Stillman, & Kirkley, 2007; Jackson, Stillman, & Atkins, 2005). One of the most commonly reported psychometric findings in the AC literature is that, among AC ratings, different-dimension same-exercise correlations tend to be stronger than same-dimension different-exercise correlations (Bowler & Woehr, 2006; Lance, Lambert, Gewin, Lievens, & Conway, 2004; Sackett & Dreher, 1982). Thus, ratings in ACs tend to accentuate performance within exercises rather than performance on the basis of allegedly stable dimensions (e.g., communication skills) assessed across exercises. Such exercise effects have been found in New Zealand (Jackson, et al., 2007; Jackson, Stillman, et al., 2005) as well as internationally (Lievens & Christiansen, in press), suggesting that there are conceptual problems associated with scoring a given dimension across multiple exercises.

Exercise effects have led several researchers to voice concerns about aggregating AC scores by dimensions summarized across exercises. Particular concerns include the meaning given to dimensions in developmental ACs (Kudisch, Ladd, & Dobbins, 1997) and limitations in terms of fostering an understanding as to the mechanism underlying AC functionality (Klimoski & Brickner, 1987). In response to the measurement issues associated with ACs, three perspectives on the meaning underlying AC ratings have emerged (Lievens & Christiansen, in press). Firstly, the traditional dimension-based approach, where dimensions are thought to form meaningful constructs when aggregated across different exercises (Arthur, Day, & Woehr, 2008). Secondly, the task-based approach, which proposes that ratings from ACs should be aggregated within exercises to form meaningful exercise-based constructs (Jackson, Stillman, et al., 2005). Thirdly, the mixed-model approach, which incorporates aspects of both dimension scores in combination with exercise scores (Hoffman, Melchers, Blair, Kleinmann, & Ladd, in press).

Despite internal measurement challenges, ACs are often found to be predictive of work outcomes. With a job performance criterion, the most recent meta-analysis suggested a criterion-related validity estimate of .36 (Arthur, Day, McNelly, & Edens, 2003) and a previous meta-analysis returned similar results with an estimate in the order of .37 (Gaugler, Rosenthal, Thornton, & Bentson, 1987). Arthur, et al. stated that their meta-analysis of the criterion-related validity of ACs focused on dimensions because of their importance to psychology and their historical use in this context (see p. 128). It is, indeed, difficult to contest the importance of the assessment of constructs such as dimensions in psychology, given the research database that has accumulated on the measurement of psychological variables. Nonetheless, some researchers have suggested that the actual constructs measured by ACs might be different from those formalized in dimension scores (Jackson, et al., 2007; Jackson, Stillman, et al., 2005; Lance, 2008a; Lowry, 1997).

The fact that dimension scores are summarized across exercises in dimension-based ACs has led to a belief that ACs tap relatively stable and enduring variables, akin to traits (Jackson, et al., 2007; Jackson, Stillman, et al., 2005; Sackett & Dreher, 1982). The concept of sampling behaviour across different measures in this way is reminiscent of trait theories presented in keystone papers on psychometrics such as that by Campbell and Fiske (1959). The urban legend that has since emerged is that, in a multitrait-multimethod matrix tradition, AC exercises (cf. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.