This article examines the experience of several agencies in Florida state government with their standards-based performance appraisal system and develops and tests a researcr strategy for determining the effect of agency investment in its implementation. Findings based on interviews of personnel managers and a survey of employees that suggest there are limits to the usefulness of performance appraisal which management must consider when deciding how to appraise and improve employee performance.
The Civil Service Reform Act of 1978 made the implementation of standards-based performance appraisal systems a priority concern for the federal government. Since that time, most state governments have followed the federal lead in assessing individual performance on the job according to work-related standards (Tyer, 1982).
This article assesses the impact of agency investments in the implementation of a standard-based performance appraisal system. It reports the results of a study in several large agencies in the state of Florida. The empirical analysis in this study focuses on the following question: Do employees in agencies with high levels of investment (training, materials and other resources) in their performance appraisal systems perceive their performance appraisals to be more helpful and effectively implemented than their counterparts in agencies with low levels of investment?
Competing purposes of performance appraisal
Despite its long history and widespread use in government organizations (Lee, 1987), performance appraisal has proven difficult to implement (Rich, 1989), and has been described as, "the weakest link in the personnel administration chain" (Hays & Tyer, 1980).
The standards-based performance appraisal system is one of the most recent approaches to resolving the problem of how to measure individual work performance in a fair and accurate manner (Locher & Teel, 1977). The appraisal focuses on "primary tasks," or the essential aspects of a position. For each primary task, there must be a performance standard which describes, in a measurable fashion, the behaviors that constitute the satisfactory accomplishment of the primary task. These standards are expected to be the basis of an objective, behavioral-based performance appraisal system (Office of Personnel Management, 1981).
However, a recent United States Government Accounting Office (GAO) report (1989) on human resource management in the federal government reports that although most federal agencies have had their standards-based performance appraisal systems in place for nearly ten years, there remain numerous, persistent problems. They include: (1) failure to develop objective and measurable performance standards; (2) limited employee involvement in the appraisal process (3) lack of acceptance and mistrust of the performance appraisal and merit pay systems; (4) lack of consistency among raters; (5) time-consuming and cumbersome processes; (6) difficulties in counseling, providing feedback, and other interpersonal issues; and (7) lack of management support (GAO, 1989). These findings parallel those of earlier assessments of the federal performance appraisal system (Perry and Pearce, 1983; Nigro, 1982).
The report concludes that the Office of Personnel Management must exercise more leadership in order for these persistent problems to be solved (GAO, 1989). It is assumed that there is nothing fundamentally wrong with the system and that it can work as intended with sufficient levels of training, resources and information (For a similar argument in regard to merit pay systems, see Seigel, 1987).
An alternative explanation for these perceived failures is that organizations are attempting to utilize performance appraisal for multiple and conflicting purposes. Figure I compares the two main purposes, judgmental and counseling, for which performance appraisal is employed in organizations (Cummings and Schwab, 1973).
The judgmental perspective focuses on performance appraisal as a tool for making administrative decisions, one-way communication from supervisor to employee, and both positive and negative incentives as means for improving performance. The counseling purpose emphasizes positive incentives, learning, and two-way communication between supervisor and employee as the means for enhancing employee performance.
These approaches to performance appraisal are fundamentally incompatible and based upon two very different sets of assumptions about employee motivation and the methods necessary to achieve high levels of performance. The judgmental approach assumes that employees are extrinsically motivated and will not perform at high levels without the expectation of material reward (McGregory, 1957). Therefore, employees must be monitored and periodically evaluated in order to assess the quantity and quality of their contribution to the organization, and to reward or punish them accordingly (Thayer, 1972). For their part, supervisors are expected to overcome their cognitive and personal limitations and provide objective appraisals of employee performance and exercise effective control over employee behavior (Glueck, 1982; Sherwood, 1982/83).
The counseling approach, on the other hand, assumes that employees are intrinsically motivated to perform well and thus want to receive feedback and collaborate with management to enhance their performance. It has also been argued that managers lack the objectivity and cognitive capacity to make fair and accurate performance judgments (Pulakos and Wexley, 1983; Thayer, 1987).
Therefore, the counseling alternative to the judgmental approach usually stresses the need for employee or peer participation in the appraisal process (Kane & Lawler, 1978; Lovrich, 1982).
Much of the literature of performance appraisal recognizes that it is very difficult for a supervisor to act as an effective coach or counselor and simultaneously take on the role of judge, particularly when the latter is tied to such important personnel decisions as pay increases and promotion. Where the supervisor is asked to do both, most of the attention and concern will be focused on the judgmental role and whether the appraisal has implications for the employee's pay or opportunities for promotion (Lovrich, 1982).
Nevertheless, performance appraisal is often expected to perform simultaneously the judgmental and counseling functions. For example, the rule governing performance appraisal of Florida's civil servants emphasizes the judgmental or administrative approach, but also encourages the use of performance appraisal for counseling and employee development:
Employee performance appraisals shall be used for, but not limited to, the following purposes;
(a) To inform the employee of strong and weak points in the employee's performance, improvements expected, and current and future training needs.
(b) To recognize the employee's potential for promotion.
(c) To determine the employee's eligibility for merit salary advances.
(d) As a basis for improving the performance of the state's workforce.
(e) To assist in determining the order of layoff and reinstatement (Florida Department of Administration, 1989).
The clear implication of the rule is that performance appraisal is intended to be a multi-purpose tool for personnel management. It does not recognize the potential for conflict among these purposes, and the attempt to implement it in state agencies provides an opportunity to empirically test the hypothesis that greater investment by state agencies in resources, training and information will enhance the ability of standards-based performance appraisal to service these multiple purposes.
Six major state agencies were asked to participate in the study, which centered on interviews of personnel directors and surveys of career service employees. All six of the agencies granted interviews with personnel directors, but only four consented to participate in the survey. Because of the size and diversity of the agencies, this sample is nevertheless generally representative of career service employees in Florida and provides contrasting perspective on the implementation of performance appraisal.
The primary purpose of the interviews with personnel directors was to identify how and to what a degree each agency invested in the implementation of performance appraisal, and thus provide a basis for making distinctions among the agencies for analysis of the survey data.
Surveys were administered to random samples of employees (including supervisors) in four of the six agencies granting interviews (N = 154). Questionnaire items (listed in the Appendix) were responded to on a 5-point Likert-like scale. They were adopted from two sources: (1) Daley's Iowa studies (1988) and, (2) a study of the federal system by Perry and Pearce (1983). The items were chosen because they reflect the stated purposes and important components of Florida's standard-based performance appraisal system.
Data from the interviews and surveys were analyzed in several steps. First, a factor analysis, using the principal components method, is performed on the survey data in order to determine empirically whether the performance appraisal system can be described more parsimoniously. Factoring was stopped when the eigenvalues obtained were less than one. Four factors for the sample were extracted and rotations to the varimax criterion were examined. Three factors, which account for 63.4% of the common factor variance, from the rotated solution were retained for analysis. Factors were identified and labeled based on items with loadings of .50 or greater.
Second, the sample was divided into two groups, according to high or low degree of agency investment in the performance appraisal system and the means and standard deviations for each questionnaire item were calculated for each group. Agencies defined as high investment were those that met the following criteria:
(1) Supervisors are required to receive training in the development and administration of standard-based performance appraisal;
(2) The agency developed and disseminated its own procedure manuals and other support materials;
(3) The agency has developed its own policy statement in regard to the importance and purpose(s) of performance appraisal.
Of the four agencies which participated in the survey, two met all of these criteria, while the other two met none of them. Those that met the criteria emphasized both the counseling and judgmental purposes of performance appraisal in their training programs, manuals, and policy statements. At the time of the study, both agencies had fully developed their materials and delivered training to all supervisors.
Third, a multivariate analysis of variance (MANOVA) was performed with each of the three factors to ascertain the effect of higher levels of agency investment on employee perceptions of performance appraisal. MANOVA produces summary statistics of the significance of the difference between the two groups, while taking into account the correlations among all of the variables simultaneously. An additional step, univariate F-tests, provides an assessment of the differences between the two groups for each individual variable.
Factor analysis/Descriptive statistics
Table 1 reports the results of the factor analysis with the loadings from the rotated solution and the labels of the items comprising the factors and the percentage of common factor variance explained. These results show that the performance appraisal can be viewed in terms of three main factors or dimensions: [Tabular Data 1 Omitted]
(1) helpfulness (for employee development and effectiveness), (2) supervisory effectiveness (how well supervisors administer and utilize performance appraisal), and (3) performance appraisal administration (administrative and judgmental aspects of performance appraisal). Factors 1 and 2 include those variables most associated with the counseling purpose of performance appraisal, while the variables that make up Factor 3 are those most associated with the judgmental aspects of the system.
Table 2 compares the means and standard deviations of each questionnaire item (listed according to factor) for the high and low investment agencies. The mean scores (scale of 1-5) indicate that, except for the merit pay provision, employees in both groups have a generally positive view of the standard-based system of performance appraisal (Factor 3). However, they do not perceive the performance appraisal system as being very helpful to them (Factor 1). Nor do they see it as an effective facilitator of performance feedback, although the scores related to the more formal aspects of supervisory effectiveness (discussing reasons for ratings, timeliness, and importance to supervisor) are considerably higher (Factor 2). [Tabular Data 2 Omitted]
Table 3, 4, 5 list the results of MANOVA analysis which was conducted for each of the three factors, with level of agency investment as the effect variable. The multivariate tests of significance for factor 1 (helpfulness), listed in Table 3, are not statistically significant (.26). That is, when taking into account the correlations among all the variables within each factors, one cannot reject the hypothesis that there are no differences between the high and low investment agencies in the helpfulness of performance appraisal. The univariate F-tests of each of the helpfulness variables provide evidence that employees in the high investment agencies view performance appraisal as more effective in helping them to receive needed training and to better serve the public. [Tabular Data 3 Omitted]
Table 4 shows that, in both the multivariate and univariate tests, there are no statistically significant differences in perceptions of supervisory effectiveness between the two groups of agencies. These results suggest that training and other support activities for performance appraisal have no discernable or consistent effect on the qualify of supervisory practices such as the frequency, quality and timeliness of performance feedback, and the importance of performance appraisal to the supervisor. [Tabular Data 4 Omitted]
Significant differences are found between high and low investment agencies in terms of the administrative aspects of performance appraisal (Factor 3). The multivariate tests of significance, listed in Table 5, are significant at the .004 level. The univariate tests suggest that the greatest difference occurs in perceptions of how important performance appraisal is to the organization (sig. F = .003). There is also evidence for differences in the perceptions of to what degree the merit pay provisions encourage job performance (sig F = .046), and of the overall effectiveness of performance appraisal (sig F = .075). Thus it appears that higher agency investment in the implementation of performance appraisal does enhance employee perceptions of the administrative aspects of the system, but primarily in terms of how important it appears to be to the organization. [Tabular Data 5 Omitted]
Perhaps the most surprising findings in this study were that there was no significant difference in supervisory effectiveness (Factor 2) between the two groups. One would expect supervisors in the high investment group to receive generally higher ratings for how they conduct appraisals than those in the low and investment group. The absence of differences suggest th at the training and additional materials added little or nothing beyond what supervisors were able to figure out on their own with the standard materials from the central personnel agency.
The most plausible reason for this lack of effect in the nature of the training and materials developed by the high investment agencies. While both used "state of the art" training programs and resources, the effect on improving supervisory competence was muted by the attempt to emphasize both the administrative and counseling aspects of the system. Apparently, the training did not spend enough time on how to conduct standards-based performance appraisals to significantly enhance perceptions of supervisory effectiveness.
The lack of effect of agency investment on the helpfulness dimension (Factor 1) is consistent with the expectation that performance appraisal is not an effective counseling tool when it is also used for judgmental decision making. Additional support for this conclusion emerged from the interviews. The personnel officers in the high investment agencies stated that the requirement to use performance appraisal for administrative decisions, especially for determining merit pay eligibility, made it difficult to use it effectively as a means for counseling employees.
In support of their experience, the recent literature on merit pay in government shows that the administration of merit pay relies on judgmental processes and assumptions, such as: (1) the requirement for unilateral yet valid distinctions between employees' performance (Perry, 1986; Thayer, 1987); (2) an emphasis on extrinsic rewards (Sherwood & Wechsler, 1986) and managerial control (Gabris, 1986); and (3) negative assumptions about the nature of employee motivation (Daley, 1987; Halachmi & Holzer, 1987; Perry, 1986).
The differences found between the two groups of agencies on the administrative dimension (Factor 3) suggest that higher levels of agency investment in performance appraisal exerts a positive effect on employees' perceptions of the legitimacy of the system. Employees are not likely to take seriously administrative decisions based on performance appraisals unless they perceive that the organization is genuinely concerned about the quality and importance of the appraisal process. Thus agency investment in performance appraisal can be important for symbolic as well as instrumental reasons.
The essential conclusion of this research is that less should be expected of performance appraisal. While there is considerable evidence that performance appraisal provides the critical input for certain types of administrative decisions, its contribution to the more positive aspects of management such as feedback and counseling is far less evident. Thus agencies should consider limiting the role of performance appraisal to that of a decision aid in such areas as merit pay distribution, promotions, hiring of probationary employees, and reductions in force. The implication of this conclusion is not that agency investments in performance appraisal are bound to be ineffective. Rather, it is that they should be focused on implementing systems, training and resources that produce valid, accurate and fair appraisals of employee performance. In this way, agencies can take the most advantage of the considerable literature on the design and implementation of performance appraisal (e.g. Morrisey, 1983; Patten, 1982; Dailey & Madsen, 1980; Stewart & Stewart, 1977; Kellogg, 1975; Mager & Pipe, 1970).
Recognizing that performance appraisals are not the best means for improving employee performance also means that other mechanisms should be identified and utilized to enable supervisors to become more effective at coaching and counseling employees. By separating the two processes, agencies are much more likely to be successful in promoting both improved employee performance and better administrative decision making.
1. My supervisor discusses with me the specific reasons for the
performance rating that I receive.
2. My performance appraisal takes into account the most important
parts of my job.
3. Information that I receive about my performance usually comes too
late for it to be of any use to me.
4. My performance rating presents a fair and accurate picture of my
actual job performance.
5. My superior considers the performance appraisal of his or her
subordinates to be an important part of his/her duties.
6. This organization considers performance appraisal to be an
important part of a supervisor's duty.
7. All in all, current merit pay provisions encourage me to perform my
8. If I perform especially well on my present job it is likely that I would
then receive a cash reward or unscheduled pay increase.
9. I would probably work harder on my job performance if I thought I
would then receive a cash reward or unscheduled pay increase.
10. All in all, I feel that the current performance appraisal process is
11. In your job how often do you receive feedback from your
supervisor that helps you improve your performance?
12. In your job how often do you receive feedback from your
supervisor for good performance?
13. In your opinion, how much did you last performance appraisal
help you to:
a) Assess your strengths and weaknesses in performing your job?
b) Establish a plan for your training and development?
c) Receive needed training?
d) Determine your contribution to the organization?
e) Make you better able to deliver service to the public?
Cummings. L. & Schwab, D. (1973). Performance in organizations: Determinants and appraisal. Glenview, IL: Scott, Foresman & Co. Daley, D. (1988). Performance appraisal and organizational success: Public employee perceptions in an MBO-based appraisal system. Review of Public Personnel Administration, 9 (1), 17-27. Daley, D. (1987). Meritpay enters with a whimper: The initial federal civil service reform experience. Review of Public Personnel Administration, 7 (2), 72-79. Daley, C. & Madsen, A. (1980). How to evaluate people in business, the track record method of making correct judgments. New York: McGraw-Hill. Florida Department of Administration. (1989). Rules of the career service system (Chapter 22A-9.02) Tallahassee, FL: Author. Gabris, G. (1986). Can merit pay systems avoid creating discord between supervisors and subordinates?--Another uneasy look at performance appraisal. Review of Public Personnel Administration, 7 (1), 70-89. Glueck, W. (1982). Personnel: A diagnostic approach. Dallas, TX Business Publications. Halachmi, A. & Holzer, M. (1987). Merit pay, performance targeting and productivity. Review of Public Personnel Administration, 7 (2), 80-91. Hays, S. & Tyer, C. (1980). Human resource management: The missing link. International Journal of Public Administration, 2 (3), 297-330. Kane, J. & Lawler, E.E. III. (1978). Methods of peer assessment. Psychological Bulletin, 85, 555-586. Kellogg, M. (1975). What to do about performance appraisal. New York: AMACOM. Lee, R.D. (1987). Public personnel systems. Rockville, MA: Aspen Publishers. Locher, A. and Teel, K. (1977). Performance appraisal: A survey of current practices. Personnel Journal, 56, 245-254. Lovrich, N. (1982). Assessing the performance of individual on the job. In Rabin, J., Vocino, T., Hildreth, W.B., & Miller, G. (Eds.), Handbook on public personnel administration and labor relations (pp. 250-275). New York: Marcel Dekker. McGregor, D. (1957). An uneasy look at performance appraisal. Harvard Business Review, 35, 89-94. Mager, R. and Pipe, P. (1970). Analyzing performance problems. Belmont, CA: Fearon Pitman. Morrisey, G. (1983). Performance appraisals in the public sector. Reading, MA: Addison-Wesley. Nigro, L. (1982). CSRA performance appraisals and merit pay: Growing uncertainty in the federal workforce. Public Personnel Management, 11, 371-375. Office of Personnel Management. (1981). Performance standards handbook: A reference for managers and supervisors. Washington, DC: U.S. Government Printing Office, pp. 1-15. Patten, T. (1982). A manager's guide to performance appraisal. New York: Free Press. Perry, J. (1986). Merit pay systems in the public sector: The case for a failure of theory. Review of Public Personnel Administration, 7 (1), 57-69. Pulakos, E. & Wexley, K. (1983). The relationship among perceputal similarity, sex and performance rating in manager-subordinate dyads. Academy of Management Journal, 26, 129-139. Rich, W. (1989). Appraising employee performance. In Perry, J. (Ed.), Handbook of public administration (pp. 388-400). San Francisco: Jossey-Bass. Sherwood, F. (1982/83). Wrong assumptions, wrong strategies. The Bureaucrat, 11, 22-27. Sherwood, F. and Wechsler, B. (1986). The |hadacol' of the eighties: Paying senior public managers for performance. Review of Public Personnel Administration, 7 (1), 27-41. Siegel, G. (1987). The jury is still out on merit pay in government. Review of Public Personnel Administration, 7 (3), 3-15. Stewart, V. and Stewart, A. (1977). Practical performance appraisal. London: Gower Press. Thayer, F. (1972). Productivity: Taylorism revisited (round three). Public Administration Review, 32, 833-840. Thayer, F. (1987). Performance appraisal and merit pay systems: The disasters multiply. Review of Public Personnel Administration, 7 (2), 36-53. Tyer, C. (1982). Employee performance appraisal in American state governments. Public Personnel Management, 11, 199-212. The United States General Accounting Office. (1989). Managing human resources: Greater OPM leadership needed to address critical challenges. Washington, DC: U.S. Government Printing Office, pp. 371-375. Danny L. Balfour is Assistant Professor of Urban Studies at the University of Akron. He is also a Research Fellow in the Center for Urban Studies and is a member of the Advisory Board to the Ohio Commission on the Public Service. He received his Ph.D. degree in Public Administration from The Florida State University.…