Measuring Return-on-Investment in Government: Issues and Procedures

Chmielewski, Todd L., Phillips, Jack J., Public Personnel Management

Much has happened in the human resources (HR) world since the 1980s, when trends such as total quality management (TQM) led many organizations to shift toward a performance-oriented approach to business practices. (1) The TQM movement spread rapidly throughout both public and private sectors, reinforced by national quality awards (e.g., the Baldrige National Quality Award). TQM was credited for turning many companies around, increasing market share and growth rates, as well as producing top-quality products while increasing profitability. In the early 1990s, TQM was followed by re-engineering/restructuring trends and drastic downsizing within the private sector. (2) The advent of TQM and similar programs of organizational change, not only transformed traditional corporate culture, but also foreshadowed a great and positive change in the way the public sector did business as well.

In 1993, the Vice President's National Performance Review--now the National Partnership for Reinventing Government (NPR)--mirrored this movement with the ultimate goal of creating a government that worked better and cost less. At the same time, an Executive Order on Customer Service Standards was issued, which reinforced the new focus on quality in performance and customer service. Notwithstanding, Congress passed the Government Performance and Results Act of 1993 (GPRA), which required government agencies to submit strategic plans and performance measures to justify their budgets. Such developments have resulted in an increased demand for government agencies to become more accountable and measure performance. The trend toward improving the way the government does business does not seem to be changing in the near future. In particular, much has been written about the need for the HR function to become more accountable, create better HR practices and tools, and measure its contribution at all organizational levels. (3)

The federal government has been creative and diligent in its evaluation efforts, despite facing persistent budgetary concerns. Under the auspices of the U.S. Office of Personnel Management, several research and development efforts have been undertaken to assist various government agencies to develop procedures and models for evaluating their HR function. (4) This work has provided a systematic approach for formulating and implementing an overall evaluation plan. One key method has been to measure return-on-investment (ROI) to assess the value of the HR function.

The application of the ROI process is a particularly useful HR tool in the federal government, since many of its agencies are mission oriented, but their organizational goals are not necessarily captured as cost savings or profit. Many agencies' strategic plans, for instance, include specific goals to increase customer service, develop the professional skills of its workforce, reduce costs, as well as improve efficiency. Documenting and reporting HR benefits through cost-benefit analysis methods, such as ROI, can help an agency meet its goals for positive change. Unfortunately, few agencies have embarked in a systematic evaluation of their HR function and its organizational impact. This article examines the basic issues concerning ROI and its applicability in the federal government.

The Return-on-Investment Process

Perhaps the best-known framework for evaluating areas of HR function (e.g., training) is Kirkpatrick's Four-Level Evaluation model (5). Kirkpatrick's approach recognizes value in: (1) the participant's reactions to the program, (2) evaluating learning, (3) measuring behavioral improvements on the job, and (4) monitoring organizational improvement, such as cost savings, work output changes, and quality changes. The classification of evaluation measures as different levels is both helpful and instructive in understanding how to best capture program impact. Although Kirkpatrick's model is widely used, research (6) has revealed some shortcomings, including relatively weak systematic evaluation of organizational impact and transfer of learning. As such, other approaches have attempted to modify and add to this basic framework. One of these models, the Phillips Five Level ROI Framework, (7) combines the best features of Kirkpatrick's approach and formulates procedures for measuring investment (cost incurred) as well as indicators of return (savings and cost avoided).

The Phillips ROI model adds a fifth level to the four levels of evaluation developed by Kirkpatrick to measure success in areas of HR function. (8) Table 1 summarizes the five-level framework used in the calculation of ROI. The first three levels of evaluation data are used to establish a chain of evidence which determines the effectiveness of the HR function (e.g., managerial training, competency-based HR Management, or career-banding) during and following the actual program implementation. The fourth level is a measure of the direct organizational impact of the HR program. At Level 5--the final and ultimate level of evaluation--the ROI measurement compares the monetary benefits from the program with the program costs. One advantage of the Phillips approach is that the process allows the conversion of both soft data (i.e., employee attitudes, organizational climate) and hard data (i.e., work output, time, costs, quality). While the ROI can be represented in several ways, it is usually presented as a percentage or cost-benefit ratio.

Although many HR program evaluations assess satisfaction of those affected by the program, few actually conduct evaluations at the ROI level. The reason stems mainly from the misconception that an ROI evaluation is a difficult and expensive process. (9) The process, however, is not …

Questia, a part of Gale, Cengage Learning. www.questia.com
Publication information: Article title: Measuring Return-on-Investment in Government: Issues and Procedures. Contributors: Chmielewski, Todd L. - Author, Phillips, Jack J. - Author. Journal title: Public Personnel Management. Volume: 31. Issue: 2 Publication date: Summer 2002. Page number: 225+. © 2009 International Personnel Management Association. COPYRIGHT 2002 Gale Group.

This material is protected by copyright and, with the exception of fair use, may not be further copied, distributed or transmitted in any form or by any means.