Taxpayers, elected officials, and current and prospective federal government employees all have a vested interest in understanding the level of performance of federal agencies. Knowing how government is performing reinforces or weakens trust in government and in the social contract and ultimately shapes the relationship between government and the general public. The U.S. federal government itself and outside groups have developed organizational assessment tools used to measure the overall performance of agencies. Our purpose is to better understand whether these tools are sending similar or divergent signals with respect to administrative performance to the federal agencies. The use of these tools is not unique or new to the U.S. federal government. However, the scale of their current application to the federal bureaucracy is arguably one of their largest utilizations ever. These organizational assessment tools were developed for different purposes including accountability, transparency, oversight, and political and coercive purposes. What they have in common are combined scores that are used as aggregate assessments of agencies. The rankings and ratings that are generated describe the overall performance of agencies in a way that is easy for the general public to understand and use.
In this paper, we look at two different organizational assessment tools--scorecards and report cards. Scorecards are completed by an organization with respect to their own organization and are usually for internal consumption. Scorecards are now used by public organizations both in the United States and abroad (Niven 2003; McIntyre 1994; Kaplan and Norton 1996; GAO 2006). In contrast, report cards are tools completed by one or more organizations to evaluate and rank multiple external organizations. Governments both undertake and are the subject of report cards (Coe & Brunet, 2006).
In this study, we compare how different entities measure the overall performance of the same government agency using aggregate measurement tools. We look at relationships between an internal organizational assessment tool of the federal government, the Office of Management and Budget's (OMB) scorecard; and two external report cards, the Best Places to Work rankings and the Mercatus Center's Annual Performance Report. We are interested in finding out if they are sending similar or divergent signals to the agencies as well as to see if they are driving administrative behavior in consistent directions. We draw on the work of Jacobs, Goddard & Smith (2006) and the discussions of Kravchuk & Shack (1996) on the complexity of aggregate performance measurement as the basis for examining how these tools measure agencies.
It is important to note that neither the OMB scorecard nor the external report cards purport to measure outcomes or outputs. Consequently, "agency performance" and similar terms in this analysis refer to performance on the scorecard and report cards, not performance in terms of outputs and outcomes. How overall performance, as measured by the scorecard and report cards correlate with each other is important in determining whether these tools applied to federal agencies tend to reinforce or undercut one another in driving administrative activity. If the organizational assessment tools undercut one another, they cannot support coherent strategies for improving federal administrative performance.
The purpose of our study is not to evaluate whether these organizational assessment tools as aggregate measurement mechanisms accurately measure performance. Instead, our study is driven by the question of whether different aggregate assessment tools are sending similar or divergent messages about general performance to the federal agencies. Although there is general agreement that federal administration should be cost-effective, it is also widely recognized that "the bureaucracy problem" requires trading off economy and efficiency with equally important values such as accountability, procedural due process, and responsiveness (Wilson, 1967). …