Desperately Seeking Management: Understanding Management Quality and Its Impact on Government Performance Outcomes under the Clean Air Act
Heckman, Alexander C., Journal of Public Administration Research and Theory
THE CHALLENGE OF MEASURING MANAGEMENT
Public management studies consistently find that measures of management quality and the use of certain management practices are positively related to government performance. However, the results of these studies often do not provide useful, generalizable, guidance about what practitioners should do to improve management practice. Unfortunately, management quality measures are frequently based on the subjective opinions of employees within the organizations studied or they have been too narrowly defined for a particular organizational or programmatic context to be of general applicability (Brudney, O'Toole, and Rainey 2000; Donahue 2004; Meier and O'Toole 2002; Nicholson-Crotty and O'Toole 2004; O'Toole, Meier, and Nicholson-Crotty 2005; Selden and Sowa 2004). Additionally, management measures used in the literature are often opaque as to what constitutes good management. For instance, Meier and O'Toole (2002) used the residual from a regression model of a superintendent's salary as a measure of managerial quality and found that good management had a positive impact on student learning outcomes. Although findings from such studies may support the notion that good management produces good performance, they shed little light on general steps for improving public management practice and policy implementation outcomes.
The lack of concrete, yet general, measures of management quality represents a major obstacle to producing pragmatic insights about the public management-government performance relationship (Boyne 2003, 2004; Boyne et al. 2005). Although Meier and O'Toole (2002) have stated that this measurement challenge is "intractable," developing more useful, general measures of management quality is essential if public administration research is to produce practical knowledge for improving public management practice that results in better government performance.
A significant exception to the lack of general and concrete measures of management quality is the management grades assigned to the 50 states in 1999, 2001, 2005, and 2008 by the Government Performance Project (GPP), which is currently managed by the Pew Center on the States. The GPP management capacity measure provides detailed, general, criteria that clearly define good management practice. The GPP grades represent the best, criteria-based, multidimensional measure that is regularly applied to a large number of governments in the United States (Borins 2005; Ingraham, Joyce, and Kneedler 2003). Studies using the GPP suggest that it is a promising way to operationalize the concept of management quality in a general way that provides practical insights into what constitutes good management practice (Burke and Wright 2002; Coggburn and Schneider 2003; Donahue, Selden, and Ingraham 2000; Knack 2002).
This study uses the GPP management measure to test management's impact on environmental outcomes--a wholly different context than past studies using the GPP. The specific questions addressed in the study are:
1) Does state management quality impact air pollution control outcomes?
2) What is the impact of management quality on air pollution control outcomes relative to other state-level factors related to problem severity, resources, and the political environment?
The study incorporates design elements and data that make it well-suited for answering these questions. The important study elements include:
* Model Specification: The model specification was developed using the Mazmanian-Sabatier implementation model (MSIM) illustrated in Figure 1, which is a well-established policy implementation model.
* Outcome Measures: The analysis uses two different measures of performance outcomes including a unique outcome measure that estimates reductions in air pollution emissions due to national Clean Air Act (CAA). This second measure better reflects the outcomes sought by policy makers and achieved by policy implementers, than similar studies using only aggregate emission outcome measures (e. …