This article is part of a growing body of research evaluating the public participation approaches and implementation efforts of governmental agencies. It compares the characteristics and performance of citizen advisory boards established by the U.S. Department of Energy's Environmental Restoration and Waste Management Program (DOE/EM) and by the U.S. Department of Defense's Army (DoD/Army). The advisory boards comprised the major component of the agencies' public participation programs that were undertaken in the early 1990s to help them plan and implement the cleanup of their contaminated installations. DOE/EM established Site Specific Advisory Boards (SSABs) and DoD established Restoration Advisory Boards (RABs).
Following this introduction, we briefly outline the methodologies used in our studies. The Acceptability Diamond is presented next, which identifies five dimensions, or objectives, of agency-public interactions and public participation programs. This evaluative framework, developed through extensive fieldwork on public-federal agency interaction, is used as the basis for describing and comparing the performance of the citizen advisory boards. We then summarize our observations about the performance of the DOE and Army boards on each of the five dimensions of the Acceptability Diamond, while the following section discusses the development of the DOE and Army advisory boards and highlights differences in the institutional context that affected the boards' performance. Finally, we summarize our conclusions about the factors that contributed to board effectiveness and points to the broader value of the Acceptability Diamond as a useful approach to evaluation and as a framework for agency managers in designing and implementing public participation programs that will enhance their ability to work effectively with local communities.
This article draws on research we have previously undertaken on public participation programs established in the 1990s by the U.S. Department of Energy, Office of Environmental Management (DOE/EM) and the U.S. Army.
Coincidental with our early work with the DOE/EM on their public participation policy, we conducted a qualitative study of U.S. Army participation programs that assessed public perceptions of the risks of alternatives for disposing of the chemical weapons stored at eight locations across the United States. The framework for the Acceptability Diamond, which emerged from this work (Bradbury, Branch, Heerwagen, & Liebow, 1994), highlighted the relationship between agency-public interactions and public participation efforts, and agency and program acceptability.
We completed an evaluation of the public participation training programs developed for DOE staff that addressed, among other issues, the importance of clarifying the decision process (Branch, Bradbury, & Silbernagel, 1995). We then conducted two surveys of 11 of the 12 SSABs during their early formative stages (Bradbury, Branch, & Zalesny, 1996a,b, 1997a,b). (1) The same survey instrument, based on a set of goals and performance criteria jointly established by the researchers, DOE staff, and board representatives was used in both surveys: Citizen participants from all boards were surveyed, together with all ex-officio Board members (DOE and regulatory members) and DOE Headquarters and Field Office staff who were closely associated with the boards.
Two subsequent, qualitative studies of DOE's public participation programs provided a more in-depth and longer-term view of DOE's public participation efforts (Bradbury & Branch, 1999; Bradbury, Branch, & Malone, 2003). (2) The 1999 study, which was based on site visits and interviews with citizen and staff members of nine SSABs, identified and analyzed six factors affecting the boards' performance. The 2003 study was broader in nature. This study evaluated the scope and effectiveness of the overall public participation program, including SSABs, at seven sites. The Acceptability Diamond was used as the framework for evaluating whether the public participation programs were addressing the full scope of community issues related to DOE cleanup plans and programs. The methodology included a review and summary of information available in DOE documents and web sites; observation of the sites' public participation activities; and interviews with of the public, SSAB members' and DOE staff. As for the 1999 study, interviews were conducted as informal discussions, following a general agenda of topics.
In 2005, we completed a second study of Army public participation, an evaluation of the effectiveness of seven Army RABs (Bradbury, Branch, Santos, & Chess, Forthcoming). (3) The overall goal was similar to that of the DOE research--specifically, to provide guidance to the Army and to board members on ways to enhance the boards' performance. The boards were selected for study from an initial pool of 20 RABs, following the collection of background data and telephone discussions with RAB Co-Chairs. The evaluation methodology included visits and observation of each RAB meeting and in-depth interviews with RAB members (community, installation staff and regulators) and members of the public who regularly attended the meetings. Interviewees were selected to represent various constituencies and points of view.
Conceptual Framework: The Acceptability Diamond--Five Critical Dimensions of Agency-Public Interaction
The literature on public participation evaluation remains largely segmented by discipline and topical or operational focus (e.g., health; community services, planning, and development; natural resource, environmental, and risk management), with particular researchers in each segment focusing on different aspects of the complex public participation process. However, this article contributes to the growing effort to create linkages across these subdivisions, and to integrate the separate studies into a broader evaluation and public involvement framework.
Our studies of DOE and Army's public participation programs have generally been undertaken as formative evaluations designed to provide information that would help both the agency sponsors and the citizen participants improve the effectiveness of their public involvement efforts and the acceptability of policies and program operation. Indeed, our research has found that both agency representatives and stakeholder participants see their public involvement programs as the principal avenue for shaping and improving the overall relationship between the agency and the affected communities and stakeholders on a whole range of difficult issues. To use Daniels and Walker's (1996) term, public involvement is being called upon to achieve "situation improvement" rather than solution of a single, specific problem.
Informed by our long-term interaction with the public involvement efforts of the DOE (more or less continuously for over 20 years) and the DoD (more episodically for over 15 years), we have arrived at a perspective that is more organizational and managerial than many public involvement researchers. From this perspective, an agency's public involvement approach--and the specific manifestations of that approach--are appropriately seen as extensions of its overall program management and operations that intersect more or less completely and effectively with the many decisions the agencies and their on-site managers make every day that affect the local communities and stakeholders. Although a particular public involvement "campaign" may be established to serve a particular decision-making episode (e.g., an Environmental Impact Statement for a siting or program decision), the historical--and future--presence of the agency in the community alters the time horizon and scope of the federal-public interaction process. In our research, stakeholders and agency representatives consistently identified agency commitment to, and performance on, five dimensions of agency-public interaction--demonstrated through policies, funding, and day-to-day behavior--as key to an agency's ability to achieve community and public acceptance of their plans, decisions, and program (Bradbury et al., 1994). Figure 1 illustrates these five dimensions. Assuring that these outcomes are achieved is seen to be an agency managerial responsibility.
In this regard, our approach focuses on outcome dimensions rather than on specific mechanisms, processes, or procedures, or on stakeholder satisfaction with these specific elements, although we recognize that these are legitimate and important foci for evaluating public involvement initiatives and have addressed them in our previous work (Butterfoss, 2006; Chess & Purcell, 1999; EPA [Environmental Protection Agency], 2001; Fiorino, 1990; Konisky & Beierle, 2001; Lynn & Busenberg, 1995; Rowe & Frewer, 2000; Steelman & Ascher, 1997; Tuler & Webler, 1999; Webler, 1995). It is more specific than, though compatible with, Beierle and Konisky's (2000, p. 588) three overall "aspirations for public involvement": (i) incorporating public values into decision making; (ii) resolving conflict among competing interests; and (iii) restoring a degree of trust in public agencies. As discussed further, many of the factors and criteria highlighted in other evaluative frameworks are important to achievement of the five objectives identified in this framework, although their role in that achievement is not specifically articulated or addressed. This includes "fairness and competence" (Murdock, Wiessner, & Sexton, 2005; Webler, 1995); deliberative and reasoned discourse (Reich, 1985; Roberts, 1997; Ryfe, 2005); a focus on the common good (Carr & Halvorsen, 2001; Poisner, 1996), access, representativeness, and equity (Beierle & Konisky, 2000; Rowe & Frewer, 2000; Van Slyke & Roch, 2004); commitment and willingness to engage (Burroughs, 1999; Murdock et al., 2005; Robertson & Tang, 1995; Sullivan & Transue, 1999); conflict resolution (Charnley & Engelbert, 2005; Rauschmayer & Wittmer, 2006); and adequate resources and structure (Rowe & Frewer, 2000; Steelman & Ascher, 1997).
[FIGURE 1 OMITTED]
The research leading to the development of the Acceptability Diamond indicated that neither individual decision processes nor the standard risk framework provided an adequate approach to public acceptability and decision legitimacy and that a more complex and nuanced framework was necessary. Rather, as shown by the Acceptability Diamond, the decision process is set firmly in its place as only one (although important) dimension of agency-public interactions and public participation programs that may have many different goals and types of activities. Similarly, the standard risk framework becomes one of many frameworks that people bring to the interactions in a public participation program and to their assessment of the acceptability of a particular activity or program (Bradbury, 1989, 1998; Krimsky & Golding, 1992; Sabatier & Jenkins-Smith, 1999; Slovic, 1993; Sullivan & Transue, 1999; Thurston et al., 2006). Participants apply a complex and dynamic combination of trust and skepticism in their decisions (Parkins & Mitchell, 2005; Slovic, 1993; Yang, 2005). The Acceptability Diamond underscores the community context in which government agency programs operate and the need for the agency to demonstrate understanding and consideration of community interests, as well as commitment to its mission, throughout the entire life cycle of the facility's existence. This broader, more imbedded perspective is reflected in the Federal Facilities Environmental Restoration (FFER) Dialogue Committee's recommendation to establish advisory committees. It also reflects appreciation, as Cowie and Borrett (2005) and Ostrom (1986) emphasize, that participation is structured by the "rule sets" that establish the position, scope, authority, information, and boundary rules that govern the specific institutional arrangements established for that engagement.
An important feature of the Acceptability Diamond is its indivisibility. That is, our research reinforced others' findings (Creighton, 1992; Yang, 2006) that people experience their interaction with an agency (or company) and with public participation programs as all-of-a-piece, not as separate pieces. An important finding of our research is that the five dimensions are closely interrelated, and though ranked differently in primacy by different stakeholders, are generally all seen as essential to a successful program. A well-designed decision process, following the many precepts laid down in the literature concerning openness, access, fairness, and competence is likely to fail if other dimensions are weak: for example, if relationships are characterized by domination on the one hand and mistrust on the other; if the substantive issues are understood differently by agency staff and community members, or if one group will experience disproportionate, severe, uncompensated impacts; and/or if institutional safeguards are considered inadequate by the participants. It is not enough to get one, two, or three of the facets right if the fourth or fifth is wrong. There is spillover from any one to all the others. Our research confirmed Habermas' observation that public participants "relate simultaneously to something in the objective, the social, and the subjective worlds, even when they thematically stress only one of the three components in their utterance" (Habermas, 1987 , p. 220).
Open Disclosure of Information
Open disclosure of information is widely recognized in the literature as a necessary, though not sufficient, component of an agency's demonstration of commitment to agency-public interactions (Beierle & Konisky, 2000; Burroughs, 1999; Charnley & Engelbert, 2005; Creighton, 1992; Rauschmayer & Wittmer, 2006; Sheppard, 2005; Tuler & Webler, 1999). Our research emphasized the central role played by timely disclosure and efficient distribution of pertinent information. Open disclosure underlies each dimension of the Acceptability Diamond. It is a prerequisite for community members' ability both to understand and confirm (or deny) the agency's identification of the issues and to understand and identify issues of importance to themselves. It is a precondition for community involvement in, and ability to influence, the decision-making process. It is a key ingredient of respectful and communicative relationships and relationship building, and indicates that the agency is indeed operating in a credible, open, and responsive manner. And, finally, it forms the basis for accountability--without accurate information, the public cannot be confident that cleanup operations are actually implemented as planned, that commitments are being kept, and that their concerns are being truly addressed.
Open disclosure has a fundamental relationship with the other four dimensions of the Acceptability Diamond. Our research found that any indication that an agency was not open in providing information, was not making an effort to present information clearly and understandably, was imposing new restrictions on the provision of information, or was intentionally refusing to release information that had been routinely provided in the past affected stakeholders' assessment of agency commitment to the interaction and the integrity of the process.
Open disclosure has both substantive and symbolic significance and is typically covered by both policy and regulations. However, not all information pertaining to cleanup decisions is covered by legal requirements and agencies therefore retain discretion over what information is made available and when it is disclosed. Information disclosure may often, therefore, be a basis for tension between an agency and its stakeholders.
Identifying, Acknowledging, and Addressing Substantive Issues
A central reason for an agency to undertake (or be required to sponsor) a public involvement or deliberation process is anticipation that the agency, regulators, and/or the public will differ in viewpoints, values, and interests in ways that are important to identify and address (Beierle & Konisky, 2000; Lawrence & Deagen, 2001; Ryfe, 2005; Vroom & Jago, 1988; Vroom & Yetton, 1973). A key motivation for stakeholders to participate in public involvement activities is to protect their own and their community's interests and values (Stephan, 2005; Tuler & Webler, 1999). If the public involvement process is seen as ineffective or as thwarting this function by failing to ensure that stakeholders have the information and forums they need to identify and act effectively to understand and protect their own and their community interests, the stakeholders are likely to participate cynically, or drop out of the process and focus their efforts elsewhere (Creighton, 1992; Rauschmayer & Wittmer, 2006; Roberts, 1997; Roberts, 2004; Susskind & Cruickshank, 1987).
Judgments about acceptability will obviously be influenced by how one's substantive interests are affected, both in the immediate instance and over the long term. Although public involvement processes generally address future policies and actions, the length and breadth of the agency-community interactions in our studies meant that many of the participants had first-hand experience with the consequences of both previous decisions and of ongoing management and operational behavior.
Important criteria for effective public involvement frequently identified in the literature are a forum, structure, schedule of interaction, and mix of participants that enable issues important to the community to be acknowledged and addressed by the agency, advocates for community and stakeholder interests to be heard, and participants to engage in genuine deliberation about alternatives that take into consideration community priorities and interests, as well as agency mission. Although typically collapsed into discussion of the decision-making process, and seldom explicitly extended to consideration of the nature and distribution of actual, substantive consequences, the ability to place issues on the agenda and to influence how they are framed is widely recognized as an important feature of effective public involvement (Creighton, 1992; Fiorino, 1990; Peterman, 2004; Poisner, 1996; Tuler & Webler, 1999; Webler, 1995; Yankelovich, 1991).
The ability to resolve or otherwise address conflicts among stakeholders or between stakeholders and the agency through a deliberative and collaborative process is also recognized as a sign of public involvement success. Characteristics of an effective process include enabling the interested public to understand how the community and different public groups will be affected by the proposed plans, including how the various technologies were selected, schedules established, and programs designed. It also includes protecting the community and less powerful stakeholders from disproportionate adverse impacts.
Establishing Clear, Fair, and Open Decision-Making Processes
Establishing a clear, fair, and open decision-making process is perhaps the most universally recognized objective and criterion for public involvement. Our research has found that understanding the decision-making process of the agency and articulating the decisions that are to be made pose difficult challenges for both program managers and community residents (Branch et al., 1995). In our research, we found that it was often not clear to anyone what decisions were being considered, who had the responsibility and authority to make which decisions, what decision method or rule was to be used, or what information was serving as the basis for a decision. Without this information, it is difficult for agency representatives or public involvement specialists to determine--or demonstrate--that the public has a genuine opportunity for involvement in, and influence on, decisions. Stakeholders care about the transparency, quality, and accessibility of an agency's decision-making process when they perceive that their well-being will be affected by those decisions. These attributes of the decision making process, along with the ability to influence the decisions being made, are widely recognized as among the important requirements for effective public involvement (Creighton, 1992; EPA, 2001; Peterman, 2004; Sheppard, 2005; Thurston et al., 2005; Tuler & Webler, 1999; Webler, 1995).
Our research has also found that an important function of agency-public interactions and public participation programs is to improve the clarity and quality of the decision-making process. This includes ensuring that all interested stakeholders are aware of the decisions being considered and know who is responsible for what aspects of the decision-making process. It requires ensuring that stakeholders have access to the information they need to determine their interests in the process and develop opinions about alternatives, as well as to have the ability to influence the process by making their interests, preferences, and arguments known to the analysts and decision makers before decisions are made.
Building Relationships that Convey Respect, Recognition, and Consideration of Others' Interests
In processes that involve people, relationships always matter: People have both needs and expectations from relationships, no matter how personal or impersonal they are. An agency's presence in a community inevitably leads to the creation of agency-public relationships (Anson, Bostrom, & Wynne, 1995; Ryfe, 2005; Sullivan & Transue, 1999). Our research found the development and maintenance of relationships that conveyed mutual respect and recognition and demonstrated understanding and consideration of one another's interests to be an important outcome of effective and successful agency-public interactions. Relationships with these characteristics both resulted from and facilitated communication and access, enhanced the perceived legitimacy of the process and the deliberations, and built social capital (Buchan, Johnson, & Croson, 2006; Sullivan & Transue, 1999). Dishonesty or disingenuousness, and poor management of the interaction processes could reverse painstakingly established relationships (Edelenbos & Klijn, 2005; Sullivan & Transue, 1999). When addressed in the public involvement literature, relationship building is often characterized as building trust (Smith & McDounough, 2001; Parkins & Mitchell, 2005; Roberts, 2004; Yang, 2005, 2006), but our research found that citizen participants in these processes were reluctant to define either the purpose or the results in these terms.
In addition to the building of relationships through face-to-face interaction and the deliberative process (Buchan et al., 2006; Rauschmayer & Wittmer, 2006; Roberts, 1997; Roberts, 2004; Yankelovich, 1991), relationships are also built through day-to-day behavior that demonstrates awareness and consideration of one another's rights and interests. In our research, we found that community members judge whether the agency has demonstrated in previous actions that the well-being of the community is a factor in its decisions and try to determine whether it will be a factor influencing their future decisions. (4) Reflecting the interrelated nature of the key factors, responsiveness and openness in providing information were identified as among the most important contributors to good relationships. In our research, relationships that gave each party, including the public, "standing," i.e., members of the public and representatives of the agency were affirmed as individuals and treated with respect were identified as an important goal of the public involvement process. With such standing, people expected that all participants would be encouraged to adhere to certain norms that are assumed to be valid, such as honesty and openness (Mesch & Talmud, 2006).
Providing Mechanisms to Assure Accountability
"Accountability refers to a relationship between those entrusted with the specific tasks (stewards) and those having power of review (reviewers). The stewards make efforts to fulfill the tasks specified by the reviewers, who are concerned with how and to what extent expected tasks are fulfilled.... Accountability is a guiding principle in administrative decision making and service delivery" (Wang, 2002, p. 351; see also Browder, 1971). Our research has found that community residents are generally sensitive to disparities in power and resources between the community and the federal agency, the federal agency and the regulators, and the agency and the individual or social group. They are therefore also sensitive to the existence, clarity, and effectiveness of mechanisms to assure accountability that would protect the community once agency decisions were made. Federal sovereignty and agency dependence upon elections and the annual funding cycle to maintain the policies and funding necessary to carry out decisions and fulfill commitments add to this sensitivity. Agency-public interactions and public participation programs are generally perceived by both agency representatives and the public to provide a measure of accountability by providing stakeholders the information they need to monitor performance and a forum for bringing issues to the attention of the agency, regulators, and the public. This type of stakeholder oversight, which provides "political accountability" (Romzek & Dubnick, 1994), has been demonstrated by Van Slyke and Roch (2004), Roberts (1997, 2004), and Bacharach and Lawler (1981) to facilitate consensual and cooperative behavior by reducing the power differential. In our research, the active participation of regulators in the interaction process increased all parties' sense of agency accountability.
Comparisons of SSAB and RAB Performance on the Dimensions of the Acceptability Diamond
A review of the data gathered for the set of studies we conducted on the SSABs and RABs shows distinct differences in performance on each of the five dimensions of the Acceptability Diamond, even taking variability among boards into account. These differences are described below.
Two caveats should be noted. First, the SSAB and RAB studies were designed and conducted separately, not as a comparative study of the differences on performance. Second, the comparisons between boards are valid only for the boards studied--some of the DOE SSABs that experienced problems, though included in the authors' early studies in 1996 and 1997, had been disbanded by the time of the more in-depth study undertaken in 2002.
Information Disclosure: Differences in Patterns of Information Distribution and Use
As emphasized in the previous section, open disclosure underlies each dimension of the Acceptability Diamond. The discussion in this section compares the DOE/EM and Army performance with respect to information disclosure in terms of patterns of distribution and use and identifies several important differences between DOE and Army information disclosure patterns.
Significantly (although with a few notable exceptions), access to information was not a persistent issue at DOE boards. Reflecting the centrality of the environmental cleanup program for DOE in the early 1990s, DOE had established multiple, agency-wide channels of information flow that extended beyond those of the EM program. There were often multiple DOE programs and associated public participation programs (including those not sponsored by EM) at any one site. Typically, DOE-wide information channels included a DOE public reading room (or more than one), newsletters, fact sheets or other publications, and a toll-free telephone number, of which SSAB members could take advantage. Most DOE sites also maintained and used an extensive mailing list for distributing information both to advisory board members and to the broader public. Of particular importance, was the highly visible, new DOE agency-wide policy and public commitment to openness and the declassification initiative that was launched by Secretary of Energy just as the SSABs were being established.
Over time, all the DOE sites developed EM and SSAB web sites through which SSAB members could access site information, documents, and meeting notices, as well as records of their board meetings and subcommittee meetings. Some sites were more proactive than others. Most, though not all, of the latter web sites were frequently updated, comprehensive, and user-friendly. Often, board members reported that they did not use, or only selectively used the DOE formal information channels, although they regarded them as an important signal of DOE's commitment to open disclosure. Many SSAB members reported that they relied on comprehensive information packets prepared by DOE/EM and its support staff for each board or subcommittee meeting. In addition, many emphasized that they viewed the information as a starting point and used the meetings as the primary opportunity for soliciting information in a detailed question-and-answer response with the site's program managers who typically participated in meetings.
Review and discussion of DOE and contractor technical reports was a primary focus of the SSABs, which had a central role in helping DOE and the regulators establish cleanup priorities and strategies. These reports were generally distributed promptly to SSAB members, who, in a number of SSABs, received documents and information at the same time as the regulators. Although there was significant conflict at several sites about the scope of issues that fell within the purview of the SSABs, by the late 1990s, most boards expected to be consulted on key site issues and to have the opportunity to comment on reports prior to their public release. It was only after a change in administration and at the time of the last evaluation of DOE/EM public participation programs (Bradbury, Branch, & Malone, 2003) that this way of doing business began to show signs of change. At this time, many interviewees expressed alarm about an EM Headquarters' directive which, contrary to previous practice, prohibited sites from releasing information and consulting with stakeholders as they were developing their Performance Management Plans concerning accelerated schedules for cleanup. Interviewees (both SSAB members and DOE and regulator staff) were concerned that this was an indication that DOE was changing its established policy of open access to information.
An important channel of information for many SSABs was the information exchanged among SSABs from different DOE sites. Initially, DOE/EM had been reluctant to encourage cross-SSAB interchange. However, after several years, DOE/EM Headquarters initiated--and provided financial support for representatives from each SSAB to attend annual (or semiannual) national meetings and training workshops on technical issues of concern (e.g., groundwater contamination and transportation). The meetings came to be seen by both DOE and the SSABs as providing a valuable opportunity for SSAB members to exchange lessons learned, develop a broader understanding of participatory processes, and develop cross-site personal relationships that could be drawn on for future information exchange and learning.
By contrast, RAB members (with the notable exception of one of the installations studied) had only limited access to information about upcoming decisions and technical documents, and review and discussion of technical documents and issues played a minor role in deliberations of most of the RABs. With some notable exceptions, less robust information packages were prepared for RAB meetings than for the DOE SSAB meetings, and this information was less likely to be distributed to RAB than SSAB members in advance of the meetings. Unlike the DOE boards, only one RAB was routinely provided access to information before the general public and at the same time as the regulators. At many RABs, a large proportion of members interviewed indicated that they did not consider review of documents to be within their purview or scope of responsibility, and for this reason they did not want to receive copies of all the documents. However, most of the RAB installations studied had persistent difficulties implementing their modest agreed-upon document and information dissemination procedures. RAB community members were generally not notified of, or provided, technical documents as a matter of course; official materials were often available locally only in a public library (as compared with an agency-sponsored reading room where specialized assistance could be obtained). At one RAB, for example, only a single copy of each document to be reviewed was given to the subcommittee designated for this review, to be shared among all the members. At some RABs, documents were not provided unless specifically requested.
Although several of the RABs prepared verbatim records of the board meetings, which they relied heavily upon as records of statements and agreements, only two of the seven RABs included in our study maintained updated web sites that included detailed board minutes and only one of these had readily available, updated, and comprehensive information about the installation's cleanup program. Only one consistently maintained and used an extensive mailing list to notify the broader public of meetings and activities and to distribute information about cleanup across the installation.
At several RABs, limitations in document access and preparation and distribution of information were explicitly attributed to a lack of staff and monetary resources. A number of the DoD installations in our study were being closed and, as a result, fewer installation staff were available to participate in RAB meetings or prepare informational materials than at the DOE installations. In general, no on-site staff support was available to make and distribute copies of material and no reimbursement was provided for costs incurred by RAB members for copying or distributing information to other RAB members.
In addition, DoD did not sponsor any cross-RAB meetings or encourage cross-RAB information exchange, in part, perhaps because of the large number of installations and RABs that would have been involved, as well as a shortage of resources. Consequently, interaction among RAB members from different installations was limited and dependent on individual RAB-member initiative. Only one cross-RAB meeting was noted by those who were interviewed during the RAB study. Community members at RABs were also less likely to be affiliated with organizations interested in and familiar with cleanup issues, and many reported having fewer independent avenues for obtaining information. However, the listserv operated by the Center for Public Environmental Oversight was identified as a resource for several RABs and many also highlighted the benefit of input from the regulators. The regulators' expertise on cleanup issues and the complexities of the regulatory requirements were widely acknowledged as an important and independent source of knowledge on which citizen RAB members could draw in providing informed advice on cleanup decisions.
Overall, however, information was less readily available from their sponsoring agency for RABs than for SSABs. And, perhaps most significantly, RAB members were much more likely than SSAB members to cite access to timely information as problematic.
Substantive Issues: Differences in Framing, Prioritizing, and Addressing Issues
Getting issues on the agenda for decision making (and for public participation) and framing them in ways that reflect the public's interests are fundamental to effective public participation. As with information disclosure, we observed different levels of performance between DOE/EM and the Army on the substantive dimension of the Acceptability Diamond.
At least in the early days of the SSABs' development, (5) DOE's agency-wide policy of open disclosure of information reinforced the EM policy of engaging the public in influencing the way site cleanup issues were framed and prioritized. This policy was interpreted as meaning that key site decision makers were to participate directly in interactions with the public and the SSABs. This gave SSAB members both access to needed information and the opportunity to present their views to and closely question management. In addition, establishment of the SSABs as Federal Advisory Committee Act (FACA) boards, combined with an emphasis highlighting the value of consensus among stakeholders, reinforced DOE/EM's stated commitment to soliciting the public's help in identifying and addressing their issues. Not all SSABs were successful, however. An issue at a number of DOE sites was the exclusion of salient community issues from the agenda. At sites where remediation was not the sole DOE mission, issues of broader national scope (e.g., weapons production) or of generic management (e.g., personnel policies) that were beyond EM's scope of authority made it difficult for the SSAB to get the public's issues on the agenda. Examples of such sites are Sandia, Los Alamos, Paducah, and Pantex. However, where sites were closing and EM had management control at the site, there were fewer battles over the board's scope (e.g., Hanford, Rocky Flats, and Fernald) and a much-increased ability by the public to get their issues on the agenda. In addition, the management at some sites such as Idaho and Savannah River recognized the value of public support in their ability to achieve their mission and demonstrated flexibility by attempting to address a broad range of community issues.
With one or two exceptions, all of the RABs studied demonstrated only very limited success in framing and getting the community's interests on the agenda. Particular problems related to the board's scope arose at almost all RAB installations studied. Although the RABs were created to focus on the restoration cleanup program, the Army developed multiple programs and decision-making processes to address the complex requirements of base cleanup, reuse, and operation. These different mandates and funding sources resulted in a distinction between in-scope RAB activities (i.e., related to the Installation Restoration Program) and activities that were deemed to be out of scope because they derived from other DoD programs and funding sources. In some cases, the distinctions applied to different cleanup activities according to the DoD program or "color of money"; in other instances, scope constraints arose from the separation of cleanup decisions from decisions about the transfer of land or from cleanup associated with ongoing installation activities. This complex--and frequently legally based and inflexibly-interpreted--separation made it difficult for the Government Co-Chairs and the community members to address the full scope of cleanup issues they perceived to be affecting their community.
Additional factors that made it more difficult for RABs to contribute to the framing and issue resolution process as compared with SSABs were the absence of alternative public participation pathways, the difficulty of obtaining adequate information about Army plans, the infrequency of meetings, and the much less extensive use of subcommittees. At many of the RABS studied, therefore, members appeared to have concluded that they had to accede to DoD's definition of scope and power. (6)
Decision Making: Differences in the Role of the Public in the Decision-Making Process
Understanding and being able to influence an agency's decision making is central to the public participation process. Did SSAB and RAB members understand what the decision-making process was--what decisions were being made, by whom, and how they could influence that process? At almost all DOE/EM and Army sites, board members who were interviewed commented on the difficulty of understanding the decision-making process. However, there were several differences between the agencies both in terms of the clarity of their intent and their performance in allowing and encouraging public influence.
DOE/EM policy related to the public's role in decision making was very clear, specifically stating that "The public will have the opportunity to participate in the EM decision-making process for program planning, design and implementation." (U.S. DOE/EM, 1998). As with substantive issues, performance varied both among the boards and also over time. Especially during the early and middle years of their existence, most SSABs were credited as striving to achieve this outcome.
SSAB members with whom we spoke generally agreed that DOE/EM's performance in explaining the decision-making process had improved over time and attributed this to increased agency staff efforts and also to the public's effective use of the public participation process to question DOE persistently. There was also general agreement among SSAB members concerning the value and influence of consensus recommendations from a body specifically established to provide input from a diversity of public views. They reported that the SSABs' focus on developing formal, written recommendations, which were authorized under FACA, was influential because it put pressure upon the board members to strive for consensus and to address differences among themselves as well as with the agency's proposals. The requirement for DOE/EM to acknowledge the recommendations, take them into account, and provide a response to show how they had been implemented--or if not, to clarify why not--was seen as an important mechanism for increasing the SSAB's influence on agency decisions and on the agency's accountability to the board and the public. A number of RAB members expressed frustration over their inability to have this type of influence on, and feedback from, the Army's decision makers.
In addition to the value of formal recommendations, many of those involved with the SSABs pointed to the importance of informal communication and access to the decision makers at both the site and Headquarters levels and the associated value of the SSABs in enabling this type of communication. DOE/EM senior staff at the site typically attended all board meetings and participated actively in subcommittee meetings where detailed discussion and review occurred. These interactions facilitated the SSABs' opportunities for early input--such interactions enabled members to help shape the framing of the issues, as well as being a mechanism for obtaining feedback. Over time, most DOE/EM boards reported that they had succeeded in gaining access to information at an early stage; were provided "heads-up" alerts about upcoming issues, decisions, and schedules; and believed that they were able to obtain clarification, if needed, and to influence site decisions. Many SSAB members nevertheless expressed concern that decisions made at the lower, site level could be overturned by Headquarters. Of particular concern at the time of the last study, conducted in 2002, was that apparent progress in understanding and being able to contribute to decisions was being eroded by decreased site authority and increased centralization of decision making at Headquarters.
In contrast to DOE, the Army's policy on the role of the public in decision making was never clearly specified nor conveyed to the installation level. Members at all except one RAB unanimously agreed that their impact on the Army's environmental decision making was very limited and many believed that their input was not sought or wanted. Since the RABs were not chartered under FACA, they lacked the authority to offer and track formal recommendations--comments and suggestions at most RABs were made informally and individually, although at some RABs, this led to an emphasis on taking minutes "for the record" in order to provide a basis for follow-up and accountability.
At most of the RABs studied, information flow tended to be unidirectional, with presentations by the Army and questions from community members. Consultation, early access to information, and "heads-up" alerts were the norm at only one of the RABs studied and most agreed that decisions were made by Headquarters staff who were unknown to the board members and distant from the Army's on-site staff. In addition to being unable to offer group advice and recommendations, most RABs studied were unable to influence decisions informally. For example, most RAB meetings were much shorter and less frequent than the SSABs; the RABs lacked subcommittees to study issues in detail; and finally, the RABs suffered from the lack of on-site decision makers to attend and participate in board and subcommittee meetings. The decision-making processes of the programs closely related to cleanup were also complex and opaque, and were becoming more so as programs multiplied and changed, and new policies were introduced. For example, the procedures for early transfer of still-contaminated property (which was not within the RABs' scope, though frequently of concern to the community) were extremely complex and poorly understood, even by agency staff.
Overall, therefore, our studies indicated that, on this dimension, RAB performance was hindered by a number of factors that did not pose similar obstacles for the SSABs. Primary among them were the lack of clear policy guidance stating the Army's intent with regard to public influence on decision making and a subsequent lack of commitment by Army decision makers to encourage and accept public influence.
Relationships: Differences in Building Relationships of Mutual Recognition and Respect
Another important function of the public participation program is to create the public space where meaningful communication can take place and relationships of mutual recognition and respect can develop. This requires effort in establishing the forum, skill in managing small group interpersonal relationships, and a commitment by each party to listen and to talk. DOE and DoD each had established relationships with nearby communities many years before EM or the Army Restoration Programs initiated their public participation programs and the SSABs and RABs were necessarily dealing with the legacy of these past relationships. Both agencies created their advisory boards, in part to create a forum for building honest, respectful relationships that could, where necessary, overcome conflicts of the past.
DOE/EM, faced with outright opposition from many communities, recognized this need and took action to address it. This was particularly true at the DOE installations with a continuing production mission (Los Alamos, Savannah River, and Oak Ridge). At the majority (though not all) of the DOE installations included in our latest study, interviews with participants in the advisory board process indicated that the boards were quite successful in creating this forum and building relationships of mutual respect and recognition. In many cases this included the use of trained facilitators and support staff to assist agency and board member chairs. This was less true of the Army RABs, and it was also less true of the SSABs that were disbanded before our last study.
DOE, more than DoD, made a concerted effort at all its sites to identify and engage the stakeholders in site cleanup, and to undertake a visible and intentional process to recruit and select board members. In addition, DOE undertook a specific and sustained effort to ensure that its stakeholders, and particularly the SSAB members, had personal familiarity with and access to installation and headquarters decision makers, and that these decision makers participated in board meetings and/or discussions with board representatives. However, successful, mutually respectful relationships were not achieved at all DOE/EM installations. For example, at Pantex, Sandia, Los Alamos, and Monticello, an inability to manage small group dynamics and interpersonal tensions, and to reach agreement among community members and between community members and ex officio members about goals and approach kept the boards from moving beyond procedural discussions to substantive issues. At these SSABs, the participants generally did not gain or evidence recognition of the legitimacy of others' viewpoints and priorities. It is notable that some of these boards were among those that disbanded without really achieving their goals. At many of the SSABs that did succeed in achieving a shared sense of purpose, focused attention and resources were directed to the management of interpersonal interactions, frequently through the use of trained facilitators and skilled chairpersons.
In general, the Army installations in our study had less intense and contentious relationships with their neighboring communities when they initiated their advisory boards, although there were notable tensions in the Army's relationship with community groups at several installations. We found that the intensity of interaction on the Army RABs was lower than on most DOE SSABs, with the exception of the Aberdeen RAB, which met frequently and worked especially intensely. An important consequence of these differences in interaction was that RAB members were less likely than SSAB members to report that they knew one another well and that they had gained an in-depth understanding of and respect for one another's viewpoints, interests, and priorities. We found that a number of RABs were seen less as forums for discussion and exchange of viewpoints and priorities with the goal of reaching some common agreement, than as an opportunity to question the Army and contractors and raise issues and problems.
However, we also found that members of some RABs viewed the building of positive relationships and interactions with other board members as the most beneficial and important outcome of their participation on the board. In addition, RAB more than SSAB members were more likely to cite the relationships they had developed with the regulators to be of great value and to identify as a benefit, their ability to call and consult with the regulators between meetings.
Overall, therefore, the record of performance on the relationship dimension was mixed. On the one hand (although with some notable exceptions for SSABs that were disbanded), the SSABs appeared to be more successful in overcoming past hostilities and suspicion and creating a base of personal relationships that supported constructive dialogue and exchange. On the other hand, some RABs identified the value of relationships they had developed with other community members and almost all RABs identified the value and benefit of their relationships with the regulators.
Accountability: Differences in Clarity, Expectations, and Enforcement
Accountability is a complex concept, implying a system of agreed-upon responsibilities and commitments, transparency, and enforcement. A perceived lack of accountability can lead to distrust, opposition, and/or the imposition of overly conservative requirements. Public participation helps clarify and build these commitments, and provides the communication channels for stakeholders to verify that established mechanisms for providing accountability are in place and are being enforced.
The issue of accountability was salient to members of both the SSABs and the RABs, and most respondents reported that the advisory boards were having a positive impact on agency accountability. To a significant extent, respondents in both the SSAB and RAB studies linked accountability with transparency--the ability of community members to find out what the agency was doing and to match its behavior to its statements and assertions over time. Both SSAB and RAB members also emphasized the important role of the federal and state regulators. Several respondents noted that the only real way to hold the agency accountable was through the courts--and that since the regulators had the most direct line to the courts, access to them and the ability to ensure that the regulators were aware of community concerns and issues were seen as particularly important. We found that, overall, the RABs had established a more substantive and constructive relationship with the regulators than the SSABs. The regulators tended to be more fully integrated into, and provided more direct advice to the RABs than the SSABs, and many RAB members noted that the regular attendance of the regulators contributed significantly both to their level of knowledge, as well as their ability to hold the Army accountable. When this was not the case, RAB members were likely to identify lack of accountability as a significant issue.
Despite this closer relationship with regulators, in general, we found that RAB members had lower expectations for the RABs' direct impact on agency accountability than SSAB members had for the SSABs. Since RAB members did not make formal recommendations to the Army or receive formal responses to the suggestions they did make, the RABs did not have a clear paper trail to which they could hold the Army accountable. The SSABs, on the other hand, did have the advantage of such formal documentation. Consequently, the SSABs were more effective in achieving accountability through direct action of the board.
Creation and Institutional Context of the SSABs and RABs
Despite a common starting point, the two agencies and the advisory boards we studied differed in a variety of ways, including their performance on the five dimensions of the Acceptability Diamond, as discussed above. While the study does not permit conclusions about causal linkages, we discerned significant differences in the institutional context in which the SSABs and RABs were implemented. These differing DOE and Army contexts affected key policy, managerial, structural, and operational decisions that appeared to be contributing factors to the boards' subsequent performance in relation to the Acceptability Diamond.
This section reviews the early development of the boards and subsequently examines agency differences in policy and organization that constrained or enhanced their effectiveness of the boards, including:
* Whether or not the boards were established under the FACA;
* The clarity and specificity of the agency's policy commitment to public participation, both for the boards and for their overall program;
* The resources, training, and staff allocated to the public participation and advisory board process; and
* The composition of the advisory boards and the frequency and intensity of advisory board activities.
Development of the Boards
Both the SSAB and RAB programs grew out of a national interest in public participation and the results of the FFER Dialogue Committee, which was convened in the early 1990s. The committee, which developed from an informal dialogue among governmental and stakeholder representatives, including the DOE and the DoD, was chartered by the EPA to develop consensus policy recommendations for improving environmental restoration at federal facilities. This dialogue reflected a growing concern by citizens and regulators about the health, safety, and environmental risks posed by hazardous materials and environmental contamination from federal facilities and a growing use by federal agencies of citizen advisory committees to address acceptability and legitimacy issues since the 1960s (Koontz & Johnson, 2004, p. 186; Lynn & Busenberg, 1995). Committee members recommended establishment of citizens' advisory boards as a way of involving stakeholders more directly in agency cleanup decisions. They viewed such boards as a means of providing a consistent opportunity for involvement; regular, early, and effective participation in federal cleanup programs; and consolidation of the many public involvement initiatives in addressing cleanup (The Keystone Center, 1993, 1996).
Both DOE and DoD adopted the committee's recommendations and during the early and mid-1990s established citizens' advisory boards as a central feature of their public involvement approaches. By 1997, SSABs were operating at 12 major sites in the DOE complex and 58 RABs were operating at Army installations.
Both agencies developed policy and guidance for the boards. DOE issued policy guidance in 1994 (subsequently updated in 1998). In 1994, the U.S. EPA and the DoD jointly issued the Restoration Advisory Board Implementation Guidelines covering the formation and operation of RABs. In addition, the guidelines addressed the expansion or modification of existing Technical Review Committees to comply with the requirements set forth under the Superfund Amendments and Reauthorization Act. Finally, in 1998, RAB policy and guidance was incorporated into the Defense Environmental Restoration Program management guidance.
Establishment under FACA
DOE established its SSABs under an umbrella FACA charter. The DoD, faced with the establishment of over 300 RABs (plus boards for other agency-public interaction issues), made a decision that was binding on all its service agencies not to establish its advisory boards as FACA boards.
The FACA decision was significant. Under FACA, advisory boards are afforded full authority to provide group advice and written recommendations, and agencies are encouraged to provide feedback on what the agency did with that advice. (7) The SSABs, once they resolved process and procedural issues, focused considerable energy on formulating recommendations. Indeed, one of DOE's principal measures of the value and effectiveness of the SSABs was their ability to make consensus recommendations.
The DoD boards, in contrast, were not established under FACA. The Army guidance, consistent with that of DoD, accordingly required the RABs to provide advice and recommendations "in their individual capacities rather than by the consensus of the RAB" (U.S. Army Environmental Center, 1998, p. 9). Lack of status as a FACA board had a wide range of implications for the performance of the RABs on the dimensions of the Acceptability Diamond. It made it more difficult for RABs to perform well on the substantive dimension (framing, identifying, and getting the community's issues on the agenda); on decision making (contributing consensus community input); and on accountability (ensuring a track record of community recommendations and agency responses). Specifically:
* The requirement for RAB members to speak as individuals reduced the impetus for the governmental members to encourage, and community members to work on resolving their differences and seek agreement among themselves about community priorities. This was arguably the most negative impact of the requirement that RAB members speak solely as individuals. In effect, it hampered the RABs from realizing their full potential value to the Army--specifically, of being able, and encouraged, to present recommendations that resulted from the reasoned and informed opinions of the entire group and reflected their assessment of community priorities, a much stronger form of input than simply a diversity of individual opinions.
* Similarly, at RABs where members' roles were defined as representing only themselves, an important function of advisory boards for both the sponsoring agencies and the affected communities could not occur--serving as a link to the issues and perspectives of the broader community. The RAB members were not asked to provide this link, and were not presented to the community as performing this function.
* The lack of authority to provide group advice and written recommendations, and the absence of a forum for discussions focused on identifying commonalities and resolving differences resulted in the RAB members providing their viewpoints and recommendations indirectly in the form of questions or statements of individuals. Significantly, this contributed to the lack of a formal record highlighting and tracking the advice and input of the RAB members concerning the issues of importance to them and relating it to the Army's considered response. This was countered at some RABs by insisting on verbatim transcripts of the meetings to provide a clear record of statements and commitments.
Clarity and Specificity of the Agency's Policy Commitment to Public Participation
Several aspects of the two agency's policies concerning public participation and the advisory boards as well as the manner in which the policies were communicated to the installations differed considerably.
For DOE, the SSABs constituted one, albeit a major, component of a broader public participation program. Significantly, the SSABs were able to draw on the communication and participation efforts of other DOE components. This was a particular advantage, for example, in ensuring ready availability of information where, at most DOE installations, multiple, agency-wide channels of information were already established. In addition, Energy Secretary O'Leary's agency-wide commitment to openness and declassification of previously inaccessible information in 1993 formed the backdrop against which DOE's public participation policy and the SSABs were established.
DOE/EM policy guidance provided a comprehensive overview of the Department's philosophy and approach to public participation. It is clear from a reading of the EM Public Participation Policy that the agency understood the meaning and implications of "meaningful" public participation and was actively seeking public advice that "provides the opportunity to participate in the EM decision-making process for program planning, design, and implementation" (U.S. DOE/EM, 1998, p. 2).
In addition, through the customized training in public participation that was provided to its managers and by their close attention to the site's SSAB activities, EM Headquarters management conveyed to their staff and the public their strong commitment to public participation in general and a continuing, strong endorsement of the purpose of the SSABs. DOE followed through on its commitment to public participation and, perhaps because there were only 12 (as compared with the Army's 58 boards), was able to provide for Headquarters' visits and frequent interaction with the site managers who were responsible for implementing the policy. This close personal attention, which did not occur at the RABs, signaled a high agency priority on providing a role for the public.
In contrast, RABs were frequently the only mechanism (apart from legally mandated information repositories) for direct interaction between management and the public and were not a central part of DoD's approach to cleanup. Of critical importance was that, in both its initial and revised guidance (U.S. Army Evironmental Center, 1998) the Army, consistent with DoD guidance (U.S. DoD/EPA, 1994), was heavily focused on the logistics of implementation. Perhaps at least in part because of the greater number of RABs than SSABs, the primary topics included in the guidance were specific rules for establishing, operating, and adjourning a RAB and for providing reporting mechanisms. The term participation was not defined; nor was explicit reference made to public participation in the decision-making process. In addition, unlike DOE management, there appears to have been only limited conveyance of the importance of public participation to Army staff and no opportunities for training in participation. The Army installations had little or no interaction with those in charge of the base closure and environmental remediation policy and program, who were located in another part of the country, and the top on-site leadership generally did not attend RAB meetings, communicate directly, or have an opportunity to develop personal relationships with RAB members.
Resources, Training, and Staff Allocated to the Public Participation and Advisory Board Process
Overall, DOE/EM placed a high organizational priority on its advisory boards, devoting considerable high-level policy attention to the establishment of the public participation policy and design of the advisory board process, as well as allocating resources to support their operation. This was less true for DoD, where public participation did not appear to have been established as a mission-critical activity, requiring the time and attention of upper level managers. These relative priority designations had a number of consequences in terms of the resources, training, and staff allocated to the public participation and advisory board processes at each agency.
DOE, for example, developed and implemented an EM-wide training program for upper-level and installation staff before the SSAB initiative was underway and provided more funding, training, and professional contractor support to the boards than did the Army. Many of the SSABs had professional staff support, including trained facilitators who were dedicated to SSAB support with annual contracts. In addition, each SSAB was allocated its own budget, which, in combination with the citizen leadership, afforded a greater a degree of independence to the members, as well as the ability to cover the cost of support for the board members and the production and distribution of information and materials among board members and their outreach to the broader community.
For most RABs, staffing was limited and funding was constrained, creating tension between time and resource requirements to support the board, as well as conducting the cleanup--a particular problem for sites that were closing and where agency staff were few and budgets were extremely limited. Unlike the DOE boards, where the use of professional facilitators for both board and subcommittee meetings was specifically suggested to all boards and implemented at about half of them, none of the RABs studied were offered or had professional facilitators. Few of the RABs had full-time assistance with document copying and distribution or interaction with RAB members, including assistance with scheduling and operating subcommittee meetings and informal discussions and assistance with group relationship building. In general, installations that were able and willing to provide additional resources performed better.
Composition of the Advisory Boards, the Membership Selection Process, and the Frequency and Intensity of Advisory Board Activities
Several factors contributed to a more limited role for the RABs than SSABs. SSABs were typically larger and more complex entities than RABs, and had longer and more frequent meetings, at least at the time of the studies. Two SSABs had a membership of over 30 people, while the RABS had a membership under 20, and often many fewer ones.
As compared with the RABs, all of the SSABs had active subcommittees that conducted detailed technical review and discussions of member recruitment and selection strategy and procedure. DOE and contractor staff participated actively in both board and subcommittee meetings, resulting in both extensive two-way communication and more opportunity for the development of personal relationships between the agency and public members. The overwhelming majority of the DOE/EM SSAB sites included upper-level management who, at least during the agency's relatively decentralized structure of the 1990s, were delegated considerable responsibility for decision making. Most SSAB meetings included senior EM staff and the site (or field office) manager attended many meetings. In contrast, the RABs included in the study typically were led by the head of the Environmental Restoration Division (who acted as co-chair), who held less decision-making authority on site and was less directly linked into the DoD's decision-making hierarchy. Senior installation staff and the garrison commander were not present at any of the observed RAB meetings and, since few RAB interviewees knew any of the headquarters staff in person, a greater disconnect was apparent between site and headquarters decisions and actions. In addition, the more infrequent and shorter RAB meetings and the reduced number of subcommittees made it difficult for members to keep abreast of events and provide input in time to be relevant for decisions.
An important difference between DOE and Army boards involved the role and participation of the installation's federal and state regulators. On the SSABs, both DOE participants and the representatives of federal and state regulators served as ex officio members--they provided information/presentations and participated actively in discussions, and jointly received advice and recommendations from the board's citizen members. In addition, the board chair, who often functioned as the executive director, was a citizen member. On the RABs, no differentiation of roles was made--regulators and Army staff participated as "regular" members, since federal regulations prevented members of non-FACA boards to provide formal advice and recommendations. Each RAB in our study was headed by co-chairs--an Army and a citizen co-chair, of which the Army co-chair was clearly the lead.
As discussed in this article, our separate studies of the SSABs and RABs indicated that agency differences in policy, managerial approach, and structure contributed to differences in performance on the five elements of the Acceptability Diamond of the boards included in our studies. More broadly, however, the studies reinforced the value of the Acceptability Diamond both in the conduct of formative evaluation and as a tool for mangers in designing and implementing public participation programs.
Differences in Approach and Performance
This comparison of DOE and Army advisory boards points to several factors under the control of management that contribute to enhanced performance. These factors include:
* Clarity and commitment to public participation in both policy and implementation, including recognition of public participation as mission critical, requiring the time and attention of upper level managers;
* Open information policies;
* Agency-wide training program for both headquarters and installation managers in public participation;
* Managers' personal attention to and interest in the public participation process and the board's progress;
* Provision of administrative support, including facilitators and professional participation specialists to help structure and manage group interactions, provide expert advice about process and procedures, and ensure that administrative and communication commitments are kept;
* Establishment as FACA boards, with authorization to provide group advice and recommendation, encouragement for members to forge agreement on issues and priorities among diverse interests of the community, and accountability for providing a track record of response to recommendations.
The DOE/EM boards benefited from the existence of all these factors, as compared to the Army boards. However, as noted, the last evaluation study of the SSABs in 2002 did not include some boards that had been disbanded and that had been struggling to reach agreement on process, procedures, and focus, and that had not proven effective in providing consensus advice. In addition, the overall context of the DOE/EM program post 2000 was very different from the early and mid-1990s. The sense of embarking on a new mission was being replaced by an emphasis on completion, closure, and long-term stewardship. Cleanup was well underway at most sites; some were nearing closure. At these sites, stewardship issues were becoming increasingly important while identifying and reaching agreement on cleanup alternatives was becoming less important. Many sites were anticipating a significantly reduced role for DOE/EM, both at their site and within DOE. Further, although there was no formal change in public participation policy, many study respondents perceived that DOE/EM Headquarters was sending a variety of signals indicating that they were placing a lower priority on consultation with community stakeholders.
Some RABs were able to develop adaptive strategies to overcome obstacles and limitations. Some managers did place a high value on public participation and community input, and were able to convey that to the RAB members. Some RAB members took the initiative to coordinate their discussions, structure effective dialogue, and make strong, timely suggestions about cleanup strategies and priorities. The lesson here is that leadership always counts.
Value of the Acceptability Diamond in Evaluating Public Participation Programs
As discussed in the second section of this article, the dimensions of the Acceptability Diamond are compatible with many of the factors and criteria identified in other evaluative frameworks. It provides an efficient, yet comprehensive means of evaluating a broad scope of information and participation activities in a variety of contexts.
By pointing to the critical role of information and the four basic types of issues that need to be addressed, the Acceptability Diamond recognizes that different activities have different and complementary functions. The key question to be resolved is whether, in combination, the various information and participation activities provide ways to meet community standards of acceptability regarding information disclosure, substantive issues, decision making, relationships, and accountability.
At the same time, this approach provides the flexibility to be used across a variety of settings that differ in geographic location, demographic, social, and cultural characteristics, and in agency missions and the historical relationships with each community. Using a broad framework permits meaningful evaluation and comparison among sites: as noted above, the key question is whether the individual programs provide mechanisms to address the four types of community concerns. The specific mechanism for how the concerns are addressed may vary, depending on the particular site; the question is whether mechanisms are in place and whether they complement one another.
Value of the Acceptability Diamond from the Organizational and Managerial Perspective
An important aspect of the Acceptability Diamond lies in the guidance it provides to program managers in designing and implementing a public participation program that will enhance their ability to work effectively with local communities. The concept of the Acceptability Diamond provides the framework for an agency to: (i) set objectives; (ii) design public participation and oversight programs; and (iii) set criteria for evaluating program effectiveness.
Our comparison of the advisory boards confirmed the need for program managers to adhere to open information policies and provide activities that address all dimensions of the Acceptability Diamond. Community members' issues and concerns are broad in scope, yet linked. Failure to address them is likely to affect both the effectiveness of the public participation program and the agency's effectiveness in achieving its mission. This article discusses the Acceptability Diamond in relation to citizen advisory board participation in agency cleanup programs. Our contention is that this general framework could logically be extended to other types of programs and other agencies. Adopting the framework could help program managers evaluate whether they are addressing the full scope of issues and community needs. Essentially, a public participation program may be deemed effective to the extent that it provides for open disclosure and addresses all four acceptability dimensions in ways that are appropriate and effective for a particular community and situation.
Our work has focused on advisory boards as a principal avenue of public participation and as a major focus of agency-public interaction. One important next step is to apply this framework to other agencies and participatory processes and to conduct a more systematic mapping of the process requirements and outcome measures identified in the public participation and deliberation literature to the elements of the Acceptability Diamond. This would contribute to the goal of creating more clear linkages across the subdivisions of this field of study and clarifying how processes contribute to outcomes. It would also provide a basis for specifying in more detail the components and metrics of each of the five dimensions of the Acceptability Diamond and for examining the interplay among them. Another important next step is to articulate and document more clearly the relationship between day-to-day program management and public participation requirements.
Kristi M. Branch and Judith A. Bradbury are at the Pacific Northwest National Laboratory. Both authors are at The Pacific Northwest National Laboratory, operated by the Battelle Memorial Institute for the U.S. Department of Energy. Kristi Branch can be contacted at firstname.lastname@example.org or at 1100 Dexter Ave N. Suite 400, Seattle, WA 98109; Judith Bradbury can be contracted at email@example.com.
The authors wish to acknowledge the contribution of Dr. Elizabeth Malone, Pacific Northwest National Laboratory, who collaborated on the final study of the U.S. Department of Energy public participation programs, which was published in 2003. Our very sincere thanks are also due to our reviewers and to Dr. Seth Tuler, of the Social and Environmental Research Institute and Dr. Thomas Webler, Antioch New England Graduate School for their very helpful comments on a previous draft.
1. The studies did not include identical sites. The 1996 study included SSABs at the following DOE sites: Fernald, Hanford, Idaho, Los Alamos, Monticello, Nevada, Pantex, Rocky Flats, St. Louis (not actually an SSAB), Sandia, and Savannah River. The 1997 study included Oak Ridge, which was established later than the other SSABs and the other sites with the exception of St. Louis. The survey measured participants' assessments of: (i) performance on the six goals (effective processes and procedures; effective exchange of information and viewpoints; provision of useful advice and recommendations; improved site decisions and decision-making process; more acceptable actions; improved trust and confidence in DOE); (ii) Board processes and outcomes; (iii) the role of DOE and the regulators; (iv) group effectiveness; and (v) overall value of the SSAB initiative.
2. The 1999 study included SSABs at the following nine sites: Hanford, Idaho, Nevada, Oak Ridge, Paducah, Pantex, Rocky Flats, Sandia, and Savannah River. The 2003 study included: Fernald, Hanford, Los Alamos, Nevada, Oak Ridge, Paducah, and Savannah River. A separate study of the Paducah SSAB was also completed in 2001.
3. The RAB study included RABs at the following Army installations: Aberdeen Proving Ground, Camp Bonneville, Fort Bliss, Fort Campbell, Sierra Army Depot, Sunflower Army Ammunition Plant, and Twin Cities Army Ammunition Plant.
4. This is consistent with research on "other regarding preferences"--trust, reciprocity, and altruism, which has found that social interactions that reduce social distance and provide mechanisms for monitoring and sanctions lead to higher other regarding preferences and more cooperation (Buchan et al., 2006; Manski, 2000; Yamagishi, 1988; Yamagishi, Cook, & Watabe, 1998).
5. At the time of our final evaluation of the SSABs and the DOE public participation program, many of those with whom we spoke commented on the change from an open to a more closed approach in 2000 as compared with the early and mid-1990s.
6. Because the RABs constituted the only DoD recognized form of public involvement at these installations, respondents often expressed frustration about this definition of scope. However, with rare exceptions, the local communities or stakeholders did not organize to create alternative mechanisms to address these concerns.
7. After several years of operation, EM required its site managers to report back to their boards how the boards' advice had been used--and if not, why not.
Anson, Robert, Robert Bostrom, and Bayard Wynne. 1995. "An Experiment Assessing Group Support System and Facilitator Effects on Meeting Outcomes." Management Science 41 (2): 189-208.
Bacharach, Samuel G., and Edward J. Lawler. 1981. Bargaining, Power, Tactics, and Outcomes. San Francisco, CA: Jossey-Bass.
Beierle, Thomas C., and David M. Konisky. 2000. "Values, Conflict, and Trust in Participatory Environmental Planning." Journal of Policy Analysis and Management 19 (4): 587-602.
Bradbury, Judith A. 1989. "The Policy Implications of Differing Concepts of Risk." Science, Technology and Human Values 14 (4): 380-399.
______. 1998. "Expanding the Rationale for Analysis and Deliberation: Looking Beyond Understanding Risk." Human Ecology Review 5 (1): 42-44.
Bradbury, Judith A., and Kristi M. Branch. 1999. An Evaluation of the Effectiveness of Local Site-Specific Advisory Boards for U.S. Department of Energy Environmental Restoration Programs, Pacific Northwest National Laboratory, PNNL-12139. Prepared for the U.S. Department of Energy, Office of Environmental Management, Office of Intergovernmental and Public Accountability, February.
Bradbury, Judith A., Kristi M. Branch, J. H. Heerwagen, and E. B. Liebow. 1994. Community Viewpoints of the Chemical Stockpile Disposal Program, Summary report and eight community reports prepared for the U.S. Army and Science Applications International Corporation.
Bradbury, Judith A., Kristi M. Branch, and Elizabeth Malone. 2003. An Evaluation of DOE/EM Public Participation Programs, Pacific Northwest National Laboratory, PNNL-14200. Prepared for the U.S. Department of Energy, Office of Environmental Management, Office of Intergovernmental and Public Accountability, February 2003.
Bradbury, Judith, Kristi Branch, Susan Santos, and Caron Chess. Forthcoming [no updated information re date of publication]. An Evaluation and Recommendations for Enhancing the U.S. Army Restoration Advisory Boards. Prepared for the U.S. Army Center for Health Promotion and Preventive Medicine.
Bradbury, Judith, Kristi Branch, and Mary Zalesny. 1996a. Site Specific Advisory Board Initiative 1996 Evaluation Survey Results: Volume I, Summary Report. Prepared by for the U.S. Department of Energy, Office of Environmental Management, Office of Intergovernmental and Public Accountability.
______. 1996b. Site Specific Advisory Board Initiative 1996 Evaluation Survey Results: Volume II, Individual Site Results. Prepared for the U.S. Department of Energy, Office of Environmental Management, Office of Intergovernmental and Public Accountability.
______. 1997a. Site Specific Advisory Board Initiative 1997 Evaluation Survey Results: Volume I, Summary Report. Pacific Northwest National Laboratory. Prepared for the U.S. Department of Energy, Office of Environmental Management, Office of Intergovernmental and Public Accountability.
______. 1997b. Site Specific Advisory Board Initiative 1996 Evaluation Survey Results: Volume II, Individual Site Results. Prepared for the U.S. Department of Energy, Office of Environmental Management, Office of Intergovernmental and Public Accountability.
Branch, Kristi M., Judith A. Bradbury, and M. Silbernagel. 1995. A Longer View: Evaluation of the Public Participation Planning for Managers Training Course, PNNL-1107, Report prepared for the U.S. Department of Energy Office of Environmental Management.
Branch, Kristi M., Judith A. Bradbury, Rochelle Williams, Marina Skumanich, and Cynthia Abrams. 2000. "Disclosure as a Strategy for Risk Reduction and Improved Worker and Public Safety: FORSCOM Experience with the CAAA Risk Management Program Rule." Proceedings of the Real World Clean Air Conference, Atlanta, GA.
Browder, Lesley H. 1971. "Introduction: Emerging Patterns of Administrative Accountability--A Point of View." In Emerging Patterns of Administrative Accountability, ed. L. H. Browder, Jr. Berkeley, CA: McCuthchan, 1-25.
Buchan, Nancy R., Eric J. Johnson, and Rachel T.A. Croson. 2006. "Let's Get Personal: An International Examination of the Influence of Communication, Culture, and Social Distance on Other Regarding Preferences." Journal of Economic Behavior and Organization 60: 373-98.
Burroughs, Richard. 1999. "When Stakeholders Choose: Process, Knowledge, and Motivation in Water Quality Decisions." Society and Natural Resources 12: 797-809.
Butterfoss, Frances Dunn. 2006. "Process Evaluation for Community Participation." Annual Review of Public Health 27: 323-40.
Carr, Deborah S., and Kathleen Halvorsen. 2001. "An Evaluation of Three Democratic, Community-Based Approaches to Citizen Participation: Surveys, Conversations with Community Groups, and Community Dinners." Society and Natural Resources 14: 107-26.
Charnley, Susan, and Bruce Engelbert. 2005. "Evaluating Public Participation in Environmental Decision-Making: EPA's Superfund Community Involvement Program." Journal of Environmental Management 77: 165-82.
Chess, Caron, and Kristen Purcell. 1999. "Public Participation and the Environment: Do We Know What Works?" Environmental Science and Technology 33 (16): 2685-92.
Cowie, Gail M., and Stuart R. Borrett. 2005. "Institutional Perspectives on Participation and Information in Water Management." Environmental Modeling and Software 20: 469-83.
Creighton, James L. 1992. Involving Citizens in Community Decision Making: A Guidebook. Washington, DC: Program for Community Problem Solving.
Daniels, Steren E., and Gregg B. Walker. 1996. "Collaborative Learning: Improving Public Deliberation in Ecosystem-Based Management." Environmental Impact Assessment Review 16:71-102.
Edelenbos, Jurian, and Erik-Hans Klijn. 2005. "Managing Stakeholder Involvement in Decision Making: A Comparative Analysis of Six Interactive Processes in the Netherlands." Journal of Public Administration Research and Theory 16: 417-46.
Environmental Protection Agency (EPA). 2001. Stakeholder Involvement and Public Participation at the U.S. EPA: Lessons Learned, Barriers, and Innovative Approaches. Washington, D.C.: Office of Policy Economics and Innovation.
Fiorino, Daniel J. 1990. "Citizen Participation and Environmental Risk: A Survey of Institutional Mechanisms. Science Technology and Human Values 15 (2): 226-43.
Habermas, Jurgen. 1987 . The Theory of Communicative Action, Volume II: Lifeworld and System, trans. Thomas McCarthy. Boston, MA: Beacon Press.
Konisky, David M., and Thomas C. Beierle. 2001. "Innovations in Public Participation and Environmental Decision Making." Society and Natural Resources 14: 815-26.
Koontz, Tomas H., and Elizabeth Moore Johnson. 2004. "One Size Does Not Fit All: Matching Breadth of Stakeholder Participation to Watershed Group Accomplishments." Policy Sciences 37: 185-204.
Krimsky, Sheldon, and Dominic Golding. 1992. Theories of Risk. Westport, CT: Praeger.
Lawrence, Rick L., and Debbie A. Deagen. 2001. "Choosing Public Participation Methods for Natural Resources: A Context-Specific Guide." Society and Natural Resources 14: 857-72.
Lynn, Frances M., and George J. Busenberg. 1995. "Citizen Advisory Committees and Environmental Policy: What We Know, What's Left to Discover." Risk Analysis 15 (2): 147-62.
Manski, Charles F. 2000. "Economic Analysis of Social Interactions." Journal of Economic Perspectives 14 (3): 115-36.
Mesch, Gustavo, and Ilan Talmud. 2006. The Quality of Online and Office Relationships: The Role of Multiplexity and Duration of Social Relationships. The Information Society 22: 137-48.
Murdock, Barbara Scott, Carol Wiessner, and Ken Sexton. 2005. Stakeholder Participation in Voluntary Environmental Agreements: Analysis of 10 Project SL Case Studies. Science, Technology & Human Values 30: 223-50.
Ostrom, Elinor. 1986. "An Agenda for the Study of Institutions." Public Choice 48 (1): 3-25.
Parkins, John R., and Ross E. Mitchell. 2005. "Public Participation as Public Debate: A Deliberative Turn in Natural Resource Management." Society and Natural Resources 18: 529-40.
Peterman, William. 2004. "Advocacy vs. Collaboration: Comparing Inclusionary Community Planning Models." Community Development Journal 39 (3): 266-76.
Poisner, J. 1996. "A Civic Republican Perspective on the National Environmental Policy Act's Process of Citizen Participation." Environmental Law 26: 53-94.
Rauschmayer, Felix, and Heidi Wittmer. 2006. "Evaluating Deliberative and Analytical Methods for the Resolution of Environmental Conflicts." Land Use Policy 23: 108-22.
Reich, Robert B. 1985. "Public Administration and Public Deliberation: An Interpretative Essay." Yale Law Journal 94: 1617-41.
Roberts, Nancy. 1997. "Public Deliberation: An Alternative Approach to Crafting Policy and Setting Direction." Public Administration Review 57: 124-32.
______. 2004. "Public Deliberation in an Age of Direct Citizen Participation." American Review of Public Administration 34 (4): 315-53.
Robertson, Peter J., and Shui-Yen Tang. 1995. "The Role of Commitment in Collective Action: Comparing the Organizational Behavior and Rational Choice Perspectives." Public Administration Review 55: 67-80.
Romzek, Barbara S., and Melvin Dubnick. 1994. "Issues of Accountability in Flexible Personnel Systems." In New Paradigms for Government: Issues for the Changing Public Service, ed. P. W. Ingraham, and B. S. Romzek San Francisco: Jossey-Bass, 263-94.
Rowe, Gene, and Lynn J. Frewer. 2000. "Public Participation Methods: A Framework for Evaluation." Science Technology and Human Values 25 (1): 3-29.
Ryfe, David M. 2005. "Does Deliberative Democracy Work?" Annual Review of Political Science 8: 49-71.
Sabatier, Paul A., and Hank Jenkins-Smith. 1999. "The Advocacy Coalition Framework: An Assessment." In Theories of the Policy Process, ed. Paul A. Sabatier. Boulder, CO: Westview Press, 117-168.
Sheppard, Stephen R. J. 2005. "Participatory Decision Support for Sustainable Forest Management: A Framework for Planning with Local Communities at the Landscape Level in Canada." Canadian Journal of Forest Research 35: 1515-26.
Slovic, Paul. 1993. "Perceived Risk, Trust, and Democracy." Risk Analysis 13 (6): 675-82.
Smith, Patrick D., and Maurean H. McDonough. 2001. "Beyond Public Participation: Fairness in Natural Resource Decision Making." Society and Natural Resources 14 (3): 239-49.
Steelman, Toddi A., and William Ascher. 1997. "Public Involvement Methods in Natural Resource Policy Making: Advantages, Disadvantages, and Trade-offs." Policy Sciences 30: 71-90.
Stephan, Mark. 2005. "Democracy in our Backyards: A Study of Community Involvement in Administrative Decision Making." Environment and Behavior 37 (5): 662-82.
Sullivan, John L., and John E. Transue. 1999. "The Psychological Underpinnings of Democracy: A Selective Review of Research on Political Tolerance, Interpersonal Trust, and Social Capital." Annual Review of Psychology 50: 625-50.
Susskind, Lawrence, and Jeffrey Cruickshank. 1987. Breaking the Impass: Consensual Approaches to Resolving Public Disputes. New York: Basic Books.
The Keystone Center. 1993. Interim Report of the Federal Facilities Environmental Restoration Dialogue Committee. The Keystone Center, Science & Public Policy Program Keystone, CO.
______. 1996. Final Report of the Federal Facilities Environmental Restoration Dialogue Committee. The Keystone Center, Science & Public Policy Program Keystone, CO.
Thurston, Wilfreda E., Gail MacKean, Ardene Vollman, Ann Casebeer, Myron Weber, Bretta Maloff, and Judy Bader. 2006. "Public Participation in Regional Health Policy: A Theoretical Framework." Health Policy 73: 237-52.
Tuler, Seth, and Thomas Webler. 1999. "Voices from the Forest: What Participants Expect of a Public Participation Process." Society and Natural Resources 12: 437-53.
U.S. Army Environmental Center. April 1998. Restoration Advisory Board and Technical Assistance for Public Participation Guidance. U.S. Army Environmental Center. Aberdeen Proving Grounds, MD.
U.S. Department of Defense and U.S. Environmental Protection Agency. 1994. Restoration Advisory Board Implementation Guidelines. Issued jointly by the U.S. Department of Defense and the U.S. Environmental Protection Agency. Washington, D.C.
U.S. Department of Energy, Office of Environmental Management. 1994. Site-Specific Advisory Board Guidance, Interim Final. November 14. Issued by the U.S. Department of Energy, Office of Enviromental Management. Washington, D.C.
______. 1998. Public Participation Policy. September 17. Directive of the U.S. Department of Energy. Office of Enviromental Management. Policy 1210. Available at url: http://www.directives.doc.gov/pdfs/doc/doctext/oldort/1210/p12101.pdf.
Van Slyke, David M., and Christine H. Roch. 2004. "What Do They Know, and Whom Do They Hold Accountable?" Journal of Public Administration Research and Theory 14 (2): 191-209.
Vroom, Victor H., and Arthur G. Jago. 1988. The New Leadership: Managing Participation in Organizations. Englewood Cliffs, NJ: Prentice Hall.
Vroom, Victor H., and Phillip W. Yetton. 1973. Leadership and Decision-Making. Pittsburgh, PA: University of Pittsburgh Press.
Wang, Xiaohu. 2002. "Assessing Administrative Accountability: Results from a National Survey." American Review of Public Administration 32 (3): 350-70.
Webler, Thomas. 1995. "'Right' Discourse in Citizen Participation: An Evaluative Yardstick." In Fairness and Competence in Citizen Participation: Evaluating Models for Environmental Discourse, ed. O. Renn, T. Webler, and P. Wiedemann. Dordrecht: Kluwer, 35-77.
Yamagishi, Toshio. 1988. "The Provision of a Sanctioning System in the United States and Japan." Social Psychology Quarterly 51: 32-42.
Yamagishi, Toshio, Karen S. Cook, and Motoki Watabe. 1998. "Uncertainty, Trust, and Commitment Formation in the United States and Japan." American Journal of Sociology 104: 165-94.
Yang, Kaifeng. 2005. "Public Administrators' Trust in Citizens: A Missing Link in Citizen Involvement Efforts." Public Administration Review 65: 273-85.
Yang, Kalfeng. 2006. "Trust and Citizen Involvement Decisions: Trust in Citizens, Trust in Institutions, and Propensity to Trust." Administration & Society 38 (5): 573-95.
Yankelovich, Daniel. 1991. Coming to Public Judgement: Making Democracy Work in a Complex World. Syracuse, NY: Syracuse University Press.…