Using Technology Absorptions as an Evaluation Criterion: Case Studies from a State Research and Development Program
Kingsley, Gordon, Farmer, Michael C., Policy Studies Journal
Consider the following scenario from an evaluation perspective.(1) An organization enters into an agreement with a state agency whose primary mission is to evaluate and fund research and development (R&D) projects.(2) The resulting project is designed as a demonstration of a technology that the state agency hopes to diffuse to other organizations located within its jurisdiction. The contractor has an interest in adopting the technology for its own use, and will be the site where the project is performed. The work involves the efforts of the contracting organization and of several subcontractors with relevant expertise. The state agency is responsible for funding the work and requires the contractor to locate cofunders for the project. This requirement serves two ends: (a) It assures the agency of the validity and relevance of the project; and (b) it creates other stakeholders through which the project outcomes can be disseminated. At the conclusion of the project, all the participants agree that the demonstration was successful. The contractor adopts the technology and incorporates it into day-today operations, but then progress stops. The technology does not transfer to any other organization. A better mousetrap was built, and the world yawned.
What is the state to make of this outcome if the ultimate goal is the diffusion of the technology to constituents? From this perspective the results are ambiguous. The state easily could take one of the following positions, supportable by the literature.
1. Declare a victory. An evaluation might argue easily that this was a tremendously successful project. After all, a technology was demonstrated successfully and transferred to an organization in the state. This view is sensible if the focus of a state agency is on the development of technology rather than dissemination (Bozeman & Fellows, 1988). Under such circumstances the state might evoke, as evidence of effectiveness, economic development indicators such as jobs created, jobs retained, sales growth, or productivity increases (Burton, 1989). Also, the state could take the long view, arguing that R&D projects are mission-oriented (Baer, Johnson, & Merrow, 1976), requiring a great deal of time between the introduction of a technology and any subsequent diffusion.
2. Write the project off as the cost of doing business. The state agency, or, more likely, evaluators from the governor's office or the legislature, just as easily might brand the project a failure because of the lack of a wider diffusion pattern (Atkinson, 1991). The state may be willing to tolerate this expenditure as one of the costs of sponsoring R&D projects. Such endeavors are fraught with uncertainty, so a percentage of project failures are expected. However, state R&D agencies tend to fund projects that are close to commercial application (Lambright & Teich, 1989). Thus, tolerance for this form of experimentation is relatively low.
3. Put things in their proper context. Alternatively, the state agency may argue that we need to understand the specific reasons why the project outputs didn't transfer and how this relates to the local context of potential adopters. The state agency might argue that the route between cause (the project) and effect (benefits to the state) might be an indirect one. For example, the demonstration may have led to the development of an even better mousetrap, or the technology was adapted to an entirely different use.
This scenario is observed in several studies of government agencies sponsoring R&D programs (Gates, 1988; MacDonald, 1986) and across a variety of institutional settings including large national laboratories (Bozeman & Fellows, 1988; Brown, Berry, & Goel, 1991), state projects (Burton, 1989; Forrer, 1989; Lambright & Teich, 1989; Wycoff & Tornatzky, 1988), and international projects (MacDonald, 1986; Supapol, 1990). It also has been observed …
Questia, a part of Gale, Cengage Learning. www.questia.com
Publication information: Article title: Using Technology Absorptions as an Evaluation Criterion: Case Studies from a State Research and Development Program. Contributors: Kingsley, Gordon - Author, Farmer, Michael C. - Author. Journal title: Policy Studies Journal. Volume: 25. Issue: 3 Publication date: Fall 1997. Page number: 436+. © 1999 Policy Studies Organization. COPYRIGHT 1997 Gale Group.