Academic journal article Innovation: Organization & Management

Unintended Consequences of Innovation Policy Programmes: Social Evaluation of Technological Projects Programme in Croatia

Academic journal article Innovation: Organization & Management

Unintended Consequences of Innovation Policy Programmes: Social Evaluation of Technological Projects Programme in Croatia

Article excerpt


The evaluation of science, technology and innovation (STI) policies and related government-supported programmes is a relatively new phenomenon which dates from the late 1960s in the United States and the late 1970s in Europe (Luukkonen 2002; Roessner 2002; Molas-Gallart & Davies 2006). The interest for STI policy evaluation is mainly driven by the need of policy makers to legitimise R&D expenses and to demonstrate the impacts of STI policies on productivity, competitiveness and economic and social welfare in general.

Nowadays, within the 'broad-based' innovation policy (Edquist et al. 2009) and 'systemic approach' (Smits & Kuhlmann 2004) which underlines innovation as a contextual and endogenous process, the performance-based evaluation suffers certain limitations in assessing innovation policy measures (Perrin 2002). It assumes a direct relationship between input and output, whereas innovation is mediated through context and interaction with many other activities. Therefore, the evaluation of innovation policy instruments should go beyond standard quantitative measures to more formative approach to perceive wider socio-economic contexts which determine the outcomes and possible impacts of policy instruments (Kuhlmann 2003, 2006). The new trends in evaluation practice and methods ranging from quantitative to qualitative measures and from summative to formative types of evaluation have been introduced and elaborated by many authors (Georghiou & Roessner 2000; Perrin 2002; Roessner 2002; Kuhlmann 2003; Arnold 2004; Bozeman 2005; Molas-Gallart & Davies 2006).

The main argument for social evaluation of the government-supported innovation programmes comes from the 'new innovation paradigm' (Mytelka & Smith 2002: 1469) drawing on the evolutionary and institutional view of innovation which is broadly elaborated by scholars like Nelson and Winter (1982), Freeman (1988), Lundvall (1992), Lundvall and Borras (1997), Edquist (1997) and others. In this new context, the ability of businesses to be competitive increasingly depends on their capacity to apply new knowledge and innovation shaped by the partnerships and interactivity among many actors of the innovation system, primarily companies and research institutes/universities. The underlying theoretical idea was that innovation through interaction occurs in specific institutional contexts in which government has a critical role by defining the 'rules of the game' through various incentive measures and polices like legal rules on intellectual property, financial assistance, transfer institutions etc. It has provided a rapid rise to development of the innovation policy programmes to stimulate the commercialisation of publicly funded research and science-industry cooperation. A numerous studies on universitybased spin-offcompanies, university patenting/ licensing, public-private joint ventures etc. have been produced to illustrate the evolution of entrepreneurial activity of both the universities in general and individual scientists in particular. Despite the clash of business and scientific culture that is rooted in the classic normative structure of science (Merton 1968) many studies like those of Etzkowitz (1998, 2002), Hsu et al. (2007) Jain et al. (2009), Stuart and Ding (2006), Van Looy et al. (2004) depicted the shifts in scientists' attitudes toward commercial involvement.

Although commercial activity in academia is not without controversy it is commonly perceived that 'academic entrepreneur' has now achieved taken-for-granted status in the scientific community (Stuart & Ding 2006) while the notion of the eentrepreneurial university and its 'third mission' has become institutionalized within the triple helix model of science-industry- government cooperation (Etzkowitz 2008). Scientists, especially those at prestigious universities seek to capitalize on their research through the different technology transfer activities (e.g. consulting, contract research, joint ventures, university spin-offs) and they manly pursue their commercial interests in tandem with their academic work (Etzkowitz 2002). Entrepreneurial scientists ' typically adopt a hybrid role identity that comprises a focal academic self and a secondary commercial persona' (Jain et al. 2009: 923).

Some studies revealed that engagement in entrepreneurial activities usually is not made on the expense of more basic research but coincides with increased publication outputs (Van Looy et al. 2004). The efficiency of the scientific entrepreneurship usually depends on the strength of the national innovation system and its institutions to support complex interactions between the actors that play a role in the process of knowledge generation and diffusion (Etzkowitz 2008).

The concept of NIS has an astonishing takeup and has been rapidly adopted by the national governments around the world as an analytical framework and practical tool on how to manage innovation processes in local economies (Albert & Laberge 2007). Influenced by the similar strategic scenarios within the European Union depicted in the Lisbon agenda (European Council 2000), the Croatian government launched the first innovation policy programme, 'Programme for Innovative Technological Development' (HITRA), at the beginning of 2001. It has marked a turning-point in science policy since it presents the first deliberate and institutionalised action of public administration to connect science and economic development (Svarc 2006). Although the initial two sub-programmes, the RAZUM, aimed at supporting knowledge-based companies, and the TEST, focused on developing new technologies, have been running for almost eight years, standard performance-based evaluations to reveal the outputs in terms of patents, start-up companies, jobs, royalties, etc. have not been performed or presented to a wider public. The lack of evaluation studies produced negative feedback on the programmes themselves and innovation policy as a whole and has given rise to disputes about the transparency of the programmes, their efficiency, socio-economic impact, grant policy, and even corruption of the administration and researchers.

Therefore, an attempt to develop a method of social evaluation of the TEST programme has been carried out in order to observe the effi- ciency, position and functioning of the innovation policy in Croatia.1 The evaluation has been limited only to the TEST programme because it had sufficient number of publicly available data accessible for the statistical analysis (212 projects) via the Ministry web site. Unlike TEST, the number of projects in the RAZUM programme was relatively small (ca. 30), still in progress and not publicly available because the business companies were the principal stakeholders.

The social evaluation is based on the analyses of the data collected by questionnaire and by this limited to the perception of the project leaders. Although the perception does not provide the complete information about the programme performance the collected data we considered suffi ciently indicative for the critical analysis of the programme implementation, especially because it is based on firsthand experience of the principal participants of the programme.

Social evaluation can be understood as a part of formative evaluation. Its aim was to identify the main bottlenecks of the programme implementation in order to assess the structural deficiencies or system failures which hinder the efficiency of the entire innovation system. The motivation is the growing criticism of the innovation system performance and the perception of poor return on public investment in the innovation policy programmes. It goes beyond standard evaluation methods focused on the quantitative measurement of outputs since social evaluation takes into account also wider socio-economic, cultural and political contexts that determine the efficiency of the system.

Dimensions of social capital like values, trust and norms rarely make a subject of standard evaluations of innovation policies despite the growing recognition of socio-cultural embeddedness of national innovation capacities (Akcomak & Ter Weel 2009) innovation system (Fagerberg & Srholec 2008) and socio-technical system (Geels 2004).2 Some pioneering studies on social capital in Croatia revealed that Croatia suffers from the deficit of social capital mainly due to the lack of generalised trust, trust in institutions and perception of corruption (Stulhofer 2004). On the other side, there is also limited civil society development related to restrictive legislation and a weak of social responsibility on the part of the corporate sector (Bezovan 2003).

Within our social evaluation we tried to provide the insight into the general value orientation, trust and other relevant attitudes of researchers - project leaders - related to the TEST programme. These project leaders, being the first in Croatia who applied for technology-oriented projects, are perceived as the agents of socio-cultural and institutional change. Therefore, the investigation of their motives and benefits, their attitudes about the programme, commercialisation of science, etc. provides a valuable contribution for analysing the positively inclined socio-cultural environment of the current innovation policy.

This social evaluation of innovation policy draws on the combination of two theoretical backgrounds. The first is the theory of unintended consequences (Merton 1936) which states that social actions in a complex system creates intended and unintended consequences. Unintended consequences are the outcomes that are not originally planned and could be either beneficial or dysfunctional for a designated system.

The next theory is the concept of institutional deficits developed by Edquist (1997: 24-26; 2001). It considers institutions to be crucial elements in all versions of the systemic innovation approach: the configuration and relationships among organisations (consciously created formal structures such as funding agencies) and institutions (sets of common habits, routines, established practices, rules, or laws) vary significantly with the national innovation systems causing differences in their performance and success. Similarly to the theory of market failures, which tends to eliminate the deficiencies of markets in generating innovation, the institutionalist approach tends to reveal institutional failures linked to the formal institutions (rules and regulations) and informal institutions (socio-political context, value orientations, norms), as well as their interaction with organisations.

The research starts from the thesis that implementation of the TEST programme produced a set of unintended and failed consequences, which shifted the focus of the TEST programme from research commercialisation and science-industry cooperation (intended consequences) towards standard research projects which mainly serve the scientific interests of researchers (undesirable unintended and failed consequences).


The social evaluation of Croatian innovation policy is an ex post evaluation performed only on the one innovation policy instrument - the TEST programme. The TEST programme was the first government programme for direct incentives and financing of research activities in both public and private sectors focused on development of new technologies (products, processes, services) up to the stage of original solutions (prototype/pilot stage). Although the overall goals of the HITRA programme were defined broadly in the strategic document (MOST 2002a) in terms of technology development and revival of industry, the Directives for the implementation of the HITRA Programme (MOST 2002b)3 emphasised that the research results of the programme are expected to be commercialised through patents/licences, strategic partnership with companies, new spinoffs/ start-up companies within the RAZUM programme or will be exploited in other ways. Besides, the TEST programme has been assigned a pioneering role to establish a new institutional framework, i.e. the legal basis, procedures and rules for supporting science-driven innovations and science-industry cooperation. In addition, a new ministerial body, the Technology Council was established in 2001 to administer and manage the TEST programme.

The barriers to successful programme implementation are captured indirectly by measuring intended and unintended outcomes of the TEST programme. In this research, the intended or expected consequences of the programme are those outputs which meet the requirements of the official goals of the programme set out in the 'Directives' (MOST 2002b) that are originally formulated such as: 'quick and effect support to applied and development research relevant for direct use in the industry and economy', 'stimulation of scientists to initiate projects that may encourage additional investments by the industry', 'permanent cooperation between scientific and industrial sector', etc. Such goals are, obviously related to the production of new technologies, research commercialisation and science-industry cooperation. Conversely, the unintended and failed consequences are those outputs that shifted the focus of the TEST programme from expected results towards standard research projects which mainly serve the scientific interests of researchers and cause so far low performance of the TEST programme.

The consequences were operationalised as a set of indicators grouped in nine dimensions of the TEST programme (Table 1). The data were collected by a self-administrated questionnaire sent by mail and with return control. It consisted of 53 questions grouped in the six thematic categories that enable measurement of the selected dimensions of the TEST programme. The respondents were exclusively project leaders who provided the information on requested data about projects and also about their motivation, attitudes and value orientations. The standard Likert scale (1 - Strongly disagree; 2 - Disagree; 3 - Cannot decide; 4 - Agree; 5 - Strongly agree) was used to measure the statements concerning attitudes and value orientation (dimensions 6-9 in the Table 1). To process the data we have used the SPSS 13.0. The missing values were excluded from the analysis.

In addition to the questionnaire, a brief analysis of the Collected programmatic papers for the HITRA programme (MOST 2002c) that determine the strategic goals as well as the guidelines for the implementation of the Programme was carried out in order to identify the influence of formal procedures and rules on the success of the programme. Finally, a total of 12 semi-structured interviews have been conducted, audio-recorded and transcribed in order to understand the rationales of the programme from both the policy makers and researchers. The interviews involve so far nine projects leaders and three policy makers - the members of the Technological council. The future research will be extended also by the representatives of the companies.

The research was conducted through a questionnaire- based survey in the winter of 2007 targeting the 212 project leaders4 who have taken part in the TEST programme and have successfully completed the projects till 2005. In the period 2001 to 2007 there were 605 applied projects, of which 298 were accepted for financing. Therefore, the sample is a self-selected purposive sample of successful applicants to TEST projects. One hundred and twenty researchers responded to the questionnaire, giving a response rate of almost 57%.

The information about the projects and project leaders has been identified from the webbased Inventory of the TEST projects provided by the Ministry of Science, Education and Sports (MSES). The majority of respondents in the sample are from university departments (76%), whereas only 17% are from public research institutes. Some 65% of respondents are male and 35% are female researchers. A large majority of them (93.3%) have a PhD in science. Most of them come from technical sciences (45%), biotechnical sciences and biomedicine (28%) and 14% are affiliated to natural science and 10% to agronomy and the veterinary field. Some of the respondents (7.5%) possess their own company and some of them (9.2%) are working parallel in another firm or institution. The sample illustrates that some Croatian researchers are quite familiar with entrepreneurship.


The social evaluation of intended and unintended consequences reveals three categories of policy effects of the TEST programme:

1. Realised intended consequences;

2. Failed intended consequences;

3. Unintended consequences.

Realised intended consequences

The following five results have been identified as the expected and intended consequences of the TEST programme. They are mostly derived from the analysis of the policy documents (MOST 2002c) and partly from the interviews with the policy makers and researchers:

1. Fulfilment of the formal outputs of the programme;

2. Share of projects with commercialised research results;

3. Strengthening of research capabilities for cooperation with industry;

4. Impact of TEST on the changing role of science;

5. Impact of particular measures of TEST on the standard science policy.

The analysis revealed that all the projects have fulfilled the three formal requirements of the programme specified in the forms on final achievements on the project: (a) final report of project which assumes standard description of research project implementation (72.5% of projects), (b) feasibility studies (20% of projects) and (c) submission of the new project proposal to the programme RAZUM5 (7.5%). Since these 'officially required' results do not say much about commercialisation or technological application of the project after its completion, a new question with a scale of nine possible follow-up activities has been posed. Three of the follow-up activities were categorised as commercial use of project results: (1) sale of patent/licence, (2) commercialisation through partnership with existing companies and (3) commercialisation in another way (e.g. online education (Figure 1).

Although the analysis revealed that the majority of project results have been used for continuation of the scientific project which was, in essence, a failed intended consequence, about 16% of projects still declared commercialisation in one of the three given ways. In addition, more than 6% of projects were used for submission of a new project proposal to the HITRA-RAZUM programme aimed at establishing a start-up company based on research results. Also, 25% of projects were declared as being used as a basis for new collaborative projects with business. Given that commercialisation is an extremely complex process with usually only a modest success rate, it could be stated that 16% of projects with commercial results provide a platform for justification of the TEST programme as the initial and pioneering innovation policy programme.

The strengthening of research capabilities for cooperation with industry is the next positive output of the TEST as revealed by the analysis of benefits by researchers participating in the TEST programme (Figure 2). Researchers estimated that their main benefits consist in testing new ideas (M = 4.06), new contacts with companies (M = 3.63) and experience in cooperation with industry (M = 3.48).

Extra material resources and procurement of new equipment are also highly appreciated by researchers. The analyses of the follow-up activities (Figure 1) and use of the equipment (Figure 8) revealed that these resources were actually used after project completion for the continuation of research projects, i.e. for scientific purposes, not for commercialisation.

The fourth positive result of the TEST programme is related to its socio-cultural impact on the role of science. It contributed to the transition of dominant 'elite-type' science in Croatia towards the productive-type known generally as the Mode 2 of science production (Gibbons et al. 1994). As a consequence, a standard science policy was extended with the innovation policy based on the triple helix model (Etzkowitz 2003). The analysis reveals (Figure 3) that respondents have recognised the TEST as a promoter of commercialisation of scientific research, change in the traditional role of science and cooperation between research and industry. They noticed, however, that TEST suffers serious shortcomings such as the lack of infrastructure needed for commercialisation of research results (e.g. technology transfer centres) and the lack of final users (companies and industry) made it not fully functional. They are also not sure about its cost-efficiency, i.e. the return on investment in the programme is doubtful.

Finally, the fifth positive result is related to the individual policy measures introduced by the TEST programme which are complete novelties in traditional science policy and aimed at higher standards in project evaluation and monitoring (Figure 4). Detailed financial reporting (with copies of relevant documents), public defence of project proposal and permanent monitoring by the expert groups nominated by the MSES were largely appreciated. They are followed by other measures such as two-phase project applications which sort out valuable ideas for full application procedures, frequent reporting in three-month cycles, return of 21% of income to MSES in case of successful commercialisation, etc.

Failed intended consequences

Regardless of the positive results of the TEST programme, some measures have not produced the expected effects. One of the main incentives for this evaluation was directed toward identification of such failures and the underlying reasons and to find the main obstacles to programmes' successful implementation. Four failures of the TEST programme have been indentified:

1. Programme fails to mobilise companies and entrepreneurs;

2. Project results serve mainly scientific, not commercial purposes;

3. Programme fails to attract investments from companies;

4. Commercialisation of research results has not been the focus of submitted projects.

The target groups of the TEST programme were both researchers and entrepreneurs who wanted to develop or test new ideas with commercial potential within the cooperative projects. It was expected that a significant number of the project proposals would come from the companies since they can benefit not only from public grants but also from research expertise, equipment and knowledge at universities, institutions and government labs. Although companies are always eager to collect public money, it turned out that the majority of projects were initiated by ideas coming from researchers (73.3%) and were in essence the extension of scientific projects financed by the government. Only about 20% of ideas are generated by industrial partners or entrepreneurs from small companies searching for technological solutions or concrete applications. The figure roughly corresponds to the percentage of commercialised projects as presented in the analysis of the follow-up activities (Figure 1). The follow-up activities also illustrate that project results serve mainly scientific and not commercial purposes since 37.5% of projects were used for the continuation of scientific projects and another 27.5% have not been used for any of the listed follow-up activities. That makes a high proportion of projects (65%) which have met formal criteria of TEST ( potential commercialisation), but were not continued to the more concrete phase of technological application or commercialisation.

Also, the analysis of published results within TEST projects shows that scientific outputs largely outnumbered technological outputs like patents. Eleven respondents protected their research results with patents and produced altogether 21 registered patents. For comparison, scientific outputs include 417 published works, primarily scientific papers (66%) published in foreign (48%) and Croatian journals (18%). Professional papers contribute with 28% and books with 6% to total scientific publishing.

Since the participation of companies in the TEST programme was rather modest, the share of their co-financing of projects was also fairly small (Figure 5). More than 62% of projects have not received any funds from firms, whereas 26% of projects have received some financial means up to the amount of 30% of total projects costs. Only about 12% of projects were co-financed by firms with more than 30% of project total costs. That roughly corresponds to the percentage of around 16% of commercialised projects. Respondents also indicated that 54% of projects were financed only by the MSES, whereas the next 38% received more than 50% of funds from the MSES. A small share of financial resources was provided by agencies and local governments. The analysis of the budget resources shows that the programme fails to attract investments from companies.

The next failed consequence of the TEST programme was that commercialisation of research results has not been in the focus of researchers who have submitted projects. Only 17% of respondents have developed concrete plans for commercialisation prior to project submission. The remaining 83% have had vague ideas of commercialisation, of whom 46% were just 'thinking about commercialisation', whereas another 35% expressed intentions to commercialise research results, but had developed no concrete plans (Figure 6).

The majority of those who had developed some kind of commercialisation strategy based their commercialisation plans on the following (Figure 6): (a) extension of the contract research with industry (21%), (b) developing business services like testing or quality control (16.7%), (c) launching production either with strategic partner (11.7%) or with the companies where researchers work part time (5%). A few respondents wanted to sell a patent/ licence and launch their own company. Almost half of respondents (49%) did not respond to this question at all, stressing, thus, that commercialisation was not really a priority for the majority of respondents

Unintended consequences

Although the intention of the policy makers when designing the programme was to promote commercialisation and immediate industrial application (that was quite clear from the official documentation), yet some rules of the programme were defined broadly (e.g. request for pre-commercial projects in the public call) which allow ambiguous understanding of the programme's requests from the applicants point of view. As a consequence, the two unintended consequences of the programme were identified, as follows:

1. The motives for participation in the TEST programme were ambiguous and mainly science-driven;

2. The TEST programme has financed mainly standard research activities, not technological outputs and cooperation with industry.

The ambiguity of motives was identified by asking the respondents about their 'additional intentions to apply for the TEST project besides presumed technological results' (Figure 7). It turned out that motives of researchers to apply for TEST projects were split between three main interests: to develop additional experience of cooperation with industry (65.8% of respondents), to buy new equipment (64.2% of respondents) and to gain additional financial means for scientific research (50% of respondents).

The combination of such mixed motives indicates the ambiguous way researchers understand the TEST programme. Their prime interest was, in essence, to secure additional funding and equipment for their scientific research, which was the raison d'être of their vocational ethos and professional existence. The analysis of budget spending supports this finding since the majority of project budgets was spent on new equipment (majority of respondents or 23.4% have spent more than 50% of the budget on equipment; Table 2). The remaining research grants were spent primarily on material and personal costs, whereas the least amount was spent on business trips and cooperation with companies. This illustrates that cooperation with industry and commercialisation was a welcome ingredient, but rarely the prime motive of researchers for participating in the programme.

Since the analysis of follow-up activities (Figure 1) revealed that the majority of projects ended up as continuations of scientific projects, it is fair to say that the grants for technology projects were mainly used for additional financing of standard research and educational activities. Almost all projects (93%) used the purchased equipment for scientific research and education, whereas only 19% of projects used it also for commercial purposes. More than a half of purchased equipment (52%) is shared among all research staffwithin research institutions and treated as a common asset, which is typical of the scientific ethos. Only 19% of projects use the equipment only within project teams (Figure 8).


The social evaluation of the TEST programme has identified a range of unexpected and failed results, which points to bottlenecks in the implantation of the programme. The bottlenecks could be summarised as low mobilisation of companies to take part in technological projects and weak commercialisation of project results. The majority of TEST projects were caught in a backward loop, going from scientific ideas to technology results and back to scientific activities. The exchange of technological outputs for essentially scientific results appeared as the common feature of many projects pointing, thus, to the persistence of some problems in the innovation system. The origin of these systemic problems can be found in the institutional frameworks that determine behaviour and interaction of the main stakeholders of the programme. In the light of the analysis of the questionnaire and official documentation of the TEST programme (public call, application rules, etc.), the three main institutional deficits can be identified, as follows:

1. Administrative deficits;

2. Lack of social capital;

3. Deficiency of the broader socio-economic environment.

1. Administrative deficits are recognised as formal institutional deficits, which include the official rules and procedures of the TEST programme defined by the official documentation (MOST 2002c). They suffer a rather important deficit which discourages participation of companies/entrepreneurs in the programme. It is related to the rule that project leader should be only from research institute or university registered by the MSES. Under this rule, research organisations are the main financial contractor, which bears all the responsibilities for project implementation whereas companies are reduced to second-hand participants, which deterred many companies from participation.

This rule is also a consequence of a fragmented innovation policy, since the innovation policy in Croatia is still remarkably defined by the division of competences between the MSES and the Ministry of Economy, Labour and Entrepreneurship (MELE). The lack of a common policy platform and communication made the MSES restrict their policy measures to science-driven innovation based on the scientific capacities of the public research sector and avoid interfering with the business-led innovation and entrepreneurship in the domain of MELE. It also significantly reduced the potential of innovation policy to provide a common ground for harmonisation of government policies towards knowledge-based economy. The simple lack of a coordinated innovation system illustrates also the low ability of governance and public administration to manage innovation processes and the correlated economic development.

2. The lack of social capital, recognised as informal institutional deficits, is one of the serious obstacles to an efficient NIS and has been analysed in the previous papers on the Croatian NIS (Svarc et al. 2009). The standard indicators of social capital such as values, norms and trust in institutions were operationalised and measured through four aspects: (1) attitudes toward traditional and entrepreneurial universities; (2) attitudes toward commercialisation of science; (3) general value orientations; and (4) trust in institutions.

The analysis revealed that participants in the TEST programme, although recognised as the agents of change towards science-industry cooperation and commercialisation of research, share the rather traditional values such as statism, antiglobalism and egalitarianism, also typical of the wider Croatian population. As expected, this general value system is accompanied by the greatest trust in science and the education system and the least trust is placed in the judicial system, political parties and public administration. Statism refers to the traditional paternalistic role of the state (not a mediator between the stakeholders) while egalitarism refers to the re-distributive ethics and antientrepreneurial spirit (Zupanov 1987). Therefore, the traditional values discourage individual initiative, willing to take risk, cooperation and networking which build social cohesion needed for the efficient innovation system in Croatia. The lack of social cohesion is also sustained by the lack of confi dence in public administration and their policy programmes such as TEST, meaning that researchers cannot be adequately mobilised to follow government actions for fostering science-industry cooperation. Although they prefer the entrepreneurial university, which is in essence based on the Triple helix paradigm and the Mode 2 of science production, they are more inclined toward the protectionist role of government that would protect them from market competition and globalisation processes. The strong value orientation towards statism seems to be in contradiction with their preferences for the commercialisation of science and the entrepreneurial university. Statism is probably driven, however, by the lack of a research market, which simply does not work in the present socio-economic circumstances. The reason is the lack of industrial research centres, which were destroyed during the privatisation process, and a lack of companies interested in research since they have largely lost their technology capabilities during transition. The only way to keep science alive is through the protective role of the state and by maintaining a restricted and closed market as reflected in the value orientation of respondents.

3. Deficiencies in the broader socio-economic environment are identified as the lack of institutions for knowledge transfer and the lack of business demand for innovation and cooperation with research sectors. It is clearly shown in the answers to the question which explicitly asked respondents to identify the barriers to commercialisation (Table 3). The most relevant barrier is the lack of business partners, since 32.5% of respondents think that the lack of business partners is the reason for the failure of commercialisation 'greatly' and 'very greatly'. The next barriers are the lack of support in the environment (30%) and the lack of business opportunities (30%).

The previous analysis of social capital and innovation system in Croatia (Svarc et al. 2009) also revealed that the deficit of technology transfer infrastructure (e.g. university offices for technology transfer) and lack of interest of industry in cooperation with research organisations are perceived as the main obstacles to more intensive science-industry cooperation and commercial use of science (Table 4). These factors are perceived as more important obstacles than the dominant culture of elitist science focused on scientific publishing and ethical principles of scientific freedom distant from application. The lack of necessary infrastructure is also emphasised as an important shortcoming of the TEST programme in the general statements about the TEST as a new policy programme (Figure 3). The respondents think that the most minor problem of commercialisation is the low quality of science.

Finally, it is worth mentioning that 30% percent of respondents stated that they would not apply for TEST again mainly because of the diffi culties in commercial implementation of the results and collaboration with firms (Figure 9).


The social evaluation of the TEST programme based on the analysis of intended and unintended consequences indicates that innovation policy in Croatia does not work well in the given socio-economic and cultural circumstances and institutional set up. Instead of expected technological outputs with possible commercial effects, the majority of TEST projects were caught in a backward loop, going from scientific ideas to technology results and back to scientific activities. The explanation of the bottlenecks of the implementation of the TEST programme visible in this backward loop draws on the institutionalist approaches developed by Edquist (1997: 24-26; 2001).

This points to the persistence of some common or system problems, which are ascribed to the three institutional deficits that determine behaviour and interaction of the main stakeholders of the TEST programme, as follows: (1) administrative deficits (formal institutions), which consists of narrowly defined programme procedures and rules which have not attracted entrepreneurs, (2) pattern of researchers' behaviour (informal institutions) driven by value orientations and norms determined by the ambiguity between scientific ethos, on the one side, and the lack of research market, on the other side: and, (3) deficiency of the broader socio-economic environment in terms of the lack of knowledge transfer institutions and weak business demand for innovation.

The institutional deficits are rooted in the broader socio-cultural and economic environment and cannot be overcome by mere redesign of the single innovation policy programmes. In contrast, the entire policy and institutional context should be changed in order to attain development based on knowledge and innovation that is essentially the long-term goal of the TEST programme. Within the given context, the most important outcome of the TEST programme consists, instead of impact on technological development, in bringing socio-cultural change in understanding the new commercial role of science, which can be described also as transition from the standard (Mode 1) to the Mode 2 of knowledge production (Gibbons et al. 1994). It also highlighted the principles of the triple helix model in Croatia (Etzkowitz 2003).

The TEST programme has introduced the institutions, which made a ground for the national innovation system in Croatia and started the irreversible processes of market orientation of science. However, after the seven years of experience in innovation policy it is high time for reviewing the achievements and for making the adequate breakthrough in institutional framework needed for accelerating economic growth based on knowledge and innovation.

It assumes also a change of the current 'sciencepush' and 'supply-side' innovation policy towards more down-to-earth policy measures, which build on existing technological, socio-cultural and governance capacities. From the strategic perspective, the focus of such policy should be transferred from research-driven innovations and translation of research into business to other areas seriously neglected in Croatia that generate structural impediments to innovation. Such areas include low technology capabilities of companies, weak international competitiveness and regional competences, neglect of the new service sectors, etc. A fresh model of innovation policy and management, which assumes a combination of supplydriven policy at the national level and user-driven regional innovation policies based on clusters, is worth exercising. An intensive process of policy learning and management of innovation is essential for such strategic changes.


1 The empirical results presented in this paper are part of the wider research project on social evaluation of the Croatian innovation system financed by the Ministry of Science, Education and Sports (MSES) in the period 2005-2009.

2 For an overview of research on innovation culture and the differences in innovation culture across Europe see the Transform project (Didero et al. 2008).

3 All official documents of the HITRA programme were published in the booklet 'Collected programmatic papers' (MOST 2002c) and consists of the (1) Croatian Program for Innovative Technological Development (HITRA) (strategic document), (2) Directives for the Implementation of the HITRA Program, (3) Regulation on the Procedure for the Implementation of the Program for Development of Knowledge-Based Companies, (4) Agreement on the Cooperation and implementation of the HITRA with research institutions, (5) public calls for project proposals. All the documents were adopted by the Government of Croatia on April 5, 2001.

4 All the project leaders of the TEST programme were coming from the public research institutes and universities since the official rules and procedures of the TEST programme allowed the business companies to submit project proposals only through the public research institutes.

5 The RAZUM programme is the follow-up phase of the TEST programme aimed at establishing a start-up company/ research spin-off.



Akcomak, S. and Ter Weel, B. (2009) Social capital, innovation and growth: Evidence from Europe, European Economic Review 53, 544-567.

Albert, M. and Laberge, S. (2007) The legitimation and dissemination process of the innovation system approach: The case of the Canadian and Quebec science and technology policy, Science, Technology & Human Values 23(2), 221-249.

Arnold, E. (2004) Evaluating research and innovation policy: A systems world needs a systems evaluation, Research Evaluation 13(1), 3-17.

Bezovan, G. (2003) Indicators of civil society development in Croatia, Drustvena Istrazivanja 12(3-4): 495-518 (in Croatian).

Bozeman, B. (2005) Public value mapping of science outcomes: Theory and method. A Monograph of the public value mapping project, Center for Science, Policy and Outcomes, Washington, DC. Columbia University.

Didero, M., Gareis, K., Marques, P. and Ratzke, M. (2008) Differences in innovation culture across Europe - A discussion paper (available on line: Differences%20in%20Innovation%20 Culture.pdf. (Last access: June 2, 2010).

Edquist, C. (1997) Systems of innovation approaches - Their emergence and characteristics. In: Edquist, C. (Ed.), Systems of innovation - Technologies, institutions and organizations, pp. 41-63, Pinter/Cassell Academic, London.

Edquist, C. (2001) The systems of innovation approach and innovation policy: An account of the state of the art, Lead paper presented at the DRUID Conference, Aalborg, June 12-15, 2001.

Edquist, C., Luukkonen, T. and Sotarauta, M. (2009) Broad-based innovation policy, Ministry of Education: Evaluation of the Finnish National Innovation System - Full Report, Helsinki University Press, Helsinki.

Etzkowitz, H. (1998) The norms of entrepreneurial science: Cognitive effects of the new university-industry linkages, Research Policy 27, 823-833.

Etzkowitz, H. (2002) MIT and the rise of entrepreneurial science, Routledge, London.

Etzkowitz, H. (2003) Learning from transition: The triple helix as an innovation system, In: Svarc, J., Laznjak, J., Sporer, Z. and Polsek, D. (Eds.), Transition countries in the knowledge society: Socioeconomic analysis, pp. 39-61, Institute of Social Sciences, Ivo Pilar, Zagreb.

Etzkowitz, H. (2008) The triple helix university- industry-government Innovation in action, Routledge, London.

European Council. (2000) Conclusions of the Presidency of the Lisbon European Council of 23 and 24 March 2000, Council document 100/1/00, European Council, Brussels.

Fagerberg, J. and Srholec, M. (2008) National innovation systems, capabilities and economic development, Research Policy 37, 1417-1435.

Freeman, C. (1988) Japan: A new national innovation system, In: Dosi, G. et al (Eds.), Technical change and economic theory, pp. 330-349, Pinter Limited, London.

Geels, F. W. (2004) From sectoral systems of innovation to socio-technical systems, insights about dynamics and change from sociology and institutional theory, Research Policy 33, 897-920.

Georghiou, L. and Roessner, D. (2000) Evaluating technology programs: Tools and methods, Research Policy 29, 657-678.

Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P. and Trow, M. (1994) The new production of knowledge: The dynamics of science and research in contemporary societies, Sage, London.

Hsu, D. H., Roberts, E. B. and Eesley, C. E. (2007) Entrepreneurs from technology-based universities: Evidence from MIT, Research Policy 36, 768-788.

Jain, S., Georg, G. and Maltarich, M. (2009) Academics or entrepreneurs? Investigating role identity modification of university scientists involved in commercialization activity, Research Policy 38, 922-935.

Kuhlmann, S. (2003) Evaluation as a source of strategic intelligence, In: Shapira, P. and Kuhlmann, S. (Eds.), Learning from science and technology policy evaluation, pp. 352-381. Edward Elgar, Cheltenham.

Kuhlmann, S. (2006) Evolution of research evaluation, Lecture in R&D Evaluation Course 2006. University of Twente, The Netherlands.

Lundvall, B. A. (1992) National system of innovations: Towards theory of innovation and interactive learning, Pinter, London.

Lundvall, B. A. and Borras, S. (1997) The globalizing learning economy: Implications for innovation policy. Report based on contributions from seven projects under TSER Program. DG XII, Commission of the European Union.

Luukkonen, T. (2002) Research evaluation in Europe: State of the art, Research Evaluation 11(2): 81-84.

Merton, R. K. (1936) The unanticipated consequences of purposive social action, American Sociological Review 1(6), 894-904.

Merton, R. K. (1968) Social theory and social structure, Free Press: New York.

Molas-Gallart, J. and Davies, A. (2006) Toward theory-led evaluation: The experience of European science, technology, and innovation policy, American Journal of Evaluation 27(1), 64-82.

MOST (2002a) Croatian program for innovative technological development (HITRA), The Ministry of Science and Technology of the Republic of Croatia, Zagreb.

MOST (2002b) Directives for the implementation of the HITRA Programme, The Ministry of Science and Technology of the Republic of Croatia, Zagreb.

MOST (2002c) Collected programmatic papers of the HITRA programme, The Ministry of Science and Technology of the Republic of Croatia, Zagreb, p. 78.

Mytelka, L. K. and Smith, K. (2002) Policy learning and innovation theory: An interactive and co-evolving process, Research Policy 31, 1567-1479.

Nelson, R. R. and Winter, S. G. (1982) An evolutionary theory of economic change, The Belknap Press of Harvard University Press, Cambridge, MA.

Perrin, B. (2002) How to - and how not to - Evaluate innovation, Evaluation 8(1), 13-28.

Roessner, J. D. (2002) Outcome measurement in the USA: State of the art, Research Evaluation 11(2), 85-93.

Smits, R. and Kuhlmann, S. (2004) The rise of systemic instruments in innovation policy, International Journal of Foresight and Innovation Policy 1(172), 4-32.

Stuart, T. E. and Ding, W. W. (2006) When do scientists become entrepreneurs? The social structural antecedents of commercial activity in the academic life sciences, American Journal of Sociology 112(1), 97-144.

Stulhofer, A. (2004) Perception of corruption and the erosion of social capital in Croatia 1995- 2003, Politiska Misao 41(5), 74-86.

Svarc, J. (2006) Socio-political factors and the failure of innovation policy in Croatia as a country in transition, Research Policy 35(1), 144-159.

Svarc, J., Laznjak, J. and Sporer, Z. (2009) Social capital and innovation policy in Croatia: Scientific community as a source of innovation, In: Tripp, G., Payne, M. and Diodorus, D. (Eds.), Social capital, pp. 15-48, Nova Science, New York.

Van Looy, B., Ranga, M., Callaert, J., Debackere, K. and Zimmermann, E. (2004) Combining entrepreneurial and scientific performance in academia: Towards a compounded and reciprocal Matthew-effect?, Research Policy 33, 425-441.

Zupanov, J. (Ed.). (1987) Egalitarism and industrialism. In: Sociology and self-management, p. 281, Skolska knjiga, Zagreb (in Croatian).

Received 23 April 2010 Accepted 02 November 2010

[Author Affiliation]


Institute of Social Science Ivo Pilar, Zagreb, Croatia


Department of Sociology, University in Zagreb, Zagreb, Croatia

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.