Academic journal article Journal of International Affairs

Natural and Quasi-Natural Experiments to Evaluate Cybersecurity Policies

Academic journal article Journal of International Affairs

Natural and Quasi-Natural Experiments to Evaluate Cybersecurity Policies

Article excerpt

Over the past decade, numerous countries around the world have developed and implemented national cybersecurity strategies. Yet very few of these strategies have been subject to evaluations. As a result, it is difficult to judge the performance of strategies, the programs that comprise them, and the cost-effectiveness of funds spent. Natural and quasi-natural experiments are a promising set of research methods for the evaluation of cybersecurity programs. This paper provides an overview of the methods used for natural or quasi-natural experiments, recounts past studies in other domains where the methods have been used effectively, and then identifies cybersecurity activities or programs for which these methods might be applied for future evaluations (e.g., computer emergency response teams in the EU, cybersecurity health checks in Australia, and data breach notification laws in the United States).

**********

Over the past decade, numerous countries across the world have developed and implemented national cybersecurity strategies. Each strategy comprises a set of objectives and various programs to achieve those objectives. Tens of billions of dollars in taxpayer funds have been diverted from other purposes to pay for these strategies. A number of countries' recent strategies are reviews of previous ones.

Unfortunately, there are still no definitive answers to questions such as: Have these strategies achieved their overall objectives? Which programs contributed the most to these objectives? By how much (or little)? Where have funds been most effectively spent? What improvements might be made?

By most accounts, the cybersecurity situation globally is getting worse, in spite of the many measures being taken. There is a real need to improve assessment and evaluation of cybersecurity policies so as to inform and guide policy change.

With a new generation of cybersecurity strategies now being rolled out, it is timely to consider what evaluation techniques might be employed at the outset, so as to better track the performance of programs and the cost-effectiveness of funds spent. In doing so, public policies might better address the present state of cybersecurity nationally and globally in the future.

One promising technique for the evaluation of some cybersecurity programs is the use of natural and quasi-natural experiments. These broad groups of research designs and methods avoid the potentially high cost, possible ethical issues, and the impracticality of randomized control trials in a domain like cybersecurity. At the same time, they provide relatively robust measures of the counterfactual and net social/economic impact of policy decisions.

This paper will start with a background on national cybersecurity strategies. This will be followed by an explanation of common evaluation techniques with a special emphasis on natural and quasi-natural experiments. Finally, the paper will identify instances in which such research designs and methods might most effectively be used to evaluate certain programs that commonly comprise cybersecurity strategies and how such evaluations might be done in practice.

CYBERSECURITY STRATEGIES

Over the past decade, at least 20 countries have developed and implemented national cybersecurity strategies. (1) At least five of them have updated their strategies since their first edition (Australia, Czech Republic, Estonia, Netherlands, and the United Kingdom).

The objectives within these strategies are broadly similar across countries. According to the European Union Agency for Network and Information Security (ENISA) in 2014, the most commonly recurring objectives of the strategies in Europe include: developing cyber defense policies and capabilities, achieving cyber resilience, reducing cybercrime, supporting industry on cybersecurity, and securing critical information infrastructures. These objectives are broadly similar to those in the strategies of non-European countries. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.