Improving Information Security Risk Analysis Practices for Small- and Medium-Sized Enterprises: A Research Agenda
Beachboard, John, Cole, Alma, Mellor, Mike, Hernandez, Steven, Aytes, Kregg, Massad, Nelson, Issues in Informing Science & Information Technology
It is commonly accepted that IT security countermeasures are imperfect thus organizations must be prepared to manage risk rather than attempt to eliminate it (Alberts & Dorofee, 2002; McCumber, 2005; Peltier, 2005; Schneier, 2004; Whitman & Mattord, 2003). A key element of the risk management process is the conduct of threat assessments and risk analyses that are tuned to the specific needs of the organization. The conduct of risk assessment and analysis is widely viewed as a necessary activity to guide the design and implementation of enterprise information security programs. The underlying framework for conducting such analyses is relatively simple. Identify and prioritize assets to be protected; identify relevant threats and the probability of their occurrence; multiply; add; then compare the expected losses with the costs of implementing relevant countermeasures. Of course, such analyses can be performed qualitatively, but the underlying logic remains largely the same.
The difficulties in effectively conducting such analyses are numerous. Identifying all relevant threats and reliably estimating the probability of occurrences have proven to be extremely difficult if not impossible. Likewise, estimating costs, even qualitatively, associated with various types of system failures or compromises is an inexact process. While the models for performing risk analyses are not difficult to understand, appropriately applying the models in given organizational contexts represents a daunting task. This is particularly true for resource- and expertiseconstrained small- and medium-sized enterprises (SME). In the U.S., the term is more typically applied to small- medium-sized businesses having less than 500 employees; the term SME is more typically used within the EU to refer to firms with less than 250 employees (Storey, 2003). Either definition works for the purposes of this paper. Under either definition, these organizations are unlikely to include large IT staffs with dedicated or extensive information security expertise. As Jaquith (2007) notes, the information security world has widely adopted the paradigm of calculating annualized cost expectancies (ALEs), but, "there is just one problem with ALE: the old dog will not hunt.... the numbers are too poor even to lie with" (p. 32). Jaquith cites three primary reasons for this (p.33):
* The inherent difficulty in modeling outliers.
* The lack of data for estimating probabilities of occurrence or loss expectancies * Sensitivity of the ALE model to small changes in assumptions.
There are numerous commercial enterprises providing software tools designed to assist with this effort. Some of them, RiskWatch[R] for example, claim to provide strong support for calculating annualized loss expectancy (ALE) and return on security investment (ROSI) (RiskWatch, 2005). While these tools may be quite effective, their use presents several practical issues for SMEs. First, they tend to be fairly expensive, although prices can vary significantly depending upon the features and support included. Second, they tend to be quite complicated. Effective use requires a significant amount of personnel training or consultant assistance as well as a significant amount of effort. Finally, for data quality problems referenced above, users have no real means of making an a priori evaluation of the quality of the final output.
Understandably, commercial companies prefer not to release their proprietary models and the knowledge bases employed in their products. However, without such information little opportunity exists for the user community to evaluate the relative efficacy of various products. Users are often permitted to download trial packages to evaluate the look and feel of program execution and reports but again lack an objective means for evaluating output quality.
To address these issues, this paper proposes the Information Assurance (IA) community adopt an "open source" approach to develop the following:
* A multi-level risk assessment methodology and set of decision heuristics designed to minimize the intellectual effort required to conduct SME infrastructure level risk assessments
* A set of decision heuristics to assist in the quantification of organizational costs, financial as well as non-financial
* A knowledge base of probability estimates associated with specified classes of threats for use in the application of the aforementioned methodology
* Automated tool(s) capable of supporting the execution of the aforementioned methodology and heuristics
At least initially, such an effort would be designed to meet the needs of profit and not-for-profit SMEs due to financial, time and intellectual constraints commonly associated with small organizations ("OCTAVE methods," 2003). …