Magazine article Risk Management

Making Risk Assessments Useful: When Uncertainty Abounds, IT Security Must Adopt a Controls-Based Approach

Magazine article Risk Management

Making Risk Assessments Useful: When Uncertainty Abounds, IT Security Must Adopt a Controls-Based Approach

Article excerpt


IN his award-winning 1921 dissertation, economist Frank Knight made an important distinction between risk and uncertainty. According to Knight, "risk" refers to a situation in which the probability of an outcome can be determined. "Uncertainty," by contrast, refers to an event with a probability that cannot be known. Thus, Knight showed that while economists wanted to present their field of study as an exact science, it is not.

Unfortunately for information security professionals, IT security falls largely within Knight's uncertainty category. IT risk assessment is an inexact science since risks are rarely quantifiable.

Take, for example, risks introduced by software vulnerabilities. I can say with near certainty that the desktop system I am currently using has vulnerabilities. I can say this even if I have patched all known vulnerabilities. How do I know this? Because my software vendor sends out periodic security alerts and releases patches to fix these vulnerabilities. Vulnerabilities often are not found until years after the software goes into service; but they were always there. They were just unknown until someone maliciously exploited them or the vendor became aware of their existence.

At its core, risk assessment has a Straightforward methodology: multiply the magnitude of a loss by the probability that loss will occur (see page 39). Obviously, quantifying risk probabilities is integral to risk assessment. But if these risk probabilities are unknowable, are we wasting our time conducting risk assessments? To quote Dirty Harry, "a man's gotta know his limitations." Ultimately, risk assessments still can be useful even when uncertainty abounds. We just need to have our priorities in order and not let the minutiae overshadow the larger approach.


Humans have a zero-risk bias. This means we will opt for a small reduction in a risk to totally eliminate it rather than a larger reduction in a more significant risk. This is because we tend to choose more certain benefits (even if they are small) over larger, less uncertain benefits.

I was once involved with a large federal system that used dozens of servers. The certification team ran a routine vulnerability scan and noticed two things. First, a number of the servers did not have critical patches installed. Second, of those that were missing patches, they were missing different patches. When the certification team reported the issue they identified the risks as "server XYZ is missing critical patch ABC."

What was the problem with this approach? They did not understand the true issue, which was that the overall security program was not set up to systematically detect vulnerabilities. It was not, by industry jargon, operating under proper "configuration management," which means that the system is being monitored for changes and adapting as necessary.

This is not to say we should not fix any unpatched systems as soon as possible. Of course we should. But we also cannot shy away from identifying and mitigating the larger risk and the flaws in security program monitoring--even if it is far more difficult to fix and its long-term benefits are less tangible.

Recognizing this is key to adopting a mentality in which we admit that, if we are to continue to do risk assessments, we should at least try to make them useful. This means understanding human cognitive biases and attempting to ask questions that dig deeper into the risks.

It also means taking into account "black swan" events that could cause significant harm--even if their likelihood is considered low. Case in point: Barings Bank.

Barings Bank was the first merchant investment bank in England. Established in 1763, the bank quickly expanded to conduct business throughout the world. It was the bank of royal families, financing the Napoleonic wars and the Louisiana Purchase. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.