So I'm having a slice today with Jon while talking about the recent dust-ups discussing IT security risk management. Some of it started in a typical securosis firestarter about asset values and ALE. The Falcon's View provides more links and commentary than you can ask for in ...Not the Devil and ...Madness.
What made lunch fun was our conclusion that by the time you have quality inputs to your risk model (aka evidence) it doesn't matter what approach you use to prioritize risks. In other words, it doesn't matter if you present a statistical distribution or placement on a 10x10 chart. Your audience will evaluate the quality of your inputs and make a judgment on your conclusion and credibility.
As my food coma set in, I sketched a little picture on a scrap of paper: Input, Process, Output process flows for quantitative and qualitative models, and it becomes even more evident. If your Process is a black box that obfuscates how your evidence translates to an assertion, I think you've lost. It's much more difficult to justify decisions when stakeholders don't understand the logic. And when you're wrong, how can you learn from something you can't explain well? Either way you need to be ready to be wrong and show how your next budget will be that much wiser.
Actually I don't care what model you use as long as it's transparent, your stakeholders understand it, and you hold yourself accountable to your conclusions (Note I did not say the model's conclusions. Models should not think for you.). Qualitative models are easier for me to meet these criteria but it all comes down to the quality of your evidence.
So let's make this actionable, how can risk assessment tools help us collect, organize, and apply evidence to future loss predictions? I'm not exactly sure but I'm going to find out. The first step is making sure you collect and organize available evidence. Here's some categories stained with pizza grease:
- Incident postmortems: lots has been written how to document data and anecdotes for future application. This is our strongest evidence source and your IR team must be prepared to collect it.
- Internal & External Assessments: control effectiveness assessment evidence. Unfortunately most people believe that if I can do it, anyone can
- Peer Incident data: very useful. You don't want to be as lame as your lame competition.
- Industry/analyst data: depends on quality but Verizon and co set a great bar.
- Metrics: metrics are probably an amplifier for other evidence e.g. we've seen trojan's on this type of machine config and our Managed for Security Metric numbers are trending down.
- Regulations: sometimes you have to step up to the lowest common denominator. Obviously the focus should be what's appropriate for the business.
- Business driven: actions the business is/will take that change risk tolerance e.g. outsource data storage to shared platforms, build all software in-house, etc. You can utilize above evidence to apply to new situations.
I think it's pretty interesting how some evidence influence either the impact or likelihood side of the risk story. We have some pretty cool ideas how to highlight evidence in the risk assessment and portfolio planning we all do. Let me know if you want to join in the mock-ups.
PS: In case you don't read the Falcon's View, here's my comment on how to determine data value from the ...Devil post linked above:
Nice post. Why all the hullabaloo for such aging topics? Is it just time to discuss them again or am I missing something?
In my experience, dealing with intrinsic value is simple: engage the business owners. Security is good at understanding threats and conducting assessments to estimate likelihood. Take this information and collaboratively work with the CIO, head of Marketing, Legal, Sales, Product Development, CEO (depending on size) once or twice a year. Include evidence from your assessments, incidents, and peers. They'll tell you what's important.
The executive team owns risk tolerance decisions, security simply provides data and drives the process. No fancy formulas or technology required. It's only hard if you go it alone.