Most organizations lack sufficient evidence or communication to assure the robustness of their security capabilities in terms that connect with and influence executive leadership. Creating rigorous security assurance requires that companies must follow the principle of cause and effect.
On the causal side we have the organization’s security capabilities. These exist to help mitigate the organization’s impact from security threats. Unfortunately, current approaches of metrics and key performance indicators are too myopic and obscure the big picture, maturity to frameworks fail on business context and weakly calibrated threat methods.
On the effect side, only attack simulations offer a true proof of result. Unfortunately, penetration tests and red/blue/purple teams are often weakly scoped or executed, or not reported in a manner that truly grips and influences.
These assurance gaps leave organizations essentially blind – or worse, a false sense of reality - to realistic cyber risk exposure, or to the most effective improvements.
We will introduce a set of measurements to better inform both sides, which will help forge a self-reinforcing, causal top-down and effect bottom-up enterprise cyber assurance system.
So what does good security assurance look like?
Assurance must matter and deliver confidence, otherwise it's not assurance at all. Good assurance will offer objective insight to these questions: What's our threat preparedness against our crown jewels? Is security strategy effectively business aligned? Are controls over or underperforming? And finally, how and where do we prioritize?
Companies are finding they can use attack simulations to analyze their environments and deliver assurance. But there are at least three points to consider. For starters, the attacker capability of the simulation must effectively mimic threat tactics up to the levels that the organization expects to protect. Second, the simulation scope must be beyond reproach. A poorly defined and justified scope undermines the credibility of the results - even if they are accurate. For best results, make the assurance big picture. Simulations must consider and leverage all reasonable means to breach. The lessons learned from broadly-scoped simulations can inform more focused simulations. Finally, time has become an essential component of the attacker toolkit, the simulation must have the appropriate time to effectively mimic a real-world attack scenario.
Here are five unifying principles to integrate SecOps, frameworks, and attack simulation into a more robust security assurance system:
- Prove that it matters.
For assurance to gain traction with executives it must pass the “so what?” test. Executives have a duty to protect the business from the biggest risks. For security risks to pique their interest, security strategy and operations must link their performance and assurance to high-value business assets. These crown jewels are justifiably where people would expect the greatest breach impacts and where executives must demonstrate due diligence.
- Demonstrate that it’s verifiable.
To instil executive-level confidence, security operations must develop an assurance capability that they can objectively measure on a repeatable basis. The team must also independently verify the capability by a non-conflicted source; and the security team must use a generally accepted verification method such as an attack simulation.
- Calibrate the results.
At the core of security strategy and operations are threats, and threats vary by sophistication levels. More advanced threats generally have more time and/or more capability to breach. Thus, they generally cost more to protect. We call this threat calibration, and it offers an objective and meaningful scale to calibrate executive expectation, security operations tactical decisions, and independent verification. Security operations can develop capabilities to calibrate and choose threat sophistication levels and coverage targets and objectively measure that development progress and capability attainment. Attacker simulators can target the most important business assets, and they know what levels they will have to effectively mimic. SecOps calibrate to the tactics that are funded. Attackers know the target tactics SecOps should protect against and can test those.
- Set realistic goals.
When done well, attack simulation works as a robust assurance capability that delivers irrefutable evidence of the performance of security strategy and security operations. When done poorly, attack simulations are often unrealistic and can lose traction and usefulness. This plagues the bulk of pen tests and red teams today.
- Make it influential.
In the simulation, the threat should start from a position of modest sophistication, from the public domain, with only publicly available information. The simulation runs as a continuous, step-by-step process of the threat actor’s discovery, perception, decision-making, and actions to advance the attack. Security influences the perception and decision-making and becomes our evidence of controls performance relative to the position of the threat actor and the level of sophistication they must use to advance. This delivers objective evidence of control strengths (where the threat was stymied), gaps (incomplete control coverage), and bypass (weak sophistication controls).
By deploying this method of calibrating an assurance capability to crown jewels, then analyzing the way the simulation breaches our crown jewels, we can peer into the effectiveness of the security strategy and better understand how to control performance. This process can help security teams clearly explain the organization’s ability to protect against threats, what they need to do about them and how much it will cost. That’s the kind of security information top management has been looking for.
Douglas Ferguson founder and CEO, Pharos Security
