Security metrics is a tricky topic to tackle. The idea behind metrics is to tell a story: How did X change over Y period of time? What impact does that affect? Looking in aggregate, are we making progress or are we regressing? In actuality, metrics are never so simple. For one thing, different people in different parts of an organization want and need to see different metrics to perform their jobs effectively. Even if everyone is (hypothetically) on the same page regarding relevance, the “meaning” of a metric can be an interpretation and carry diverging consequences depending on one’s perspective.
When it comes to cybersecurity metrics, practitioners have always struggled. Data such as number of incidents that occurred during X timeframe, number of attacks blocked/prevented by Y technology, and time to patch critical vulnerabilities are all imperative to track. These technical details are, after all, the cornerstones of an effective security organization. You can’t manage what you don’t measure, and all that. However, even though these data—and the metrics which are formulated using these data—are part and parcel of a security team’s functionality, they are significantly less useful to anyone outside of the security organization. Yet too many board presentation decks include information about alerts triaged or how many employees passed the annual security awareness training test.
Hone your focus
When it comes to the board of directors, cybersecurity is just another business area that needs to facilitate growth and opportunity. As such, board presentations on security should focus on how the security program is reducing risk, not the bits and bytes of how that is happening. Think of it this way: If you are an Amazon stockholder, you’d (first and foremost) want to know if revenue is increasing, thus driving up your stock holdings. You might also be interested in which major categories of products are driving growth—electronics vs. clothing vs. home goods, etc. What you likely wouldn’t be so interested in, however, is how many pairs of Women’s High Waist Leggings the company sold last month. When you report to your board how much malware was blocked by your firewall or IPS last month, you’re reporting the “leggings equivalent.” In other words, if you wouldn’t see the information in a 10-K, it doesn’t belong in your board presentation. This doesn’t mean certain data or metrics not included in the presentation aren’t valuable; it simply means that these aren’t the metrics the board wants to know.
Kristy Westphal, Security Manager, Security Tools at Charles Schwab says that where board-level security presentations often fall down is in showing risk reduction in a simple, easy-to-understand format. Board members don’t necessarily have a security background—nor should it be a requirement. It is the responsibility of the security presenter (CISO or otherwise) to clearly explain how security is affecting business. “One important thing to keep in mind when presenting any metrics,” says Westphal, “is to keep them simple and visual. Tell a story with one glance instead of a lot of tiny little details.” She advises security executives to craft metrics “like an elevator pitch” –concise and uncomplicated, because security is typically just one item on the meeting agenda and it must feed the bigger risk management picture.
One way to accomplish this, says Westphal, is to “measure [security’s] progress against compliance to a security framework,” like the NIST Cybersecurity Framework (CSF) or the CIS Critical Security Controls. Using industry standards as a benchmark is a simple way to set a baseline and demonstrate where your organization is—and how it’s been progressing—relative to that baseline. As with compliance, the framework doesn’t equal security, but it’s easy for board members to grasp. Showing upwards, downwards, or stalled trends “can be telling, and may help make the argument for more investment in resources” says Westphal.
{tweetme}When it comes to the board of directors, cybersecurity is just another business area that needs to facilitate growth and opportunity. #InfoSecInsider #infosec{/tweetme}
Link metrics to business priorities
Drilling deeper, Security Director at MRK Technologies, Chris Clymer, advises companies to ensure that any metrics presented outside of the immediate security team need to “be specific to your organization” and “always tie back to business priorities.” Clymer agrees that “the board doesn’t care how many patches you’ve applied or the number of firewall rules you’ve processed.” If, however, the security team can show how it has helped reduce the likelihood of a breach, find cybersecurity risks that could affect merger and acquisition activity (e.g., Yahoo, Nortel Networks), or disrupt the disruption of a core customer-facing operation (e.g., website availability, payment processing), the board will recognize value, he says.
When Clymer presents to his board (or helps clients with their own presentations), he uses a risk heatmap that is tied to “big-picture, board-level risks” that are being addressed through people, processes, and technology. Though metrics like number of attacks auto-prevented by IPS or total number of malicious websites auto-blocked in a month inform the bigger picture, they are not part of Clymer’s presentations. He says that to be useful, metrics must be:
- Easily measured
- Easily understood
- Meaningful
- Consistent
- Quantitative
Most importantly, though, a good metric enables decision making, which is why technical details are not appropriate for board-level presentations.
Clymer uses a 5-step process for building his board deck heatmap:
- Define the problem
- Identify your resources
- Build a rough draft
- Review with stakeholders
- Start back at step 1 and repeat
Audience interests – first and foremost
When executing this process, it’s important to always keep in mind the board’s interests, goals, and concerns. Too often security presentations are focused around what matters to the security team, but remember that the board presentation isn’t for you—it’s for non-security people who are making large-scale business decisions into which the security program must fit. Map your metrics to stated business objectives and show how a specific action/process/technology is helping or hurting the business as it strives to achieve forward progress, and you’ll be well on your way to gaining trust and influence with the decision makers who can support (or not) your security program
Chris Clymer will teach a a half day workshop on Making Meaningful Metrics at InfoSec World 2018 in Orlando, Florida, on Wednesday, March 21, 2018.
