Allie Mellen, an analyst at Forrester, former researcher at MIT and Boston University and all-around cybersecurity practitioner, has been laser-focused on trends in the security analytics and tooling markets.
In a recent blog she tackles the growing evolution of security information and event management systems over the past decade and calls out a number of outdated or dishonest criticisms that are regularly lobbed at the technology today.
SC Media caught up with Mellen to learn more about the way SIEMs are mischaracterized by rival marketers, the increasing convergence of security analytics tooling and why automation needs are poised to loom large over the market in the next decade.
So the headline for your article is “The top five lies security vendors tell about SIEM.” Do you see some of these criticisms as more than just conventional wisdom gone wrong? Are vendors being actively dishonest about some of these challenges and if so, what are they hoping to get out? Who benefits from perceptions that SIEMs are a waste of time?
Mellen: I hope that they’re not trying to actively be deceptive. I do hope that this is something where these vendors are used to being on the practitioner side from a long time ago and they think this is still the experience that security teams have. I find that highly doubtful, because a lot of this revolves around just not having the pulse of the end users and not really understanding how end users currently work.
These questions come up quite frequently and these comments [like that security teams hate their SIEMs] really come up quite frequently. It’s fun to say that SIEMs are bad, it makes you seem like the cool guy in the room and everybody laughs, but the reality is when you look at the data, when you talk to practitioners, they all use the SIEM, a lot of them love their SIEM and it plays a really pivotal role in the [Security Operations Center].
That’s what I was trying to get across here because I think that, as the operating system of the SOC, SIEMs often get thrown under the bus in marketing messages from these other vendors trying to really push the message [that] ‘we can be your replacement SIEM.’ But we haven’t actually seen something that’s able to take the place of the SIEM…so ultimately it is a tad overblown, especially with these five lies that I highlighted.
You write a number of times that some of the myths you describe may have been true at one point, but not anymore. I’m curious what drove those changes?
Mellen: This definitely reflects changing end user needs, for sure. That explains a lot of the integrations that we’ve seen SIEMs really expand with, it also explains the convergence of SIEM and [Security Orchestration and Automation] into security analytics platforms, which has been such a big and important shift so that engineers can not only detect threats but also investigate them and actually respond to them in one place.
So all of that is really driving transformation of SIEMs to security analytics platforms, and then on top of that you also have compliance and other use cases. It’s arguably been slow going, but it is something that has been 100% driven by the need for security teams to understand more and to be able to dig into what’s going on in their environment better.
When you look at the different types of platforms and tools available today between SIEM, security orchestration and response, endpoint or extended detection and response, to what extent are these tools competing with one another? Is there a competitive mindset between say, an XDR vendor and a SIEM vendor?
Mellen: It’s a question I often get from security teams trying to decide what tools they should be using and prioritizing in the SOC. Ultimately, SIEM and SOAR are not necessarily competing, but now that they have reached the point of convergence… standalone SOAR capabilities are competing with security analytics platforms that can not only have the same elements but also the log management, the compliance use cases and the detection and response use cases.
Endpoint detection and response is an example of an input into the SIEM, through this granular collection and detection and response attributes that EDR is able to provide. It can be an input to the SIEM, it can also be standalone; but it alone is not enough to meet the use cases that the SIEM is able to give security teams in the SOC.
Extended detection and response systems is an attempt for EDR vendors to take on the SIEM for more and more use cases over time. It’s not ready to replace SIEM right now but that is definitely the future path that XDR vendors talk about in terms of what it’s going to become.
It gets kind of murky and it’s almost like you need a Venn diagram to understand where each of these is, but [security analytics platforms] are definitely on a collision course to be main competitors, and SIEM and SOAR are converging into one solution that’s supposed to address more detection investigation and response use cases as well as compliance.
How would you describe or relate EDR systems to that larger group? They’re more about measuring things at the endpoint and device level than the network, but do some of their functions or roles overlap with the others in ways that are worth discussing?
Mellen: I would separate it and not put it in the same category, but I can understand why people do that because as we’ve seen time and time again, incident responders love using EDR technology to detect and respond to threats.
Ultimately, there are other sources of telemetry that they use both for detection and then also for deeper investigation like the network. I classify them differently because typically with SIEMs, with security analytics platforms and XDR, they are taking in a wide variety of different security data as opposed to EDR which is restricted to endpoint data that’s collected by an agent. So there are architectural differences and there’s also outcome differences based around what the data is.
One of the complaints I often hear about SIEMs is that they bury you in security alerts and it’s difficult to find that needle in the haystack or that one alert that is truly problematic. Do you buy that premise and if so, does that jive or conflict in any way with the second myth you bust: that SIEMs can’t scale up?
Mellen: In some cases I absolutely do buy that. I think that it is absolutely a challenge and it is one reason why XDR rose in recognition in the past two to three years. That said, there are many security teams using their SIEM right now and not struggling under a massive load of false positives.
It’s all about choosing the data that you’re bringing in carefully, making it very use case driven, so that you’re not just bringing all of the data into the SIEM; you’re bringing the most important data in and that will help reduce those false positives.
Doing that in conjunction with tuning…when you successfully tune the alerts that are happening in the SIEM, you can get higher efficacy alerts. The challenge with this is it’s an ongoing process. You have to be regularly tuning, you have to be regularly building playbooks for your SOAR capability.
And that’s the thing that XDR is looking to take to the next level. You need these aspects but let’s see if we can automate them better so you don’t have to be constantly tuning, so you don’t have to be constantly building playbooks, so that the tool can do some of that work for you.
There are some security teams that are struggling with their SIEM. A lot of times that is a problem that is inflicted on them by bringing in as many different data sources as they can hoping that will solve their problem rather than doing the necessary tuning and strategic work.
Where are you seeing mistakes in implementation? Is it a matter of proper configuration, or a staffing and headcount issue? How are organizations doing SIEM wrong and how does it exacerbate some of the problems you’ve talked about?
Mellen: Honestly, it comes back to those resource constraints…where if you don’t have someone who knows what they’re doing with the SIEM, you’re going to run into issues.
Because that’s when security teams are like ‘we’re just going to funnel everything into the SIEM, and hopefully we’re going to catch stuff…and the tool will be able to support us.’ As opposed to having someone who is strategically thinking ‘what are the most important sources of data coming into the SIEM? What are the tools that we need to support it?' [Is it] something like an EDR as opposed to just pulling all the Windows event logs off of the endpoints you have in your environment? I’m not really a fan of this term but it’s about having someone dedicated to that caring and feeding of the SIEM.
Automation hype is sometimes viewed cynically by segments of the information security community or pegged as an oversimplification of how threat intelligence work operates in practice. But will that end up being true in this case, where you’re dealing with ever-increasing data processing requirements for some of these tools?
Mellen: I absolutely think that automation is one of the key ways that we’re going to be able to do this. I say that with the caveat that the way I think about automation is fairly different from the way it gets characterized in certain places.
The challenge right now is we don’t have enough people, we don’t have enough skills and we don’t have enough time. So the way that I think about automation is to automate all the things that are happening in the SOC that are just mundane tasks that the analyst has to do but doesn’t want to do. To take away all of those aspects and leave them with the final decision making that takes the creativity, that takes the accountability.
I don’t want to automate response. I want to automate response recommendations, so that the analyst can say this is what we need to do for the situation, we’re just going to execute it. Make their focus be on using their own creativity, their own intelligence to solve these problems.
You say that even as SIEMs have made a tremendous amount of progress over the year, there remain challenges and shortcomings. What are some of the realities about SIEM failings today that the industry will need to work to turn into myths 10 years from now?
Mellen: It really all comes back to automation. One of the biggest challenges that I see with SIEM technology is speaking to what we just talked about. SIEMs do automation but they do a very fragile form of automation, which is really just about having a human being doing the manual effort that gets put into the SIEM. But they still have to do things like tuning; they still have to do things like rule development, building playbooks for their SOAR. Those are not adaptable the way we need them to be adaptable.
One of the first pieces that I wrote for Forrester was on humanizing security operations, and one of the reasons is because the tools that we use today are not giving us what we need from an adaptability perspective to stay resilient in the face of not only a constantly changing tech landscape – and part of our job is enabling that tech landscape – but also from the standpoint of the attacks that we’re seeing.
One of the biggest things holding back security analytics platforms today is the crutch that they have on human generated automation as opposed to a solution that is able to automate these minute actions in order to not rely on a pre-built playbook... to say these are the little actions you need to take that add up to a whole. So we can recommend these more specifically and more closely based on different circumstances, not rely on one general circumstance to define how we approach this problem time and time again.
You say at the end of your piece that detection, exceptional user experience, and automation for investigation and response as areas where SIEM vendors should focus on improving. Why are those the most important?
Mellen: Detection is always an important one, because ultimately the challenge is that we have – whether we like it or not – built this system where SIEMs want you to bring in more data, and when you bring in more data it costs more. So, they need an approach that doesn’t rely on this excessive amount of data coming in to perform these detections, because ultimately what they’ve built is a big data problem that is a very challenging thing to solve.
At the end of the day someone has to use this stuff, and when thinking about user experience, I want to make sure the analyst can see what caused this attack, what are the steps that this attack took? How do I need to respond? Those are things that should be done with automation…without [users] having to go into deeper investigation.
It’s about automating the collection of data points that are going to be beneficial for the analyst to initiate deeper investigation or just to open up the alert and see what’s going on. That’s ultimately where XDR is headed and one of the reasons why security analytics needs to pick up the pace in that direction as well.
