Cybercriminals are getting smarter by the day. Attack surfaces are expanding rapidly, and the attacks themselves are becoming more sophisticated and frequent. At the same time, organizations are generating and storing more data than ever before. Companies of all sizes need the ability to aggregate and analyze their data across multiple systems in order to stay secure and compliant, and SIEM (security information and event management) tools are many businesses’ preferred way of accomplishing that.
SIEM technology not only helps enterprises meet compliance requirements (e.g., SOC 2, HIPAA), it’s also an essential part of an organization’s threat management program. Its ability to gather and analyze data from a variety of sources — from network devices, to servers, endpoints, and more — makes it an indispensable tool for uncovering and addressing security threats.
But this strategy isn’t without its challenges: When used in a traditional manner, SIEM tools can become extremely costly. This is due to the fact that most SIEM vendors charge based on the amount of data ingested; usually gigabytes per day or events per second (EPS). These are tough numbers for organizations to predict, so they most often sign up for an estimated amount of EPS initially and end up needing more later, causing costs to skyrocket. If left unchecked, these soaring costs can lead security professionals to make critical decisions based on dollars and cents rather than the organization’s actual security needs.
I personally know people who’ve experienced this firsthand: Several years ago, a member of my team used SIEM tools in their role as a chief security architect for a SaaS company. Their data bill would creep higher and higher each month, and in time the company was burning more than $1 million per year on SIEM data ingest alone. This was just their data spend — their overall security spend was higher still. There was a feeling that nothing could be done to control this cost; there were no knobs they could turn to dial their solution up or down, so they continued paying the bill.
Clearly, not every company has the budget to spend over a million dollars on data ingest (or their security program as a whole, for that matter) each year — nor do they need to. Thankfully, solutions exist today that let organizations adjust the volume of data being processed by their SIEM system, reducing the overall cost while still providing security functionality. It’s the adjustable, knobs-and-dials capability I could’ve benefitted from greatly in the prior role I mentioned.
These solutions cut down on SIEM data ingest costs by giving organizations the following options:
Agentless only
If an organization doesn’t have the budget for their SIEM system to process all of its data for whatever reason, they can opt to go agentless. This means the log data transfer occurs without an agent — the log-generating host might transmit its logs to the SIEM directly, or an intermediate logging server could be involved, such as a syslog server. This option won’t let a company detect every single threat that might pop up, but some amount of data is available, therefore some level of detection is possible. And — let’s face it — having some level of detection at a lower cost is certainly better than having none, i.e., turning all data ingest off. Should a security incident occur, organizations will be able to investigate its root cause.
Agents in query mode with data turned down
Another option is having agents in query mode with data turned down. This is ideal for organizations who have leaner security teams, or perhaps aren’t yet ready to run a security operations center (SOC) for whatever reason. The amount of data ingested can be turned up or down on an as-needed basis, and since the agents are running in query mode, they’re not shipping an excess amount of event data. Again, some security tooling is always better than none, and this is an effective way to detect and remediate more potential threats without breaking the bank. Real-time alerts may not be possible, but if something happens, the security team can use the agent to run queries to get insight into what occurred.
Agents with data turned up
There’s also the option to run agents with data turned up. This could be the preferred option for a company with a robust budget, a large security team, or in instances where security has become more critical for whatever reason. This setting can be applied to an organization’s entire fleet, or just specific areas for enhanced threat detection capabilities. While considerably more expensive than the aforementioned options, the data can always be turned back down if the company has a bad quarter or needs to conserve costs for some other reason.
Structuring telemetry at its source
These solutions also provide companies the option to structure their telemetry at its source. By organizing telemetry data in a clear, predefined format or schema, the SIEM system will be able to efficiently process the data without the need for extensive transformation during the ingestion process. Ultimately, this option can enable organizations to eliminate expensive ETL (extract, transform, and load) down the road for security data lakes, as well as indexing costs if leveraging unstructured search.
Focused log collection
Finally, there’s the option to selectively collect logs that are most relevant to security monitoring, reducing the overall volume of data flowing into the SIEM. There’s a common misconception that collecting every single log for the SIEM will result in better analytics, but all this really does is increase costs and generate more false alerts. Instead, companies can opt to send only artifacts (detections, alerts, events, etc.) and then use SOAR (security orchestration, automation, and response) to pull additional data from the lake when more context is required for a specific investigation.
Whichever strategy a company chooses, having this range of options to fall back on lets security teams become more closely aligned with business stakeholders. When security teams can present stakeholders with more choices, having conversations surrounding security is invariably easier.
Gone are the days when organizations had to choose between either uncontrolled costs due to a firehose of data overwhelming their SIEM system, or shutting the switch off altogether and missing potential security threats. Solutions that let companies adjust the amount of data ingested by SIEM systems are providing the best of both worlds: Cost-effective security capabilities that can be adjusted based on their specific needs, keeping both security teams and stakeholders happy.
