If organizations aren't currently incorporating log management and monitoring as a critical part of their cybersecurity strategy, they aren't seeing the forest for the trees.
Logs record every system event that takes place across a network, as well as when and how it happened. And because logs are so detailed, security teams can determine whether their devices have been breached by closely examining them, making log management a proven, proactive approach to detecting intrusions and safeguarding data.
Given the evolving nature of cyber threats, the shift from on-premises data storage to the cloud, and the sheer amount of data generated daily — more than 2.5 billion gigabytes globally — it's not surprising that this cybersecurity practice has accelerated.
According to Research and Markets, organizations worldwide are projected to nearly double their log management and monitoring spend from $1.83 billion in 2021 to $3.43 billion by 2026. Yet, while log management and monitoring spend has increased, understanding why it's a wise decision requires exploring the benefits that come with investing in these tools.
The importance of log management
Logs tie events together within a system and correlate what's happened across multiple systems. Log management keeps track of system and application logs to gain insights into user activity, vulnerabilities, security breaches, and other potential issues. They are often structured as encrypted text files, but can also include binary data such as images and videos.
Security analysts can review logs in the moment to determine if a system has been breached. The WannaCry ransomware attacks, for instance, infected hundreds of thousands of devices worldwide in May 2017 and brought organizations such as FedEx, Renault, and the United Kingdom's National Health Service to a standstill. Users could detect whether the malware was present on their computers by scanning server logs for files that were encrypted with a specific extension or connections that were made through certain TCP ports.
SOC analysts can also preserve logs for forensic analysis following intrusions. After Equifax was breached in 2017 and the Bank of Montreal was attacked in 2020, investigators reviewed log files to determine how attackers accessed each company's systems. Closer scrutiny of those logs by each organization in the moment would have detected the entries earlier and limited damage.
How the cloud changed log management
Log management and monitoring have become increasingly complex as more companies adopt cloud storage solutions. According to Netwrix, organizations have moved 41% of their workloads to the cloud and expect to increase that figure to 54% within the next 12-18 months. Additionally, 80% of businesses use the cloud to store sensitive data, including personally identifiable information of employees and customers.
Organizations have traditionally had unlimited access to very detailed logs because their data has been stored on-premises. When data gets hosted in the cloud, tenancy agreements prevent access to the underlying systems, thereby reducing visibility and making conventional logging methods impractical.
Companies also have to consider cost because organizations are limited by the price and capacity of their cloud storage plan. They must decide which logs are most relevant for a cloud-based monitoring system to review since a massive number of logs gets generated, and sending them all to the cloud isn't feasible.
That results in security teams having less insight into potential threats, so they must trust that cloud providers can detect any malicious activity before it becomes an issue.
Ensuring successful log management
Even the best-trained security operations analysts need help reviewing the significant number of logs created. That's why many companies turn to a security information and event management system, or SIEM.
A SIEM uses machine learning algorithms to correlate data from various sources, filter out routine events, and alert analysts to suspicious activities or patterns that may signal malicious actors are present.
Companies need to make smart decisions about how they allocate resources. Understand what the budget allows and the licensing models the company’s cloud provider offers, as some models can restrict access to certain types of logs that could get used for meaningful monitoring efforts. If the provider gives the team logging tools or sets up a SIEM, find out what the gaps are and identify limitations. Even then, consider whether the team wants to move its logs back on-premises or find another provider that can offer better visibility into potential threats.
Many companies still find it difficult to automate incident response, so training the team’s analysts to incorporate a SIEM and review and act on its alerts has become critical to successful log management. With these procedures in place, the company would have taken valuable steps toward reacting quickly once incidents occur — or helping to prevent them altogether.
Three questions for organizations to ask
Ultimately, any log management and monitoring strategy still boils down to what the organization needs. Here are three questions for the team to ask:
Is the company trying to maintain compliance with regulatory standards? Is it trying to meet requirements for cyber insurance? And, does the company have a weak security posture that needs strengthening? A responsible approach to log management and monitoring can address them all.
It’s important to comply with data protection laws and industry standards given how much sensitive information gets stored. Understand the cloud provider's approach to compliance and the scope of the agreement to ensure the organization is protected should an incident occur.
Likewise, the rising frequencies and costs of data breaches have led to rising cyber coverage thresholds. Most insurers now require staunch log management processes, and some even go as far as mandating a SIEM to monitor and alert potential threats before a policy can be issued.
Price and storage capacity shouldn't serve as limits when it comes to effective log management. Companies need access to as much data as possible for effective monitoring and protection –the more data the company has, the sooner it can spot and address network intrusions.
Data breaches are surging in frequency today, and they're more costly than ever. A more vigilant approach to log management and monitoring, including the use of a reliable SIEM, can further safeguard the company against emerging cyber threats, regardless of whether data gets stored on-premises or in the cloud.
Brian Knudtson, director, product market intelligence; product manager, 11:11 Systems
