Cybersecurity enterprise solutions are getting better at recognizing malicious activity conducted via APIs and Windows Management Instrumentation tools, but they still need improvement in terms of identifying and stopping defense evasion techniques, according to Frank Duff, director of ATT&CK evaluations at Mitre.
This week, Mitre Engenuity – The Mitre Corporation’s tech foundation for public good – released the results of its independent evaluation of 29 vendors to see how their products were able to detect and in some cases block known Mitre ATT&CK techniques associated with the financially motivated cybercriminal groups FIN7 and Carbanak.
This is the third such evaluation performed by Mitre Engenuity, after previously looking at solutions’ ability to spots tactics associated the Chinese threat actor Gothic Panda (APT3) and the Russian nation-state group Cozy Bear (APT29). But it’s the first time the foundation’s evaluations focused on financial cybercriminal activity, and the first time that product solutions’ effectiveness were tested in Linux-based servers as well as in Windows environments.
Individual vendor results can be found here in this report, although MITRE Engenuity does not actively rank the solutions or compare them against each other. (For the record, Check Point Software Solutions had the most detections: 330 across 174 substeps.) But Duff did inform SC Media of several key takeaways from the collective data. For starters, he said, vendors are leveraging the ATT&CK framework better, in that they are “figuring out how to incorporate ATT&CK into their dashboards in a smarter way, so it's not necessarily leading to alert fatigue, but it's still enriching the data.”
In other words, it’s no longer as common for users to be bombarded with alerts for every action that might be connected to a known malicious technique. “So you don't just see that ‘this process opened’ or ‘this file got read.’ You're now getting the context of what [those actions] could potentially be in a way that's not just flashing lights in your face,” Duff continued.
Malicious actors leveraging WMI and directly accessing APIs have historically been “high-noise events” that have been tough to pinpoint as malicious activity amongst all the heavy volumes of data, but solutions are getting better at this too, Duff said.
“That's really where the EDR marketplace is moving towards – trying to collect these high-volume logs in a more effective way that will allow [malicious actions] to be exposed, versus a couple years ago when they would have just said, ‘I can't do API monitoring like that. That's way too much data. Maybe one day.’ And I think we're getting to the point where it's starting to be that one day,” said Duff.
On the other hand, the ability to identify and thwart defense evasion techniques is an area that “definitely needs a lot more attention,” said Duff, especially the “scanning for which software is on your system, so they know how to avoid it.”
“That obviously is a very deep concern, because we're relying a lot on this software to defend us,” said Duff. “And if adversaries know what's on a box and they know what these capabilities are, [then] they potentially know ways of getting around them. And so I think the defense evasion needs to have a spotlight under it and continue to improve how it is, or how those detections happen.”
Mitre refers to Carbanak as a financial cybercrime group that has primarily targeted banks, often using its own eponymous malware in the process. FIN7 similarly uses Carbanak malware, but has largely targeted the U.S. retail, restaurant, and hospitality sectors, also using point-of-sale malware. These two groups are sometimes lumped together, but are considered separate entities.
Mitre Engenuity selected FIN7 and Carbanak for its latest evaluations due to heavy interest among the business community.
“They are both heavily documented across industry. So [this evaluation] allowed us the opportunity to address a new threat, one that was affecting the public as a whole,” said Duff. “That's really what the main drive was.”
The inclusion of Linux-based environments in the evaluation was also a significant development, and representative of the increasingly hybrid nature of IT environments.
“There is still not a huge amount of information publicly available on how [malware is] executed on Linux, which makes it very challenging for us since we're doing emulation and we really want to do it in the spirit of the specific adversary,” said Duff. However, “there was some pointing to Carbanak threat group specifically using Linux, and so we were able to pull from those techniques and create what we feel is a pretty faithful representation of what they could do.”
The scenario Mitre Engenuity cooked up is that the imaginary attackers first infiltrate a Windows box, but upon discovering a Linux server, they pivot there and then pivot back out to another Windows machine. That was the foundation’s “put-the-toe-in-the-water” attempt to understand vendors’ Linux coverage, Duff stated.
The vendors involved in the evaluation seem to understand and appreciate the value of the exercise.
“We know that cybercriminals are always evolving their tradecraft,” said Ismael Valenzuela, senior principal and head of AC3, the applied countermeasures team at McAfee. “In the most comprehensive evaluation to date, the Mitre ATT&CK team demonstrated their expertise completing four days of rigorous testing. This has a tremendous value to both our customers and our threat content engineers.”
“Fortinet is a firm believer in independent security testing of all kinds – effectiveness, performance and capability,” said John Maddison, executive vice president of products and chief marketing officer at Fortinet. “What we really like about ATT&CK Evaluations by Mitre Engenuity is that they not only show what a security product detects – and now protects – but also identify when, how and why. This insight “under the hood” of security products helps organizations to confidently apply the Evaluation results well beyond the specific campaigns emulated, to campaigns using similar… techniques, today and tomorrow.”
Meanwhile, members of the end-user community also benefit by being able to research each individual vendor and see which ones hold up best against the particular threat actors and threat techniques that they are most concerned about.
For the evaluation, vendors were provided with fake host environments – one a hospitality mock-up and the other a bank mock-up – which were set up on a Microsoft Azure cloud platform. The vendors then deployed their solutions on these environments, to see how they responded to threat behavior emulations. Mitre Engenuity essentially served as the red team, while also observing what the solutions missed and what got flagged as a false positive.
Last month, Mitre launched a new training and certification program that could finally provide the much-needed guidance security professionals need to more effectively and comprehensively integrate the respected ATT&CK framework into their security operations center assessments and threat intelligence operations.
