SE Labs tested VMware NSX Network Detection and Response against a range of advanced persistent threats designed to compromise systems and penetrate target networks in the same way as criminals and other attackers breach systems and networks.
Full chains of attack were used, meaning that testers behaved as real attackers, probing targets using a variety of tools, techniques, and vectors before attempting to gain lower-level and more powerful access. Finally, the testers/attackers attempted to complete their missions, which might include stealing information, damaging systems, and connecting to other systems on the network.
The following is the first of a two-part preview of SE Labs’ report on the exercise, which can be downloaded here. In this installment, we look at SE Labs’ testing methodology, as explained by SE Labs CEO Simon Edwards. In the next installment, we delve into what researchers learned about attack groups and techniques along the way.
How testing was done
Edwards noted that NDR products are designed to recognize attacks as they pass through one or more networks -- like CCTV systems monitoring the flow of information running through an organization, data center or other infrastructure.
Testing challenges: sensor accuracy
Edwards explained that there are a few different ways to test NDR solutions, “many of which are so synthetic as to be misleading.” You could run a tool that pushes network packets containing elements of an attack, for example, and that could trigger a detection by the NDR sensors. “Or it might not,” he said. “It depends how those sensors are designed.”
He said a very accurate sensor might not generate an alert when analyzing such fake test traffic. Ideally it would only alert on a real attack so that the team in the Security Operations Centre (SOC) focuses on significant events only.
“Parts of an exploit, malware or suspicious login are not a threat,” he said. “Only a real attack looks like a real attack. A basic sensor might report problems with every packet that appears to be bad without looking at the context.”
For example, he said, if a user logs into a system that they use regularly, an unsophisticated system might register that as a problem. A more intelligent one would recognize that all is well and hold back the alert. But it might sound the alarm if the same user logs in from an unusual part of the network. This could be a sign of an attacker moving between systems and using stolen login credentials.
Use of MITRE ATT&CK framework
Edwards said SE Labs’ tests we make no assumptions about how security products work and run full attacks, from the beginning stages to completing the final mission, which might be data damage, theft, or the creation of a persistent presence.
“We replicate the behaviors of real-world attackers and use the MITRE ATT&CK framework to map out the attack chains used in every test case,” he said. “We also perform benign activities to ensure that the product we are testing isn’t just alerting without discrimination. By running the most realistic set of attacks possible we put NDR products to a significant challenge. Can they detect real attacks in real time, often using unique scripts and malware?”
The testers behaved as attackers, pivoting between systems (and generating lateral movement traffic), attempting to use credentials, exfiltrating data and creating command and control data flows.
The report’s findings offer insight into the techniques used by several prolific attack groups, including Fin7 and Carbanak, OilRig, APT3 and APT29.
Next up: Popular techniques used by these attack groups, and how NSX Network Detection and Response held up against them.
