A new open-source threat hunting language designed to help relieve the workload of security operations center analysts through an embrace of automation and a platform-agnostic approach is now available to the greater cyber community.
Developed by programmers at IBM Research and IBM Security, the language, Kestrel, has just been officially approved for use by members of the Open Cybersecurity Alliance. The OCA is an “open project” launched in late 2019 by tech standardization organization OASIS with the goal of addressing the lack of integration between cyber solutions and promoting interoperability across the security industry.
According to an OCA announcement of IBM’s contribution, threat hunting has historically involved a siloed, manual approach to detection that requires tech skills and knowledge that can be difficult to find in prospective SOC employees.
“Instead of benefiting from the threat hunting community’s collective knowledge and sharing code, threat hunters often end up working in isolation rewriting the same programs following each attack,” the release states. However, Kestrel allows threat hunters “to express hunts in an open, composable threat hunting language,” thereby facilitating better collaboration moving forward. The language also leverages automation “to execute tedious hunting tasks, allowing threat hunters to focus on higher priority tasks,” while efficiently reusing best practices again and again as needed.
Dee Schur, senior manager, development and advocacy, stressed to SC Media in an interview that the open-source nature of the project is a core benefit. “Sometimes standards development can be like parallel play,” said Schur, referring to the concept of children that play alongside each other and observe, but don’t interact.
But “when you start talking about open source, you're really talking about a lot more interactive and intuitive kind of play,” which ultimately can result in a more universal solution with the potential to be accepted by a de jure body.
“The future of cybersecurity automation is in analyst augmentation and platform interoperability. Kestrel embodies both of these traits, enabling SOC analysts to hunt threats at scale using a standardized language,” said Vaughan Shanks, CEO of Cydarm Technologies, which along with IBM is a member of the OCA governing board. Other members of the OCA, which formed in later 2019, include the Center for Internet Security (CIS), Cybereason, CyberNB, Cydarm, Cyware, EclecticIQ, EPRI, F5, IBM Security, McAfee, NewContext, Rapid7, S-Fractal Consulting, SafeBreach, SAIC, Tenable, ThreatQuotient, Tripwire and TruSTAR.
SC Media this week spoke with Jason Keirstead, distinguished engineer and chief technology officer of IBM Security Threat Management, to dive deeper into what differentiates Kestrel as a threat hunting language and the value he expects it to bring to the security community.
Let's begin with a little background on OCA and its recent endeavors as an organization.
The mission of the organization is really around increasing interoperability in the cybersecurity industry through the use of open source – and using that to accelerate the adoption of open standards. What led to the creation of this organization was an identification of a problem area over a number of years… and that is that the majority of our clients [and our industry partners’ clients] have a very large number of cybersecurity tools. Those tools are essential, yet they don't work seamlessly together.
Typically, our customers have to spend a lot of time money and energy getting their tools to talk to each other… First of all, that is investment that could be spent actually detecting threats, and instead they're spending it maintaining the integration between their tools. Secondly, because the tools are poorly integrated, there's the potential that they're missing threats, because they don't have complete visibility into their enterprise, as the information doesn't flow seamlessly from tool to tool, so the context gets lost. Things slip through the cracks.
What we are trying to do with OCA is bring together stakeholders to tackle this challenge head on. Because what we've identified is: standards alone don't solve for complete interoperability… and having open-source tools and libraries and code that implement these standards, and that products can consume of the box, can greatly accelerate the adoption of them.
So that’s why we formed this organization as an OASIS open project. Oasis open projects are focused on this nexus of standards and source code where you can create an open-source source project or pieces of code, and then use that code to create the standard later on. The notion is “code before paper,” and that's very much what we're trying to do in the OCA. We're focused on getting working solutions out to the market. And then, if those solutions evolve into standards, that's spectacular.
Explain the concept behind the Kestrel threat-hunting language, and the value it introduces.
Kestrel has been under development at IBM for close to two years now. The origins of the project started with some of our IBM research colleagues who work with [the Defense Advanced Research Projects Agency]. It is a purpose-built language that can be used by threat hunters to search for threats in a platform-agnostic way. The challenge that threat hunters have is that all of the different tools that they use in their day-to-day jobs speak different APIs, different languages. So if you're going to go and create a hunt around Crowdstrike, you have to learn one language, and then you want to run the same hunt in Carbon Black it's a different language. You want to run the same on Microsoft Sentinel or Microsoft ATP, it's a different language and API.
And all of these languages are extremely esoteric. So, becoming a subject matter expert on all these different things and keeping up with them is a huge learning curve. Most analysts are only able to specialize in one, to be frank. And, as a result of this, threat hunters, as opposed to focusing… on threat management and threat hunting, they're focusing on learning an API, and debugging esoteric language.
Kestrel leverages another OCA project that already exists called STIX Shifter, which was one of the founding projects with OCA. It's a data abstraction layer that does translation between native APIs and the Oasis STIX 2 standard… But it’s kind of a dumb translation… like a basic query-response. It's like: “Go and give me the data, and then here's your data table…” Kestrel runs on top of that, and it adds another layer of language and analytics on top of it so that you can do true threat hunting… So the combination of the two is what's powerful.
In the OCA GitHub repository, we're going to be adding more and more analytics to the Kestrel project over time to do things like outlier detection, behavioral analytics and process beaconing. And what's great about this project is every single one of those things that we, or anyone in the community, adds, you can now run that against any tool. So it's not beaconing that works with Crowdstrike; it’s beaconing that works with 20 different platforms, because every single time you add something to Kestrel it works everywhere. And that's really what's unique about it.
Can you also explain in a further detail how Kestrel leverages automation to execute tedious hunting tasks while allowing the threat hunters to focus on higher priority undertakings?
Because it is an orchestration language, Kestrel lets you create these chains of hunts.
Threat hunting is usually a multi-stage endeavor. It's not just: “Run this query and get your results back.” You run something, you get results back, you then want to pivot around this piece of data and then go and fetch more data to mine those data sets, etc, etc.
Kestrel allows you to orchestrate all of these activities in a chain. So you can go and get data set A, get data set B, combine those together, run an outlier detection algorithm on it, pull that back, do another level of enrichment by pulling in third party threat intelligence feeds, and then run that through machine learning-based beaconing detection. And then take the results and output them. You can orchestrate all of that in a single notebook with a series of commands and then automate it. Because now once you have orchestrated it, you can now automate that and run that on a regular basis, whenever you see any of those patterns of activity.
Kestrel leverages Jupyter notebooks – and Jupyter is a tool that's becoming very popular with threat hunters, because of the ability to slice and dice the data inside the tool… With Kestrel you can write a hunt and execute it inside a Jupyter notebook, and have it run across all of your connected data sources, bring back that data, and then apply chains of analytics, all in the same notebook.
And what's really cool is, we envision a community, building up around this project. Because these analytics that are developed, these hunts, they can then be shared with other users via GitHub.
Was the language in its earlier stages only available to IBM clients?
Correct. It was internally built. We'd been getting feedback from some of our clients, as well as our internal security teams... They helped with the design of the language and using the product as we've been building it out. So they were the primary stakeholders before the donation. [But] it was never intended to be internal only.
We identified some time ago that we planned to donate this to the OCA. IBM Security is very much invested in the mission of open security, and things of this nature that can help raise the bar against the threat adversaries.
But because the OCA is an open project with open governance, IBM does not control the OCA… we couldn't just come in as IBM and say, “we're giving you this.” What we had to run through the process… We had to go through a vote balloting process. All of the other OCA governing board members voted to accept the project.
Why do you think the project was met with such approval? What value did the members see in the language, and what current threat hunting challenges does Kestrel tackle?
The project resonates so well with the OCA mission because it's very aligned with what we're trying to accomplish, and it's consumable. It’s a tool that SOC practitioners can use today. Some of the existing OCA projects – they haven't been tools; they're more libraries focused on improving interoperability. But this is the first OCA project that's actually specifically targeted at SOC practitioners.
Take, for example, an enterprise that has multiple, different EDR and SIEM solutions – which is really the majority of cases. Most enterprises have at a minimum a SEIM and an EDR, and we know a lot that actually have multiples of each.
[In such a case,] when you have a hunt team or even a SOC team whose looking for a specific pattern of activity, you really don't have a lot of options other than to create that analytic in each of these platforms and then execute them… And each one of those is a different data model with a different language, a different way you'll have to go and analyze that data in that tool, and you run into what a colleague of mine calls “Swivel-Chair-itis where you're just swiveling your chair from tool to tool, attempting to get a holistic picture of your enterprise.
We're trying to basically bring more of the power of the crowd to the cybersecurity challenge. Because… across the industry… we do not collaborate enough around detection engineering. Whenever there is a new zero day threat, or a new Mitre attack chain that needs to be discovered, enterprises go and build those detections for these things in a vacuum.
Because when there is a [critical] Microsoft Windows printer spooler vulnerability dropped like there was [this week], every CISO team on the planet is going around trying to build detections to find that in their environment. And there's not really any value add to the cybersecurity industry that everyone is doing the same thing over and over and over again. And the reason that we do that is… we are unable to collaborate, because we don't have any way to do detection engineering that's cross platform. All the detection engineering is proprietary and tied to the platform that you happen to have deployed.
