Collectively racking up a victim count in the tens of thousands, high-profile attacks targeting users of SolarWinds Orion and Microsoft Exchange serve as a harsh reminder that threats to software security remain one of the biggest issues facing the security landscape today.
And yet, even as both government and industry acknowledge the severity of the situation, strategies are fragmented at best, elusive at worst.
Indeed, evidence shows that the attack surface around applications is getting larger. Bugcrowd, which offers a platform allowing companies to connect their applications to a community of thousands of security researchers who root out for bugs and vulnerabilities, reported a 50 percent increase in total bug bounty submissions in 2020 compared to 2019. That tracks with other research that has found a record number of new vulnerabilities reported over the past year, many of which target faulty or shoddy software programs.
At the very least, the issue appears to be gaining more attention in board rooms and among policymakers. In a survey of more than 2,400 security technology decision-makers conducted by Forrester in 2020, improving application security capabilities and services was listed as the top tactical IT security priority over the next 12 months, a sign that businesses are starting to confront the growing threat head on.
The Biden administration is also poised to take action, with Reuters reporting that an upcoming executive order is likely to implement a raft of software-related security measures designed to prevent breaches like the one that hit SolarWinds and its customers. Specifically, the order would obligate software vendors who do business with the federal government to report a breach of their systems, and also require a software bill of materials on critical government IT programs.
But there’s a long way to go – and much more work to be done – if industry and government are going to succeed in stemming the rising tide of software-based attacks they’re facing on a daily basis.
A complex picture
The introduction of the cloud, APIs, open-source code and containerization has only added further complexity to the software development process. More recently, in the wake of the COVID-19 pandemic, many businesses rushed to put new apps online to continue serving their customers or moved existing ones to more unfamiliar cloud environments in ways that have created new security holes and oversights.
Software applications have always been vulnerable – hence the development of concepts like DevSecOps – but a number of factors have combined in recent years to supercharge existing concerns.
Sandy Carielli, a principal analyst at Forrester who serves as the lead author for the company’s annual report on the state of application security, told SC Media that applications are still one of the most common attack vectors in external data breaches, but awareness is rising at the executive level and newer tools like static and dynamic application security testing and SOAR (security orchestration, automation and response) have made it easier than ever to integrate security during the code-writing process. That being said, the aforementioned changes to the software development process over the years means there are always new considerations or weaknesses to which practitioners have had to adapt.
“If I was going to summarize it in one sentence, I would say ‘Not great, but moving in the right direction,'” said Carielli when asked to evaluate the current state of app security. “One caution, though, is that yes, it’s slowly starting to get better, but every time we change the way we build applications, every time we advance in terms of how we architect, every time we make it easier to build and manage and introduce new architectures and new structures – whether it’s containers or serverless or Infrastructure as Code or APIs – every time we do that, we introduce new risk and we discover that there are new ways to breach an application that perhaps we hadn’t thought about.”
The rising rate of automation in the software development process could also be creating new holes. Timur Gilmullin, DevOps team lead at security research company Positive Technologies, said most major software vendors have more or less fully automated their Continuous Integration/Continuous Development processes over the past five years, from building components and installers to deploying them on testbeds, testing and publishing updates.
“Each of these stages is susceptible to a targeted attack,” similar to the attack that corrupted an update of SolarWinds’ Orion software last year, said Gilmullin.
Bad security can also create a negative feedback loop, where damaging vulnerabilities are exploited by bad actors, those successes are observed by new groups and those actors then dedicate more time and resources towards finding more vulnerabilities.
Ransomware actors routinely look for easy vulnerabilities to exploit in victim organizations. Traditionally that has meant phishing lures, credential theft and other low-effort pathways, but some observers point to episodes like the recent weaponization of the Microsoft Exchange vulnerabilities by multiple ransomware groups and worry that attacks at the application software level could become a more attractive option in the near future.
“That’s a space we’re going to really see attackers take more advantage of moving forward,” said Jen Miller-Osborn, deputy director of threat intelligence at Palo Alto Networks’ Unit 42 research team. “I think that’s an area we’re going to see ransomware actors move into unfortunately, because it tends to be very successful and it’s got a relatively low barrier to entry once there are [proof of concept exploits] published on the internet and that’s another way that attackers can potentially make a lot of money quickly.”
Concepts like DevSecOps, a framework for weaving security teams and principles earlier and more naturally into the software development process, have been around for years and were supposed to address many of the concerns around software security. However, while the concept is heavily pushed in some security circles and evangelized at conferences, many dev teams still fail to incorporate the ideas into their process, particularly for cloud-based projects. Some developers feel a lack of standardization for this methodology has hampered more widespread adoption.
“There is a set of general recommendations and many specialized programs for monitoring security during development. Each of them solves one minor problem, but requires a lot of time to learn and implement,” said Gilmullin, explaining why some organizations struggle to incorporate DevSecOps into their workflow. “Implementing a set of tools for secure development is not easy – in the absence of proper support, training and outreach activities by the DevSecOps specialists, these tools simply will not be used.”
The more complex the development environment, the more complex the security tools used to scan, test and analyze the code integrity. While that reality can allow for more granular security testing, it can also muck up the development process and create awkward tradeoffs between security and other business goals.
Integrating security tools into the dev process "is not a one-button click thing," said Reed Loden, chief open source evangelist at HackerOne, a vulnerability coordination and bug bounty platform. It takes work and if its not done correctly, "it breaks the pipeline and that blocks developers from actually doing work," Loden said.
"Security has always kind of been seen as that blocking factor in a lot of ways, and so people are less apt to actually care about it, no matter the company," said Loden. "They just say 'Hey, if security is going to block me from doing something, then that's not helpful to me and I'm not going to be interested in actually dealing with this [problem].'"
A note of cautious optimism
In speaking to a range of experts, many offer the same general outlook: while the tooling and practices around software security are slowly getting better, a number of changing trends and evolutions in software development over the past decade have combined to decrease visibility and increase the vulnerability and attack surface.
Gary McGraw, a software security expert and co-founder of the Berryville Institute of Machine Learning, lays out a “Trinity of Trouble” that is affecting the ability of software developers to spot and fix problems in their code: complexity, extensibility and widespread networking.
As software has become more complex, it has become harder to understand how all the different pieces of code interact together and create openings for attackers. Since many applications are designed to be perpetually upgraded and expanded over time, they eventually grow beyond the analytical capabilities of most security teams. Finally, the widespread networking of IT systems and assets – particularly in the post-COVID-19 era – means that a single compromise today is often more impactful than in years past, with the potential to infect multiple systems or victims.
One of the features expected to come out of the Biden administration’s upcoming executive order is a software bill of materials. Allan Friedman, director of cybersecurity initiatives at the National Telecommunications and Information Administration, has spent years working with other stakeholders on a framework for a software bill of materials that could introduce more transparency into the software world. A software bill of materials (or “S-BOM” as Friedman and colleagues call it) is essentially a list of all the different pieces of code that go into making a software application.
Virtually all applications are composed of chunks or snippets of older code that are stitched together by developers to perform a new function. These pieces come from different places – previous internal software, open-source code libraries or licensed third-party applications – and are recycled so much that it is often hard to know where they originally came from, or whether they share commonalities with other vulnerable software products that are regularly reported by security researchers.
“Our scope is quite ambitious. We are trying to foster an attitude of transparency in all software on the planet,” said Friedman in an interview. “Not just your traditional modern enterprise software, but also in areas of critical infrastructure, in automotive and energy and healthcare, where especially devices are going on be on-premise, they’re going to be embedded in systems that might have a long lifespan and it’s very important to know what’s under the hood.”
Friedman and others believe that breaking down and tracking the provenance of these different bits of code can have numerous, multiplying effects of software security. It can feed into allow lists and deny lists to protect networks from risky code, be used to monitor potential end-of-life software issues and, once it’s implemented widely enough, and become a factor in consumers' security evaluations of software program lacking a SBOM. It can also help inform cyber insurers, who may choose to raise premiums for companies that can’t document where their code comes from.
The research Friedman’s group has done found that there are actually few implementation hurdles that would prevent many organizations from implementing an SBOM for their software beyond the general need to marshal awareness and support for the idea. One big complication is the need to harmonize and standardize the process to ensure organizations are putting out the same information consistently and in a way that can facilitate follow up security actions.
“The basics of SBOM are there and an organization can implement it. The challenge is if we want to implement it in a machine-readable, automatable capacity, there’s still a little more work we need to do so that an SBOM from one vendor looks enough like an SBOM from another vendor that a company can integrate them,” Friedman said.
One thing that is unlikely to change is the agile nature of modern software development, which tends to emphasize speed and continuous updates in the development process. Carielli said the DevOps concept is deeply entrenched in the software development community and aligns with the larger business needs of most company executives.
“Development teams are tasked with getting features in customers’ hands, and ultimately that’s the job of the business,” said Carielli. “So, they’re going to move fast, but the challenge is when security doesn’t have the integration and the tooling and the relationship with development to move at that same speed.”
That doesn’t mean nothing can be done, and Carielli said tighter integration between security and development teams can address a lot of these problems without fundamentally changing the nature of the modern software development cycle. Ironically, SolarWinds may now be one of the few companies that has given its CISO the authority to hit pause on any software update where speed and time-to-market are the prime considerations and there are outstanding security questions.
Despite these trends, McGraw and others sounded a note of optimism that good security is still possible.
“I’m optimistic that we’re making progress in the software security field,” said McGraw during a March 25 virtual event hosted by Neil Daswani, co-director of Stanford Online’s Advanced Cybersecurity Program and author of "Big Breaches.” “Though there will continue to be breaches and we’re going to continue to have problems, we actually do know what to do to build secure software. Now it’s up to us all as a society to do it.”
