The most pervasive wisdom about preventing damage from ransomware is to backup systems. FujiFilm and Colonial Pipeline in fact, restored from back-ups. So in an era of increased concern about ransomware, is solving the ransomware scourge as simple as investing in some backups?
"If it was that easy, [ransomware] just wouldn't be an issue," said Riley Stauffer, security and incident response analyst at managed detection and response firm Pondurance.
Indeed, recovering from ransomware can be tough. Backups can make it easier. But they can't make it easy. Backups can be damaged, untested, prohibitively difficult to deploy, encrypted by attackers, or restore to the same breached state they backed up. They don't rid hackers from systems. They don't address secondary forms of disruption.
In fact, the law firm BakerHostetler calculated that 20% of its clients who restore from backups also end up paying the ransom. Executives from Colonial Pipeline and Mandiant, the firm running Colonial's recovery and remediation efforts, testified at a Congressional hearing last week that, even though backups ended up being sufficient to restore the network, the company still paid $4.4 million in ransom.
"Relying on the perceived wisdom about backups being enough is highly problematic," Jeremy Kennelly, senior managing director of analysis at Mandiant told SC Media. "Any organization that's expecting backups alone to resolve or allow them to get back up and running is going to have challenges."
Remediation is more than recovering files
"If we're called in on a Friday, a lot of executives' first questions is, 'Are we going to be up and going on Monday?' Just know you're not," said Stauffer. "I usually tell them to prepare for about two weeks of your IT team not getting a lot of sleep just to get a container on the thing and get you to a point where you can start to stand everything back up."
Restoring encrypted files might seem like the most important thing to do when faced with a ransomware attack. Unfortunately, recovering from backup might just mean recovering systems to a point where the attacker already had access. It doesn't fix the vulnerabilities that led to the breach. Resolving a ransomware attack isn't just a matter of the files any more than resolving a flooded basement is just a matter of pumping out the water. You also need to fix the leaky pipes.
"If your entire recovery strategy is based around restoring the data that has been encrypted, what you're doing is you're closing the door on understanding how that person obtained access, whether they still have access with information they've learned about your network, and what they touched in your network," said Kennelly.
In fact, the vast majority of companies that pay ransom are revictimized, according to a new Cybereason report.
Kennelly said organizations need to prepare to run simultaneous recovery and investigation operations, and anticipate that the investigation might force the recovery to pause or reverse course.
One of the worst negotiating positions to be in, said Kurtis Minder, CEO of GroupSense and a well known ransomware negotiator, is to defy the ransomware group, only to later find out that the restoration left the door wide open for the group to return.
"A partial backup can help in negotiation, because we can communicate that we're partially restored and it's not worth it to us to pay the full amount. But you also have to be careful. If you say that to the threat actor, you better be super sure they can't get back in and mess things up. We've seen that a bunch of times," said Minder.
There may be more to the extortion than losing access to files
Backups prepare you to recover files. They do not prepare you for so-called double extortion, where hackers threaten to leak files they have stolen from your network.
Ransomware operators have been using leaks as a motivator for years, most famously when the Dark Overlord group leaked Orange is the New Black episodes in 2017 after a post-production studio involved law enforcement after paying a ransom. It is now a pervasive component of ransomware, with operators hosting dedicated leak sites.
Recently, DDoS protection firm Netscout has seen a new element of triple-extortion — encrypting files, threatening to leak files and running a DDoS attack while victims have been mulling over the ransom note.
"It certainly drives a sense of urgency," said Hardik Modi, assistant vice president of threat and mitigation products at Netscout. "You're trying to make the decision about whether to pay up or whether to go for backups. And the ransomware actor sends a DDoS to say 'We haven't forgotten about you.' These are the times where you're trying to communicate with the world about what happened. They are not good times for a website to go down."
Modi says Netscout started to see DDoS being used as an additional stressor in attacks last year.
"We've now seen multiple groups that are using DDoS alongside their encryption and the breach aspects," he said.
Backups will not stop these kinds of threats. Attackers employ them, in part, because they know companies first line of defense is on a tape drive.
Prepare for your backups to let you down
"Right now I'm dealing with a situation where the backups that were networked were all encrypted," said Chris Ballod, associate managing director at Kroll's cyber risk practice. "And then they said 'no problem we've got tape backups.' But we find out that, of course, the tape backups are like a year old. That's not helpful."
Even if they were, he added, the software needed to see what's on those tape backups was encrypted. And all tape backups are sequential. "You have to restore the systems based on when they were put on the tape," Ballod said. "You don't get to select critical systems over others."
Backups are great when they work. But there are a lot of ways they might fail.
"At the end of the day, just because Bob from IT said we do the backups weekly — I've been in more circumstances where that's actually not the case, or Bob is no longer with us so we don't know where he put those backups," said Ballod.
If backups are networked, there is a good chance the ransomware group will have encrypted them, too. The best practice is to keep some form of offline back up just in case. But that can create its own problems. For example, what happens when the off site storage facility is only open on weekdays and you have been attacked on a Friday night?
Backups can easily be out of date to the point they are no longer useful or even compatible with current systems. Even companies that employ regular backups can sometimes lack the testing regime to ensure that the most critical backups are usable when the time is dire.
"What's pretty consistent with clients we've dealt with is that they don't test their backups," said Chad Vicknair, a backup and recovery expert at industrial networks cybersecurity firm aeCyberSolutions.
There are myriad ways for a backup to fail. Within the cybersecurity industry are jokes about "Schrodinger's Backup," the backup you don't know will work until your systems need to be restored. Vicknair said he has seen backups rendered unusable from firewalls creating timeouts between servers and storage.
And when backups do work, they might not work exactly the way people expect. Vicknair notes that, especially in the operational technology space, a growing amount of interdependency between systems makes it harder to just restore a critical system first. A manufacturing plant might not work without the just-in-time billing system located on a business network also up and running. And, said Ballod, the recovery process can often be days and millions of dollars in lost work product slower than paying a ransom.
So, then what?
Everyone that SC Media spoke to for this story believes backups are a critical component to defend against ransomware and other types of attacks. The key, they say, is to understand its limits and the preparations that need to be in place to successfully use a back up.
Vicknair advocates for a 3-2-1 approach — three copies of data, using two different technologies, with copy kept off site.
Backups need to be tested regularly. Good logging practices can accelerate the incident investigation process. Plans need to be in place for multiple types of extortion and for when every plan falls apart.
"Backups are essential — they're a key part of the risk management plan in general, much less if you get hit by ransomware," said Ballod. "But there's more going on."
