The Log4j vulnerability requires security teams across all sectors to respond quickly, with speed of exploits demonstrating critical risk to those that fail to do so. So what did the last week look like for chief information security officers?
We asked Dawn Cappelli, CISO at manufacturing giant Rockwell Automation and member of SC Media sister organization Cybersecurity Collaborative, to pull the curtain back on her own experiences safeguarding systems and evaluating risk.
We all learned about the Log4j vulnerability Friday. What did you do first?
Cappelli: We first looked at all of the security measures that we had in place. The firewall vendors are coming up with updated firewall rules — so from the start we've been making sure that they're all being updated and that our endpoint detection and response solution is going to block exploits. We’re constantly looking at any IOCs that come in. We’ve run threat hunting queries in our network, going back to December 1, just looking for any evidence of any past exploit. So far we haven't found anything.
Our IT team has been taking the lead on patching, so the security team can focus on the security stuff [monitoring the network, indicators of compromise, and so on]. They are looking at our application inventory, looking at the GitHub site that CISA put together, checking on all of the applications that we use. And we also have our security liaison in our businesses involved, because they have developers — and these are their tools. We said, "Hey, give us a list of all of those tools," and now we are making sure no one is missing anything that they may be running in their labs.
When the IT team identifies an application that is iffy, they pull us. Some of these are really complicated, requiring a risk-based decision. Do we want to apply that remediation or is it riskier to do that?
On top of that, we have a product security team looking at all of our products; when you're a company as old as we are, we have a lot of products, so that involves looking at the software bill of materials for each. It’s great that we have the SBOMs, because that's been a big issue since the executive order came out — that a lot of companies don't have them.
So during the weekend, we were focusing on protecting our enterprise, doing that assessment of our products and all of our services for our customers and looking at what we had to patch — reacting to vendor notifications and then actively looking for every one we could find.
Then Monday rolled around. I'm sure that work continued, but what other considerations came up?
On Monday we started to look at acquisitions. We've been doing a lot of acquisitions. Right away, we realized some of them have been integrated into Rockwell, so they're part of us; but other ones are running their own infrastructure. So from day one, we realized we need to be reaching out, making sure that they're a part of all of this. We needed to figure out, are they doing everything that we're doing, or at least are we're doing it for them. So that was a big piece.
Then we started focusing on all of our manufacturing suppliers, our distributors — looking at everybody we can think of in that third-party ecosystem and reaching out to them. Do they have software bill of materials and do they have secure development life cycles? If they’re using Log4j, are they vulnerable?
What worries me the most about this whole thing is the small and medium businesses out there that either don't even know this is happening or they know, but they don't know what to do about it.
A company like Rockwell Automation has an enormous number of suppliers. How do you get those reassurances?
We have a third-party risk application system that we use, so we are going through that and saying, "Who haven't we reached out to?" For instance, companies that have our confidential or private information; we need to make sure that they're aware and they're taking measures. We do third-party risk assessments before we sign any contracts, so you would hope they should have a good security program or we wouldn't have signed the contract.
The balance of managing your own internal risk, and working with suppliers to ensure they do the same, seems incredibly overwhelming.
We waited to reach out for that reason — we figured if we reach out on Monday or Tuesday, they're just going to say, "We're on it. We're investigating. We'll let you know." Wednesday was a reasonable time to start reaching out. We’re trying to be a little considerate of how much we're bothering or not bothering; everybody is frantically working on this just like we are. We get customer requests and we have a statement that we send to them. So we're just trying to crawl in this together. I try to think of what kind of a burden are we going to place on companies if we bombard them with emails.
Internally, we have a meeting everyday, and every day we add more people to that meeting. In fact, I have to add someone else this afternoon because she's taking care of our software supply chain. But I also want the security team to rest. I told them anything that isn't an urgent priority we can push off till the new year. We're a week and a half from Christmas; I don’t want them to feel like we're getting behind on all this other stuff we should have been doing. We have to prioritize, and make those difficult decisions. We're back to a normal pace, but the worst is probably yet to come.
You and I have spoken before about how highly targeted the manufacturing sector is by ransomware groups. How does that factor in to response?
We're always looking for that suspicious activity. Lateral movement, privilege escalation. But what we came to realize when a few suppliers were hit in 2019 is that wait a second, we do have to care about their security program, because when they go down and they can't supply products for a month, that impacts us. We extended our third-party risk program to those critical manufacturing suppliers last year. But I'm afraid for the companies that haven't done that; they really need to reach out to those critical manufacturing suppliers to make sure. Some of them aren't very big. Since we extended our own program, we found some where we said, "Whoa, you have a lot that you're going to have to do if we're going to keep working with you." We need to spread the word; even if they don't touch your network, they don't touch your information, they can impact your business.
