As critical infrastructure facilities increasingly converge their IT and OT systems, visibility into traditionally isolated operational systems is turning into a key security challenge. Kristin Sanders, chief information security officer for the Albuquerque Bernalillo County Water Utility Authority, revealed last week how New Mexico’s largest water and wastewater utility has been addressing this challenge by leveraging a series of software solutions, sensors and internet-of-things tech.
Recognizing that the ABCWUA is “ahead of a lot of the water authorities” across the U.S. in terms of IT/OT modernization and compliance with the Water Infrastructure Act of 2018, Sanders offered advice to utilities that are seeking to make similar progress. She recommended to start by focusing on the Center for Internet Security’s top 20 controls and resources, and then see how you can implement in some different solutions to really knock out some of that low-hanging fruit.”
From an economics point of view, solutions that can be simultaneously implemented across both IT and OT environments – such as secure-access platforms with two-factor or multi-factor authentication – is a good place for a utility to start, she added, speaking in an online webinar organized by Cisco Systems.
“You can really make sure that you use this product across multiple things – RDP, VPN, email – all that are constantly being attacked,” said Sanders, noting that ABCWUA’s solution from Cisco and Duo Security processes over 12,000 authorizations per month.
The same philosophy applies to ABCWUA’s installation of its cloud-based enterprise network security software. “We’re able to roll that out not only for our desktop computers and for laptops and for VPN clients, but even for mobile devices,” said Sanders. “So we're able to take this one product and use it across a whole bunch of different endpoints to ensure that we're getting full coverage.”
Another key step is investing in training for employees so they understand both IT and OT operations, not just one or the other. “It wasn’t something that we were ever expected to need to know in the past,” said Sanders. But times change, so “one of the great things that we did was we actually hired somebody who was familiar with the operation side, and actually brought him in on the IT side” to help train the IT staff, said Sanders.
The authority, which serves more than 650,000 consumers and has had more than 100,000 smart meters installed since fall 2012, had historically kept its OT processes air gapped and separate from IT. “Now we're starting to see a convergence of these two into IoT, [although] traditionally the two groups never really worked a whole lot with each other,” said Sanders.
So far, “it’s been going really well,” she said. However, such modernization is not without risk. Infosec professionals at the plant must worry about malicious actors potentially sabotaging OT systems using the connected IT systems as an initial vector of compromise. Such an attack could theoretically affect the utility’s 3,000+ miles of water supply pipeline, 2,400 miles of sewer collector pipeline or its dual groundwater/surface water supply system.
Such dangers were highlighted last February when it was revealed that a malicious hacker attempted to poison the Oldsmar, Florida water supply after hijacking a remote access system used by employees at the city’s water treatment plant.
To control this threat, a utility’s security team must have visibility into OT activity. However, “there tends to be very antiquated equipment that runs within these industrial control environments,” and monitoring at the ABCWUA has historically been conducted manually, with employees monitoring operations on a screen, Sanders explained. “A lot of times, the security was kind of an afterthought; it was not built into the product originally because it was never intended to ever talk to a network,” she continued.
As IT and OT converged, untrained IT staffers were uncertain at first as to what an attack might look like. “Because there's no way of knowing that there's an anomaly if you have no clue what normal even looks like,” explained Sanders.
But the utility’s staff has started to gain improved network traffic visibility after deploying the industrial IoT security and visibility solution Cyber Vision from Cisco and integrating it with smart sensors and newly implemented industrial switches.
“It will do the baselining for you so you can start to build out this idea of what normal traffic is,” said Sanders. “That way you can see when something abnormal happens.” Now, the authority has visibility into its inventory of OT assets and endpoints, and it can detect new devices connecting to its systems and send alerts accordingly.
As part of its modernization, the authority also implemented a firewall management center, a secure access and policy management platform, a network controller and management dashboard, and a video conferencing platform.
According to Sanders, the improved security infrastructure has placed the utility in a position to ensure “staff safety and also the safety of our water.”
