Case study: Network clarity

A critical infrastructure company in Northern California gained visibility into its network...while securing its clients, reports Greg Masters. 

As one of the largest natural gas and electric utilities in the United States, Pacific Gas and Electric Co. (PG&E) is the very definition of critical infrastructure. And, like any large business, it has to defend its tens of thousands of devices on different network segmentations from cyber attacks. 

PG&E has more than 20,000 employees working at 1,000 facilities and branches across northern California. The San Francisco-based utility company provides electricity and natural gas to approximately 16 million people and has more than five million business and residential accounts spanning more than 100,000 square miles of service territory. Its annual revenue is around $17 billion. 

As might be expected, certain of its networks need to be kept separated for security reasons and its team managing the implementation needs full visibility at all times. To that end, it was agreed that a solution was required that would allow the PG&E security team to visualize both networks on an ongoing basis with limited connectivity between them, monitor for suspicious activity and quickly understand and isolate risk.

“Ensuring the safety of the public is paramount,” says James Sample (left), senior director and CISO at PG&E. “We must ensure our systems are reliable and resilient to be able to do this.”

He says that one of the things his team was trying to drive toward was a comprehensive risk management solution. “There's a lot of controversy with those who say that risk management is too ‘pie in the sky,' and you have to do the whole practical implementation,” he says. “Our view is that if you are doing risk management properly, you are doing that whole lifecycle. You are doing the analysis component of it, as well as the detailed execution component of it.”

In moving toward this holistic risk management solution, one thing that is really important is visibility, he says. “The more you can see, the more you understand about your environment, the more data you have and the more data points you have – the more informed decisions you can make and determine where you want or need to invest.”

Sample says he and his team sought to ensure they weren't focused only on chasing vulnerabilities. “We could use all of these tools in a reactive fashion and spend our time chasing vulnerabilities, but instead, we want to plug in trends, plug in more data and be more proactive.”

His team – which consists of 1,700 full-time technology personnel, plus 500 contractors – is dealing with a complex network. So, for his team to be able to protect it and perform computer network defense activities and monitor for vulnerabilities, they needed a better way of understanding and visualizing the activity. “And on top of that, we need to provide that visibility to folks who didn't design or operate our network, but really need to understand our network topography,” he says.

A search for an appropriate solution began. His team compared offerings from RedSeal Networks side-by-side with other vendors in this space. RedSeal, he says, for a variety of reasons was their choice “hands down” to join its security solutions as an overall component of a comprehensive risk management strategy.

“First, RedSeal was readily able to show that they could do the things that we needed the solution to do, and it seemed to be far more mature and built-out versus conceptual,” he says.

Second, he says, because PG&E is a Cisco shop, one of the key requirements was support and integration with Cisco. Another key requirement was that the tool also integrate with its vulnerability management solution, which is nCircle's IP360. This satisfied a need for visibility into network activity – with intelligence on the vulnerabilities that exist on the systems and how exposed those systems are to attack  – that could be shared with partners. “RedSeal excelled in its ability to demonstrate these capabilities,” Sample says.

Third, he says, in working with RedSeal throughout the evaluation process, his team came away confident with how it works and supports business partners. “RedSeal was on the same page with us in terms of strategic thinking about how to integrate into a larger system,” he says. “They demonstrated their worth not just operationally, but tactically as well.” 

The RedSeal platform automatically creates an end-to-end picture of the network infrastructure, based on the “as-built” configurations of the live networking equipment, says Mike Lloyd, CTO at Santa Clara, Calif.-based RedSeal Networks. “The RedSeal engine then adds data about endpoints, generally obtained from vulnerability scanners. This combines the ‘chess board' of the network with the ‘chess pieces' – the endpoints on which the business runs.”

Combining them answers four major types of questions, Lloyd says: Visibility, including the ability to see what is missing from the defensive big picture; checking individual elements against best practice rules; testing the network as a whole to ensure it meets stated policies for network access; and simulating attacks on the infrastructure to find weak points.

Smooth deployment

Deployment of the RedSeal solution went smoothly, says PG&E's Sample. “RedSeal sent its team in to do some complex tuning work and they were a tremendous help with the queries and analysis. Instead of saying, ‘Hey, here's an upgrade every quarter,' RedSeal provided excellent hands-on support out of the gate, and to this day as needed.”

He adds that he is finding the RedSeal solution easy to manage and operate. “The system is meeting our expectations and is playing an integral role as it feeds right into our overall risk management framework to be able to prioritize and drive investment decisions,” he says.

Overall network visibility is probably the biggest thing that RedSeal Networks has done for his team, Sample says, because it imports in all of the configurations of the entirety of PG&E's firewalls and routers into RedSeal. 

For example, he might detect a machine that is behaving suspiciously, so he can quickly search in RedSeal to determine if that is a machine at a particular physical location and he can run some queries to determine the type of access that exists to and from that given machine.

On top of that, the offering assists with compliance regulations, which as the company must comply with Critical Infrastructure Protection (CIP) regulations from the North American Electric Reliability Corp. (NERC), have specific callouts around managing ports and services and in-scope networks, Sample says. “RedSeal helps with our monitoring and analysis to achieve and maintain compliance.”

Sample and his team stay in close contact with their RedSeal account team quarterly to talk about what's coming. And to give them new ideas on how PG&E can better use the system they regularly discuss interesting use cases that RedSeal sees with other customers.

“We are continuously seeking to bridge the two paradigms of real-time security operations with more periodic risk management,” Sample says. “We've had some excellent conversations with the RedSeal engineers with how we might be able to do that.”

The boardroom discussion

The implementation is just part of a big-picture game plan as Sample and his team are moving toward a non-traditional approach to security, as traditional security has only gotten them so far. “We are looking at how do we really take it to the next level? Security has become a boardroom discussion, so how do you take it to the boardroom?”

Traditionally, he says, security organizations have performed the maintenance and support and tactical components of the tools, as well as trying to do some of the analytic components. But, his team looks at these as two distinct functions. Maintaining the tool and keeping it up and running, and keeping it healthy have really become a commodity in a sense and an IT operational top task. The consumption of the data and doing all of the analytics that come out of the tool are the core competency of a security risk management type group, he explains. “We've done a lot to separate those two functions and spreading those out across the organization like they should be.” 

As an example, he points out how the human resources department operated years ago. They would go out and get their software, run it and support it, he says. Today, that is no longer done by HR. Instead, they are a user of the software. “Security is growing up into that function and RedSeal Networks is well positioned and useful for organizations seeking to make that evolution.”

RedSeal releases product software upgrades once or twice per year, with patch updates in between as needed, adds RedSeal's Lloyd. There is also a weekly published Threat Reference Library to keep the security analytics up to date.

“Don't confuse end-to-end network access control with the more basic firewall management,” says Lloyd. “There is a paradigm shift going on. World-class organizations prefer an end-to-end, holistic approach because they cannot possibly manage every detail in their infrastructure. Medium and smaller organizations are beginning to realize the limits on their scaling potential if they stick with the old compliance-centric approach, ignoring the potential of security and risk analytics.”