The Power Behind a European Cisco-Based Electrical Substation

When you deliver electricity, safety and security are top of mind. Unless you protect the grid, you cannot ensure reliable power to your customers. Today, protecting the grid means both ensuring the internal integrity of the electrical system and protecting it from external threats, including cyberattacks that can shut down or take control of power plants and distribution networks.

CKW AG is a Swiss utility that provides electricity to industrial, enterprise, and residential clients, serving homes, hospitals, data centers, supply critical plants, and municipal distributors. Our smart energy portfolio for homes includes solar panels and batteries, heat pumps, and apps that monitor and control power consumption. We also operate hydroelectric power plants, and our thriving IT practice provides fiber, data center services, and other enterprise services to Swiss businesses.  

I’ve been with CKW AG for 30 years. I work primarily on the data transmission side of the business and am currently employed as a senior system engineer focused on securing our OT assets. We place a lot of value on operational technology (OT), hardware and software that monitors and controls the physical assets of the power grid, including network infrastructure. But we managed our OT and IT infrastructure separately, so we lacked the segmentation and automation features we needed to secure and protect everything.

In 2018, I joined the team building a new substation—a facility that transforms high-voltage electricity into usable power for distribution lines—in the Rothenburg industrial area of Lucerne. Given the increased threat of cyberattacks on electric system infrastructure, this new facility required us to rethink how we secure our OT assets. 

Rethinking Security at Our Rothenburg Substation

Previously, CKW AG used Siemens Ruggedcom switches at our substations. These devices were designed at a time when the networks in the substations were still operated autonomously and had no monitoring and no segmentation. This was at a time when safety functions in substations were not yet considered relevant.

The new substation uses the IEC 61850 standard, the International Electrotechnical Commission’s reference architecture that defines communication protocols for intelligent electronic devices at electrical substations. It's a comprehensive standard that approaches OT with security in mind, but IEC 61850 doesn't take account of the advances in manageability and security that have been happening in the world of IT networking. To that end, we wanted to go beyond the base standard, automating the management of our IT infrastructure and segmenting our data network into the smallest possible sections. In some cases, that means only allowing a single host on a network segment.  

The reason for keeping segments small is simple. Should we ever encounter a technical issue or detect lateral movement, we can isolate only the affected segment without impacting the rest of the network. In the event of a system-wide failure, we can focus on bringing back the most critical hosts or segments first, before shifting our attention to secondary systems. This level of segmentation also allows us to fine-tune access to OT infrastructure at the substation level. We can set firewall rules and access privileges for specific hosts, thus creating air walls that isolate substation network segments from our primary network and the outside world. 

Increasing our automation capabilities allows us to avoid human error, which often leads to security concerns. Automation is essential for managing large numbers of devices and configurations, and it enables us to respond faster when we detect suspicious activity.

Partnerships for the Future

To move forward with our new Rothenburg facility, we needed OT network equipment that would work well in the walled garden of the substation while also offering IT tools that provided robust protection against outside threats. We wanted to strengthen and simplify network management while improving security, optimization, and automation. The lifecycle of a substation is 10–15 years, so we also needed hardware and software with longevity.

After creating an initial design, we sought a partner who could help us realize our vision. At the time, the previous supplier lacked the robust offerings we needed for the scope for our project, and its knowledge of network technology was limited to applications within the strict confines of substation operational technology. That’s when another power utility recommended Netcloud AG, a Swiss IT service provider with extensive expertise in enterprise networks, cybersecurity, and infrastructure optimization.  
 

After our initial discussions with Netcloud AG, we knew we’d found a company with the breadth of experience that could help bring our plans to fruition. The team at Netcloud AG did more than analyze our plans. They considered the big picture, searching for potential cracks in our network infrastructure and security posture. After carefully considering our needs, Netcloud AG suggested that we work with Cisco. 

We’d long worked with Cisco in our core network, and Netcloud AG helped us realize that Cisco also had what we needed to extend into our substation. The networking giant offered a wide range of solutions that were well suited to OT security and automation. We also knew Cisco products were highly reliable, easy to upgrade, incredibly flexible, and perfectly adapted to the extended lifecycle of our substations. By expanding our Cisco environment to include industrial and OT networking and security solutions, we were confident we could develop a setup that would last. 

Building Substation Infrastructure Using Cisco OT and IoT Solutions

CKW AG engineers worked with the Netcloud AG team to build and deploy a substation solution using Cisco technology. We started with Cisco Secure Firewall Management Center to unify and streamline firewall management, application control, intrusion detection, and port and protocol control. We built our substation around a Cisco Secure Firewall ISA 3000, an internet appliance explicitly designed for OT and IoT security applications. Almost everything was set up and tested in the laboratory and only then moved to the high-voltage facilities. Much of the work was done remotely, so there was little need for the Netcloud AG team to be present on site.
 

The ISA3000 is IEC 61850-compliant and offers full visibility and control of industrial protocols and applications for a range of automation vendors, including Omron, Rockwell, GE, Schneider, and Siemens. It also leverages Cisco Talos to protect unpatched OT devices. 

We paired the ISA 3000 with Cisco ASA-5500-X with FirePOWER services, the industry’s first threat-focused next-generation firewall (NGFW) to protect against advanced cyberattacks. We use Cisco ASA-5500-X in the central locations (data centers). The networking of the ISA 3000 via the transit network with the Cisco ASA-5500X is normally interrupted and is only switched on by hardware relay if required. The syslog messages from the ISA 3000 are sent to the Cisco CSM via data diode. This ensures that the system is monitored at all times. 

Next, we added Cisco IE (Industrial Ethernet) 4000 and 4010 switches for layer 2 and layer 3 Gigabit Ethernet. These switches are centrally managed via Cisco DNA Center and double as SD-Access nodes.
 

In addition, we deployed Cisco Secure Network Analytics within our core network. As we continue to familiarize ourselves with the solution and roll out Secure Network Analytics into our substations throughout 2022, we will be able to see everything within our substation infrastructure, advanced telemetry, and smarter network segmentation. When the ISA 3000 firewall is connected to the core network, Secure Network Analytics checks the respective connections.

We also purchased Cisco Identity Services Engine (ISE) to enable software-defined network access control. ISE provides the AAA for the control technology in the substations. The network is statically segmented with VLAN and the ISA 3000 firewall controls the network transitions. A third-party IDS provides AI endpoint analytics in the substations, and that IDS monitors all flow controls in the network. The big advantage of the Cisco Identity Services Engine (ISE) is that it can provide AAA in full. We can therefore implement MAB as well as certificate-based access controls. Furthermore, the ISE is used as a radius server for RBAC, which we can implement fully. ISE has all the necessary functions to provide a fully comprehensive AAA in the substations as they evolve.

The result is a single-vendor solution that provides deep visibility into our substation network, enhanced OT and IT infrastructure control, and a scalable environment with enhanced cybersecurity standards. 

Working with a single vendor like Cisco means that we know where to turn if we ever have an issue. We also benefit from working with an industry giant that has been in the business for decades. Unlike smaller vendors, Cisco isn’t going anywhere.

Building the Future of Electricity in Switzerland

I’m proud that CKW AG is the first electric utility in Europe to build our substation infrastructure around Cisco products, and I’m confident we won’t be the last. We recently presented a talk about our substation network at a conference in Berlin, and my industry peers were stunned at our achievement. They had never seen a single-vendor setup that seamlessly integrated substation OT and IT management, segmentation, and automation until we walked them through our deployment. 

CKW AG took a chance by partnering with companies that don’t deal exclusively with IEC 61850 technologies, and our collective efforts have been a great success. We have already rolled out a second substation using Cisco technology, and 10 more are in the planning stages. With help from Cisco and Netcloud AG, we are pioneering better ways to deliver electricity to the Swiss market.