Implementation of an Advanced Remote Engineering Platform (AREP)

The Estonian Distribution System Operator, Elektrilevi, has been working on improving remote engineering of substation devices for over a decade. This article details the stages of these efforts and how they evolved towards the realization of their new Advanced Remote Engineering Platform (AREP). This AREP platform implements a cyber-secure Privileged Access Management (PAM) system for managing the fleet of Intelligent Electronic Devices (IEDs) that automate most of the Estonian distribution grid.

Elektrilevi Background

Elektrilevi covers 95% of Estonia with its electrical grid and brings electricity to almost all Estonians. Elektrilevi has more than 533,000 electricity network service customers. To ensure electricity supply, Elektrilevi maintains and upgrades 63,000 kilometres of power lines and 25,300 substations throughout Estonia.

Due to Estonia's low population density, a single consumer has to support a much larger portion of the network in comparison to many other European countries. This also affects the network-related costs.

Elektrilevi aims to continuously improve their network service while maintaining a reasonable price range. To assure the quality of their service, Elektrilevi uses effective and efficient solutions, continuously works on weatherproofing the network and digitizing the power distribution grid management.

In order to improve the quality of Elektrilevi network service, Elektrilevi makes daily efforts to enhance the automation of the network, develop innovative solutions and carry out data analysis. Elektrilevi believes that through the implementation of smart IT solutions, they make life easier and more secure for everyone. The AREP deployment is one of the programs driven by this belief.

Remote Engineering Vision: the Beginning

In the early 2010s, Elektrilevi began investigating efficiency improvements and cost savings that may be achieved through the deployment of remote engineering functionality of IP-connected IEDs.

At CIRED in 2017, Elektrilevi presented an article reporting on its two initial trial deployments of remote engineering systems using different user interface and communication architectures.

First Generation (Until 2014)

The first approach enabled specialists to use either their own PC or two dedicated workstations to securely connect to IEDs in 40% of Elektrilevi's Primary Stations via the utility's secure SCADA WAN and Station VLANs. This approach added more flexibility for users but did require each user to have their own copy of IED management software installed and licensed. Also, it was considered not secure enough, since the user computers were simultaneously connected to SCADA WAN and corporate network. As a result, the system usage was limited to 30% (12/40) of potential users.

Second Generation

The second approach added a centrally administered Terminal Server to which authorized users could RDP from their own PCs. This expanded system enabled users to remotely access IEDs at 320 sites. This system was more usable since it only required the central system to have all the necessary IED Management software installed and licensed.

This new system was more secure since the terminal server had no access to public networks. This approach also improved and simplified the user experience, resulting in usage statistics improving up to 70% (28/40) of all potential users. As this second system was expanded and usage increased, so did the technical challenges of managing all the users and the manuals that describe site-specific connection details.

These initial internal approaches did, however, validate the benefits of Remote Maintenance. It was estimated that 1.5 hours were saved on each task that would have otherwise required a site visit.

The Advanced Remote Engineering Platform (AREP) Vision

The two initial efforts were primarily about enabling remote access to devices, thus avoiding the time and expense to travel to the field device. The AREP vision was expanded to be much broader and more ambitious than these initial approaches. AREP is an engineering platform envisaged to provide a flexible, efficient, and secure way of working in heavily digitized critical infrastructure.

The overall objective of the AREP is to deploy a software system that enables remote engineering of operational technology (OT) devices used by the power distribution grid operators to monitor, update and automate management tasks of the respective devices.

The solution defined requirements for the ability to automatically change passwords and retrieve Firmware Versions, Configuration Files, Event Logs and Fault Files on a user-initiated or scheduled basis. Other use cases included managing IED password policies, supporting a password checkout capability, and performing automated IED password updates.

The core of the AREP is based on a Privileged Access Management (PAM) software solution customized to support specific OT devices not available out-of-the-box. To support contractors and vendors, the PAM supports a centralized work-order-based access management capability to provide them secure, temporary access to specific devices.

The AREP vision also included European Union eID identification capability to identify the specific person performing work and eSignature for signing work-orders.

System Requirements and Selection

Elektrilevi developed a comprehensive set of business use cases and functional and non-functional requirements for the Advanced Remote Engineering Platform:

Initial production capacity targets were set at 80 users, 30 simultaneous sessions, 10,000 managed IEDs, and 70 management applications. Five-year expansion targets: 300 users, 50 simultaneous sessions, 100,000 IEDs, and 300 applications.

After comprehensive market research, Elektrilevi selected SUBNET Solutions Inc.'s PowerSYSTEM Center product, which best aligned with their cybersecurity architecture and functional needs.

Cybersecurity Architecture

The system isolates external and corporate users from operational technology networks using DMZ and jump servers, limiting connection duration to work-order specifications and automatically managing credentials to eliminate user knowledge of passwords and network addresses.

By implementing the AREP, the need for physical travel to the IED on the grid is noticeably lower, resulting in lower maintenance costs, quicker access to detailed network diagnostic data, and faster reaction on power distribution grid faults.

Project Funding and ROI

Co-financed through the European Union's Connecting Europe Facility (CEF Digital), the project quantified savings over seven years:

Additional unrealized savings potential exists from manual task automation, prevention of IED obsolescence, and improved grid reliability metrics.

Deployment Timeline

Platform Scale