A Reinforcement-Learning-based Agent to discover Safety-Critical States in Smart Grid Environments

The complexity of industrial systems and processes has grown significantly in recent years, due to the integration of Information Technology (IT) and Operational Technology (OT) to monitor and control interconnected equipment in critical infrastructures, improving their production processes. Smart Grids are one of the possible examples of technology enabled by IT/OT integration. However, such energy distribution systems are exposed to several vulnerabilities that make them particularly susceptible to cyber threats with critical implications for human safety. Sophisticated attacks against OT infrastructures show few observable indicators in a large timeframe and traditional fault detection methods are ineffective in discovering safety-critical states, especially in large observation and action spaces.This paper presents a methodology to identify safety-critical command patterns within a Smart Grid ICS network. In particular, a modular simulation framework that embeds physical processes simulation, industrial-specific control protocols virtualization, as well as a Reinforcement Learning algorithm has been developed. The preliminary results in training and test cases have demonstrated the modeling and learning capability of the proposed approach.