Voltage Control on Distributed Generation Systems based on Multi-Agent Reinforcement learning approach

Voltage Control on Distributed Generation Systems based on Multi-Agent Reinforcement learning approach

© 2023 by IJETT Journal
Volume-71 Issue-2
Year of Publication : 2023
Author : Tlotlollo S Hlalele, Yanxia Sun, Zenghui Wang
DOI : 10.14445/22315381/IJETT-V71I2P222

How to Cite?

Tlotlollo S Hlalele, Yanxia Sun, Zenghui Wang, "Voltage Control on Distributed Generation Systems based on Multi-Agent Reinforcement learning approach," International Journal of Engineering Trends and Technology, vol. 71, no. 2, pp. 186-196, 2023. Crossref, https://doi.org/10.14445/22315381/IJETT-V71I2P222

The voltage control problem due to bidirectional power flows is more apparent when heterogeneous distributed generation systems (DGS) are integrated into the grid. In this paper, a novel method of voltage control in distributed generation systems based on a reinforcement learning technique is proposed. DGS incorporating renewable energy resources are highly complicated nonlinear dynamic systems. There are several challenges in employing the existing control methods. The novel method presented in this paper entrenches the Q learning algorithm into the voltage control problem of DGS. The Q-learning algorithm teaches agents responsible for decision taking in controlling the voltage and award the reward if the aim is achieved. The IEEE 9 bus test system with DG’s integrated is used with various controlling agents connected. The results show significant improvement in the reliability of agent communication and the efficiency of the proposed method.

Distributed generation, Reinforcement learning, Voltage control.

[1] Chongjie Zhang, and Victor Lesser, “Coordinating Multi-Agent Reinforcement Learning with Limited Communication,” Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems, pp. 1101–1108, 2013.
[2] Hasan Can Kiliçkiran, Bedri Kekezoglu, and Nikolaos G. Paterakis, “Reinforcement Learning for Optimal Protection Coordination,” 2018 International Conference on Smart Energy Systems and Technologies, pp. 1-6, 2018. Crossref, http://doi.org/10.1109/SEST.2018.8495830
[3] Praveen Prakash Singh, and Ivo Palu, “State Coordinated Voltage Control in an Active Distribution Network with On-Load Tap Changers and Photovoltaic Systems,” Global Energy Interconnection, vol. 4, no. 2, pp. 117–125, 2021. Crossref, http://doi.org/10.1016/j.gloei.2021.05.005
[4] Carlos Guestrin, Michail Lagoudakis, and Ronald Parr, “Coordinated Reinforcement Learning”.
[5] Patrick T. Manditereza, and Ramesh C. Bansal, “Multi-Agent Based Distributed Voltage Control Algorithm for Smart Grid Applications,” Electric Power Components and Systems, vol. 44, no. 20, pp. 2352–2363, 2016. Crossref, http://doi.org/10.1080/15325008.2016.1219889
[6] Won Nam Koong et al., “Voltage Control of Distribution Networks to Increase their Hosting Capacity in South Korea,” Journal of Electrical Engineering & Technology, vol. 16, no. 3, pp. 1305–1312, 2021. Crossref, http://doi.org/10.1007/s42835-021-00682-z.
[7] Nicholas S. Coleman, Christian Schegan, and Karen N. Miu, “A Study of Power Distribution System Fault Classification with Machine Learning Techniques,” 2015 North American Power Symposium, pp. 1–6, 2015. Crossref, http://doi.org/10.1109/NAPS.2015.7335264
[8] M.Reza Tousi et al., “A Multi-Agent-Based Voltage Control in Power Systems Using Distributed Reinforcement Learning,” SIMULATION, vol. 87, no. 7, pp. 581–599, 2011. Crossref, http://doi.org/10.1177/0037549710367904
[9] S. Anudevi, and Vinayak N. Shet, "Distribution Network with Optimal DG Placement and Protection Impacts: Review Analysis," SSRG International Journal of Electrical and Electronics Engineering, vol. 4, no. 2, pp. 12-16, 2017. Crossref, https://doi.org/10.14445/23488379/IJEEE-V4I2P103
[10] Zhen Ni, and Shuva Paul, “A Multistage Game in Smart Grid Security: A Reinforcement Learning Solution,” IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 9, pp. 2684–2695, 2019. Crossref, https://doi.org/10.1109/TNNLS.2018.2885530
[11] Ali Moradi Amani et al., “Voltage Control in Distributed Generation Systems Based on Complex Network Approach,” Energy Procedia, vol. 110, pp. 334–339, 2017. Crossref, https://doi.org/10.1016/j.egypro.2017.03.149
[12] Kyriaki E. Antoniadou-Plytaria et al., “Distributed and Decentralized Voltage Control of Smart Distribution Networks : Models, Methods, and Future Research,” IEEE Transactions on Smart Grid, vol. 8, no. 6, pp. 2999–3008, 2017. Crossref, https://doi.org/10.1109/TSG.2017.2679238
[13] Mulumudi Rajesh, and A. Lakshmi Devi, "Wind, PV Solar, Hydro and Hybrid Energy Storage System-Based Intelligent Adaptive Control for Standalone Distributed Generation System," SSRG International Journal of Electrical and Electronics Engineering, vol. 9, no. 11, pp. 67-94, 2022. Crossref, https://doi.org/10.14445/23488379/IJEEE-V9I11P108
[14] Ali Bidram et al., “Distributed Cooperative Secondary Control of Microgrids Using Feedback Linearization,” IEEE Transactions on Power Systems, vol. 28, no. 3, pp. 3462–3470, 2013. Crossref, https://doi.org/10.1109/TPWRS.2013.2247071
[15] Zhibin Liu et al., “Simulation Analysis of Distributed Micro-Energy Clusters Participating in Voltage Regulation and Peak Regulation,” Energy Reports, vol. 7, pp. 1529–1543, 2021. Crossref, https://doi.org/10.1016/j.egyr.2021.09.089
[16] Tine L. Vandoorn et al., “Voltage Control in Islanded Microgrids by means of a Linear-Quadratic Regulator,” IEEE Benelux Young Researchers Symposium, 2010.
[17] John W. Simpson-Porco et al., “Secondary Frequency and Voltage Control of Islanded Microgrids via Distributed Averaging,” IEEE Transactions on Industrial Electronics, vol. 62, no. 11, pp. 7025–7038, 2015. Crossref, https://doi.org/10.1109/TIE.2015.2436879
[18] Xiangyu Wu, Chen Shen, and Reza Iravani, “A Distributed, Cooperative Frequency and Voltage Control for Microgrids,” IEEE Transactions on Smart Grid, vol. 9, no. 4, pp. 2764–2776, 2018. Crossref, https://doi.org/10.1109/TSG.2016.2619486
[19] Juan S. Gómez et al., “Distributed Predictive Control for Frequency and Voltage Regulation in Microgrids,” IEEE Transactions on Smart Grid, vol. 11, no. 2, pp. 1319–1329, 2020. Crossref, https://doi.org/10.1109/TSG.2019.2935977
[20] Jonas Engels, Hamada Almasalma, and Geert Deconinck, “A Distributed Gossip-based Voltage Control Algorithm for Peer-to-Peer Microgrids,” 2016 IEEE International Conference on Smart Grid Communications, pp. 370–375, 2016. Crossref, https://doi.org/10.1109/SmartGridComm.2016.7778789
[21] Modu Abba Gana, Ishaku Abdul Dalyop, and Musa Mustapha, "Assessment of Reliability of Distribution Network with Embedded Generation," SSRG International Journal of Electrical and Electronics Engineering, vol. 7, no. 2, pp. 34-37, 2020. Crossref, https://doi.org/10.14445/23488379/IJEEE-V7I2P107
[22] Nader A. El-Taweel, Hadi Khani, and Hany E.Z. Farag, “Electrical Power and Energy Systems Voltage Regulation in Active Power Distribution Systems Integrated with Natural Gas Grids Using Distributed Electric and Gas Energy Resources,” International Journal of Electrical Power & Energy Systems, vol. 106, pp. 561–571, 2019. Crossref, https://doi.org/10.1016/j.ijepes.2018.10.037
[23] Ge Shaoyun et al., “Coordinated Voltage Control for Active Distribution Network on District Distribution Heating and Cooling Coordinated Voltage Control for Active Network Considering the Impact of Energy Storage,” Energy Procedia, vol. 158, pp. 1122–1127, 2019. Crossref, https://doi.org/10.1016/j.egypro.2019.01.277
[24] P. C. Olival, A. G. Madureira, and M. Matos, “Advanced Voltage Control for Smart Microgrids Using Distributed Energy Resources,” Electric Power Systems Research, vol. 146, pp. 132–140, 2017. Crossref, https://doi.org/10.1016/j.epsr.2017.01.027
[25] Um-e-Batool et al., "Controller for Voltage Profile Improvement of Double Fed Induction Generator based Wind Generator," SSRG International Journal of Electrical and Electronics Engineering, vol. 7, no. 12, pp. 21-26, 2020. Crossref, https://doi.org/10.14445/23488379/IJEEE-V7I12P104
[26] Josep M. Guerrero et al., “Hierarchical Control of Droop-Controlled AC and DC Microgrids - A General Approach Toward Standardization,” IEEE Transactions on Industrial Electronics, vol. 58, no. 1, pp. 158–172, 2011. Crossref, https://doi.org/10.1109/TIE.2010.2066534
[27] Xiaorui Liu, and Charalambos Konstantinou, “Reinforcement Learning for Cyber-Physical Security Assessment of Power Systems,” 2019 IEEE Milan PowerTech, PowerTech, pp. 1–6, 2019. Crossref, https://doi.org/10.1109/PTC.2019.8810568
[28] Aleksandar Boričić, Jose Luis Rueda Torres, and Marjan Popov, “Fundamental Study on the Influence of Dynamic Load and Distributed Energy Resources on Power System Short-Term Voltage Stability,” International Journal of Electrical Power & Energy Systems, vol. 131, 2021. Crossref, https://doi.org/10.1016/j.ijepes.2021.107141
[29] Ana Paula Carboni de Mello, Luciano Lopes Pfitsche, and Daniel Pinheiro Bernardon, “Coordinated Volt / VAr Control for Real-Time Operation of Smart Distribution Grids,” Electric Power Systems Research, vol. 151, pp. 233–242, 2017. Crossref, https://doi.org/10.1016/j.epsr.2017.05.040
[30] Madalina M. Drugan, “Reinforcement Learning Versus Evolutionary Computation : A Survey on Hybrid Algorithms,” Swarm and Evolutionary Computation, vol. 44, pp. 228–246, 2019. Crossref, https://doi.org/10.1016/j.swevo.2018.03.011