research paper power generation

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Published: 09 February 2024

Artificial intelligence-based methods for renewable power system operation

• Yuanzheng Li   ORCID: orcid.org/0000-0001-8052-1233 1 , 2 ,
• Yizhou Ding   ORCID: orcid.org/0000-0003-3121-1487 3 ,
• Shangyang He   ORCID: orcid.org/0009-0000-2380-8617 3 ,
• Fei Hu   ORCID: orcid.org/0000-0003-2386-9035 1 ,
• Juntao Duan   ORCID: orcid.org/0000-0003-3957-7248 1 ,
• Guanghui Wen   ORCID: orcid.org/0000-0003-0070-8597 4 ,
• Hua Geng   ORCID: orcid.org/0000-0002-8336-6731 5 ,
• Zhengguang Wu 6 ,
• Hoay Beng Gooi   ORCID: orcid.org/0000-0002-5983-2181 7 ,
• Yong Zhao   ORCID: orcid.org/0009-0001-0977-6580 1 , 2 ,
• Chenghui Zhang   ORCID: orcid.org/0000-0003-2317-5930 8 ,
• Shengwei Mei   ORCID: orcid.org/0000-0002-2757-5977 9 &
• Zhigang Zeng   ORCID: orcid.org/0000-0003-4587-3588 1 , 2  

Nature Reviews Electrical Engineering volume  1 ,  pages 163–179 ( 2024 ) Cite this article

844 Accesses

3 Citations

2 Altmetric

Metrics details

• Energy grids and networks
• Energy management
• Renewable energy

Carbon neutrality goals are driving the increased use of renewable energy (RE). Large-scale use of RE requires accurate energy generation forecasts; optimized power dispatch, which minimizes costs while satisfying operational constraints; effective system control to ensure a stable power supply; and electricity markets that support bidding and trading decisions associated with RE. However, the uncertainties in RE generation make renewable power systems challenging to operate. For example, the intermittent nature of wind power can make it difficult to balance the supply and demand of electricity in real time; therefore, traditional power sources could be needed to meet the demand, which can increase electricity prices. This Review outlines the potential of artificial intelligence-based methods for supporting renewable power system operation. We discuss the ability of machine learning, deep learning and reinforcement learning methods to facilitate power system forecasts, dispatch, control and markets to support the use of RE. We also emphasize the applicability of these techniques to different operational problems. Finally, we discuss potential trends in renewable power system development and approaches to address the associated operational challenges such as the increasingly distributed nature of RE installations, diversification of energy storage systems and growing market complexity.

The large variabilities in renewable energy (RE) generation can make it challenging for renewable power systems to provide stable power supplies; however, artificial intelligence (AI)-based methods can help overcome these challenges.

Deep learning methods can provide accurate RE generation forecasts to help balance the supply of and demand for electricity.

Reinforcement learning techniques can effectively handle the increased computational complexity associated with optimizing power dispatch for renewable power systems to ensure that costs are minimized and operational constraints are met.

Renewable power systems are subject to greater instabilities than traditional systems, which can lead to voltage and frequency fluctuations in the power supply. AI-based techniques can provide real-time control signals to facilitate generation-to-demand control.

Reinforcement learning techniques can also be used to analyse market behaviours and optimize decision-making to support the effective integration of RE into power markets.

Future AI-based methods will need to solve the challenges that could arise from increases in the number of entities supplying RE and the diversity of energy storage systems, which will further complicate renewable power systems.

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 12 digital issues and online access to articles

111,21 € per year

only 9,27 € per issue

Buy this article

• Purchase on SpringerLink
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Machine learning-based energy management and power forecasting in grid-connected microgrids with multiple distributed energy sources

Developing a three stage coordinated approach to enhance efficiency and reliability of virtual power plants

Data-driven assisted real-time optimal control strategy of submerged arc furnace via intelligent energy terminals considering large-scale renewable energy utilization

BP p.l.c. Energy outlook. BP https://www.bp.com/en/global/corporate/energy-economics/energy-outlook.html (2023).

Li, Y. et al. Deep reinforcement learning for smart grid operations: algorithms, applications, and prospects. Proc. IEEE 111 , 1055–1096 (2023).

Google Scholar  

Sharma, S., Xu, Y., Verma, A. & Panigrahi, B. K. Time-coordinated multi-energy management of smart buildings under uncertainties. IEEE Trans. Ind. Inform. 15 , 4788–4798 (2019).

China Monitoring and Early Warning Centre for Renewable Energy Absorption. Evaluation and analysis of Chinese renewable energy utilization in the fourth quarters of 2021. China Power http://www.chinapower.com.cn/zx/hyfx/20220315/138719.html (2022).

US Energy lnformation Administration. As Texas wind and solar capacity increase, energy curtailments are also likely to rise. EIA https://www.eia.gov/todayinenergy/detail.php?id=57100 (2023).

Li, Y., Zhao, Y., Wu, L. & Zeng, Z. Artificial Intelligence Enabled Computational Methods for Smart Grid Forecast and Dispatch (Springer Nature, 2023).

Razavi, S. & Tolson, B. A. A new formulation for feedforward neural networks. IEEE Trans. Neural Netw. 22 , 1588–1598 (2011).

PubMed   Google Scholar  

Ren, Z. et al. Embedding physics domain knowledge into a Bayesian network enables layer-by-layer process innovation for photovoltaics. NPJ Comput. Mater. 6 , 9 (2020).

ADS   CAS   Google Scholar  

Kingsford, C. & Salzberg, S. L. What are decision trees? Nat. Biotechnol. 26 , 1011–1013 (2008).

CAS   PubMed   PubMed Central   Google Scholar  

Noble, W. S. What is a support vector machine? Nat. Biotechnol. 24 , 1565–1567 (2006).

CAS   PubMed   Google Scholar  

Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J. & Müller, K.-R. Explaining deep neural networks and beyond: a review of methods and applications. Proc. IEEE 109 , 247–278 (2021).

Parlos, A. G., Chong, K. T. & Atiya, A. F. Application of the recurrent multilayer perceptron in modeling complex process dynamics. IEEE Trans. Neural Netw. 5 , 255–266 (1994).

Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. Commun. ACM 60 , 84–90 (2017).

He, K., Zhang, X., Ren, S. & Sun, J. in Proc. IEEE Conf. Computer Vision and Pattern Recognition 770–778 (IEEE, 2016).

Pascanu, R., Mikolov, T. & Bengio, Y. in Int. Conf. Machine Learning 1310–1318 (2013).

Li, C. et al. Long short-term memory networks in memristor crossbar arrays. Nat. Mach. Intell. 1 , 49–57 (2019).

Ravanelli, M., Brakel, P., Omologo, M. & Bengio, Y. Light gated recurrent units for speech recognition. IEEE Trans. Emerg. Top. Comput. Intell. 2 , 92–102 (2018).

Bellemare, M. G., Dabney, W. & Munos, R. in Int. Conf. Machine Learning 449–458 (ACM, 2017).

Henderson, P. et al. in Proc. AAAI Conf. Artificial Intelligence Vol. 32 (AAAI, 2018).

Li, Y. et al. Dense skip attention based deep learning for day-ahead electricity price forecasting. IEEE Trans. Power Syst. 38 , 4308–4327 (2023).

ADS   Google Scholar  

Hong, T. et al. Energy forecasting: a review and outlook. Res. Pap. Econ. 7 , 376–388 (2020).

Liu, H. & Zhang, Z. A bi-party engaged modeling framework for renewable power predictions with privacy-preserving. IEEE Trans. Power Syst. 38 , 5794–5805 (2022).

Gu, Y. & Green, T. C. Power system stability with a high penetration of inverter-based resources. Proc. IEEE 111 , 832–853 (2023).

Wang, Y. et al. Short-term load forecasting for industrial customers based on TCN-LightGBM. IEEE Trans. Power Syst. 36 , 1984–1997 (2021).

Li, C., Tang, G., Xue, X., Saeed, A. & Hu, X. Short-term wind speed interval prediction based on ensemble GRU model. IEEE Trans. Sustain. Energy 11 , 1370–1380 (2019).

Zheng, Z. & Zhang, Z. A stochastic recurrent encoder decoder network for multistep probabilistic wind power predictions. IEEE Trans. Neural Netw. Learn. Syst. https://doi.org/10.1109/TNNLS.2023.3234130 (2023).

Article   PubMed   Google Scholar  

Chai, S., Xu, Z., Jia, Y. & Wong, W. K. A robust spatiotemporal forecasting framework for photovoltaic generation. IEEE Trans. Smart Grid 11 , 5370–5382 (2020).

Wang, J. et al. (eds.) Advances in Neural Information Processing Systems Vol. 35 5941–5954 (Curran Associates, 2022).

Wu, F., Jing, R., Zhang, X.-P., Wang, F. & Bao, Y. A combined method of improved grey BP neural network and MEEMD-ARIMA for day-ahead wave energy forecast. IEEE Trans. Sustain. Energy 12 , 2404–2412 (2021).

Dudek, G., Pełka, P. & Smyl, S. A hybrid residual dilated LSTM and exponential smoothing model for midterm electric load forecasting. IEEE Trans. Neural Netw. Learn. Syst. 33 , 2879–2891 (2021).

Dhiman, H. S., Deb, D. & Guerrero, J. M. Hybrid machine intelligent SVR variants for wind forecasting and ramp events. Renew. Sustain. Energy Rev. 108 , 369–379 (2019).

Tantithamthavorn, C., McIntosh, S., Hassan, A. E. & Matsumoto, K. The impact of automated parameter optimization on defect prediction models. IEEE Trans. Softw. Eng. 45 , 683–711 (2018).

Li, Z. et al. Deep learning based densely connected network for load forecasting. IEEE Trans. Power Syst. 36 , 2829–2840 (2021).

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521 , 436–444 (2015).

ADS   CAS   PubMed   Google Scholar  

Yan, J. et al. Frequency-domain decomposition and deep learning based solar PV power ultra-short-term forecasting model. IEEE Trans. Ind. Appl. 57 , 3282–3295 (2021).

Zhou, X., Pang, C., Zeng, X., Jiang, L. & Chen, Y. A short-term power prediction method based on temporal convolutional network in virtual power plant photovoltaic system. IEEE Trans. Instrum. Meas. 72 , 1–10 (2023).

Abdel-Nasser, M., Mahmoud, K. & Lehtonen, M. Reliable solar irradiance forecasting approach based on choquet integral and deep LSTMs. IEEE Trans. Ind. Inform. 17 , 1873–1881 (2020).

Zheng, Z., Yang, L. & Zhang, Z. Conditional variational autoencoder informed probabilistic wind power curve modeling. IEEE Trans. Sustain. Energy 14 , 2445–2460 (2023).

Ziyabari, S., Du, L. & Biswas, S. K. Multibranch attentive gated ResNet for short-term spatio-temporal solar irradiance forecasting. IEEE Trans. Ind. Appl. 58 , 28–38 (2021).

Sun, C., Shrivastava, A., Singh, S. & Gupta, A. in Proc. IEEE Int. Conf. Computer Vision 843-852 https://doi.org/10.1109/ICCV.2017.97 (IEEE, 2017).

Zhou, H. et al. Informer: beyond efficient Transformer for long sequence time-series forecasting. Proc. AAAI Conf. Artif. Intell. 35 , 11106–11115 (2021).

Wang, C. et al. A transformer-based method of multi-energy load forecasting in integrated energy system. IEEE Trans. Smart Grid 13 , 2703–2714 (2022).

Verde, S. F. & Rossetto, N. The Future of Renewable Energy Communities in the EU: An Investigation at the Time of the Clean Energy Package (European Univ. Institute, 2020).

Sharda, S., Singh, M. & Sharma, K. RSAM: robust self-attention based multi-horizon model for solar irradiance forecasting. IEEE Trans. Sustain. Energy 12 , 1394–1405 (2020).

Li, Y., He, S., Li, Y., Ding, Q. & Zeng, Z. Renewable energy absorption oriented many-objective probabilistic optimal power flow. IEEE Trans. Netw. Sci. Eng . https://doi.org/10.1109/TNSE.2023.3290147 (2023).

Dommel, H. W. & Tinney, W. F. Optimal power flow solutions. IEEE Trans. Power Appar. Syst. PAS-87 , 1866–1876 (1968).

Frangioni, A., Gentile, C. & Lacalandra, F. Tighter approximated MILP formulations for unit commitment problems. IEEE Trans. Power Syst. 24 , 105–113 (2009).

Hou, H. et al. Data-driven economic dispatch for islanded micro-grid considering uncertainty and demand response. Int. J. Electr. Power Energy Syst. 136 , 107623 (2022).

Du, Y., Li, F., Li, J. & Zheng, T. Achieving 100× acceleration for n  – 1 contingency screening with uncertain scenarios using deep convolutional neural network. IEEE Trans. Power Syst. 34 , 3303–3305 (2019).

Zhou, M., Chen, M. & Low, S. H. DeepOPF-FT: one deep neural network for multiple AC-OPF problems with flexible topology. IEEE Trans. Power Syst. 38 , 964–967 (2023).

Li, Y., Wan, C., Chen, D. & Song, Y. Nonparametric probabilistic optimal power flow. IEEE Trans. Power Syst. 37 , 2758–2770 (2022).

Li, Y. et al. Optimal operation of multimicrogrids via cooperative energy and reserve scheduling. IEEE Trans. Ind. Inform. 14 , 3459–3468 (2018).

Owerko, D., Gama, F. & Ribeiro, A. in ICASSP 2020 — 2020 IEEE Int. Conf. Acoustics, Speech and Signal Processing 5930–5934 (IEEE, 2020).

Diehl, F. in 33rd Conf. Neural Information Processing Systems (NeurIPS 2019) 1–6 (MIT Press, 2019).

Yan, Z. & Xu, Y. Real-time optimal power flow: a lagrangian based deep reinforcement learning approach. IEEE Trans. Power Syst. 35 , 3270–3273 (2020).

Liu, S. et al. Varying condition SCOPF based on deep learning and knowledge graph. IEEE Trans. Power Syst. 38 , 3189–3200 (2023).

Singh, M. K., Kekatos, V. & Giannakis, G. B. Learning to solve the AC-OPF using sensitivity-informed deep neural networks. IEEE Trans. Power Syst. 37 , 2833–2846 (2022).

Liu, T. et al. A Bayesian learning based scheme for online dynamic security assessment and preventive control. IEEE Trans. Power Syst. 35 , 4088–4099 (2020).

Velloso, A. & Van Hentenryck, P. Combining deep learning and optimization for preventive security-constrained DC optimal power flow. IEEE Trans. Power Syst. 36 , 3618–3628 (2021).

Donti, P., Agarwal, A., Bedmutha, N. V., Pileggi, L. & Kolter, J. Z. Adversarially robust learning for security-constrained optimal power flow. Adv. Neural Inf. Process. Syst. 34 , 28677–28689 (2021).

Zeng, L. et al. Physics-constrained vulnerability assessment of deep reinforcement learning-based SCOPF. IEEE Trans. Power Syst. 38 , 2690–2704 (2023).

Yang, H., Yi, J., Zhao, J. & Dong, Z. Extreme learning machine based genetic algorithm and its application in power system economic dispatch. Neurocomputing 102 , 154–162 (2013).

Chen, W., Park, S., Tanneau, M. & Van Hentenryck, P. Learning optimization proxies for large-scale security-constrained economic dispatch. Electr. Power Syst. Res. 213 , 108566 (2022).

Han, J., Yan, L. & Li, Z. A task-based day-ahead load forecasting model for stochastic economic dispatch. IEEE Trans. Power Syst. 36 , 5294–5304 (2021).

Zhou, S. et al. Combined heat and power system intelligent economic dispatch: a deep reinforcement learning approach. Int. J. Electr. Energy Syst. 120 , 106016 (2020).

Yu, T., Zhou, B., Chan, K. W., Chen, L. & Yang, B. Stochastic optimal relaxed automatic generation control in non-Markov environment based on multi-step Q ( λ ) learning. IEEE Trans. Power Syst. 26 , 1272–1282 (2011).

Han, X., He, H., Wu, J., Peng, J. & Li, Y. Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle. Appl. Energy 254 , 113708 (2019).

CAS   Google Scholar  

Duan, J. et al. Deep-reinforcement-learning-based autonomous voltage control for power grid operations. IEEE Trans. Power Syst. 35 , 814–817 (2020).

Li, Y., He, S., Li, Y., Shi, Y. & Zeng, Z. Federated multiagent deep reinforcement learning approach via physics-informed reward for multi-microgrid energy management. IEEE Trans. Neural Netw. Learn. Syst . 1–13 https://doi.org/10.1109/TNNLS.2022.3232630 (2023).

Du, Y. & Li, F. Intelligent multi-microgrid energy management based on deep neural network and model-free reinforcement learning. IEEE Trans. Smart Grid 11 , 1066–1076 (2020).

Wang, Z., Liu, Y., Ma, Z., Liu, X. & Ma, J. LiPSG: lightweight privacy-preserving Q-learning-based energy management for the IoT-enabled smart grid. IEEE Internet Things J. 7 , 3935–3947 (2020).

Lee, S. & Choi, D.-H. Dynamic pricing and energy management for profit maximization in multiple smart electric vehicle charging stations: a privacy-preserving deep reinforcement learning approach. Appl. Energy 304 , 117754 (2021).

Chen, P., Liu, S., Chen, B. & Yu, L. Multi-agent reinforcement learning for decentralized resilient secondary control of energy storage systems against dos attacks. IEEE Trans. Smart Grid 13 , 1739–1750 (2022).

Liang, Y., Ding, Z., Zhao, T. & Lee, W.-J. Real-time operation management for battery swapping-charging system via multi-agent deep reinforcement learning. IEEE Trans. Smart Grid 14 , 559–571 (2022).

Salari, A., Ahmadi, S. E., Marzband, M. & Zeinali, M. Fuzzy Q-learning-based approach for real-time energy management of home microgrids using cooperative multi-agent system. Sustain. Cities Soc. 95 , 104528 (2023).

Padhy, N. Unit commitment—a bibliographical survey. IEEE Trans. Power Syst. 19 , 1196–1205 (2004).

Zhou, M., Wang, B. & Watada, J. Deep learning-based rolling horizon unit commitment under hybrid uncertainties. Energy 186 , 115843 (2019).

Ajagekar, A. & You, F. Deep reinforcement learning based unit commitment scheduling under load and wind power uncertainty. IEEE Trans. Sustain . Energy 14 , 803–812 (2023).

Li, F., Qin, J. & Zheng, W. X. Distributed Q-learning-based online optimization algorithm for unit commitment and dispatch in smart grid. IEEE Trans. Cybern. 50 , 4146–4156 (2020).

de Mars, P. & O’Sullivan, A. Applying reinforcement learning and tree search to the unit commitment problem. Appl. Energy 302 , 117519 (2021).

Dalal, G. & Mannor, S. Reinforcement learning for the unit commitment problem. 2015 IEEE Eindh. PowerTech . 1–6 https://doi.org/10.1109/PTC.2015.7232646 (2015).

Li, F., Jiahu, Q. & Wei, X. Z. Distributed Q-learning-based online optimization algorithm for unit commitment and dispatch in smart grid. IEEE Trans. Cybern. 50.9 , 4146–4156 (2019).

Ajagekar, A. & Fengqi, Y. Deep reinforcement learning based unit commitment scheduling under load and wind power uncertainty. IEEE Trans. Sustain. Energy 14.2 , 803–812 (2022).

Guedes, L. S., de Mendonça Maia, P., Lisboa, A. C., Vieira, D. A. G. & Saldanha, R. R. A unit commitment algorithm and a compact MILP model for short-term hydro-power generation scheduling. IEEE Trans. Power Syst. 32 , 3381–3390 (2016).

Shi, J. & Oren, S. S. Stochastic unit commitment with topology control recourse for power systems with large-scale renewable integration. IEEE Trans. Power Syst. 33 , 3315–3324 (2017).

Chen, Y. et al. Security-constrained unit commitment for electricity market: modeling, solution methods, and future challenges. IEEE Trans. Power Syst. 38 , 4668–4681 (2022).

Qu, K., Zheng, X. & Yu, T. Environmental–economic unit commitment with robust diffusion control of gas pollutants. IEEE Trans. Power Syst. 38 , 818–834 (2022).

Chu, Z. & Teng, F. Voltage stability constrained unit commitment in power systems with high penetration of inverter-based generators. IEEE Trans. Power Syst. 38 , 1572–1582 (2022).

Bertsimas, D., Litvinov, E., Sun, X. A., Zhao, J. & Zheng, T. Adaptive robust optimization for the security constrained unit commitment problem. IEEE Trans. Power Syst. 28 , 52–63 (2012).

Zhang, J. et al. A survey for solving mixed integer programming via machine learning. Neurocomputing 519 , 205–217 (2023).

Gasse, M., Chételat, D., Ferroni, N., Charlin, L. & Lodi, A. Exact combinatorial optimization with graph convolutional neural networks. Adv. Neural Information Processing Systems . 32, 1–13 (2019).

Paulus, M. B., Zarpellon, G., Krause, A., Charlin, L. & Maddison, C. in Int. Conf. Machine Learning 17584–17600 (PMLR, 2022).

Hatziargyriou, N. et al. Definition and classification of power system stability—revisited & extended. IEEE Trans. Power Syst. 36 , 3271–3281 (2021).

Parvez, I., Aghili, M., Sarwat, A. I., Rahman, S. & Alam, F. Online power quality disturbance detection by support vector machine in smart meter. J. Mod. Power Syst. Clean. Energy 7 , 1328–1339 (2019).

Impram, S., Varbak Nese, S. & Oral, B. Challenges of renewable energy penetration on power system flexibility: a survey. Energy Strateg. Rev. 31 , 100539 (2020).

Shi, Z. et al. Artificial intelligence techniques for stability analysis and control in smart grids: methodologies, applications, challenges and future directions. Appl. Energy 278 , 115733 (2020).

Shi, Z. et al. Bidirectional active transfer learning for adaptive power system stability assessment and dominant instability mode identification. IEEE Trans. Power Syst . 1–15 https://doi.org/10.1109/TPWRS.2022.3220955 (2022).

Xi, L., Wu, J., Xu, Y. & Sun, H. Automatic generation control based on multiple neural networks with actor–critic strategy. IEEE Trans. Neural Netw. Learn. Syst. 32 , 2483–2493 (2021).

Xi, L., Yu, L., Xu, Y., Wang, S. & Chen, X. A novel multi-agent DDQN-AD method-based distributed strategy for automatic generation control of integrated energy systems. IEEE Trans. Sustain. Energy 11 , 2417–2426 (2020).

Buşoniu, L., de Bruin, T., Tolić, D., Kober, J. & Palunko, I. Reinforcement learning for control: performance, stability, and deep approximators. Annu. Rev. Control. 46 , 8–28 (2018).

MathSciNet   Google Scholar  

Hwang, M., Muljadi, E., Jang, G. & Kang, Y. C. Disturbance-adaptive short-term frequency support of a DFIG associated with the variable gain based on the ROCOF and rotor speed. IEEE Trans. Power Syst. 32 , 1873–1881 (2017).

Kheshti, M. et al. Toward intelligent inertial frequency participation of wind farms for the grid frequency control. IEEE Trans. Ind. Inform. 16 , 6772–6786 (2020).

Liang, Y., Zhao, X. & Sun, L. A multiagent reinforcement learning approach for wind farm frequency control. IEEE Trans. Ind. Inform. 19 , 1725–1734 (2023).

Dong, H. & Zhao, X. Data-driven wind farm control via multiplayer deep reinforcement learning. IEEE Trans. Control. Syst. Technol. 31 , 1468–1475 (2023).

Yan, J., Nuertayi, A., Yan, Y., Liu, S. & Liu, Y. Hybrid physical and data driven modeling for dynamic operation characteristic simulation of wind turbine. Renew. Energy 215 , 118958 (2023).

Su, Y. et al. An adaptive PV frequency control strategy based on real-time inertia estimation. IEEE Trans. Smart Grid 12 , 2355–2364 (2021).

Chen, Y. & Xu, D. Review of soft-switching topologies for single-phase photovoltaic inverters. IEEE Trans. Power Electron. 37 , 1926–1944 (2022).

Zhao, B., Zhang, X. & Huang, J. AI algorithm-based two-stage optimal design methodology of high-efficiency CLLC resonant converters for the hybrid AC–DC microgrid applications. IEEE Trans. Ind. Electron. 66 , 9756–9767 (2019).

Cao, D. et al. Attention enabled multi-agent DRL for decentralized volt-VAR control of active distribution system using PV inverters and SVCs. IEEE Trans. Sustain. Energy 12 , 1582–1592 (2021).

Khan, M. A., Haque, A. & Kurukuru, V. S. B. Intelligent transition control approach for different operating modes of photovoltaic inverter. IEEE Trans. Ind. Appl. 58 , 2332–2340 (2022).

Alam, M. J. E., Muttaqi, K. M. & Sutanto, D. A multi-mode control strategy for VAr support by solar PV inverters in distribution networks. IEEE Trans. Power Syst. 30 , 1316–1326 (2015).

Weckx, S. & Driesen, J. Optimal local reactive power control by PV inverters. IEEE Trans. Sustain. Energy 7 , 1624–1633 (2016).

Fu, X., Li, S. & Jaithwa, I. Implement optimal vector control for LCL-filter-based grid-connected converters by using recurrent neural networks. IEEE Trans. Ind. Electron. 62 , 4443–4454 (2015).

Song, Y. et al. A Q-learning based robust MPC method for DFIG to suppress the rotor overcurrent. Int. J. Electr. Power Energy Syst. 141 , 108106 (2022).

Yin, Z., Wang, S. & Zhao, Q. Sequential reconfiguration of unbalanced distribution network with soft open points based on deep reinforcement learning. J. Mod. Power Syst. Clean. Energy 11 , 107–119 (2023).

Peng, F. Z. Flexible AC transmission systems (FACTS) and resilient AC distribution systems (RACDS) in smart grid. Proc. IEEE 105 , 2099–2115 (2017).

Zheng, D.-D., Madani, S. S. & Karimi, A. Data-driven distributed online learning control for islanded microgrids. IEEE J. Emerg. Sel. Top. Circuits Syst. 12 , 194–204 (2022).

Zamzam, A. S. & Sidiropoulos, N. D. Physics-aware neural networks for distribution system state estimation. IEEE Trans. Power Syst. 35 , 4347–4356 (2020).

Chen, Q., Lin, N., Bu, S., Wang, H. & Zhang, B. Interpretable time-adaptive transient stability assessment based on dual-stage attention mechanism. Power Syst. IEEE Trans. 38 , 2776–2790 (2023).

Ye, X., Yan, J., Wang, Y., Lu, L. & He, R. A novel capsule convolutional neural network with attention mechanism for high-voltage circuit breaker fault diagnosis. Electr. Power Syst. Res. 209 , 108003 (2022).

Wang, W., Yu, N., Gao, Y. & Shi, J. Safe off-policy deep reinforcement learning algorithm for Volt-VAR control in power distribution systems. IEEE Trans. Smart Grid 11 , 3008–3018 (2020).

Wang, Y., Mao, M., Chang, L. & Hatziargyriou, N. D. Intelligent voltage control method in active distribution networks based on averaged weighted double deep Q-network algorithm. J. Mod. Power Syst. Clean. Energy 11 , 132–143 (2023).

Sun, X. & Qiu, J. Two-stage Volt/Var control in active distribution networks with multi-agent deep reinforcement learning method. IEEE Trans. Smart Grid 12 , 2903–2912 (2021).

Cao, D. et al. Data-driven multi-agent deep reinforcement learning for distribution system decentralized voltage control with high penetration of PVs. IEEE Trans. Smart Grid 12 , 4137–4150 (2021).

Hu, D. et al. Multi-agent deep reinforcement learning for voltage control with coordinated active and reactive power optimization. IEEE Trans. Smart Grid 13 , 4873–4886 (2022).

Li, Y. et al. Many-objective distribution network reconfiguration via deep reinforcement learning assisted optimization algorithm. IEEE Trans. Power Deliv. 37 , 2230–2244 (2022).

Kushwaha, A., Gopal, M. & Singh, B. Q-learning based maximum power extraction for wind energy conversion system with variable wind speed. IEEE Trans. Energy Convers. 35 , 1160–1170 (2020).

Liu, H. & Wu, W. Federated reinforcement learning for decentralized voltage control in distribution networks. IEEE Trans. Smart Grid 13 , 3840–3843 (2022).

Poudyal, A. et al. Multiarea inertia estimation using convolutional neural networks and federated learning. IEEE Syst. J. 16 , 6401–6412 (2022).

Molina-García, A., Bouffard, F. & Kirschen, D. S. Decentralized demand-side contribution to primary frequency control. IEEE Trans. Power Syst. 26 , 411–419 (2011).

Wang, X., Wang, J. & Liu, J. Vehicle to grid frequency regulation capacity optimal scheduling for battery swapping station using deep Q-network. IEEE Trans. Ind. Inform. 17 , 1342–1351 (2021).

Li, S. et al. Battery protective electric vehicle charging management in renewable energy system. IEEE Trans. Ind. Inform. 19 , 1312–1321 (2023).

Kabir, M. E., Ghafouri, M., Moussa, B. & Assi, C. A two-stage protection method for detection and mitigation of coordinated EVSE switching attacks. IEEE Trans. Smart Grid 12 , 4377–4388 (2021).

Du, Y., Li, F., Kurte, K., Munk, J. & Zandi, H. Demonstration of intelligent HVAC load management with deep reinforcement learning: real-world experience of machine learning in demand control. IEEE Power Energy Mag. 20 , 42–53 (2022).

Wang, B., Li, Y., Ming, W. & Wang, S. Deep reinforcement learning method for demand response management of interruptible load. IEEE Trans. Smart Grid 11 , 3146–3155 (2020).

Hu, Z. et al. Intelligent and rapid event-based load shedding pre-determination for large-scale power systems: knowledge-enhanced parallel branching dueling Q-network approach. Appl. Energy 347 , 121468 (2023).

Zhang, B., Hu, W., Ghias, A. M. Y. M., Xu, X. & Chen, Z. Multi-agent deep reinforcement learning-based coordination control for grid-aware multi-buildings. Appl. Energy 328 , 120215 (2022).

Guelpa, E. & Verda, V. Demand response and other demand side management techniques for district heating: a review. Energy 219 , 119440 (2021).

Zhang, Z. et al. A review of technologies and applications on versatile energy storage systems. Renew. Sustain. Energy Rev. 148 , 111263 (2021).

Calero, F. et al. A review of modeling and applications of energy storage systems in power grids. Proc. IEEE. 111 , 806–831 (2023).

Yao, F., Zhao, J., Li, X., Mao, L. & Qu, K. RBF neural network based virtual synchronous generator control with improved frequency stability. IEEE Trans. Ind. Inform. 17 , 4014–4024 (2021).

Wang, Y. & Wai, R.-J. Adaptive fuzzy-neural-network power decoupling strategy for virtual synchronous generator in micro-grid. IEEE Trans. Power Electron. 37 , 3878–3891 (2022).

Saadatmand, S., Shamsi, P. & Ferdowsi, M. Adaptive critic design-based reinforcement learning approach in controlling virtual inertia-based grid-connected inverters. Int. J. Electr. Power Energy Syst. 127 , 106657 (2021).

Bui, V.-H., Hussain, A. & Kim, H.-M. Double deep Q -learning-based distributed operation of battery energy storage system considering uncertainties. IEEE Trans. Smart Grid 11 , 457–469 (2020).

Shuai, H., Li, F., Pulgar-Painemal, H. & Xue, Y. Branching dueling Q-network-based online scheduling of a microgrid with distributed energy storage systems. IEEE Trans. Smart Grid 12 , 5479–5482 (2021).

Hosseini, M. M. & Parvania, M. Hierarchical intelligent operation of energy storage systems in power distribution grids. IEEE Trans. Sustain. Energy 14 , 741–750 (2023).

Yao, Z. et al. Machine learning for a sustainable energy future. Nat. Rev. Mater. 8 , 202–215 (2022).

ADS   PubMed   PubMed Central   Google Scholar  

Ye, Y., Qiu, D., Sun, M., Papadaskalopoulos, D. & Strbac, G. Deep reinforcement learning for strategic bidding in electricity markets. IEEE Trans. Smart Grid 11 , 1343–1355 (2019).

Liang, Y., Guo, C., Ding, Z. & Hua, H. Agent-based modeling in electricity market using deep deterministic policy gradient algorithm. IEEE Trans. Power Syst. 35 , 4180–4192 (2020).

Zhang, Z., Chen, Z. & Lee, W.-J. Soft actor–critic algorithm featured residential demand response strategic bidding for load aggregators. IEEE Trans. Ind. Appl. 58 , 4298–4308 (2022).

Peng, F. et al. Review on bidding strategies for renewable energy power producers participating in electricity spot markets. Sustain. Energy Technol. Assess. 58 , 103329 (2023).

Longoria, G., Davy, A. & Shi, L. Subsidy-free renewable energy trading: a meta agent approach. IEEE Trans. Sustain. Energy 11 , 1707–1716 (2019).

Ochoa, T., Gil, E., Angulo, A. & Valle, C. Multi-agent deep reinforcement learning for efficient multi-timescale bidding of a hybrid power plant in day-ahead and real-time markets. Appl. Energy 317 , 119067 (2022).

Jeong, J., Kim, S. W. & Kim, H. Deep reinforcement learning based real-time renewable energy bidding with battery control. IEEE Trans. Energy Mark. Policy Regul. 1 , 85–96 (2023).

Tang, Q., Guo, H. & Chen, Q. Multi-market bidding behavior analysis of energy storage system based on inverse reinforcement learning. IEEE Trans. Power Syst. 37 , 4819–4831 (2022).

Lin, Z. & Liu, X. Wind power forecasting of an offshore wind turbine based on high-frequency SCADA data and deep learning neural network. Energy 201 , 117693 (2020).

Jin, X., Xu, Z. & Qiao, W. Condition monitoring of wind turbine generators using SCADA data analysis. IEEE Trans. Sustain. Energy 12 , 202–210 (2021).

Yu, Y., Chen, L. & Liu, R. The source of wind power producers’ market power. Energy Policy 173 , 113401 (2023).

Tellidou, A. C. & Bakirtzis, A. G. Agent-based analysis of capacity withholding and tacit collusion in electricity markets. IEEE Trans. Power Syst. 22 , 1735–1742 (2007).

Razmi, P., Buygi, M. O. & Esmalifalak, M. Collusion strategy investigation and detection for generation units in electricity market using supervised learning paradigm. IEEE Syst. J. 15 , 146–157 (2020).

Qiu, D. et al. Strategic retail pricing and demand bidding of retailers in electricity market: a data-driven chance-constrained programming. Adv. Appl. Energy 7 , 100100 (2022).

Tsaousoglou, G. et al. Flexibility aggregation of temporally coupled resources in real-time balancing markets using machine learning. IEEE Trans. Ind. Inform. 18 , 4342–4351 (2021).

Chen, T. & Su, W. Indirect customer-to-customer energy trading with reinforcement learning. IEEE Trans. Smart Grid 10 , 4338–4348 (2018).

Taghizadeh, A., Montazeri, M. & Kebriaei, H. Deep reinforcement learning-aided bidding strategies for transactive energy market. IEEE Syst. J. 16 , 4445–4453 (2022).

Qian, T., Shao, C., Shi, D., Wang, X. & Wang, X. Automatically improved VCG mechanism for local energy markets via deep learning. IEEE Trans. Smart Grid 13 , 1261–1272 (2021).

Zhao, Z., Feng, C. & Liu, A. L. Comparisons of auction designs through multiagent learning in peer-to-peer energy trading. IEEE Trans. Smart Grid 14 , 593–605 (2022).

Ye, Y., Papadaskalopoulos, D., Yuan, Q., Tang, Y. & Strbac, G. Multi-agent deep reinforcement learning for coordinated energy trading and flexibility services provision in local electricity markets. IEEE Trans. Smart Grid 14 , 1541–1554 (2023).

Wang, J., Mishra, D. K., Li, L. & Zhang, J. Demand side management and peer-to-peer energy trading for industrial users using two-level multi-agent reinforcement learning. IEEE Trans. Energy Mark. Policy Regul. 1 , 23–36 (2023).

Gao, G., Wen, Y. & Tao, D. Distributed energy trading and scheduling among microgrids via multiagent reinforcement learning. IEEE Trans. Neural Networks Learn. Syst . 1–15 https://doi.org/10.1109/TNNLS.2022.3170070 (2022).

Nunna, H. K., Sesetti, A., Rathore, A. K. & Doolla, S. Multiagent-based energy trading platform for energy storage systems in distribution systems with interconnected microgrids. IEEE Trans. Ind. Appl. 56 , 3207–3217 (2020).

Lu, X. et al. Reinforcement learning-based microgrid energy trading with a reduced power plant schedule. IEEE Internet Things J. 6 , 10728–10737 (2019).

Yan, L., Chen, X., Chen, Y. & Wen, J. A hierarchical deep reinforcement learning-based community energy trading scheme for a neighborhood of smart households. IEEE Trans. Smart Grid 13 , 4747–4758 (2022).

Wu, Y., Zhao, T., Yan, H., Liu, M. & Liu, N. Hierarchical hybrid multi-agent deep reinforcement learning for peer-to-peer energy trading among multiple heterogeneous microgrids. IEEE Trans. Smart Grid 14 , 4649–4665 (2023).

Zamee, M. A., Han, D. & Won, D. Online hour-ahead load forecasting using appropriate time-delay neural network based on multiple correlation–multicollinearity analysis in IoT energy network. IEEE Internet Things J. 9 , 12041–12055 (2022).

Zhang, X., Pipattanasomporn, M., Chen, T. & Rahman, S. An IoT-based thermal model learning framework for smart buildings. IEEE Internet Things J. 7 , 518–527 (2020).

Lin, W., Wu, D. & Boulet, B. Spatial-temporal residential short-term load forecasting via graph neural networks. IEEE Trans. Smart Grid 12 , 5373–5384 (2021).

Zheng, X. et al. A multi-scale time-series dataset with benchmark for machine learning in decarbonized energy grids. Sci. Data 9 , 359 (2022).

PubMed   PubMed Central   Google Scholar  

Yin, X., Zhu, Y. & Hu, J. A comprehensive survey of privacy-preserving federated learning: a taxonomy, review, and future directions. ACM Comput. Surv. 54 , 1–36 (2021).

Hu, Q., Guo, Z. & Li, F. Imitation learning based fast power system production cost minimization simulation. IEEE Trans. Power Syst. 38 , 2951–2954 (2023).

Bellemare, M. G., Dabney, W. & Rowland, M. Distributional Reinforcement Learning (MIT Press, 2023).

Hu, Q., Zhang, S., Yu, M. & Xie, Z. Short-term wind speed or power forecasting with heteroscedastic support vector regression. IEEE Trans. Sustain. Energy 7 , 241–249 (2016).

Download references

Acknowledgements

This work was supported in part by the National Key R&D Program of China (grant 2021ZD0201300), National Natural Science Foundation of China (grants 62325304 and 62073148) and Key Project of the National Natural Science Foundation of China (grant 62233006).

Author information

Authors and affiliations.

Key Laboratory of lmage Information Processing and Intelligent Control of Ministry of Education of China, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China

Yuanzheng Li, Fei Hu, Juntao Duan, Yong Zhao & Zhigang Zeng

National Key Laboratory of Multispectral Information Intelligent Processing Technology, Huazhong University of Science and Technology, Wuhan, China

Yuanzheng Li, Yong Zhao & Zhigang Zeng

China-EU Institute for Clean and Renewable Energy, Huazhong University of Science and Technology, Wuhan, China

Yizhou Ding & Shangyang He

Department of Systems Science, School of Mathematics, Southeast University, Nanjing, China

Guanghui Wen

Department of Automation, Tsinghua University, Beijing, China

State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, China

Zhengguang Wu

School of Electrical and Electronics Engineering, Nanyang Technological University, Singapore, Singapore

Hoay Beng Gooi

School of Control Science and Engineering, Shandong University, Jinan, China

Chenghui Zhang

Department of Electrical Engineering, Tsinghua University, Beijing, China

Shengwei Mei

You can also search for this author in PubMed   Google Scholar

Contributions

Y.L., Y.D., S.H., F.H. and J.D. researched data for the article. All authors contributed substantially to discussion of the content. Y.L., Y.D., S.H., F.H., J.D., G.W. and H.B.G. wrote the article. All authors edited the manuscript before submission.

Corresponding authors

Correspondence to Guanghui Wen , Hoay Beng Gooi or Zhigang Zeng .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Reviews Electrical Engineering thanks Bhuvanagiri Ramesh and the other, anonymous, reviewer for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Li, Y., Ding, Y., He, S. et al. Artificial intelligence-based methods for renewable power system operation. Nat Rev Electr Eng 1 , 163–179 (2024). https://doi.org/10.1038/s44287-024-00018-9

Download citation

Accepted : 20 December 2023

Published : 09 February 2024

Issue Date : March 2024

DOI : https://doi.org/10.1038/s44287-024-00018-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

We apologize for the inconvenience...

To ensure we keep this website safe, please can you confirm you are a human by ticking the box below.

If you are unable to complete the above request please contact us using the below link, providing a screenshot of your experience.

https://ioppublishing.org/contacts/

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Research on energy management in hydrogen–electric coupled microgrids based on deep reinforcement learning.

1. Introduction

• Intelligent hydrogen–electric coupled microgrid energy management strategy: This paper proposes an energy management strategy based on the DDPG. A deep neural network is used to simulate and optimize the energy management strategy of the microgrid by combining the forecast data of PV generation and load demand. The strategy can effectively cope with the influence of uncertain factors, such as PV generation, EV charging loads, and hydrogen charging loads on the optimization results, and ensure that the system supply and demand are balanced throughout the dispatch cycle.
• Optimization of system operation economics and the reduction in light shedding: The DDPG algorithm operates hydrogen production from excess power during peak PV generation hours, which achieves full utilization of PV power and reduces light shedding. In addition, the method achieves a reduction in the system power purchase cost and improves the overall economic efficiency through the operation of charging and hydrogen production during low-price hours and discharging and selling power during high-price hours.
• Load smoothing and grid stability enhancement: Through the optimal scheduling of EV charging loads, the time and magnitude of peak loads are reduced, and the optimized charging load curves are smoother, which significantly reduces the gap between the peaks and valleys of the grid loads and thus enhances the stability and operational efficiency of the grid.
• The effectiveness and superiority of the DDPG algorithm are verified: The accuracy and effectiveness of the DDPG algorithm over the traditional DQN in dealing with continuous action decision-making problems are verified through case studies. The DDPG algorithm is more capable of optimizing the energy management of the microgrid under complex constraints, which significantly reduces the operating cost of the microgrid.

2. Hydrogen–Electric Coupled Microgrid Structure

3. distributed energy system models, 3.1. photovoltaic power generation model, 3.2. battery energy storage system model, 3.3. electrolytic hydrogen production model, 3.4. hydrogen fuel cell model, 3.5. model of hydrogen storage facilities, 4. decision-making model for microgrid energy management, 4.1. objective function, 4.2. constraints, 4.2.1. power and energy balance constraints, 4.2.2. constraints on the operation of photovoltaic power generation systems, 4.2.3. electrolytic hydrogen production system operational constraints.

• Operational Constraints of Electrolytic Cells

4.2.4. Electrochemical Energy Storage Operational Constraints

4.2.5. constraints on the operation of charging/hydrogen cells.

• Charging Load Constraints

5. Optimization Algorithms for Deep Reinforcement Learning

5.1. the principles of the ddpg algorithm, 5.2. implementation of the ddpg algorithm.

• Definition of the State Space

6. Case Study Analysis

6.1. case description, 6.2. simulation analysis, 7. conclusions.

• In hydrogen–electric coupled microgrids, the energy management system can intelligently adjust charging and discharging strategies based on electricity price signals and photovoltaic generation through the DDPG algorithm, achieving “buy low, sell high” operations.
• The DDPG algorithm takes into account the volatility of photovoltaic generation, the uncertainties of charging/hydrogen loads, and other uncertain factors, ensuring supply–demand balance between photovoltaic generation, electric vehicle charging/hydrogen loads, and the energy storage system during the scheduling period, thus enhancing the reliability and stability of system operation.
• Through the DDPG algorithm, hydrogen–electric coupled microgrids can participate in flexible grid regulation based on electricity price incentive signals by adjusting charging loads and energy storage systems, reducing peak loads, and improving grid stability and economic efficiency.
• The accuracy of the DDPG algorithm in continuous action problems has been validated through comparisons with the DQN algorithm.

Author Contributions

Data availability statement, conflicts of interest.

• Shi, T.; Sheng, J.; Chen, Z.; Zhou, H. Simulation Experiment Design and Control Strategy Analysis in Teaching of Hydrogen-Electric Coupling System. Processes 2024 , 12 , 138. [ Google Scholar ] [ CrossRef ]
• Cai, G.; Chen, C.; Kong, L.; Peng, L.; Zhang, H. Modeling and Control of grid-connected system of wind power/photovoltaic/Hydrogen production/Supercapacitor. Power Syst. Technol. 2016 , 40 , 2982–2990. [ Google Scholar ] [ CrossRef ]
• Zhang, R.; Li, X.; Wang, X.; Wang, Q.; Qi, Z. Optimal scheduling for hydrogen-electric hybrid microgrid with vehicle to grid technology. In 2021 China Automation Congress (CAC) ; IEEE: Piscataway, NJ, USA, 2021; pp. 6296–6300. [ Google Scholar ]
• Guanghui, L. Research on Modeling and Optimal Control of Wind-Wind Hydrogen Storage Microgrid System. Master’s Thesis, North China University of Technology, Beijing, China, 2024. [ Google Scholar ]
• Huo, Y.; Wu, Z.; Dai, J.; Huo, Y.; Wu, Z.; Dai, J.; Duan, W.; Zhao, H.; Jiang, J.; Yao, R. An Optimal Dispatch Method for the Hydrogen-Electric Coupled Energy Microgrid. In World Hydrogen Technology Convention ; Springer Nature Singapore: Singapore, 2023; pp. 69–75. [ Google Scholar ]
• Hou, L.; Dong, J.; Herrera, O.E.; Mérida, W. Energy management for solar-hydrogen microgrids with vehicle-to-grid and power-to-gas transactions. Int. J. Hydrogen Energy 2023 , 48 , 2013–2029. [ Google Scholar ] [ CrossRef ]
• Yu, L.; Qin, S.; Zhang, M.; Shen, C.; Jiang, T.; Guan, X. Deep reinforcement learning for smart building energy management: A survey. arXiv 2020 , arXiv:2008.05074. [ Google Scholar ]
• Zheng, J.; Song, Q.; Wu, G.; Chen, H.; Hu, Z.; Chen, Z.; Weng, C.; Chen, J. Low-carbon operation strategy of regional integrated energy system based on the Q learning algorithm. J. Electr. Power Sci. Technol. 2022 , 37 , 106–115. [ Google Scholar ]
• Xu, H.; Lu, J.; Yang, Z.; Li, Y.; Lu, J.; Huang, H. Decision optimization model of incentive demand response based on deep reinforcement learning. Autom. Electr. Power Syst. 2021 , 45 , 97–103. [ Google Scholar ]
• Shuai, C. Microgrid Energy Management and Scheduling Based on Reinforcement Learning. Ph.D. Thesis, University of Science and Technology Beijing, Beijing, China, 2023. [ Google Scholar ]
• Kim, B.; Zhang, Y.; Van Der Schaar, M.; Lee, J.W. Dynamic pricing and energy consumption scheduling with reinforcement learning. IEEE Trans. Smart Grid 2016 , 7 , 2187–2198. [ Google Scholar ] [ CrossRef ]
• Shi, T.; Xu, C.; Dong, W.; Zhou, H.; Bokhari, A.; Klemeš, J.J.; Han, N. Research on energy management of hydrogen electric coupling system based on deep reinforcement learning. Energy 2023 , 282 , 128174. [ Google Scholar ] [ CrossRef ]
• Liu, J.; Chen, J.; Wang, X.; Zeng, J.; Huang, Q. Research on Energy Management and Optimization Strategy of micro-energy networks based on Deep Reinforcement Learning. Power Syst. Technol. 2020 , 44 , 3794–3803. [ Google Scholar ] [ CrossRef ]
• Ji, Y.; Wang, J.; Xu, J.; Fang, X.; Zhang, H. Real-Time Energy Management of a Microgrid Using Deep Reinforcement Learning. Energies 2019 , 12 , 2291. [ Google Scholar ] [ CrossRef ]
• Darshi, R.; Shamaghdari, S.; Jalali, A.; Arasteh, H. Decentralized Reinforcement Learning Approach for Microgrid Energy Management in Stochastic Environment. Int. Trans. Electr. Energy Syst. 2023 , 2023 , 1190103. [ Google Scholar ] [ CrossRef ]
• Kolodziejczyk, W.; Zoltowska, I.; Cichosz, P. Real-Time Energy Purchase Optimization for a Storage-Integrated Photovoltaic System by Deep Reinforcement Learning. Control Eng. Pract. 2021 , 106 , 104598. [ Google Scholar ] [ CrossRef ]
• Nicola, M.; Nicola, C.I.; Selișteanu, D. Improvement of the Control of a Grid Connected Photovoltaic System Based on Synergetic and Sliding Mode Controllers Using a Reinforcement Learning Deep Deterministic Policy Gradient Agent. Energies 2022 , 15 , 2392. [ Google Scholar ] [ CrossRef ]
• Wang, C.; Zhang, J.; Wang, A.; Wang, Z.; Yang, N.; Zhao, Z.; Lai, C.S.; Lai, L.L. Prioritized sum-tree experience replay TD3 DRL-based online energy management of a residential microgrid. Appl. Energy 2024 , 368 , 123471. [ Google Scholar ] [ CrossRef ]
• Guo, C.; Wang, X.; Zheng, Y.; Zhang, F. Real-time optimal energy management of microgrid with uncertainties based on deep reinforcement learning. Energy 2022 , 238 , 121873. [ Google Scholar ] [ CrossRef ]
• Benhmidouch, Z.; Moufid, S.; Ait-Omar, A.; Abbou, A.; Laabassi, H.; Kang, M.; Chatri, C.; Ali, I.H.O.; Bouzekri, H.; Baek, J. A novel reinforcement learning policy optimization based adaptive VSG control technique for improved frequency stabilization in AC microgrids. Electr. Power Syst. Res. 2024 , 230 , 110269. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Shi, T.; Zhou, H.; Shi, T.; Zhang, M. Research on Energy Management in Hydrogen–Electric Coupled Microgrids Based on Deep Reinforcement Learning. Electronics 2024 , 13 , 3389. https://doi.org/10.3390/electronics13173389

Shi T, Zhou H, Shi T, Zhang M. Research on Energy Management in Hydrogen–Electric Coupled Microgrids Based on Deep Reinforcement Learning. Electronics . 2024; 13(17):3389. https://doi.org/10.3390/electronics13173389

Shi, Tao, Hangyu Zhou, Tianyu Shi, and Minghui Zhang. 2024. "Research on Energy Management in Hydrogen–Electric Coupled Microgrids Based on Deep Reinforcement Learning" Electronics 13, no. 17: 3389. https://doi.org/10.3390/electronics13173389

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Maintenance work is planned from 21:00 BST on Tuesday 20th August 2024 to 21:00 BST on Wednesday 21st August 2024, and on Thursday 29th August 2024 from 11:00 to 12:00 BST.

During this time the performance of our website may be affected - searches may run slowly, some pages may be temporarily unavailable, and you may be unable to log in or to access content. If this happens, please try refreshing your web browser or try waiting two to three minutes before trying again.

We apologise for any inconvenience this might cause and thank you for your patience.

Journal of Materials Chemistry A

Electric power generation using paper materials †.

* Corresponding authors

a Beijing Key Laboratory of Photoelectronic/Electrophotonic Conversion Materials, Key Laboratory of Cluster Science, Ministry of Education, School of Chemistry and Chemical Engineering, Beijing Institute of Technology, Beijing 100081, P. R. China E-mail: [email protected] , [email protected]

Power generation from renewable sources is important for sustainable development due to the depletion of traditional fossil fuels and related environmental pollution. In this work, paper is used to generate electricity under moisture ingress. As a result, a piece of untreated print paper (1.5 cm 2 in area) can induce a voltage of 0.25 V and a current of 15 nA. The power output can be conveniently tuned by changing the humidity, temperature and number of devices by simple series/parallel connections. Such paper-based moist-electric generators (PMEGs) are expected to find their applications in the daily ambient environment owing to the wide availability and low cost of paper materials. A ‘power’ book is then fabricated to prove the concept, providing new insights into moist-electric power generation and a viable approach for designing extremely simple power generators for various applications.

Supplementary files

• Supplementary movie MOV (2121K)
• Supplementary information PDF (1805K)

Article information

Download citation, permissions.

Electric power generation using paper materials

X. Gao, T. Xu, C. Shao, Y. Han, B. Lu, Z. Zhang and L. Qu, J. Mater. Chem. A , 2019,  7 , 20574 DOI: 10.1039/C9TA08264F

To request permission to reproduce material from this article, please go to the Copyright Clearance Center request page .

If you are an author contributing to an RSC publication, you do not need to request permission provided correct acknowledgement is given.

If you are the author of this article, you do not need to request permission to reproduce figures and diagrams provided correct acknowledgement is given. If you want to reproduce the whole article in a third-party publication (excluding your thesis/dissertation for which permission is not required) please go to the Copyright Clearance Center request page .

Read more about how to correctly acknowledge RSC content .

Social activity

Search articles by author, advertisements.

Suggestions or feedback?

MIT News | Massachusetts Institute of Technology

• Machine learning
• Sustainability
• Black holes
• Classes and programs

Departments

• Aeronautics and Astronautics
• Brain and Cognitive Sciences
• Architecture
• Political Science
• Mechanical Engineering

Centers, Labs, & Programs

• Abdul Latif Jameel Poverty Action Lab (J-PAL)
• Picower Institute for Learning and Memory
• Lincoln Laboratory
• School of Architecture + Planning
• School of Engineering
• School of Humanities, Arts, and Social Sciences
• Sloan School of Management
• School of Science
• MIT Schwarzman College of Computing

MIT engineers’ new theory could improve the design and operation of wind farms

Press contact :, media download.

*Terms of Use:

Images for download on the MIT News office website are made available to non-commercial entities, press and the general public under a Creative Commons Attribution Non-Commercial No Derivatives license . You may not alter the images provided, other than to crop them to size. A credit line must be used when reproducing images; if one is not provided below, credit the images to "MIT."

Previous image Next image

The blades of propellers and wind turbines are designed based on aerodynamics principles that were first described mathematically more than a century ago. But engineers have long realized that these formulas don’t work in every situation. To compensate, they have added ad hoc “correction factors” based on empirical observations.

Now, for the first time, engineers at MIT have developed a comprehensive, physics-based model that accurately represents the airflow around rotors even under extreme conditions, such as when the blades are operating at high forces and speeds, or are angled in certain directions. The model could improve the way rotors themselves are designed, but also the way wind farms are laid out and operated. The new findings are described today in the journal Nature Communications , in an open-access paper by MIT postdoc Jaime Liew, doctoral student Kirby Heck, and Michael Howland, the Esther and Harold E. Edgerton Assistant Professor of Civil and Environmental Engineering.

“We’ve developed a new theory for the aerodynamics of rotors,” Howland says. This theory can be used to determine the forces, flow velocities, and power of a rotor, whether that rotor is extracting energy from the airflow, as in a wind turbine, or applying energy to the flow, as in a ship or airplane propeller. “The theory works in both directions,” he says.

Because the new understanding is a fundamental mathematical model, some of its implications could potentially be applied right away. For example, operators of wind farms must constantly adjust a variety of parameters, including the orientation of each turbine as well as its rotation speed and the angle of its blades, in order to maximize power output while maintaining safety margins. The new model can provide a simple, speedy way of optimizing those factors in real time.

“This is what we’re so excited about, is that it has immediate and direct potential for impact across the value chain of wind power,” Howland says.

Modeling the momentum

Known as momentum theory, the previous model of how rotors interact with their fluid environment — air, water, or otherwise — was initially developed late in the 19th century. With this theory, engineers can start with a given rotor design and configuration, and determine the maximum amount of power that can be derived from that rotor — or, conversely, if it’s a propeller, how much power is needed to generate a given amount of propulsive force.

Momentum theory equations “are the first thing you would read about in a wind energy textbook, and are the first thing that I talk about in my classes when I teach about wind power,” Howland says. From that theory, physicist Albert Betz calculated in 1920 the maximum amount of energy that could theoretically be extracted from wind. Known as the Betz limit, this amount is 59.3 percent of the kinetic energy of the incoming wind.

But just a few years later, others found that the momentum theory broke down “in a pretty dramatic way” at higher forces that correspond to faster blade rotation speeds or different blade angles, Howland says. It fails to predict not only the amount, but even the direction of changes in thrust force at higher rotation speeds or different blade angles: Whereas the theory said the force should start going down above a certain rotation speed or blade angle, experiments show the opposite — that the force continues to increase. “So, it’s not just quantitatively wrong, it’s qualitatively wrong,” Howland says.

The theory also breaks down when there is any misalignment between the rotor and the airflow, which Howland says is “ubiquitous” on wind farms, where turbines are constantly adjusting to changes in wind directions. In fact, in an  earlier paper in 2022, Howland and his team found that deliberately misaligning some turbines slightly relative to the incoming airflow within a wind farm significantly improves the overall power output of the wind farm by reducing wake disturbances to the downstream turbines.

In the past, when designing the profile of rotor blades, the layout of wind turbines in a farm, or the day-to-day operation of wind turbines, engineers have relied on ad hoc adjustments added to the original mathematical formulas, based on some wind tunnel tests and experience with operating wind farms, but with no theoretical underpinnings.

Instead, to arrive at the new model, the team analyzed the interaction of airflow and turbines using detailed computational modeling of the aerodynamics. They found that, for example, the original model had assumed that a drop in air pressure immediately behind the rotor would rapidly return to normal ambient pressure just a short way downstream. But it turns out, Howland says, that as the thrust force keeps increasing, “that assumption is increasingly inaccurate.”

And the inaccuracy occurs very close to the point of the Betz limit that theoretically predicts the maximum performance of a turbine — and therefore is just the desired operating regime for the turbines. “So, we have Betz’s prediction of where we should operate turbines, and within 10 percent of that operational set point that we think maximizes power, the theory completely deteriorates and doesn’t work,” Howland says.

Through their modeling, the researchers also found a way to compensate for the original formula’s reliance on a one-dimensional modeling that assumed the rotor was always precisely aligned with the airflow. To do so, they used fundamental equations that were developed to predict the lift of three-dimensional wings for aerospace applications.

The researchers derived their new model, which they call a unified momentum model, based on theoretical analysis, and then validated it using computational fluid dynamics modeling. In followup work not yet published, they are doing further validation using wind tunnel and field tests.

Fundamental understanding

One interesting outcome of the new formula is that it changes the calculation of the Betz limit, showing that it’s possible to extract a bit more power than the original formula predicted. Although it’s not a significant change — on the order of a few percent — “it’s interesting that now we have a new theory, and the Betz limit that’s been the rule of thumb for a hundred years is actually modified because of the new theory,” Howland says. “And that’s immediately useful.” The new model shows how to maximize power from turbines that are misaligned with the airflow, which the Betz limit cannot account for.

The aspects related to controlling both individual turbines and arrays of turbines can be implemented without requiring any modifications to existing hardware in place within wind farms. In fact, this has already happened, based on earlier work from Howland and his collaborators two years ago that dealt with the wake interactions between turbines in a wind farm, and was based on the existing, empirically based formulas.

“This breakthrough is a natural extension of our previous work on optimizing utility-scale wind farms,” he says, because in doing that analysis, they saw the shortcomings of the existing methods for analyzing the forces at work and predicting power produced by wind turbines. “Existing modeling using empiricism just wasn’t getting the job done,” he says.

In a wind farm, individual turbines will sap some of the energy available to neighboring turbines, because of wake effects. Accurate wake modeling is important both for designing the layout of turbines in a wind farm, and also for the operation of that farm, determining moment to moment how to set the angles and speeds of each turbine in the array.

Until now, Howland says, even the operators of wind farms, the manufacturers, and the designers of the turbine blades had no way to predict how much the power output of a turbine would be affected by a given change such as its angle to the wind without using empirical corrections. “That’s because there was no theory for it. So, that’s what we worked on here. Our theory can directly tell you, without any empirical corrections, for the first time, how you should actually operate a wind turbine to maximize its power,” he says.

Because the fluid flow regimes are similar, the model also applies to propellers, whether for aircraft or ships, and also for hydrokinetic turbines such as tidal or river turbines. Although they didn’t focus on that aspect in this research, “it’s in the theoretical modeling naturally,” he says.

The new theory exists in the form of a set of mathematical formulas that a user could incorporate in their own software, or as an open-source software package that can be freely downloaded from GitHub . “It’s an engineering model developed for fast-running tools for rapid prototyping and control and optimization,” Howland says. “The goal of our modeling is to position the field of wind energy research to move more aggressively in the development of the wind capacity and reliability necessary to respond to climate change.”

The work was supported by the National Science Foundation and Siemens Gamesa Renewable Energy.

Share this news article on:

Related links.

• Unified momentum model software
• Howland Lab
• Department of Civil and Environmental Engineering

Related Topics

• Civil and environmental engineering
• Renewable energy
• Climate change

Related Articles

Michael Howland gives wind energy a lift

A new method boosts wind farms’ energy output, without new equipment

The answer may be blowing in the wind

Previous item Next item

More MIT News

President Kornbluth welcomes the Class of 2028

Read full story →

Scientists find neurons that process language on different timescales

Pursuing the secrets of a stealthy parasite

Study of disordered rock salts leads to battery breakthrough

Toward a code-breaking quantum computer

Uphill battles: Across the country in 75 days

• More news on MIT News homepage →

Massachusetts Institute of Technology 77 Massachusetts Avenue, Cambridge, MA, USA

• Map (opens in new window)
• Events (opens in new window)
• People (opens in new window)
• Careers (opens in new window)
• Accessibility
• Social Media Hub
• MIT on Facebook
• MIT on YouTube
• MIT on Instagram

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

• Submit Paper
• Check Paper Status
• Download Certificate/Paper

• --> --> --> --> --> E-mail [email protected] --> -->