Capacity Configuration of Rural Wind-PV-Hydro-Storage Microgrids Based on Deep Reinforcement Learning

FANG Yong, WANG Guorui, XI Haikuo

Distributed Energy ›› 2025, Vol. 10 ›› Issue (5) : 82-91.

PDF(2966 KB)
PDF(2966 KB)
Distributed Energy ›› 2025, Vol. 10 ›› Issue (5) : 82-91. DOI: 10.16513/j.2096-2185.DE.25100080

Capacity Configuration of Rural Wind-PV-Hydro-Storage Microgrids Based on Deep Reinforcement Learning

Author information +
History +

Abstract

To address weak infrastructure, poor voltage stability, and low renewable-energy utilization in rural areas, this paper proposes a siting-and-sizing model for distributed generation (DG) that simultaneously optimizes voltage quality and economic performance. One objective aims to minimize voltage deviations caused by DG integration, thereby enhancing distribution-network power quality; the other seeks to minimize the levelized cost of energy (LCOE) over the full life cycle of the DG portfolio, accounting for investment, operation and maintenance expenses, and energy yield. The model is solved with a double deep Q-network (DDQN), yielding a configuration that balances voltage stability and cost. Simulation on a modified IEEE 33-bus rural feeder shows that the DDQN-based scheme markedly improves voltage profiles while reducing upgrade costs. Furthermore, comparative analyses with the deep Q-network (DQN), non-dominated sorting genetic algorithm II (NSGA-II), and multi-objective particle swarm optimization (MOPSO) methods verify the superiority of the proposed approach, highlighting the efficiency, adaptability, and robustness of reinforcement learning for complex energy-system optimization.

Key words

distribution networks / renewable energy microgrids / reinforcement learning / levelized cost of energy (LCOE) / capacity allocation

Cite this article

Download Citations
FANG Yong , WANG Guorui , XI Haikuo. Capacity Configuration of Rural Wind-PV-Hydro-Storage Microgrids Based on Deep Reinforcement Learning[J]. Distributed Energy Resources. 2025, 10(5): 82-91 https://doi.org/10.16513/j.2096-2185.DE.25100080

References

[1]
国家发展和改革委员会. 关于实施农村电网巩固提升工程的指导意见[EB/OL].(2023-07-04)[2025-10-16]. https://www.gov.cn/zhengce/zhengceku/202307/content_6891875.htm.
[2]
宋美琪, 李洪全, 肖赫, 等. 面向高比例新能源接入主动配电网的多层模型预测电压控制[J]. 分布式能源, 2025, 10(1): 53-61.
SONG Meiqi, LI Hongquan, XIAO He, et al. Multilayer model predictive voltage control for high proportion new energy integrated active distribution network[J]. Distributed Energy, 2025, 10(01): 53-61.
[3]
方圆, 陈洁, 田小壮, 等. 孤岛模式下风-氢互补微电网的容量经济优化[J]. 计算机仿真, 2020, 37(2): 110-114.
FANG Yuan, CHEN Jie, TIAN Xiaozhuang, et al. Capacity economical optimization of non-grid-connected wind/hydrogen hybrid micro power grid[J]. Computer Simulation, 2020, 37(2): 110-114.
[4]
周京华, 翁志鹏, 宋晓通. 兼顾可靠性与经济性的孤岛型光储微电网容量配置方法[J]. 电力系统自动化, 2021, 45(8): 166-174.
ZHOU Jinghua, WENG Zhipeng, SONG Xiaotong. Capacity configuration method of islanded microgrid with photovoltaic and energy storage system considering reliability and economy[J]. Automation of Electric Power Systems, 2021, 45(8): 166-174.
[5]
王世震, 窦迅, 王俊. 考虑微电网群和系统经济运行的配电网储能优化配置[J]. 现代电力, 2022, 39(1): 88-94.
WANG Shizhen, DOU Xun, WANG Jun. Energy storage optimization configuration of distribution network considering microgrid clusters and system economic operation[J]. Modern Electric Power, 2022, 39(1): 88-94.
[6]
韩莹, 于三川, 李荦一, 等. 计及阶梯式碳交易的风光氢储微电网低碳经济配置方法[J]. 高电压技术, 2022, 48(7): 2523-2533.
HAN Ying, YU Sanchuan, LI Luoyi, ea al. Low-carbon and economic configuration method for solar hydrogen storage microgrid including stepped carbon trading[J]. High Voltage Engineering, 2022, 48(7): 2523-2533.
[7]
李争, 罗晓瑞, 徐若思, 等. 风光-氢储微电网系统多目标容量优化配置[J]. 热能动力工程, 2023, 38(4): 131-138.
LI Zheng, LUO Xiaorui, XU Ruosi, et al. Multi-objective capacity optimization of wind-pv-hydrogen energy storage micro-grid system[J]. Journal of Engineering for Thermal Energy and Power, 2023, 38(4): 131-138.
[8]
屈克庆, 乔敬茂, 毛玲, 等. 共享储能电站优化配置及选址评价方法[J]. 现代电力, 2025, 42(3):614-622.
QU Keqing, QIAO Jingmao, MAO Ling, et al. Optimal configuration and site-selection evaluation method for shared energy storage stations[J]. Modern Electric Power, 2025, 42(3):614-622..
[9]
张莲, 赵梦琪, 廖宗毅, 等. 计及多因素聚合储能寿命的微电网容量优化配置[J]. 重庆理工大学学报(自然科学), 2023, 37(1): 196-203.
ZHANG Lian, ZHAO Mengqi, LIAO Zongyi, et al. Optimal configuration of microgrid capacity for multi-factor polymerized energy storage life[J]. Journal of Chongqing University of Technology (Natural Science), 2023, 37(1): 196-203.
[10]
张世旭, 李姚旺, 刘伟生, 等. 面向微电网群的云储能经济-低碳-可靠多目标优化配置方法[J]. 电力系统自动化, 2024, 48(1): 21-30.
ZHANG Shixu, LI Yaowang, LIU Weisheng, et al. Economic, low-carbon and reliable multi-objective optimal configuration method of cloud energy storage for microgrid clusters[J]. Automation of Electric Power Systems, 2024, 48(1): 21-30.
[11]
辛曦, 欧阳森, 黄祎, 等. 考虑储能容量衰减的多保供电型微网最优经济配置及可靠性评估[J]. 电力建设, 2024, 45(10): 100-113.
XIN Xi, OUYANG Sen, HUANG Yi, et al. Optimal economic configuration and reliability evaluation of multiple power supply ensuring microgrid considering energy storage capacity attenuation[J]. Electric Power Construction, 2024, 45(10): 100-113.
[12]
孙建梅, 陈璐. 基于LCOE的分布式光伏发电并网效益分析[J]. 中国电力, 2018, 51(3): 88-93.
SUN Jianmei, CHEN Lu. Analysis on grid-connected benefit of distributed photovoltaic power generation based on LCOE model[J]. Electric Power, 2018, 51(3): 88-93.
[13]
昌敦虎, 田川, 张泽阳, 等. 基于LCOE模型的光伏发电经济效益分析:以宜昌农村地区光伏扶贫电站项目为例[J]. 环境科学研究, 2020, 33(10): 2412-2420.
CHANG Dunhu, TIAN Chuan, ZHANG Zeyang, et al. Economic benefit analysis on photovoltaic power generation with LCOE model: The case of poverty alleviation project in rural areas of Yichang city[J]. Research of Environmental Sciences, 2020, 33(10): 2412-2420.
[14]
赵振宇, 张玉洁. 光储项目成本效益模型及平价上网预测研究[J]. 太阳能学报, 2023, 44(7): 214-220.
Abstract
为科学、准确地确定光储项目的经济性和平准化度电成本(LCOE),考虑技术进步、政策调控等因素的影响建立光储项目成本效益模型、LCOE模型以及系统动力学仿真模型。以河北省光储开发项目为案例,分别在有无补贴的情况下应用模型,对光储项目LCOE、内部收益率和综合效益等指标进行仿真分析。选取青海省、云南省光储开发项目做对比研究,测算出光储项目在上述3省中实现平价上网的不同临界点。所建模型可用于分析光储项目成本效益、测算LCOE值、预测光储发电实现平价上网的初始年份。
ZHAO Zhenyu, ZHANG Yujie. Study on cost-benefit model and grid parity prediction of photovoltaic energy storage power project[J]. Acta Energiae Solaris Sinica, 2023, 44(7): 214-220.
It is especially urgent to calculate the cost and benefit of photovoltaic energy storage power project accurately. In order to scientifically and accurately determine the economic and levelized cost of energy(LCOE) of photovoltaic energy storage power project, in this paper, the cost benefit model, LCOE model and system dynamics simulation model of photovoltaic energy storage power project are established considering the influence of technological progress and policy regulation. Taking the photovoltaic energy storage power project in Hebei province as an example, the LCOE, internal rate of return and comprehensive benefit of the photovoltaic energy storage power project are simulated and analyzed by using the model with or without subsidies. It also selects the photovoltaic energy storage power projects in Qinghai province and Yunnan province for comparative study, and calculates the different critical points for photovoltaic energy storage power projects to achieve grid parity in the above three provinces. The model can be used to analyze the cost benefit of photovoltaic energy storage power project, to measure LCOE, and to predict the initial year when photovoltaic energy storage power project will realize grid parity in the future.
[15]
张有兵, 林一航, 黄冠弘, 等. 深度强化学习在微电网系统调控中的应用综述[J]. 电网技术, 2023, 47(7): 2774-2788.
ZHANG Youbing, LIN Yihang, HUANG Guanhong, et al. Review on applications of deep reinforcement learning in regulation of microgrid systems[J]. Power System Technology, 2023, 47(7): 2774-2788.
[16]
陈维江, 靳晓凌, 吴鸣, 等. 双碳目标下我国配电网形态快速演进的思考[J]. 中国电机工程学报, 2024, 44(17): 6811-6818.
CHEN Weijiang, JIN Xiaoling, WU Ming, et al. Thinking on the rapid evolution of distribution network form under the carbon peaking and carbon neutrality goals[J]. Proceedings of the CSEE, 2024, 44(17): 6811-6818.
[17]
郭一帆, 欧阳森, 张晋铭, 等. 含小水电配电网重要负荷评估及其水光储优化配置方法[J]. 广东电力, 2024, 37(5):32-42.
GUO Yifan, OUYANG Sen, ZHANG Jinming, et al. Assessment of critical load and optimization configuration method for small hydro power distribution network with integrated hydro-PV-energy storage[J]. Guangdong Electric Power, 2024, 37(5):32-42.
[18]
李鑫伟, 陈彬剑, 于明志, 等. 基于多目标优化的多能互补冷热电联产系统运行优化研究[J]. 热力发电, 2024, 53(7): 73-81.
LI Xinwei, CHEN Binjian, YU Mingzhi, et al. Research on operation optimization of multi-energy complementary cogeneration system based on multi-objective optimization[J]. Thermal Power Generation, 2024, 53(7): 73-81.
[19]
智筠贻, 凌浩恕, 吴昊, 等. 风光储多能互补能源系统容量配置优化[J]. 储能科学与技术, 2024, 13(11): 3874-3888.
Abstract
风光储多能互补能源系统可充分利用可再生能源提高供能的经济性和环保性。本文提出了一种风光储多能互补能源系统,建立了系统的能量模型;综合考虑系统运行的经济性和环保性,提出了系统综合成本和碳排放量最低的目标;开发了改进型非支配遗传算法求解仿真模型,得到了多目标问题的帕累托最优解集,并通过逼近理想解排序法获得了系统的最优容量配置运行方案;利用线性规划软件CPLEX求解器开展了系统的运行调度优化,验证了该系统框架和优化调度模型的有效性和正确性。研究结果表明,本文所提出的风光储多能互补能源系统容量配置优化方法有效提高了可再生能源利用率,实现了经济成本和碳排放量最低,提高了系统的经济性和环保性。本文为可再生能源系统实现持续稳定可靠的供能和园区的低碳化转型提供了参考。
ZHI Junyi, LING Haoshu, WU Hao, et al. Optimization of capacity configuration for multi-energy complementary systems using wind, solar, and energy storage[J]. Energy Storage Science and Technology, 2024, 13(11): 3874-3888.

The multi-energy complementary system integrating wind, solar, and energy storage technologies optimizes the use of renewable energy resources, enhancing both economic and environmental benefits. This study proposes a multi-energy complementary system model that incorporates wind, solar, and energy storage. The objective is to minimize the system's overall cost and carbon emissions, addressing both economic and environmental concerns. An improved non-dominated genetic algorithm is developed to obtain the Pareto optimal solution set for the multi-objective optimization problem. The optimal capacity configuration and operation scheme are determined using the technique for order preference by similarity to ideal solution. The system's operation scheduling is optimized using the CPLEX solver, a linear programming software, to validate the effectiveness and accuracy of the proposed system framework and scheduling model. Results demonstrate that the proposed optimization method significantly enhances renewable energy utilization, minimizes economic costs and carbon emissions, and improves the system's economic and environmental performance. This research offers valuable insights for the sustainable, stable, and reliable energy supply of renewable energy systems and supports the low-carbon transition of industrial parks.

[20]
谢雨龙, 罗逸飏, 李智威, 等. 考虑微网新能源经济消纳的共享储能优化配置[J]. 高电压技术, 2022, 48(11): 4403-4413.
XIE Yulong, LUO Yiyang, LI Zhiwei, et al. Optimal allocation of shared energy storage considering the economic consumption of microgrid new energy[J]. High Voltage Engineering, 2022, 48(11): 4403-4413.
[21]
刘健辰, 刘山林. 基于二阶锥松弛和Big-M法的配电网分布式电源优化配置[J]. 电网技术, 2018, 42(8): 2604-2611.
LIU Jianchen, LIU Shanlin. Optimal distributed generation allocation in distribution network based on second order conic relaxation and big-M method[J]. Power System Technology, 2018, 42(8): 2604-2611.
[22]
高冠中, 姚建国, 严嘉豪, 等. 基于多智能体深度强化学习的配-微网协同优化调度研究[J]. 智慧电力, 2024, 52(9): 80-87.
GAO Guanzhong, YAO Jianguo, YAN Jiahao, et al. Collaborative optimization scheduling of distribution network and microgrids based on multi agent deep reinforcement learning[J]. Smart Power, 2024, 52(9): 80-87.
[23]
杜明, 任建国, 张清杨. 基于双Q学习算法的安全容量的优化[J]. 现代电子技术, 2024, 47(11): 181-186.
DU Ming, REN Jianguo, ZHANG Qingyang. Secrecy capacity optimization algorithm based on dueling double deep Q-network[J]. Modern Electronics Technique, 2024, 47(11): 181-186.

Funding

Key Scientific Research Project of State Grid Jibei Electric Power Co., Ltd.(SGTYHT/21-JS-223)
PDF(2966 KB)

Accesses

Citation

Detail

Sections
Recommended

/