FS-DDPG: Optimal Control of a Fan Coil Unit System Based on Safe Reinforcement Learning
To optimize the control of fan coil unit (FCU) systems under model-free conditions, researchers have integrated reinforcement learning (RL) into the control processes of system pumps and fans. However, traditional RL methods can lead to significant fluctuations in the flow of pumps and fans, posing...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Buildings |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-5309/15/2/226 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | To optimize the control of fan coil unit (FCU) systems under model-free conditions, researchers have integrated reinforcement learning (RL) into the control processes of system pumps and fans. However, traditional RL methods can lead to significant fluctuations in the flow of pumps and fans, posing a safety risk. To address this issue, we propose a novel FCU control method, Fluctuation Suppression–Deep Deterministic Policy Gradient (FS-DDPG). The key innovation lies in applying a constrained Markov decision process to model the FCU control problem, where a penalty term for process constraints is incorporated into the reward function, and constraint tightening is introduced to limit the action space. In addition, to validate the performance of the proposed method, we established a variable operating conditions FCU simulation platform based on the parameters of the actual FCU system and ten years of historical weather data. The platform’s correctness and effectiveness were verified from three aspects: heat transfer, the air side and the water side, under different dry and wet operating conditions. The experimental results show that compared with DDPG, FS-DDPG avoids 98.20% of the pump flow and 95.82% of the fan flow fluctuations, ensuring the safety of the equipment. Compared with DDPG and RBC, FS-DDPG achieves 11.9% and 51.76% energy saving rates, respectively, and also shows better performance in terms of operational performance and satisfaction. In the future, we will further improve the scalability and apply the method to more complex FCU systems in variable environments. |
|---|---|
| ISSN: | 2075-5309 |