FS-DDPG: Optimal Control of a Fan Coil Unit System Based on Safe Reinforcement Learning
To optimize the control of fan coil unit (FCU) systems under model-free conditions, researchers have integrated reinforcement learning (RL) into the control processes of system pumps and fans. However, traditional RL methods can lead to significant fluctuations in the flow of pumps and fans, posing...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Buildings |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2075-5309/15/2/226 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832588840241463296 |
|---|---|
| author | Chenyang Li Qiming Fu Jianping Chen You Lu Yunzhe Wang Hongjie Wu |
| author_facet | Chenyang Li Qiming Fu Jianping Chen You Lu Yunzhe Wang Hongjie Wu |
| author_sort | Chenyang Li |
| collection | DOAJ |
| description | To optimize the control of fan coil unit (FCU) systems under model-free conditions, researchers have integrated reinforcement learning (RL) into the control processes of system pumps and fans. However, traditional RL methods can lead to significant fluctuations in the flow of pumps and fans, posing a safety risk. To address this issue, we propose a novel FCU control method, Fluctuation Suppression–Deep Deterministic Policy Gradient (FS-DDPG). The key innovation lies in applying a constrained Markov decision process to model the FCU control problem, where a penalty term for process constraints is incorporated into the reward function, and constraint tightening is introduced to limit the action space. In addition, to validate the performance of the proposed method, we established a variable operating conditions FCU simulation platform based on the parameters of the actual FCU system and ten years of historical weather data. The platform’s correctness and effectiveness were verified from three aspects: heat transfer, the air side and the water side, under different dry and wet operating conditions. The experimental results show that compared with DDPG, FS-DDPG avoids 98.20% of the pump flow and 95.82% of the fan flow fluctuations, ensuring the safety of the equipment. Compared with DDPG and RBC, FS-DDPG achieves 11.9% and 51.76% energy saving rates, respectively, and also shows better performance in terms of operational performance and satisfaction. In the future, we will further improve the scalability and apply the method to more complex FCU systems in variable environments. |
| format | Article |
| id | doaj-art-66fcb40a137f4d8dbeafa9e5444d3f57 |
| institution | Kabale University |
| issn | 2075-5309 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Buildings |
| spelling | doaj-art-66fcb40a137f4d8dbeafa9e5444d3f572025-01-24T13:26:15ZengMDPI AGBuildings2075-53092025-01-0115222610.3390/buildings15020226FS-DDPG: Optimal Control of a Fan Coil Unit System Based on Safe Reinforcement LearningChenyang Li0Qiming Fu1Jianping Chen2You Lu3Yunzhe Wang4Hongjie Wu5School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaSchool of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaSchool of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaSchool of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaSchool of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaSchool of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, ChinaTo optimize the control of fan coil unit (FCU) systems under model-free conditions, researchers have integrated reinforcement learning (RL) into the control processes of system pumps and fans. However, traditional RL methods can lead to significant fluctuations in the flow of pumps and fans, posing a safety risk. To address this issue, we propose a novel FCU control method, Fluctuation Suppression–Deep Deterministic Policy Gradient (FS-DDPG). The key innovation lies in applying a constrained Markov decision process to model the FCU control problem, where a penalty term for process constraints is incorporated into the reward function, and constraint tightening is introduced to limit the action space. In addition, to validate the performance of the proposed method, we established a variable operating conditions FCU simulation platform based on the parameters of the actual FCU system and ten years of historical weather data. The platform’s correctness and effectiveness were verified from three aspects: heat transfer, the air side and the water side, under different dry and wet operating conditions. The experimental results show that compared with DDPG, FS-DDPG avoids 98.20% of the pump flow and 95.82% of the fan flow fluctuations, ensuring the safety of the equipment. Compared with DDPG and RBC, FS-DDPG achieves 11.9% and 51.76% energy saving rates, respectively, and also shows better performance in terms of operational performance and satisfaction. In the future, we will further improve the scalability and apply the method to more complex FCU systems in variable environments.https://www.mdpi.com/2075-5309/15/2/226safe reinforcement learningfan coil unit systemmodel-free controlbuilding energy efficiency |
| spellingShingle | Chenyang Li Qiming Fu Jianping Chen You Lu Yunzhe Wang Hongjie Wu FS-DDPG: Optimal Control of a Fan Coil Unit System Based on Safe Reinforcement Learning Buildings safe reinforcement learning fan coil unit system model-free control building energy efficiency |
| title | FS-DDPG: Optimal Control of a Fan Coil Unit System Based on Safe Reinforcement Learning |
| title_full | FS-DDPG: Optimal Control of a Fan Coil Unit System Based on Safe Reinforcement Learning |
| title_fullStr | FS-DDPG: Optimal Control of a Fan Coil Unit System Based on Safe Reinforcement Learning |
| title_full_unstemmed | FS-DDPG: Optimal Control of a Fan Coil Unit System Based on Safe Reinforcement Learning |
| title_short | FS-DDPG: Optimal Control of a Fan Coil Unit System Based on Safe Reinforcement Learning |
| title_sort | fs ddpg optimal control of a fan coil unit system based on safe reinforcement learning |
| topic | safe reinforcement learning fan coil unit system model-free control building energy efficiency |
| url | https://www.mdpi.com/2075-5309/15/2/226 |
| work_keys_str_mv | AT chenyangli fsddpgoptimalcontrolofafancoilunitsystembasedonsafereinforcementlearning AT qimingfu fsddpgoptimalcontrolofafancoilunitsystembasedonsafereinforcementlearning AT jianpingchen fsddpgoptimalcontrolofafancoilunitsystembasedonsafereinforcementlearning AT youlu fsddpgoptimalcontrolofafancoilunitsystembasedonsafereinforcementlearning AT yunzhewang fsddpgoptimalcontrolofafancoilunitsystembasedonsafereinforcementlearning AT hongjiewu fsddpgoptimalcontrolofafancoilunitsystembasedonsafereinforcementlearning |