Optimizing intelligent residential scheduling based on policy black box and adaptive clustering federated deep reinforcement learning
In the context of user-side demand response, flexible resources in buildings such as air conditioners and electric vehicles are characterized by small individual capacities, large aggregate scales, and geographically dispersed distributions, necessitating integration by intelligence buildings (IRs)....
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-02-01
|
| Series: | Engineering Science and Technology, an International Journal |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2215098625000060 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832087484422422528 |
|---|---|
| author | Wei Zhang Yiyang Li |
| author_facet | Wei Zhang Yiyang Li |
| author_sort | Wei Zhang |
| collection | DOAJ |
| description | In the context of user-side demand response, flexible resources in buildings such as air conditioners and electric vehicles are characterized by small individual capacities, large aggregate scales, and geographically dispersed distributions, necessitating integration by intelligence buildings (IRs). However, the optimization scheduling of IR clusters often involves detailed energy consumption data, posing privacy issues such as revealing household routines. The traditional aggregator-IRs bi-level architecture typically employs centralized or game-theoretic strategies for optimization scheduling, which struggle to balance efficiency and privacy simultaneously. To address this issue, this paper proposes a bi-level optimization scheduling strategy that balances efficiency and privacy. First, deep reinforcement learning models are established for both the aggregator and the IRs to address efficiency. Then, the trained demand response models of the IRs are encapsulated into strategy black boxes and uploaded to the aggregator’s deep reinforcement learning model. Throughout this process, the aggregator remains unaware of the user-side data, thus protecting user privacy. Additionally, considering that training IR strategy black box models is a parallel and similar process, this paper introduces the paradigm of federated learning to reduce learning costs and improve training efficiency on the IRs side. Furthermore, an adaptive clustering federated deep reinforcement learning method is proposed to address the heterogeneity of the IRs. Finally, case studies demonstrate the feasibility and effectiveness of the proposed method. |
| format | Article |
| id | doaj-art-416e85e84edb42c1a6735e1f6187bd89 |
| institution | Kabale University |
| issn | 2215-0986 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Engineering Science and Technology, an International Journal |
| spelling | doaj-art-416e85e84edb42c1a6735e1f6187bd892025-02-06T05:11:52ZengElsevierEngineering Science and Technology, an International Journal2215-09862025-02-0162101951Optimizing intelligent residential scheduling based on policy black box and adaptive clustering federated deep reinforcement learningWei Zhang0Yiyang Li1Corresponding author.; School of Mechanical Engineering, University of Shanghai for Science and Technology, Shanghai 200093, ChinaSchool of Mechanical Engineering, University of Shanghai for Science and Technology, Shanghai 200093, ChinaIn the context of user-side demand response, flexible resources in buildings such as air conditioners and electric vehicles are characterized by small individual capacities, large aggregate scales, and geographically dispersed distributions, necessitating integration by intelligence buildings (IRs). However, the optimization scheduling of IR clusters often involves detailed energy consumption data, posing privacy issues such as revealing household routines. The traditional aggregator-IRs bi-level architecture typically employs centralized or game-theoretic strategies for optimization scheduling, which struggle to balance efficiency and privacy simultaneously. To address this issue, this paper proposes a bi-level optimization scheduling strategy that balances efficiency and privacy. First, deep reinforcement learning models are established for both the aggregator and the IRs to address efficiency. Then, the trained demand response models of the IRs are encapsulated into strategy black boxes and uploaded to the aggregator’s deep reinforcement learning model. Throughout this process, the aggregator remains unaware of the user-side data, thus protecting user privacy. Additionally, considering that training IR strategy black box models is a parallel and similar process, this paper introduces the paradigm of federated learning to reduce learning costs and improve training efficiency on the IRs side. Furthermore, an adaptive clustering federated deep reinforcement learning method is proposed to address the heterogeneity of the IRs. Finally, case studies demonstrate the feasibility and effectiveness of the proposed method.http://www.sciencedirect.com/science/article/pii/S2215098625000060Strategy black-boxDemand responsePrivacy preservationDeep reinforcement learningFederated learning |
| spellingShingle | Wei Zhang Yiyang Li Optimizing intelligent residential scheduling based on policy black box and adaptive clustering federated deep reinforcement learning Engineering Science and Technology, an International Journal Strategy black-box Demand response Privacy preservation Deep reinforcement learning Federated learning |
| title | Optimizing intelligent residential scheduling based on policy black box and adaptive clustering federated deep reinforcement learning |
| title_full | Optimizing intelligent residential scheduling based on policy black box and adaptive clustering federated deep reinforcement learning |
| title_fullStr | Optimizing intelligent residential scheduling based on policy black box and adaptive clustering federated deep reinforcement learning |
| title_full_unstemmed | Optimizing intelligent residential scheduling based on policy black box and adaptive clustering federated deep reinforcement learning |
| title_short | Optimizing intelligent residential scheduling based on policy black box and adaptive clustering federated deep reinforcement learning |
| title_sort | optimizing intelligent residential scheduling based on policy black box and adaptive clustering federated deep reinforcement learning |
| topic | Strategy black-box Demand response Privacy preservation Deep reinforcement learning Federated learning |
| url | http://www.sciencedirect.com/science/article/pii/S2215098625000060 |
| work_keys_str_mv | AT weizhang optimizingintelligentresidentialschedulingbasedonpolicyblackboxandadaptiveclusteringfederateddeepreinforcementlearning AT yiyangli optimizingintelligentresidentialschedulingbasedonpolicyblackboxandadaptiveclusteringfederateddeepreinforcementlearning |