Elastic Optimization for Stragglers in Edge Federated Learning
To fully exploit enormous data generated by intelligent devices in edge computing, edge federated learning (EFL) is envisioned as a promising solution. The distributed collaborative training in EFL deals with delay and privacy issues compared to traditional centralized model training. However, the e...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Tsinghua University Press
2023-12-01
|
| Series: | Big Data Mining and Analytics |
| Subjects: | |
| Online Access: | https://www.sciopen.com/article/10.26599/BDMA.2022.9020046 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832557683766460416 |
|---|---|
| author | Khadija Sultana Khandakar Ahmed Bruce Gu Hua Wang |
| author_facet | Khadija Sultana Khandakar Ahmed Bruce Gu Hua Wang |
| author_sort | Khadija Sultana |
| collection | DOAJ |
| description | To fully exploit enormous data generated by intelligent devices in edge computing, edge federated learning (EFL) is envisioned as a promising solution. The distributed collaborative training in EFL deals with delay and privacy issues compared to traditional centralized model training. However, the existence of straggling devices, responding slow to servers, degrades model performance. We consider data heterogeneity from two aspects: high dimensional data generated at edge devices where the number of features is greater than that of observations and the heterogeneity caused by partial device participation. With large number of features, computation overhead on the devices increases, causing edge devices to become stragglers. And incorporation of partial training results causes gradients to be diverged which further exaggerates when more training is performed to reach local optima. In this paper, we introduce elastic optimization methods for stragglers due to data heterogeneity in edge federated learning. Specifically, we define the problem of stragglers in EFL. Then, we formulate an optimization problem to be solved at edge devices. We customize a benchmark algorithm, FedAvg, to obtain a new elastic optimization algorithm (FedEN) which is applied in local training of edge devices. FedEN mitigates stragglers by having a balance between lasso and ridge penalization thereby generating sparse model updates and enforcing parameters as close as to local optima. We have evaluated the proposed model on MNIST and CIFAR-10 datasets. Simulated experiments demonstrate that our approach improves run time training performance by achieving average accuracy with less communication rounds. The results confirm the improved performance of our approach over benchmark algorithms. |
| format | Article |
| id | doaj-art-04c8c805ac8047f88be967a161576621 |
| institution | Kabale University |
| issn | 2096-0654 |
| language | English |
| publishDate | 2023-12-01 |
| publisher | Tsinghua University Press |
| record_format | Article |
| series | Big Data Mining and Analytics |
| spelling | doaj-art-04c8c805ac8047f88be967a1615766212025-02-03T02:57:52ZengTsinghua University PressBig Data Mining and Analytics2096-06542023-12-016440442010.26599/BDMA.2022.9020046Elastic Optimization for Stragglers in Edge Federated LearningKhadija Sultana0Khandakar Ahmed1Bruce Gu2Hua Wang3Institute for Sustainable Industries and Liveable Cities (ISILC), Victoria University, Melbourne 3011, AustraliaIntelligent Technology Innovation Lab (ITIL), Institute for Sustainable Industries and Liveable Cities (ISILC), Victoria University, Melbourne 3011, AustraliaShandong Computer Science Center (National Supercomputer Center), Jinan 250101, ChinaInstitute for Sustainable Industries and Liveable Cities (ISILC), Victoria University, Melbourne 3011, AustraliaTo fully exploit enormous data generated by intelligent devices in edge computing, edge federated learning (EFL) is envisioned as a promising solution. The distributed collaborative training in EFL deals with delay and privacy issues compared to traditional centralized model training. However, the existence of straggling devices, responding slow to servers, degrades model performance. We consider data heterogeneity from two aspects: high dimensional data generated at edge devices where the number of features is greater than that of observations and the heterogeneity caused by partial device participation. With large number of features, computation overhead on the devices increases, causing edge devices to become stragglers. And incorporation of partial training results causes gradients to be diverged which further exaggerates when more training is performed to reach local optima. In this paper, we introduce elastic optimization methods for stragglers due to data heterogeneity in edge federated learning. Specifically, we define the problem of stragglers in EFL. Then, we formulate an optimization problem to be solved at edge devices. We customize a benchmark algorithm, FedAvg, to obtain a new elastic optimization algorithm (FedEN) which is applied in local training of edge devices. FedEN mitigates stragglers by having a balance between lasso and ridge penalization thereby generating sparse model updates and enforcing parameters as close as to local optima. We have evaluated the proposed model on MNIST and CIFAR-10 datasets. Simulated experiments demonstrate that our approach improves run time training performance by achieving average accuracy with less communication rounds. The results confirm the improved performance of our approach over benchmark algorithms.https://www.sciopen.com/article/10.26599/BDMA.2022.9020046edge computingfederated learningdistributed machine learningregularizationstragglers |
| spellingShingle | Khadija Sultana Khandakar Ahmed Bruce Gu Hua Wang Elastic Optimization for Stragglers in Edge Federated Learning Big Data Mining and Analytics edge computing federated learning distributed machine learning regularization stragglers |
| title | Elastic Optimization for Stragglers in Edge Federated Learning |
| title_full | Elastic Optimization for Stragglers in Edge Federated Learning |
| title_fullStr | Elastic Optimization for Stragglers in Edge Federated Learning |
| title_full_unstemmed | Elastic Optimization for Stragglers in Edge Federated Learning |
| title_short | Elastic Optimization for Stragglers in Edge Federated Learning |
| title_sort | elastic optimization for stragglers in edge federated learning |
| topic | edge computing federated learning distributed machine learning regularization stragglers |
| url | https://www.sciopen.com/article/10.26599/BDMA.2022.9020046 |
| work_keys_str_mv | AT khadijasultana elasticoptimizationforstragglersinedgefederatedlearning AT khandakarahmed elasticoptimizationforstragglersinedgefederatedlearning AT brucegu elasticoptimizationforstragglersinedgefederatedlearning AT huawang elasticoptimizationforstragglersinedgefederatedlearning |