FL-Joint: joint aligning features and labels in federated learning for data heterogeneity
Abstract Federated learning is a distributed machine learning paradigm that trains a shared model using data from various clients, it faces a core challenge in data heterogeneity arising from diverse client settings and environments. Existing methods typically focus on weight divergence mitigation a...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2024-11-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-024-01636-4 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571195091845120 |
|---|---|
| author | Wenxin Chen Jinrui Zhang Deyu Zhang |
| author_facet | Wenxin Chen Jinrui Zhang Deyu Zhang |
| author_sort | Wenxin Chen |
| collection | DOAJ |
| description | Abstract Federated learning is a distributed machine learning paradigm that trains a shared model using data from various clients, it faces a core challenge in data heterogeneity arising from diverse client settings and environments. Existing methods typically focus on weight divergence mitigation and aggregation strategy enhancements, they overlook the mixed skew in label and feature distributions prevalent in real-world data. To address this, we present FL-Joint, a federated learning framework that aligns label and feature distributions using auxiliary loss functions. This framework involves a class-balanced classifier as the local model. It aligns label and feature distributions locally by using auxiliary loss functions based on class-conditional information and pseudo-labels. This alignment drives client feature distributions to converge towards a shared feature space, refining decision boundaries and boosting the global model’s generalization ability. Extensive experiments across diverse datasets and heterogeneous data settings show that our method significantly improves accuracy and convergence speed compared to baseline approaches. |
| format | Article |
| id | doaj-art-c0a2d98072c74564b090f10c162dd700 |
| institution | Kabale University |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-c0a2d98072c74564b090f10c162dd7002025-02-02T12:49:32ZengSpringerComplex & Intelligent Systems2199-45362198-60532024-11-0111111410.1007/s40747-024-01636-4FL-Joint: joint aligning features and labels in federated learning for data heterogeneityWenxin Chen0Jinrui Zhang1Deyu Zhang2School of Computer Science, Central South UniversityDepartment of Computer Science and Technology, BNRist, Tsinghua UniversitySchool of Computer Science, Central South UniversityAbstract Federated learning is a distributed machine learning paradigm that trains a shared model using data from various clients, it faces a core challenge in data heterogeneity arising from diverse client settings and environments. Existing methods typically focus on weight divergence mitigation and aggregation strategy enhancements, they overlook the mixed skew in label and feature distributions prevalent in real-world data. To address this, we present FL-Joint, a federated learning framework that aligns label and feature distributions using auxiliary loss functions. This framework involves a class-balanced classifier as the local model. It aligns label and feature distributions locally by using auxiliary loss functions based on class-conditional information and pseudo-labels. This alignment drives client feature distributions to converge towards a shared feature space, refining decision boundaries and boosting the global model’s generalization ability. Extensive experiments across diverse datasets and heterogeneous data settings show that our method significantly improves accuracy and convergence speed compared to baseline approaches.https://doi.org/10.1007/s40747-024-01636-4Federated learningData heterogeneityJoint alignmentMixed distribution skew |
| spellingShingle | Wenxin Chen Jinrui Zhang Deyu Zhang FL-Joint: joint aligning features and labels in federated learning for data heterogeneity Complex & Intelligent Systems Federated learning Data heterogeneity Joint alignment Mixed distribution skew |
| title | FL-Joint: joint aligning features and labels in federated learning for data heterogeneity |
| title_full | FL-Joint: joint aligning features and labels in federated learning for data heterogeneity |
| title_fullStr | FL-Joint: joint aligning features and labels in federated learning for data heterogeneity |
| title_full_unstemmed | FL-Joint: joint aligning features and labels in federated learning for data heterogeneity |
| title_short | FL-Joint: joint aligning features and labels in federated learning for data heterogeneity |
| title_sort | fl joint joint aligning features and labels in federated learning for data heterogeneity |
| topic | Federated learning Data heterogeneity Joint alignment Mixed distribution skew |
| url | https://doi.org/10.1007/s40747-024-01636-4 |
| work_keys_str_mv | AT wenxinchen fljointjointaligningfeaturesandlabelsinfederatedlearningfordataheterogeneity AT jinruizhang fljointjointaligningfeaturesandlabelsinfederatedlearningfordataheterogeneity AT deyuzhang fljointjointaligningfeaturesandlabelsinfederatedlearningfordataheterogeneity |