Resource-Efficient Personalization in Federated Learning With Closed-Form Classifiers
Statistical heterogeneity in Federated Learning (FL) often leads to client drift and biased local solutions. Prior work in the literature shows that client drift particularly affects the parameters of the classification layer, hindering both convergence and accuracy. While Personalized FL (PFL) addr...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10946159/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849731639364550656 |
|---|---|
| author | Eros Fani Raffaello Camoriano Barbara Caputo Marco Ciccone |
| author_facet | Eros Fani Raffaello Camoriano Barbara Caputo Marco Ciccone |
| author_sort | Eros Fani |
| collection | DOAJ |
| description | Statistical heterogeneity in Federated Learning (FL) often leads to client drift and biased local solutions. Prior work in the literature shows that client drift particularly affects the parameters of the classification layer, hindering both convergence and accuracy. While Personalized FL (PFL) addresses this by allowing client-specific models, it can overlook valuable global knowledge. This paper introduces Federated Recursive Ridge Regression (<monospace>Fed3R</monospace>), a fast and efficient method to construct a closed-form classifier that effectively incorporates global knowledge while being inherently robust to statistical heterogeneity. <monospace>Fed3R</monospace> leverages a pre-trained feature extractor and a recursive ridge regression formulation to achieve exact aggregation of local classifiers and recover the centralized solution. We demonstrate that <monospace>Fed3R</monospace> serves as a robust initialization for further fine-tuning with various FL and PFL algorithms, accelerating convergence and boosting performance. Furthermore, we propose Only Local Labels (<monospace>OLL</monospace>), a novel PFL technique that simplifies local classifiers by focusing only on locally relevant classes, preventing misclassifications and improving efficiency. Our empirical evaluation on real-world cross-device datasets shows that <monospace>Fed3R</monospace>, combined with <monospace>OLL</monospace>, significantly improves performance and reduces training costs in heterogeneous FL and PFL scenarios. |
| format | Article |
| id | doaj-art-59f7ea047d5444168ca93dbf1204e6c4 |
| institution | DOAJ |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-59f7ea047d5444168ca93dbf1204e6c42025-08-20T03:08:28ZengIEEEIEEE Access2169-35362025-01-0113619286195710.1109/ACCESS.2025.355658710946159Resource-Efficient Personalization in Federated Learning With Closed-Form ClassifiersEros Fani0https://orcid.org/0000-0002-4486-1086Raffaello Camoriano1https://orcid.org/0000-0002-8890-2732Barbara Caputo2https://orcid.org/0000-0001-7169-0158Marco Ciccone3https://orcid.org/0000-0002-3306-1323Department of Control and Computer Engineering, Polytechnic University of Turin, Turin, ItalyDepartment of Control and Computer Engineering, Polytechnic University of Turin, Turin, ItalyDepartment of Control and Computer Engineering, Polytechnic University of Turin, Turin, ItalyVector Institute, Toronto, ON, CanadaStatistical heterogeneity in Federated Learning (FL) often leads to client drift and biased local solutions. Prior work in the literature shows that client drift particularly affects the parameters of the classification layer, hindering both convergence and accuracy. While Personalized FL (PFL) addresses this by allowing client-specific models, it can overlook valuable global knowledge. This paper introduces Federated Recursive Ridge Regression (<monospace>Fed3R</monospace>), a fast and efficient method to construct a closed-form classifier that effectively incorporates global knowledge while being inherently robust to statistical heterogeneity. <monospace>Fed3R</monospace> leverages a pre-trained feature extractor and a recursive ridge regression formulation to achieve exact aggregation of local classifiers and recover the centralized solution. We demonstrate that <monospace>Fed3R</monospace> serves as a robust initialization for further fine-tuning with various FL and PFL algorithms, accelerating convergence and boosting performance. Furthermore, we propose Only Local Labels (<monospace>OLL</monospace>), a novel PFL technique that simplifies local classifiers by focusing only on locally relevant classes, preventing misclassifications and improving efficiency. Our empirical evaluation on real-world cross-device datasets shows that <monospace>Fed3R</monospace>, combined with <monospace>OLL</monospace>, significantly improves performance and reduces training costs in heterogeneous FL and PFL scenarios.https://ieeexplore.ieee.org/document/10946159/Federated learningpersonalized federated learningpre-trained modelsridge regressionclosed-form classifiers |
| spellingShingle | Eros Fani Raffaello Camoriano Barbara Caputo Marco Ciccone Resource-Efficient Personalization in Federated Learning With Closed-Form Classifiers IEEE Access Federated learning personalized federated learning pre-trained models ridge regression closed-form classifiers |
| title | Resource-Efficient Personalization in Federated Learning With Closed-Form Classifiers |
| title_full | Resource-Efficient Personalization in Federated Learning With Closed-Form Classifiers |
| title_fullStr | Resource-Efficient Personalization in Federated Learning With Closed-Form Classifiers |
| title_full_unstemmed | Resource-Efficient Personalization in Federated Learning With Closed-Form Classifiers |
| title_short | Resource-Efficient Personalization in Federated Learning With Closed-Form Classifiers |
| title_sort | resource efficient personalization in federated learning with closed form classifiers |
| topic | Federated learning personalized federated learning pre-trained models ridge regression closed-form classifiers |
| url | https://ieeexplore.ieee.org/document/10946159/ |
| work_keys_str_mv | AT erosfani resourceefficientpersonalizationinfederatedlearningwithclosedformclassifiers AT raffaellocamoriano resourceefficientpersonalizationinfederatedlearningwithclosedformclassifiers AT barbaracaputo resourceefficientpersonalizationinfederatedlearningwithclosedformclassifiers AT marcociccone resourceefficientpersonalizationinfederatedlearningwithclosedformclassifiers |