The Library
FedProf : selective federated learning based on distributional representation profiling
Tools
Wu, Wentai, He, Ligang, Lin, Weiwei and Maple, Carsten (2023) FedProf : selective federated learning based on distributional representation profiling. IEEE Transactions on Parallel and Distributed Systems, 34 (6). pp. 1942-1953. doi:10.1109/TPDS.2023.3265588 ISSN 1045-9219.
|
PDF
WRAP-FedProf-selective-federated-learning-representation-polling-23.pdf - Accepted Version - Requires a PDF viewer. Download (1659Kb) | Preview |
Official URL: https://doi.org/10.1109/TPDS.2023.3265588
Abstract
Federated Learning (FL) has shown great potential as a privacy-preserving solution to learning from decentralized data that are only accessible to end devices (i.e., clients). The data locality constraint offers strong privacy protection but also makes FL sensitive to the condition of local data. Apart from statistical heterogeneity, a large proportion of the clients, in many scenarios, are probably in possession of low-quality data that are biased, noisy or even irrelevant. As a result, they could significantly slow down the convergence of the global model we aim to build and also compromise its quality. In light of this, we first present a new view of local data by looking into the representation space and observing that they converge in distribution to Normal distributions before activation. We provide theoretical analysis to support our finding. Further, we propose FedProf , a novel algorithm for optimizing FL over non-IID data of mixed quality. The key of our approach is a distributional representation profiling and matching scheme that uses the global model to dynamically profile data representations and allows for low-cost, lightweight representation matching. Using the scheme we sample clients adaptively in FL to mitigate the impact of low-quality data on the training process. We evaluated our solution with extensive experiments on different tasks and data conditions under various FL settings. The results demonstrate that the selective behavior of our algorithm leads to a significant reduction in the number of communication rounds and the amount of time (up to 2.4× speedup) for the global model to converge and also provides accuracy gain.
Item Type: | Journal Article | |||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Subjects: | Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software | |||||||||||||||||||||||||||
Divisions: | Faculty of Science, Engineering and Medicine > Science > Computer Science Faculty of Science, Engineering and Medicine > Engineering > WMG (Formerly the Warwick Manufacturing Group) |
|||||||||||||||||||||||||||
Library of Congress Subject Headings (LCSH): | Electronic data processing -- Distributed processing, Machine learning, Neural networks (Computer science), Artificial intelligence, Application software | |||||||||||||||||||||||||||
Journal or Publication Title: | IEEE Transactions on Parallel and Distributed Systems | |||||||||||||||||||||||||||
Publisher: | IEEE | |||||||||||||||||||||||||||
ISSN: | 1045-9219 | |||||||||||||||||||||||||||
Official Date: | June 2023 | |||||||||||||||||||||||||||
Dates: |
|
|||||||||||||||||||||||||||
Volume: | 34 | |||||||||||||||||||||||||||
Number: | 6 | |||||||||||||||||||||||||||
Page Range: | pp. 1942-1953 | |||||||||||||||||||||||||||
DOI: | 10.1109/TPDS.2023.3265588 | |||||||||||||||||||||||||||
Status: | Peer Reviewed | |||||||||||||||||||||||||||
Publication Status: | Published | |||||||||||||||||||||||||||
Reuse Statement (publisher, data, author rights): | © 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | |||||||||||||||||||||||||||
Access rights to Published version: | Restricted or Subscription Access | |||||||||||||||||||||||||||
Date of first compliant deposit: | 3 April 2023 | |||||||||||||||||||||||||||
Date of first compliant Open Access: | 4 April 2023 | |||||||||||||||||||||||||||
RIOXX Funder/Project Grant: |
|
|||||||||||||||||||||||||||
Related URLs: |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year