The Library
Accelerating federated learning via momentum gradient descent
Tools
Liu, Wei, Chen, Li, Chen, Yunfei and Zhang, Wenyi (2020) Accelerating federated learning via momentum gradient descent. IEEE Transactions on Parallel and Distributed Systems, 31 (8). pp. 1754-1766. doi:10.1109/TPDS.2020.2975189 ISSN 1045-9219.
|
PDF
WRAP-accelerating-federated-learning-momentum-gradient-descent-Chen-2020.pdf - Accepted Version - Requires a PDF viewer. Download (1676Kb) | Preview |
Official URL: https://doi.org/10.1109/TPDS.2020.2975189
Abstract
Federated learning (FL) provides a communication-efficient approach to solve machine learning problems concerning distributed data, without sending raw data to a central server. However, existing works on FL only utilize first-order gradient descent (GD) and do not consider the preceding iterations to gradient update which can potentially accelerate convergence. In this article, we consider momentum term which relates to the last iteration. The proposed momentum federated learning (MFL) uses momentum gradient descent (MGD) in the local update step of FL system. We establish global convergence properties of MFL and derive an upper bound on MFL convergence rate. Comparing the upper bounds on MFL and FL convergence rates, we provide conditions in which MFL accelerates the convergence. For different machine learning models, the convergence performance of MFL is evaluated based on experiments with MNIST and CIFAR-10 datasets. Simulation results confirm that MFL is globally convergent and further reveal significant convergence improvement over FL.
Item Type: | Journal Article | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Subjects: | Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software | |||||||||
Divisions: | Faculty of Science, Engineering and Medicine > Engineering > Engineering | |||||||||
Library of Congress Subject Headings (LCSH): | Machine learning, Computer science | |||||||||
Journal or Publication Title: | IEEE Transactions on Parallel and Distributed Systems | |||||||||
Publisher: | IEEE | |||||||||
ISSN: | 1045-9219 | |||||||||
Official Date: | 1 August 2020 | |||||||||
Dates: |
|
|||||||||
Volume: | 31 | |||||||||
Number: | 8 | |||||||||
Page Range: | pp. 1754-1766 | |||||||||
DOI: | 10.1109/TPDS.2020.2975189 | |||||||||
Status: | Peer Reviewed | |||||||||
Publication Status: | Published | |||||||||
Reuse Statement (publisher, data, author rights): | © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | |||||||||
Access rights to Published version: | Restricted or Subscription Access | |||||||||
Date of first compliant deposit: | 5 March 2020 | |||||||||
Date of first compliant Open Access: | 10 March 2020 | |||||||||
RIOXX Funder/Project Grant: |
|
|||||||||
Related URLs: |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year