
The Library
Accelerating the processing of deep neural networks
Tools
Li, Junyu (2020) Accelerating the processing of deep neural networks. PhD thesis, University of Warwick.
|
PDF
WRAP_Theses_Li_J_2020.pdf - Submitted Version - Requires a PDF viewer. Download (1239Kb) | Preview |
Official URL: http://webcat.warwick.ac.uk/record=b3517414~S15
Abstract
Artificial Intelligent (AI) has become the most potent and forward-looking force in the technologies over the past decade. It allowed breakthrough applications that have truly changed the world for the better. Deep Neural Network (DNN), the back-end of the AI, plays a significant role to bestow computers mind-boggling abilities in many tasks from computer vision, natural language processing, audio understanding to complex signal processing. Despite the rapid development of AI, training increasingly complex DNN is constrained by many limiting factors including learning algorithms, data sizes and computing resources. These issues have attracted a range of research on algorithm optimisation and efficient usage of computing resources. In this thesis, we aim to accelerate the processing of DNN from three aspects: data pruning, distributed processing and network pruning. Particularly, we first develop a real-time data pruning approach together with an automatic strategy for adjusting learning rate. The approach temporarily prunes specific training data items based on real-time analysis and automatically adjust the learning rate according to analysed learning trends. The work reduces training time for popular neural networks by around 24.46% without sacrificing the training accuracy. Second, we propose a novel distributed learning strategy to speed up the training process. It uses the predictors residing in the parameter server to forecast the loss of the model, which is used to compensate for the delayed model updating in distributed learning methods. Our strategy outperforms other remarkable methods in the accuracy. Lastly, we propose a fine-grained pruning method together with a feature-aware weight pruning method to reduce the complexity of the trained neural network. The methods prune less important feature maps and weights for each convolutional layer. The proposed methods can prune 74.2% of operations on average while maintaining the accuracy close to the original model.
Item Type: | Thesis (PhD) | ||||
---|---|---|---|---|---|
Subjects: | Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software | ||||
Library of Congress Subject Headings (LCSH): | Machine learning, Neural networks (Computer science) | ||||
Official Date: | October 2020 | ||||
Dates: |
|
||||
Institution: | University of Warwick | ||||
Theses Department: | Department of Computer Science | ||||
Thesis Type: | PhD | ||||
Publication Status: | Unpublished | ||||
Supervisor(s)/Advisor: | He, Ligang | ||||
Format of File: | |||||
Extent: | xiv, 126 leaves : colour illustrations | ||||
Language: | eng |
Request changes or add full text files to a record
Repository staff actions (login required)
![]() |
View Item |
Downloads
Downloads per month over past year