Skip to content Skip to navigation
University of Warwick
  • Study
  • |
  • Research
  • |
  • Business
  • |
  • Alumni
  • |
  • News
  • |
  • About

University of Warwick
Publications service & WRAP

Highlight your research

  • WRAP
    • Home
    • Search WRAP
    • Browse by Warwick Author
    • Browse WRAP by Year
    • Browse WRAP by Subject
    • Browse WRAP by Department
    • Browse WRAP by Funder
    • Browse Theses by Department
  • Publications Service
    • Home
    • Search Publications Service
    • Browse by Warwick Author
    • Browse Publications service by Year
    • Browse Publications service by Subject
    • Browse Publications service by Department
    • Browse Publications service by Funder
  • Help & Advice
University of Warwick

The Library

  • Login
  • Admin

Accelerating the processing of deep neural networks

Tools
- Tools
+ Tools

Li, Junyu (2020) Accelerating the processing of deep neural networks. PhD thesis, University of Warwick.

[img]
Preview
PDF
WRAP_Theses_Li_J_2020.pdf - Submitted Version - Requires a PDF viewer.

Download (1239Kb) | Preview
Official URL: http://webcat.warwick.ac.uk/record=b3517414~S15

Request Changes to record.

Abstract

Artificial Intelligent (AI) has become the most potent and forward-looking force in the technologies over the past decade. It allowed breakthrough applications that have truly changed the world for the better. Deep Neural Network (DNN), the back-end of the AI, plays a significant role to bestow computers mind-boggling abilities in many tasks from computer vision, natural language processing, audio understanding to complex signal processing. Despite the rapid development of AI, training increasingly complex DNN is constrained by many limiting factors including learning algorithms, data sizes and computing resources. These issues have attracted a range of research on algorithm optimisation and efficient usage of computing resources. In this thesis, we aim to accelerate the processing of DNN from three aspects: data pruning, distributed processing and network pruning. Particularly, we first develop a real-time data pruning approach together with an automatic strategy for adjusting learning rate. The approach temporarily prunes specific training data items based on real-time analysis and automatically adjust the learning rate according to analysed learning trends. The work reduces training time for popular neural networks by around 24.46% without sacrificing the training accuracy. Second, we propose a novel distributed learning strategy to speed up the training process. It uses the predictors residing in the parameter server to forecast the loss of the model, which is used to compensate for the delayed model updating in distributed learning methods. Our strategy outperforms other remarkable methods in the accuracy. Lastly, we propose a fine-grained pruning method together with a feature-aware weight pruning method to reduce the complexity of the trained neural network. The methods prune less important feature maps and weights for each convolutional layer. The proposed methods can prune 74.2% of operations on average while maintaining the accuracy close to the original model.

Item Type: Thesis (PhD)
Subjects: Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software
Library of Congress Subject Headings (LCSH): Machine learning, Neural networks (Computer science)
Official Date: October 2020
Dates:
DateEvent
October 2020UNSPECIFIED
Institution: University of Warwick
Theses Department: Department of Computer Science
Thesis Type: PhD
Publication Status: Unpublished
Supervisor(s)/Advisor: He, Ligang
Format of File: pdf
Extent: xiv, 126 leaves : colour illustrations
Language: eng

Request changes or add full text files to a record

Repository staff actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics

twitter

Email us: wrap@warwick.ac.uk
Contact Details
About Us