The Library
TurboGNN : improving the end-to-end performance for sampling-based GNN training on GPUs
Tools
Wu, W., Shi, X., He, Ligang and Jin, H. (2023) TurboGNN : improving the end-to-end performance for sampling-based GNN training on GPUs. IEEE Transactions on Computers, 72 (9). pp. 2571-2584. doi:10.1109/TC.2023.3257507 ISSN 0018-9340.
|
PDF
WRAP-TurboGNN-improving-end-to-end-performance-sampling-GNN-23.pdf - Accepted Version - Requires a PDF viewer. Download (5Mb) | Preview |
Official URL: https://doi.org/10.1109/TC.2023.3257507
Abstract
Graph Neural Networks (GNN) have evolved as powerful models for graph representation learning. Sampling-based training methods have been introduced to train large graphs without compromising accuracy. However, it is challenging for the existing GNN systems to effectively utilize multi-core accelerators, especially GPUs, due to a large number of atomic operations and unbalanced workload originating from the serial execution of multiple GNN processing stages. In this paper, we propose a combination of optimization techniques to accelerate the end-to-end performance of the sampling-based GNN training process. Specifically, we propose an adaptive share memory-based sampling technique and a degree-guided thread block scheduling strategy to optimize the graph sampling. Further, based on the observations of resource demand in different training stages, we propose an asynchronous pipeline-based scheduling method, which accelerates the GNN training by decoupling different training stages into a pipeline and therefore improves the GPU resource utilization significantly. The experimental results show that compared with the existing work, the proposed methods can achieve up to 5.6X performance speedup in the end-to-end performance.
Item Type: | Journal Article | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Subjects: | Q Science > QA Mathematics Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software |
|||||||||
Divisions: | Faculty of Science, Engineering and Medicine > Science > Computer Science | |||||||||
Library of Congress Subject Headings (LCSH): | Neural networks (Computer science), Graph theory, Graphics processing units | |||||||||
Journal or Publication Title: | IEEE Transactions on Computers | |||||||||
Publisher: | IEEE | |||||||||
ISSN: | 0018-9340 | |||||||||
Official Date: | 1 September 2023 | |||||||||
Dates: |
|
|||||||||
Volume: | 72 | |||||||||
Number: | 9 | |||||||||
Page Range: | pp. 2571-2584 | |||||||||
DOI: | 10.1109/TC.2023.3257507 | |||||||||
Status: | Peer Reviewed | |||||||||
Publication Status: | Published | |||||||||
Re-use Statement: | © 2023 Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. | |||||||||
Access rights to Published version: | Restricted or Subscription Access | |||||||||
Date of first compliant deposit: | 20 February 2023 | |||||||||
Date of first compliant Open Access: | 20 February 2023 | |||||||||
RIOXX Funder/Project Grant: |
|
|||||||||
Related URLs: |
Request changes or add full text files to a record
Repository staff actions (login required)
View Item |
Downloads
Downloads per month over past year