The Library
GraphM : an efficient storage system for high throughput of concurrent graph processing
Tools
Tools
Tools
Zhao, Jin, Zhang, Yu, Liao, Xiaofei, He, Ligang, He, Bingsheng, Jin, Hai, Liu, Haikun and Chen, Yicheng (2019) GraphM : an efficient storage system for high throughput of concurrent graph processing. In: SC '19 International Conference for High Performance Computing, Networking, Storage and Analysis, Denver, Colorado, 17-19 Nov 2019. Published in: SC '19 Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis pp. 1-14. ISBN 9781450362290. doi:10.1145/3295500.3356143
|
PDF
WRAP-GraphM-efficient-storage-system-high-throughput-graph-He-2019.pdf - Published Version - Requires a PDF viewer. Download (1558Kb) | Preview |
Official URL: http://dx.doi.org/10.1145/3295500.3356143
Abstract
With the rapidly growing demand of graph processing in the real world, a large number of iterative graph processing jobs run concurrently on the same underlying graph. However, the storage engines of existing graph processing frameworks are mainly designed for running an individual job. Our studies show that they are inefficient when running concurrent jobs due to the redundant data storage and access overhead. To cope with this issue, we develop an efficient storage system, called GraphM. It can be integrated into the existing graph processing systems to efficiently support concurrent iterative graph processing jobs for higher throughput by fully exploiting the similarities of the data accesses between these concurrent jobs. GraphM regularizes the traversing order of the graph partitions for concurrent graph processing jobs by streaming the partitions into the main memory and the Last-Level Cache (LLC) in a common order, and then processes the related jobs concurrently in a novel fine-grained synchronization. In this way, the concurrent jobs share the same graph structure data in the LLC/memory and also the data accesses to the graph, so as to amortize the storage consumption and the data access overhead. To demonstrate the efficiency of GraphM, we plug it into state-of-the-art graph processing systems, including GridGraph, GraphChi, PowerGraph, and Chaos. Experiments results show that GraphM improves the throughput by 1.73~13 times.
| Item Type: | Conference Item (Paper) | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Subjects: | Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software Z Bibliography. Library Science. Information Resources > ZA Information resources |
||||||||||||||||||
| Divisions: | Faculty of Science, Engineering and Medicine > Science > Computer Science | ||||||||||||||||||
| Library of Congress Subject Headings (LCSH): | Graph theory -- Data processing, Information retrieval , Computer storage devices | ||||||||||||||||||
| Journal or Publication Title: | SC '19 Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis | ||||||||||||||||||
| ISBN: | 9781450362290 | ||||||||||||||||||
| Book Title: | Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis on - SC '19 | ||||||||||||||||||
| Official Date: | 2019 | ||||||||||||||||||
| Dates: |
|
||||||||||||||||||
| Page Range: | pp. 1-14 | ||||||||||||||||||
| Article Number: | 3 | ||||||||||||||||||
| DOI: | 10.1145/3295500.3356143 | ||||||||||||||||||
| Status: | Peer Reviewed | ||||||||||||||||||
| Publication Status: | Published | ||||||||||||||||||
| Reuse Statement (publisher, data, author rights): | ACM New York, NY, USA ©2019 This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in SC '19 Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis http://dx.doi.org/10.1145/3295500.3356143 | ||||||||||||||||||
| Access rights to Published version: | Restricted or Subscription Access | ||||||||||||||||||
| Date of first compliant deposit: | 2 December 2019 | ||||||||||||||||||
| Date of first compliant Open Access: | 3 December 2019 | ||||||||||||||||||
| RIOXX Funder/Project Grant: |
|
||||||||||||||||||
| Conference Paper Type: | Paper | ||||||||||||||||||
| Title of Event: | SC '19 International Conference for High Performance Computing, Networking, Storage and Analysis | ||||||||||||||||||
| Type of Event: | Conference | ||||||||||||||||||
| Location of Event: | Denver, Colorado | ||||||||||||||||||
| Date(s) of Event: | 17-19 Nov 2019 |
Request changes or add full text files to a record
Repository staff actions (login required)
 |
View Item |
Downloads
Downloads per month over past year
