Skip to content Skip to navigation
University of Warwick
  • Study
  • |
  • Research
  • |
  • Business
  • |
  • Alumni
  • |
  • News
  • |
  • About

University of Warwick
Publications service & WRAP

Highlight your research

  • WRAP
    • Home
    • Search WRAP
    • Browse by Warwick Author
    • Browse WRAP by Year
    • Browse WRAP by Subject
    • Browse WRAP by Department
    • Browse WRAP by Funder
    • Browse Theses by Department
  • Publications Service
    • Home
    • Search Publications Service
    • Browse by Warwick Author
    • Browse Publications service by Year
    • Browse Publications service by Subject
    • Browse Publications service by Department
    • Browse Publications service by Funder
  • Help & Advice
University of Warwick

The Library

  • Login
  • Admin

Parallel file system analysis through application I/O tracing

Tools
- Tools
+ Tools

Wright, Steven A., Hammond, Simon D., Pennycook, Simon J., Bird, Robert F., Herdman, J. A., Miller, I., Vadgama, A., Bhalerao, Abhir and Jarvis, Stephen A. (2013) Parallel file system analysis through application I/O tracing. Computer Journal, Volume 56 (Number 2). pp. 141-155. doi:10.1093/comjnl/bxs044

[img]
Preview
Text
WRAP_Wright_The Computer Journal-2013-Wright-141-55.pdf - Published Version
Available under License Creative Commons Attribution.

Download (743Kb) | Preview
[img] Text
Wright_paper.pdf - Accepted Version
Embargoed item. Restricted access to Repository staff only

Download (456Kb)
Official URL: http://dx.doi.org/10.1093/comjnl/bxs044

Request Changes to record.

Abstract

Input/Output (I/O) operations can represent a significant proportion of the run-time of parallel scientific computing applications. Although there have been several advances in file format libraries, file system design and I/O hardware, a growing divergence exists between the performance of parallel file systems and the compute clusters that they support. In this paper, we document the design and application of the RIOT I/O toolkit (RIOT) being developed at the University of Warwick with our industrial partners at the Atomic Weapons Establishment and Sandia National Laboratories. We use the toolkit to assess the performance of three industry-standard I/O benchmarks on three contrasting supercomputers, ranging from a mid-sized commodity cluster to a large-scale proprietary IBM BlueGene/P system. RIOT provides a powerful framework in which to analyse I/O and parallel file system behaviour—we demonstrate, for example, the large file locking overhead of IBM's General Parallel File System, which can consume nearly 30% of the total write time in the FLASH-IO benchmark. Through I/O trace analysis, we also assess the performance of HDF-5 in its default configuration, identifying a bottleneck created by the use of suboptimal Message Passing Interface hints. Furthermore, we investigate the performance gains attributed to the Parallel Log-structured File System (PLFS) being developed by EMC Corporation and the Los Alamos National Laboratory. Our evaluation of PLFS involves two high-performance computing systems with contrasting I/O backplanes and illustrates the varied improvements to I/O that result from the deployment of PLFS (ranging from up to 25× speed-up in I/O performance on a large I/O installation to 2× speed-up on the much smaller installation at the University of Warwick).

Item Type: Journal Article
Subjects: Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software
Divisions: Faculty of Science > Computer Science
Library of Congress Subject Headings (LCSH): High performance computing -- Great Britain, Electronic data processing, Parallel processing (Electronic computers), Input-output analysis -- Computer programs, File organization (Computer science)
Journal or Publication Title: Computer Journal
Publisher: Oxford University Press
ISSN: 0010-4620
Official Date: 2013
Dates:
DateEvent
2013Published
Volume: Volume 56
Number: Number 2
Page Range: pp. 141-155
DOI: 10.1093/comjnl/bxs044
Status: Peer Reviewed
Publication Status: Published
Access rights to Published version: Open Access
Description:

This article appears in: Special Focus on Performance Engineering

Funder: Royal Society (Great Britain), TSB Bank (Great Britain), Atomic Weapons Establishment (Great Britain) (AWE), United States. Department of Energy
Grant number: IF090020/AM (RS) ; KTP006740 (TSB) ; CDK0660, CDK0724 (AWE) ; DE-AC04-94AL85000 (USDE)

Request changes or add full text files to a record

Repository staff actions (login required)

View Item View Item
twitter

Email us: wrap@warwick.ac.uk
Contact Details
About Us