Skip to content Skip to navigation
University of Warwick
  • Study
  • |
  • Research
  • |
  • Business
  • |
  • Alumni
  • |
  • News
  • |
  • About

University of Warwick
Publications service & WRAP

Highlight your research

  • WRAP
    • Home
    • Search WRAP
    • Browse by Warwick Author
    • Browse WRAP by Year
    • Browse WRAP by Subject
    • Browse WRAP by Department
    • Browse WRAP by Funder
    • Browse Theses by Department
  • Publications Service
    • Home
    • Search Publications Service
    • Browse by Warwick Author
    • Browse Publications service by Year
    • Browse Publications service by Subject
    • Browse Publications service by Department
    • Browse Publications service by Funder
  • Help & Advice
University of Warwick

The Library

  • Login
  • Admin

LDPLFS : improving I/O performance without application modification

Tools
- Tools
+ Tools

Wright, Steven A., Hammond, Simon D., Pennycook, Simon J., Miller, I., Herdman, J. A. and Jarvis, Stephen A. (2012) LDPLFS : improving I/O performance without application modification. In: 13th IEEE International Workshop on Parallel and Distributed Scientific and Engineering Computing, Shanghai, China, 21-25 May 2012. Published in: 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum (IPDPSW) pp. 1352-1359. ISBN 9781467309745. doi:10.1109/IPDPSW.2012.172

[img]
Preview
PDF
WRAP-LDPLFS-improving--performance-without-application-modification-Wright-2012.pdf - Accepted Version - Requires a PDF viewer.

Download (657Kb) | Preview
Official URL: http://dx.doi.org/10.1109/IPDPSW.2012.172

Request Changes to record.

Abstract

Input/Output (I/O) operations can represent a significant proportion of run-time when large scientific applications are run in parallel and at scale. In order to address the growing divergence between processing speeds and I/O performance, the Parallel Log-structured File System (PLFS) has been developed by EMC Corporation and the Los Alamos National Laboratory (LANL) to improve the performance of parallel file activities. Currently, PLFS requires the use of either (i) the FUSE Linux Kernel module; (ii) a modified MPI library with a customised ROMIO MPI-IO library; or (iii) an application rewrite to utilise the PLFS API directly. In this paper we present an alternative method of utilising PLFS in applications. This method employs a dynamic library to intercept the low-level POSIX operations and retarget them to use the equivalents offered by PLFS. We demonstrate our implementation of this approach, named LDPLFS, on a set of standard UNIX tools, as well on as a set of standard parallel I/O intensive mini-applications. The results demonstrate almost equivalent performance to a modified build of ROMIO and improvements over the FUSE-based approach. Furthermore, through our experiments we demonstrate decreased performance in PLFS when ran at scale on the Lustre file system.

Item Type: Conference Item (Paper)
Subjects: Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software
Divisions: Faculty of Science, Engineering and Medicine > Science > Computer Science
Journal or Publication Title: 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum (IPDPSW)
Publisher: IEEE
ISBN: 9781467309745
Official Date: 21 May 2012
Dates:
DateEvent
21 May 2012Published
Page Range: pp. 1352-1359
DOI: 10.1109/IPDPSW.2012.172
Status: Peer Reviewed
Publication Status: Published
Reuse Statement (publisher, data, author rights): © 2012 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Access rights to Published version: Restricted or Subscription Access
Conference Paper Type: Paper
Title of Event: 13th IEEE International Workshop on Parallel and Distributed Scientific and Engineering Computing
Type of Event: Workshop
Location of Event: Shanghai, China
Date(s) of Event: 21-25 May 2012

Request changes or add full text files to a record

Repository staff actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics

twitter

Email us: wrap@warwick.ac.uk
Contact Details
About Us