The Library

Parallel file system analysis through application I/O tracing

Tools

Wright, S. A., Hammond, Simon D., Pennycook, Simon J., Bird, Robert F., Herdman, J. A., Miller, I. (Ian), Vadgama, A., Bhalerao, Abhir and Jarvis, Stephen A., 1970- . (2013) Parallel file system analysis through application I/O tracing. Computer Journal, Volume 56 (Number 2). pp. 141-155. ISSN 0010-4620

[img]
Preview
 Text
WRAP_Wright_The Computer Journal-2013-Wright-141-55.pdf - Published Version
Available under License Creative Commons Attribution.
Download (743Kb) | Preview
[img] Text
Wright_paper.pdf - Accepted Version
Restricted to Repository staff only
Download (456Kb)

Abstract

Input/Output (I/O) operations can represent a significant proportion of the run-time of parallel scientific computing applications. Although there have been several advances in file format libraries, file system design and I/O hardware, a growing divergence exists between the performance of parallel file systems and the compute clusters that they support. In this paper, we document the design and application of the RIOT I/O toolkit (RIOT) being developed at the University of Warwick with our industrial partners at the Atomic Weapons Establishment and Sandia National Laboratories. We use the toolkit to assess the performance of three industry-standard I/O benchmarks on three contrasting supercomputers, ranging from a mid-sized commodity cluster to a large-scale proprietary IBM BlueGene/P system. RIOT provides a powerful framework in which to analyse I/O and parallel file system behaviour—we demonstrate, for example, the large file locking overhead of IBM's General Parallel File System, which can consume nearly 30% of the total write time in the FLASH-IO benchmark. Through I/O trace analysis, we also assess the performance of HDF-5 in its default configuration, identifying a bottleneck created by the use of suboptimal Message Passing Interface hints. Furthermore, we investigate the performance gains attributed to the Parallel Log-structured File System (PLFS) being developed by EMC Corporation and the Los Alamos National Laboratory. Our evaluation of PLFS involves two high-performance computing systems with contrasting I/O backplanes and illustrates the varied improvements to I/O that result from the deployment of PLFS (ranging from up to 25× speed-up in I/O performance on a large I/O installation to 2× speed-up on the much smaller installation at the University of Warwick).

Item Type: Journal Article
Subjects: Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software
Divisions: Faculty of Science > Computer Science
Library of Congress Subject Headings (LCSH): High performance computing -- Great Britain, Electronic data processing, Parallel processing (Electronic computers), Input-output analysis -- Computer programs, File organization (Computer science)
Journal or Publication Title: Computer Journal
Publisher: Oxford University Press
ISSN: 0010-4620
Official Date: 2013
Volume: Volume 56
Number: Number 2
Page Range: pp. 141-155
Identification Number: 10.1093/comjnl/bxs044
Status: Peer Reviewed
Publication Status: Published
Access rights to Published version: Open Access
Description: This article appears in: Special Focus on Performance Engineering
Funder: Royal Society (Great Britain), TSB Bank (Great Britain), Atomic Weapons Establishment (Great Britain), United States. Dept. of Energy
Grant number: IF090020/AM (RS) ; KTP006740 (TSB) ; CDK0660, CDK0724 (AWE) ; DE-AC04-94AL85000 (USDE)
References:

[1] Message Passing Interface Forum (1998) MPI2: a message
passing interface standard. High Perf. Comput. Appl., 12, 1–299.
[2] ANL/MCS-TM-234 (1997) ROMIO: A High-Performance,
Portable MPI-IO Implementation. Argonne, IL.
[3] Gabriel, E. et al. (2004) Open MPI: goals, concept, and design of
a next generation MPI implementation. Lect. Notes Comput. Sci.
(LNCS), 3241, 97–104.
[4] Gropp,W. (2002) MPICH2: a newstart for MPI implementations.
Lect. Notes Comput. Sci. (LNCS), 2474, 7–42.
[5] Almási, G. et al. (2004) Implementing MPI on the BlueGene/L
supercomputer. Lect. Notes Comput. Sci. (LNCS), 3149, 833–845.
[6] Bull (2010) BullX Cluster Suite Application Developer’s Guide.
Les Clayes-sous-Bois, Paris.
[7] Koziol, Q. and Matzke, R. (1998) HDF5—A New Generation of
HDF: Reference Manual and User Guide. Champaign, IL, USA.
[8] Rew, R.K. and Davis, G.P. (1990) NetCDF: an interface for
scientific data access. IEEE Comput. Graph. Appl., 10, 76–82.
[9] Li, J., Liao,W., Choundhary, A., Ross, R., Thakur, R., Gropp,W.,
Latham, R., Siegel, A., Gallagher, B. and Zingale, M. (2003)
Parallel netCDF: A High-performance Scientific I/O Interface.
Proc. ACM/IEEE Int. Conf. Supercomputing (SC’03), Pheonix,
AZ, November, pp. 39–50. ACM, NewYork, NY.
[10] Wright, S.A., Pennycook, S.J., Hammond, S.D. and Jarvis, S.A.
(2011) RIOT—A Parallel Input/Output Tracer. Proc. 27th
Annual UK Performance Engineering Workshop (UKPEW’11),
Bradford, UK, July, pp. 25–39. The University of Bradford,
Bradford, UK.
[11] Shan, H., Antypas, K. and Shalf, J. (2008) Characterizing and
Predicting the I/O Performance of HPC Applications using a
Parameterized Synthetic Benchmark. Proc. ACM/IEEE Int. Conf.
Supercomputing (SC’08), November. IEEE Press, Piscataway,
NJ, Austin, TX.
[12] LBNL-62647 (2007) Using IOR to Analyze the I/O Performance
for HPC Platforms. Lawrence Berkeley National Laboratory,
Berkeley, CA.
[13] Bent, J., Gibson, G., Grider, G., McClelland, B., Nowoczynski, P.,
Nunez, J., Polte, M. andWingate, M. (2009) PLFS:ACheckpoint
Filesystem for Parallel Applications. Proc. ACM/IEEE Int.
Conf. Supercomputing Conf. (SC’09), Portland, OR, November,
pp. 21:1–21:12. ACM, NewYork, NY.
[14] Polte, M., Lofstead, J., Bent, J., Gibson, G., Klasky, S.A.,
Liu, Q., Parashar, M., Schwan, K. and Wolf, M. (2009) . . . And
Eat It Too: High Read Performance in Write-Optimized HPC
I/O Middleware File Formats. Proc. 4th Annual Workshop on
Petascale Data Storage (PDSW’09), Portland, OR, November,
pp. 21–25. ACM, NewYork, NY.
[15] Wolman, B. and Olson, T. (1989) IOBENCH: a system
independent IO benchmark. ACM SIGARCH Comput. Archit.
News, 17, 55–70.
[16] Bailey, D.H. et al. (1991) The NAS parallel benchmarks. Int. J.
High Perf. Comput. Appl., 5, 63–73.
[17] Rosner, R. et al. (2000) Flash code: studying astrophysical
thermonuclear flashes. Comput. Sci. Eng., 2, 33–41.
[18] Layton, J. (2012) HPC storage—getting started with I/O profiling.
http://hpc.admin-magazine.com/Articles/HPC-Storage-IO-
Profiling (accessed 2 February 2012).
[19] (2011) Collectl. http://collectl.sourceforge.net (accessed 16
January 2012).
[20] (2011) Ganglia monitoring system. http://ganglia.sourceforget.
net (accessed 16 January 2012).
[21] Carns, P., Latham, R., Ross, R., Iskra, K., Lang, S. and Riley,
K. (2009) 24/7 Characterization of Petascale I/O Workloads.
Proc. IEEE Int. Conf. Cluster Computing and Workshops
(CLUSTER’09), New Orleans, LA, September, pp. 1–10. IEEE
Computer Society, Los Alamitos, CA.
[22] Fuerlinger, K., Wright, N. and Skinner, D. (2010) Effective
Performance Measurement at Petascale Using IPM. Proc. IEEE
16th Int. Conf. Parallel and Distributed Systems (ICPADS’10),
Shanghai, China, December, pp. 373–380. IEEE Computer
Society,Washington, DC.
[23] Uselton, A., Howison, M., Wright, N.J., Skinner, D., Keen, N.,
Shalf, J., Karavanic, K.L. and Oliker, L. (2010) Parallel I/O
Performance: From Events to Ensembles. Proc. IEEE Int. Symp.
Parallel Distributed Processing (IPDPS’10), Atlanta, GA, April,
pp. 1–11. IEEE Computer Society, Los Alamitos, CA.
[24] Schmuck, F. and Haskin, R. (2002) GPFS: A Shared-
Disk File System for Large Computing Clusters. Proc. 1st
USENIX Conf. File and Storage Technologies (FAST’02),
Monterey, CA, January, pp. 231–244. USENIX Association
Berkeley, CA.
[25] Schwan, P. (2003) Lustre: building a file system for 1000-node
clusters. http://lustre.org (accessed 23 October 2011).
[26] Nowoczynski, P., Stone, N., Yanovich, J. and Sommerfield, J.
(2008) Zest Checkpoint Storage System for Large Supercomputers.
Proc. 3rd Annual Workshop on Petascale Data Storage
(PDSW’08), Austin, TX, November, pp. 1–5. IEEE Computer
Society, Los Alamitos, CA.
[27] Polte, M., Simsa, J., Tantisiriroj, W., Gibson, G., Dayal, S.,
Chainani, M. and Uppugandla, D.K. (2008) Fast Logbased
Concurrent Writing of Checkpoints. Proc. 3rd Annual
Workshop on Petascale Data Storage (PDSW’08), Austin, TX,
November, pp. 1–4. IEEE Computer Society, Los Alamitos, CA.
[28] Wang, Y. and Kaeli, D. (2003) Source Level Transformations
to Improve I/O Data Partitioning. Proc. 1st Int. Workshop on
Storage Network Architecture and Parallel I/Os (SNAPI’03),
New Orleans, LA, September–October, pp. 27–35. ACM,
NewYork, NY.
[29] Wang, Y. and Kaeli, D. (2003) Profile-Guided I/O Partitioning.
Proc. 17th Annual Int. Conf. Supercomputing (ICS’03),
San Francisco, CA, June, pp. 252–260. ACM, NewYork, NY.
[30] Wright, S.A., Hammond, S.D., Pennycook, S.J. and Jarvis, S.A.
(2011) Light-weight parallel I/O analysis at scale. Lect. Notes
Comput. Sci. (LNCS), 6977, 235–249.
[31] Thakur, R., Gropp, W. and Lusk, E. (1996) An Abstract-
Device Interface for Implementing Portable Parallel-I/O
Interfaces. Proc. 6th Symp. Frontiers of Massively Parallel
Computation (FRONTIERS’96), Annapolis, MD, October,
pp. 180–187. IEEE Computer Society, Los Alamitos, CA.
[32] Zingale, M. (2011) FLASH I/O benchmark routine—parallel
HDF 5. http://www.ucolick.org/∼zingale/flash_benchmark_io/
(accessed 21 February 2011).
[33] Argonne National Laboratory (2011) Parallel I/O benchmarking
consortium. http://www.mcs.anl.gov/research/projects/piobenchmark/
(accessed 21 February 2011).
[34] Fryxell, B., Olson, K., Ricker, P., Timmes, F., Zingale, M.,
Lamb, D., MacNeice, P., Rosner, R., Truran, J. and H.Tufo (2000)
FLASH: an adaptive mesh hydrodynamics code for modeling
astrophysical thermonuclear flashes. Astrophys. J. Suppl. Ser.,
131, 273.
[35] RNR-94-007 (1994) TheNAS Parallel Benchmarks.NASAAmes
Research Center, Moffet Field, CA.
[36] Thakur, R., Gropp, W. and Lusk, E. (1999) Data Sieving and
Collective I/O in ROMIO. Proc. 7th Symp. Frontiers of Massively
Parallel Computation (FRONTIERS’99), Annapolis, MD,
February, pp. 182–191. IEEE Computer Society, Los Alamitos,
CA.
[37] Nitzberg, B. and Lo, V. (1997) Collective Buffering: Improving
Parallel I/O Performance. Proc. 6th IEEE Int. Symp. High
Performance Distributed Computing (HPDC’97), Portland, OR,
August, pp. 148–157. IEEE Computer Society,Washington, DC.
The Computer Journal, Vol. 56 No. 2, 2013
URI: http://wrap.warwick.ac.uk/id/eprint/45580

Request changes to a record

Actions (login required)

View Item View Item

Document Downloads

More statistics for this item...