Wright, S. A., Hammond, Simon D., Pennycook, Simon J., Bird, Robert F., Herdman, J.A., Miller, I., Vadgama, A., Bhalerao, Abhir and Jarvis, Stephen A., 1970- . (2013) Parallel file system analysis through application I/O tracing. Computer Journal, Volume 56 (Number 2). pp. 141-155. ISSN 0010-4620

Preview

Text WRAP_Wright_The Computer Journal-2013-Wright-141-55.pdf - Published Version Available under License Creative Commons Attribution. Download (743Kb) | Preview

Abstract

Input/Output (I/O) operations can represent a significant proportion of the run-time of parallel scientific computing applications. Although there have been several advances in file format libraries, file system design and I/O hardware, a growing divergence exists between the performance of parallel file systems and the compute clusters that they support. In this paper, we document the design and application of the RIOT I/O toolkit (RIOT) being developed at the University of Warwick with our industrial partners at the Atomic Weapons Establishment and Sandia National Laboratories. We use the toolkit to assess the performance of three industry-standard I/O benchmarks on three contrasting supercomputers, ranging from a mid-sized commodity cluster to a large-scale proprietary IBM BlueGene/P system. RIOT provides a powerful framework in which to analyse I/O and parallel file system behaviour—we demonstrate, for example, the large file locking overhead of IBM's General Parallel File System, which can consume nearly 30% of the total write time in the FLASH-IO benchmark. Through I/O trace analysis, we also assess the performance of HDF-5 in its default configuration, identifying a bottleneck created by the use of suboptimal Message Passing Interface hints. Furthermore, we investigate the performance gains attributed to the Parallel Log-structured File System (PLFS) being developed by EMC Corporation and the Los Alamos National Laboratory. Our evaluation of PLFS involves two high-performance computing systems with contrasting I/O backplanes and illustrates the varied improvements to I/O that result from the deployment of PLFS (ranging from up to 25× speed-up in I/O performance on a large I/O installation to 2× speed-up on the much smaller installation at the University of Warwick).

Item Type:	Journal Article
Subjects:	Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software
Divisions:	Faculty of Science > Computer Science
Library of Congress Subject Headings (LCSH):	High performance computing -- Great Britain, Electronic data processing, Parallel processing (Electronic computers), Input-output analysis -- Computer programs, File organization (Computer science)
Journal or Publication Title:	Computer Journal
Publisher:	Oxford University Press
ISSN:	0010-4620
Date:	2013
Volume:	Volume 56
Number:	Number 2
Page Range:	pp. 141-155
Identification Number:	10.1093/comjnl/bxs044
Status:	Peer Reviewed
Publication Status:	Published
Access rights to Published version:	Open Access
Description:	This article appears in: Special Focus on Performance Engineering
Funder:	Royal Society (Great Britain), TSB Bank (Great Britain), Atomic Weapons Establishment (Great Britain), United States. Dept. of Energy
Grant number:	IF090020/AM (RS) ; KTP006740 (TSB) ; CDK0660, CDK0724 (AWE) ; DE-AC04-94AL85000 (USDE)
Related URLs:	Other Repository
References:	[1] Message Passing Interface Forum (1998) MPI2: a message passing interface standard. High Perf. Comput. Appl., 12, 1–299. [2] ANL/MCS-TM-234 (1997) ROMIO: A High-Performance, Portable MPI-IO Implementation. Argonne, IL. [3] Gabriel, E. et al. (2004) Open MPI: goals, concept, and design of a next generation MPI implementation. Lect. Notes Comput. Sci. (LNCS), 3241, 97–104. [4] Gropp,W. (2002) MPICH2: a newstart for MPI implementations. Lect. Notes Comput. Sci. (LNCS), 2474, 7–42. [5] Almási, G. et al. (2004) Implementing MPI on the BlueGene/L supercomputer. Lect. Notes Comput. Sci. (LNCS), 3149, 833–845. [6] Bull (2010) BullX Cluster Suite Application Developer’s Guide. Les Clayes-sous-Bois, Paris. [7] Koziol, Q. and Matzke, R. (1998) HDF5—A New Generation of HDF: Reference Manual and User Guide. Champaign, IL, USA. [8] Rew, R.K. and Davis, G.P. (1990) NetCDF: an interface for scientific data access. IEEE Comput. Graph. Appl., 10, 76–82. [9] Li, J., Liao,W., Choundhary, A., Ross, R., Thakur, R., Gropp,W., Latham, R., Siegel, A., Gallagher, B. and Zingale, M. (2003) Parallel netCDF: A High-performance Scientific I/O Interface. Proc. ACM/IEEE Int. Conf. Supercomputing (SC’03), Pheonix, AZ, November, pp. 39–50. ACM, NewYork, NY. [10] Wright, S.A., Pennycook, S.J., Hammond, S.D. and Jarvis, S.A. (2011) RIOT—A Parallel Input/Output Tracer. Proc. 27th Annual UK Performance Engineering Workshop (UKPEW’11), Bradford, UK, July, pp. 25–39. The University of Bradford, Bradford, UK. [11] Shan, H., Antypas, K. and Shalf, J. (2008) Characterizing and Predicting the I/O Performance of HPC Applications using a Parameterized Synthetic Benchmark. Proc. ACM/IEEE Int. Conf. Supercomputing (SC’08), November. IEEE Press, Piscataway, NJ, Austin, TX. [12] LBNL-62647 (2007) Using IOR to Analyze the I/O Performance for HPC Platforms. Lawrence Berkeley National Laboratory, Berkeley, CA. [13] Bent, J., Gibson, G., Grider, G., McClelland, B., Nowoczynski, P., Nunez, J., Polte, M. andWingate, M. (2009) PLFS:ACheckpoint Filesystem for Parallel Applications. Proc. ACM/IEEE Int. Conf. Supercomputing Conf. (SC’09), Portland, OR, November, pp. 21:1–21:12. ACM, NewYork, NY. [14] Polte, M., Lofstead, J., Bent, J., Gibson, G., Klasky, S.A., Liu, Q., Parashar, M., Schwan, K. and Wolf, M. (2009) . . . And Eat It Too: High Read Performance in Write-Optimized HPC I/O Middleware File Formats. Proc. 4th Annual Workshop on Petascale Data Storage (PDSW’09), Portland, OR, November, pp. 21–25. ACM, NewYork, NY. [15] Wolman, B. and Olson, T. (1989) IOBENCH: a system independent IO benchmark. ACM SIGARCH Comput. Archit. News, 17, 55–70. [16] Bailey, D.H. et al. (1991) The NAS parallel benchmarks. Int. J. High Perf. Comput. Appl., 5, 63–73. [17] Rosner, R. et al. (2000) Flash code: studying astrophysical thermonuclear flashes. Comput. Sci. Eng., 2, 33–41. [18] Layton, J. (2012) HPC storage—getting started with I/O profiling. http://hpc.admin-magazine.com/Articles/HPC-Storage-IO- Profiling (accessed 2 February 2012). [19] (2011) Collectl. http://collectl.sourceforge.net (accessed 16 January 2012). [20] (2011) Ganglia monitoring system. http://ganglia.sourceforget. net (accessed 16 January 2012). [21] Carns, P., Latham, R., Ross, R., Iskra, K., Lang, S. and Riley, K. (2009) 24/7 Characterization of Petascale I/O Workloads. Proc. IEEE Int. Conf. Cluster Computing and Workshops (CLUSTER’09), New Orleans, LA, September, pp. 1–10. IEEE Computer Society, Los Alamitos, CA. [22] Fuerlinger, K., Wright, N. and Skinner, D. (2010) Effective Performance Measurement at Petascale Using IPM. Proc. IEEE 16th Int. Conf. Parallel and Distributed Systems (ICPADS’10), Shanghai, China, December, pp. 373–380. IEEE Computer Society,Washington, DC. [23] Uselton, A., Howison, M., Wright, N.J., Skinner, D., Keen, N., Shalf, J., Karavanic, K.L. and Oliker, L. (2010) Parallel I/O Performance: From Events to Ensembles. Proc. IEEE Int. Symp. Parallel Distributed Processing (IPDPS’10), Atlanta, GA, April, pp. 1–11. IEEE Computer Society, Los Alamitos, CA. [24] Schmuck, F. and Haskin, R. (2002) GPFS: A Shared- Disk File System for Large Computing Clusters. Proc. 1st USENIX Conf. File and Storage Technologies (FAST’02), Monterey, CA, January, pp. 231–244. USENIX Association Berkeley, CA. [25] Schwan, P. (2003) Lustre: building a file system for 1000-node clusters. http://lustre.org (accessed 23 October 2011). [26] Nowoczynski, P., Stone, N., Yanovich, J. and Sommerfield, J. (2008) Zest Checkpoint Storage System for Large Supercomputers. Proc. 3rd Annual Workshop on Petascale Data Storage (PDSW’08), Austin, TX, November, pp. 1–5. IEEE Computer Society, Los Alamitos, CA. [27] Polte, M., Simsa, J., Tantisiriroj, W., Gibson, G., Dayal, S., Chainani, M. and Uppugandla, D.K. (2008) Fast Logbased Concurrent Writing of Checkpoints. Proc. 3rd Annual Workshop on Petascale Data Storage (PDSW’08), Austin, TX, November, pp. 1–4. IEEE Computer Society, Los Alamitos, CA. [28] Wang, Y. and Kaeli, D. (2003) Source Level Transformations to Improve I/O Data Partitioning. Proc. 1st Int. Workshop on Storage Network Architecture and Parallel I/Os (SNAPI’03), New Orleans, LA, September–October, pp. 27–35. ACM, NewYork, NY. [29] Wang, Y. and Kaeli, D. (2003) Profile-Guided I/O Partitioning. Proc. 17th Annual Int. Conf. Supercomputing (ICS’03), San Francisco, CA, June, pp. 252–260. ACM, NewYork, NY. [30] Wright, S.A., Hammond, S.D., Pennycook, S.J. and Jarvis, S.A. (2011) Light-weight parallel I/O analysis at scale. Lect. Notes Comput. Sci. (LNCS), 6977, 235–249. [31] Thakur, R., Gropp, W. and Lusk, E. (1996) An Abstract- Device Interface for Implementing Portable Parallel-I/O Interfaces. Proc. 6th Symp. Frontiers of Massively Parallel Computation (FRONTIERS’96), Annapolis, MD, October, pp. 180–187. IEEE Computer Society, Los Alamitos, CA. [32] Zingale, M. (2011) FLASH I/O benchmark routine—parallel HDF 5. http://www.ucolick.org/∼zingale/flash_benchmark_io/ (accessed 21 February 2011). [33] Argonne National Laboratory (2011) Parallel I/O benchmarking consortium. http://www.mcs.anl.gov/research/projects/piobenchmark/ (accessed 21 February 2011). [34] Fryxell, B., Olson, K., Ricker, P., Timmes, F., Zingale, M., Lamb, D., MacNeice, P., Rosner, R., Truran, J. and H.Tufo (2000) FLASH: an adaptive mesh hydrodynamics code for modeling astrophysical thermonuclear flashes. Astrophys. J. Suppl. Ser., 131, 273. [35] RNR-94-007 (1994) TheNAS Parallel Benchmarks.NASAAmes Research Center, Moffet Field, CA. [36] Thakur, R., Gropp, W. and Lusk, E. (1999) Data Sieving and Collective I/O in ROMIO. Proc. 7th Symp. Frontiers of Massively Parallel Computation (FRONTIERS’99), Annapolis, MD, February, pp. 182–191. IEEE Computer Society, Los Alamitos, CA. [37] Nitzberg, B. and Lo, V. (1997) Collective Buffering: Improving Parallel I/O Performance. Proc. 6th IEEE Int. Symp. High Performance Distributed Computing (HPDC’97), Portland, OR, August, pp. 148–157. IEEE Computer Society,Washington, DC. The Computer Journal, Vol. 56 No. 2, 2013
URI:	http://wrap.warwick.ac.uk/id/eprint/45580

Request changes to a record

Actions (login required)

View Item