The Library
On the throughput optimization in large-scale batch-processing systems
Tools
Tools
Tools
Kar, Sounak, Rehrmann, Robin, Mukhopadhyay, Arpan, Alt, Bastian, Ciucu, Florin, Koeppl, Heinz, Binnig, Carsten and Rizk, Amr (2020) On the throughput optimization in large-scale batch-processing systems. In: IFIP Performance 2020, Online, 2-6 Nov 2020. Published in: Performance Evaluation, 144 ISSN 0166-5316. doi:10.1016/j.peva.2020.102142
|
PDF
WRAP-throughput-optimization-large-scale-batch-processing-systems-Mukhopadhyay-2020.pdf - Accepted Version - Requires a PDF viewer. Available under License Creative Commons Attribution Non-commercial No Derivatives 4.0. Download (1483Kb) | Preview |
|
![[img]](http://wrap.warwick.ac.uk/style/images/fileicons/application_pdf.png) |
PDF
WRAP-throughput-optimization-large-scale-batch-processing-systems-Mukhopadhyay-2020.pdf - Accepted Version Embargoed item. Restricted access to Repository staff only - Requires a PDF viewer. Download (1482Kb) |
Official URL: https://doi.org/10.1016/j.peva.2020.102142
Abstract
We analyse a data-processing system with clients producing jobs which are processed in batches by parallel servers; the system throughput critically depends on the batch size and a corresponding sub-additive speedup function. In practice, throughput optimization relies on numerical searches for the optimal batch size, a process that can take up to multiple days in existing commercial systems. In this paper, we model the system in terms of a closed queueing network; a standard Markovian analysis yields the optimal throughput in time. Our main contribution is a mean-field model of the system for the regime where the system size is large. We show that the mean-field model has a unique, globally attractive stationary point which can be found in closed form and which characterizes the asymptotic throughput of the system as a function of the batch size. Using this expression we find the asymptotically optimal throughput in time. Numerical settings from a large commercial system reveal that this asymptotic optimum is accurate in practical finite regimes.
| Item Type: | Conference Item (Paper) | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Subjects: | Q Science > QA Mathematics Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software T Technology > TA Engineering (General). Civil engineering (General) |
||||||||
| Divisions: | Faculty of Science > Computer Science | ||||||||
| Library of Congress Subject Headings (LCSH): | Information storage and retrieval systems, Operating systems (Computers), Markov processes , Structural optimization , Queuing networks (Data transmission) | ||||||||
| Journal or Publication Title: | Performance Evaluation | ||||||||
| Publisher: | Elsevier Science BV | ||||||||
| ISSN: | 0166-5316 | ||||||||
| Official Date: | December 2020 | ||||||||
| Dates: |
|
||||||||
| Volume: | 144 | ||||||||
| Article Number: | 102142 | ||||||||
| DOI: | 10.1016/j.peva.2020.102142 | ||||||||
| Status: | Peer Reviewed | ||||||||
| Publication Status: | Published | ||||||||
| Access rights to Published version: | Restricted or Subscription Access | ||||||||
| Conference Paper Type: | Paper | ||||||||
| Title of Event: | IFIP Performance 2020 | ||||||||
| Type of Event: | Conference | ||||||||
| Location of Event: | Online | ||||||||
| Date(s) of Event: | 2-6 Nov 2020 | ||||||||
| Related URLs: | |||||||||
| Open Access Version: |
Request changes or add full text files to a record
Repository staff actions (login required)
 |
View Item |
