Skip to content Skip to navigation
University of Warwick
  • Study
  • |
  • Research
  • |
  • Business
  • |
  • Alumni
  • |
  • News
  • |
  • About

University of Warwick
Publications service & WRAP

Highlight your research

  • WRAP
    • Home
    • Search WRAP
    • Browse by Warwick Author
    • Browse WRAP by Year
    • Browse WRAP by Subject
    • Browse WRAP by Department
    • Browse WRAP by Funder
    • Browse Theses by Department
  • Publications Service
    • Home
    • Search Publications Service
    • Browse by Warwick Author
    • Browse Publications service by Year
    • Browse Publications service by Subject
    • Browse Publications service by Department
    • Browse Publications service by Funder
  • Help & Advice
University of Warwick

The Library

  • Login
  • Admin

On the throughput optimization in large-scale batch-processing systems

Tools
- Tools
+ Tools

Kar, Sounak, Rehrmann, Robin, Mukhopadhyay, Arpan, Alt, Bastian, Ciucu, Florin, Koeppl, Heinz, Binnig, Carsten and Rizk, Amr (2020) On the throughput optimization in large-scale batch-processing systems. In: IFIP Performance 2020, Online, 2-6 Nov 2020. Published in: Performance Evaluation, 144 ISSN 0166-5316. doi:10.1016/j.peva.2020.102142

[img]
Preview
PDF
WRAP-throughput-optimization-large-scale-batch-processing-systems-Mukhopadhyay-2020.pdf - Accepted Version - Requires a PDF viewer.
Available under License Creative Commons Attribution Non-commercial No Derivatives 4.0.

Download (1483Kb) | Preview
[img] PDF
WRAP-throughput-optimization-large-scale-batch-processing-systems-Mukhopadhyay-2020.pdf - Accepted Version
Embargoed item. Restricted access to Repository staff only - Requires a PDF viewer.

Download (1482Kb)
Official URL: https://doi.org/10.1016/j.peva.2020.102142

Request Changes to record.

Abstract

We analyse a data-processing system with clients producing jobs which are processed in batches by parallel servers; the system throughput critically depends on the batch size and a corresponding sub-additive speedup function. In practice, throughput optimization relies on numerical searches for the optimal batch size, a process that can take up to multiple days in existing commercial systems. In this paper, we model the system in terms of a closed queueing network; a standard Markovian analysis yields the optimal throughput in time. Our main contribution is a mean-field model of the system for the regime where the system size is large. We show that the mean-field model has a unique, globally attractive stationary point which can be found in closed form and which characterizes the asymptotic throughput of the system as a function of the batch size. Using this expression we find the asymptotically optimal throughput in time. Numerical settings from a large commercial system reveal that this asymptotic optimum is accurate in practical finite regimes.

Item Type: Conference Item (Paper)
Subjects: Q Science > QA Mathematics
Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software
T Technology > TA Engineering (General). Civil engineering (General)
Divisions: Faculty of Science, Engineering and Medicine > Science > Computer Science
Library of Congress Subject Headings (LCSH): Information storage and retrieval systems, Operating systems (Computers), Markov processes , Structural optimization , Queuing networks (Data transmission)
Journal or Publication Title: Performance Evaluation
Publisher: Elsevier Science BV
ISSN: 0166-5316
Official Date: December 2020
Dates:
DateEvent
December 2020Published
8 October 2020Available
25 September 2020Accepted
Volume: 144
Article Number: 102142
DOI: 10.1016/j.peva.2020.102142
Status: Peer Reviewed
Publication Status: Published
Access rights to Published version: Restricted or Subscription Access
Conference Paper Type: Paper
Title of Event: IFIP Performance 2020
Type of Event: Conference
Location of Event: Online
Date(s) of Event: 2-6 Nov 2020
Related URLs:
  • Publisher
Open Access Version:
  • ArXiv

Request changes or add full text files to a record

Repository staff actions (login required)

View Item View Item
twitter

Email us: wrap@warwick.ac.uk
Contact Details
About Us