Skip to content Skip to navigation
University of Warwick
  • Study
  • |
  • Research
  • |
  • Business
  • |
  • Alumni
  • |
  • News
  • |
  • About

University of Warwick
Publications service & WRAP

Highlight your research

  • WRAP
    • Home
    • Search WRAP
    • Browse by Warwick Author
    • Browse WRAP by Year
    • Browse WRAP by Subject
    • Browse WRAP by Department
    • Browse WRAP by Funder
    • Browse Theses by Department
  • Publications Service
    • Home
    • Search Publications Service
    • Browse by Warwick Author
    • Browse Publications service by Year
    • Browse Publications service by Subject
    • Browse Publications service by Department
    • Browse Publications service by Funder
  • Help & Advice
University of Warwick

The Library

  • Login
  • Admin

Goal-based composition of scalable hybrid analytics for heterogeneous architectures

Tools
- Tools
+ Tools

Coetzee, Peter and Jarvis, Stephen A. (2017) Goal-based composition of scalable hybrid analytics for heterogeneous architectures. Journal of Parallel and Distributed Computing, 108 . pp. 59-73. doi:10.1016/j.jpdc.2016.11.009

[img]
Preview
PDF
WRAP_1-s2.0-S0743731516301666-main.pdf - Published Version - Requires a PDF viewer.
Available under License Creative Commons Attribution 4.0.

Download (1557Kb) | Preview
Official URL: http://dx.doi.org/10.1016/j.jpdc.2016.11.009

Request Changes to record.

Abstract

Crafting scalable analytics in order to extract actionable business intelligence is a challenging endeavour, requiring multiple layers of expertise and experience. Often, this expertise is irreconcilably split between an organisation’s engineers and subject matter domain experts. Previous approaches to this problem have relied on technically adept users with tool-specific training.

Such an approach has a number of challenges: Expertise — There are few data-analytic subject domain experts with in-depth technical knowledge of compute architectures; Performance — Analysts do not generally make full use of the performance and scalability capabilities of the underlying architectures; Heterogeneity — calculating the most performant and scalable mix of real-time (on-line) and batch (off-line) analytics in a problem domain is difficult; Tools — Supporting frameworks will often direct several tasks, including, composition, planning, code generation, validation, performance tuning and analysis, but do not typically provide end-to-end solutions embedding all of these activities.

In this paper, we present a novel semi-automated approach to the composition, planning, code generation and performance tuning of scalable hybrid analytics, using a semantically rich type system which requires little programming expertise from the user. This approach is the first of its kind to permit domain experts with little or no technical expertise to assemble complex and scalable analytics, for hybrid on- and off-line analytic environments, with no additional requirement for low-level engineering support.

This paper describes (i) an abstract model of analytic assembly and execution, (ii) goal-based planning and (iii) code generation for hybrid on- and off-line analytics. An implementation, through a system which we call Mendeleev, is used to (iv) demonstrate the applicability of this technique through a series of case studies, where a single interface is used to create analytics that can be run simultaneously over on- and off-line environments. Finally, we (v) analyse the performance of the planner, and (vi) show that the performance of Mendeleev’s generated code is comparable with that of hand-written analytics.

Item Type: Journal Article
Subjects: Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software
Divisions: Faculty of Science > Computer Science
Library of Congress Subject Headings (LCSH): Computer science—Mathematics
Journal or Publication Title: Journal of Parallel and Distributed Computing
Publisher: Elsevier Science BV
ISSN: 0743-7315
Official Date: October 2017
Dates:
DateEvent
October 2017Published
7 December 2016Available
18 November 2016Accepted
15 June 2015Submitted
Volume: 108
Page Range: pp. 59-73
DOI: 10.1016/j.jpdc.2016.11.009
Status: Peer Reviewed
Publication Status: Published
Access rights to Published version: Open Access
Funder: Engineering and Physical Sciences Research Council (EPSRC)
Grant number: EP/K503204/1(EPSRC);
RIOXX Funder/Project Grant:
Project/Grant IDRIOXX Funder NameFunder ID
EP/K503204/1Engineering and Physical Sciences Research Councilhttp://dx.doi.org/10.13039/501100000266

Request changes or add full text files to a record

Repository staff actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics

twitter

Email us: wrap@warwick.ac.uk
Contact Details
About Us