The Library
Learning graphical models from a distributed stream
Tools
Tools
Tools
Zhang, Y., Tirthapura, S. and Cormode, Graham (2018) Learning graphical models from a distributed stream. In: International Conference on Data Engineering (ICDE), 2018, Paris, France, 16–19 Apr 2018
|
PDF
WRAP-learning-graphical-models-distributed-stream-Cormode-2018.pdf - Accepted Version - Requires a PDF viewer. Download (1034Kb) | Preview |
Official URL: https://icde2018.org/
Abstract
A current challenge for data management systems is to support the construction and maintenance of machine learning models over data that is large, multi-dimensional, and evolving. While systems that could support these tasks are emerging, the need to scale to distributed, streaming data requires new models and algorithms. In this setting, as well as computational scalability and model accuracy, we also need to minimize the amount of communication between distributed processors, which is the chief component of latency. We study Bayesian Networks, the workhorse of graphical models, and present a communication-efficient method for continuously learning and maintaining a Bayesian network model over data that is arriving as a distributed stream partitioned across multiple processors. We show a strategy for maintaining model parameters that leads to an exponential reduction in communication when compared with baseline approaches to maintain the exact MLE (maximum likelihood estimation). Meanwhile, our strategy provides similar prediction errors for the target distribution and for classification tasks.
| Item Type: | Conference Item (Paper) | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Subjects: | Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software | |||||||||||||||
| Divisions: | Faculty of Science > Computer Science | |||||||||||||||
| Library of Congress Subject Headings (LCSH): | Database management, Machine learning, Big data, Bayesian statistical decision theory | |||||||||||||||
| Publisher: | IEEE | |||||||||||||||
| Official Date: | 2018 | |||||||||||||||
| Dates: |
|
|||||||||||||||
| Date of first compliant deposit: | 27 February 2018 | |||||||||||||||
| Status: | Peer Reviewed | |||||||||||||||
| Publication Status: | Published | |||||||||||||||
| Access rights to Published version: | Restricted or Subscription Access | |||||||||||||||
| RIOXX Funder/Project Grant: |
|
|||||||||||||||
| Conference Paper Type: | Paper | |||||||||||||||
| Title of Event: | International Conference on Data Engineering (ICDE), 2018 | |||||||||||||||
| Type of Event: | Conference | |||||||||||||||
| Location of Event: | Paris, France | |||||||||||||||
| Date(s) of Event: | 16–19 Apr 2018 |
Request changes or add full text files to a record
Repository staff actions (login required)
 |
View Item |
Downloads
Downloads per month over past year
