Boustati, Ayman (2021) Compositionality, stability and robustness in probabilistic machine learning. PhD thesis, University of Warwick.

Preview

PDF WRAP_Theses_Boustati_2021.pdf - Submitted Version - Requires a PDF viewer. Download (6Mb) | Preview

Request Changes to record.

Abstract

Probability theory plays an integral part in the field of machine learning. Its use has been advocated by many [MacKay, 2002; Jaynes, 2003] as it allows for the quantification of uncertainty and the incorporation of prior knowledge by simply applying the rules of probability [Kolmogorov, 1950]. While probabilistic machine learning has been originally restricted to simple models, the advent of new computational technologies, such as automatic differentiation, and advances in approximate inference, such as Variational Inference [Blei et al., 2017], has made it more viable in complex settings. Despite this progress, there remain many challenges to its application to real-world tasks. Among those are questions about the ability of probabilistic models to model complex tasks and their reliability both in training and in the face of unexpected data perturbation. These three issues can be addressed by examining the three properties of compositionality, stability and robustness in these models. Hence, this thesis explores these three key properties and their application to probabilistic models, while validating their importance on a range of applications.

The first contribution in this thesis studies compositionality. Compositionality enables the construction of complex and expressive probabilistic models from simple components. This increases the types of phenomena that one can model and provides the modeller with a wide array of modelling options. This thesis examines this property through the lens of Gaussian processes [Rasmussen and Williams, 2006]. It proposes a generic compositional Gaussian process model to address the problem of multi-task learning in the non-linear setting.

Additionally, this thesis contributes two methods addressing the issue of stability. Stability determines the reliability of inference algorithms in the presence of noise. More stable training procedures lead to faster, more reliable inferences, especially for complex models. The two proposed methods aim at stabilising stochastic gradient estimation in Variational Inference using the method of control variates [Owen, 2013].

Finally, the last contribution of this thesis considers robustness. Robust machine learning methods are unaffected by unaccounted-for phenomena in the data. This makes such methods essential in deploying machine learning on real-world datasets. This thesis examines the problem of robust inference in sequential probabilistic models by combining the ideas of Generalised Bayesian Inference [Bissiri et al., 2016] and Sequential Monte Carlo sampling [Doucet and Johansen, 2011].

Item Type:	Thesis (PhD)
Subjects:	Q Science > QA Mathematics
Library of Congress Subject Headings (LCSH):	Distribution (Probability theory), Probabilities, Machine learning, Computational learning theory, Gaussian processes
Official Date:	January 2021
Dates:	Date Event January 2021 UNSPECIFIED
Institution:	University of Warwick
Theses Department:	Mathematics for Real-World Systems Centre for Doctoral Training
Thesis Type:	PhD
Publication Status:	Unpublished
Supervisor(s)/Advisor:	Damoulas, Theodoro
Format of File:	pdf
Extent:	xv, 1, 201 leaves : illustrations
Language:	eng

Request changes or add full text files to a record

Repository staff actions (login required)

View Item

Downloads