The Library

Particle methods for maximum likelihood estimation in latent variable models

Tools

Johansen, Adam M., Doucet, Arnaud and Davy, Manuel. (2008) Particle methods for maximum likelihood estimation in latent variable models. Statistics and Computing, Vol.18 (No.1). pp. 47-57. ISSN 0960-3174

Preview

PDF
WRAP_Johansen_corrected2.pdf - Accepted Version - Requires a PDF viewer.
Download (236Kb)

Official URL: http://dx.doi.org/10.1007/s11222-007-9037-8

Abstract

Standard methods for maximum likelihood parameter estimation
in latent variable models rely on the Expectation-Maximization algorithm and
its Monte Carlo variants. Our approach is different and motivated by similar
considerations to simulated annealing; that is we build a sequence of artificial
distributions whose support concentrates itself on the set of maximum likelihood estimates. We sample from these distributions using a sequential Monte
Carlo approach. We demonstrate state of the art performance for several applications of the proposed approach.

Item Type:

Journal Article

Subjects:

Q Science > QA Mathematics

Divisions:

Faculty of Science > Statistics

Library of Congress Subject Headings (LCSH):

Parameter estimation, Latent variables

Journal or Publication Title:

Statistics and Computing

Publisher:

Springer

ISSN:

0960-3174

Official Date:

2008

Dates:

Date	Event
2008	Published

Volume:

Vol.18

Number:

No.1

Page Range:

pp. 47-57

Identifier:

10.1007/s11222-007-9037-8

Status:

Peer Reviewed

Access rights to Published version:

Restricted or Subscription Access

References:

B. Amzal, F. Y. Bois, E. Parent, and C. P. Robert. Bayesian optimal design via
interacting particle systems. Journal of the American Statistical Association, 101
(474):773–785, June 2006.
N. Chopin. Central limit theorem for sequential Monte Carlo methods and its applications
to Bayesian inference. Annals of Statistics, 32(6):2385–2411, December
2004.
P. Del Moral. Feynman-Kac formulae: genealogical and interacting particle systems
with applications. Probability and Its Applications. Springer Verlag, New York,
2004.
P. Del Moral, A. Doucet, and A. Jasra. Sequential Monte Carlo samplers. Journal
of the Royal Statistical Society B, 63(3):411–436, 2006.
A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete
data via the EM Algorithm. Journal of the Royal Statistical Society, Series
B, 39:2–38, 1977.
A. Doucet, N. de Freitas, and N. Gordon, editors. Sequential Monte Carlo Methods
in Practice. Statistics for Engineering and Information Science. Springer Verlag,
New York, 2001.
A. Doucet, S. J. Godsill, and C. P. Robert. Marginal maximum a posteriori estimation
using Markov chain Monte Carlo. Statistics and Computing, 12:77–84,
2002.
A. Doucet, M. Briers, and S. S´en´ecal. Efficient block sampling strategies for sequential
Monte Carlo methods. Journal of Computational and Graphical Statistics,
15(3):693–711, 2006.
M. D. Escobar and M. West. Bayesian density estimation and inference using
mixtures. Journal of the American Statistical Association, 90(430):577–588, June
1995.
C. Gaetan and J.-F. Yao. A multiple-imputation Metropolis version of the EM
algorithm. Biometrika, 90(3):643–654, 2003.
C.-R. Hwang. Laplace’s method revisited: Weak convergence of probability measures.
The Annals of Probability, 8(6):1177–1182, December 1980.
E. Jacquier, M. Johannes, and N. Polson. MCMC maximum likelihood for latent
state models. Journal of Econometrics, 137(2):615–640, April 2007.
A. M. Johansen. Some Non-Standard Sequential Monte Carlo Methods With Ap-
plications. Ph.D. thesis, University of Cambridge Department of Engineering,
2006.
J. S. Liu and R. Chen. Sequential Monte Carlo methods for dynamic systems.
Journal of the American Statistical Association, 93(443):1032–1044, September
1998.
P. M¨uller, B. Sans´o, and M. de Iorio. Optimum Bayesian design by inhomogeneous
Markov chain simulation. Journal of the American Statistical Association, 99:
788–798, 2004.
C. P. Robert and G. Casella. Monte Carlo Statistical Methods. Springer Verlag,
New York, second edition, 2004.
C. P. Robert and D. M. Titterington. Reparameterization strategies for hidden
Markov models and Bayesian approaches to maximum likelihood estimation. Sta-
tistics and Computing, 8:145–158, 1998.
K. Roeder. Density estimation with cofidence sets exemplified by superclusters
and voids in galaxies. Journal of the American Statistical Association, 85(411):
617–624, September 1990.
K. Roeder and L.Wasserman. Practical Bayesian density estimation using mixtures
of normals. Journal of the American Statistical Association, 92(439):894–902,
September 1997.