A variational bayes approach to variable selection

John T. Ormerod, Chong You, Samuel Müller

Research output: Contribution to journalArticlepeer-review

34 Citations (Scopus)
43 Downloads (Pure)

Abstract

We develop methodology and theory for a mean field variational Bayes approximation to a linear model with a spike and slab prior on the regression coefficients. In particular we show how our method forces a subset of regression coefficients to be numerically indistinguishable from zero; under mild regularity conditions estimators based on our method consistently estimate the model parameters with easily obtainable and (asymptotically) appropriately sized standard error estimates; and select the true model at an exponential rate in the sample size. We also develop a practical method for simultaneously choosing reasonable initial parameter values and tuning the main tuning parameter of our algorithms which is both computationally efficient and empirically performs as well or better than some popular variable selection approaches. Our method is also faster and highly accurate when compared to MCMC.

Original languageEnglish
Pages (from-to)3549-3594
Number of pages46
JournalElectronic Journal of Statistics
Volume11
Issue number2
DOIs
Publication statusPublished - 2017
Externally publishedYes

Bibliographical note

Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

Keywords

  • Mean field variational Bayes
  • Bernoulli-Gaussian model
  • Markov Chain Monte Carlo

Fingerprint

Dive into the research topics of 'A variational bayes approach to variable selection'. Together they form a unique fingerprint.

Cite this