Fast and approximate exhaustive variable selection for generalised linear models with APES

Kevin Y. X. Wang, Garth Tarr, Jean Y. H. Yang, Samuel Mueller

Research output: Contribution to journalArticle

Abstract

We present APproximated Exhaustive Search (APES), which enables fast and approximated exhaustive variable selection in Generalised Linear Models (GLMs). While exhaustive variable selection remains as the gold standard in many model selection contexts, traditional exhaustive variable selection suffers from computational feasibility issues. More precisely, there is often a high cost associated with computing maximum likelihood estimates (MLE) for all subsets of GLMs. Efficient algorithms for exhaustive searches exist for linear models, most notably the leaps-and-bound algorithm and, more recently, the mixed integer optimisation (MIO) algorithm. The APES method learns from observational weights in a generalised linear regression super-model and reformulates the GLM problem as a linear regression problem. In this way, APES can approximate a true exhaustive search in the original GLM space. Where exhaustive variable selection is not computationally feasible, we propose a best-subset search, which also closely approximates a true exhaustive search. APES is made available in both as a standalone R package as well as part of the already existing mplot package.

Original languageEnglish
Pages (from-to)445-465
Number of pages21
JournalAustralian and New Zealand Journal of Statistics
Volume61
Issue number4
DOIs
Publication statusPublished - Dec 2019
Externally publishedYes

Keywords

  • best subset search
  • exhaustive search
  • generalised linear model
  • variable selection

Fingerprint Dive into the research topics of 'Fast and approximate exhaustive variable selection for generalised linear models with APES'. Together they form a unique fingerprint.

  • Cite this