Joint selection in mixed models using regularized PQL

Francis K. C. Hui*, Samuel Müller, A. H. Welsh

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

33 Citations (Scopus)


The application of generalized linear mixed models presents some major challenges for both estimation, due to the intractable marginal likelihood, and model selection, as we usually want to jointly select over both fixed and random effects. We propose to overcome these challenges by combining penalized quasi-likelihood (PQL) estimation with sparsity inducing penalties on the fixed and random coefficients. The resulting approach, referred to as regularized PQL, is a computationally efficient method for performing joint selection in mixed models. A key aspect of regularized PQL involves the use of a group based penalty for the random effects: sparsity is induced such that all the coefficients for a random effect are shrunk to zero simultaneously, which in turn leads to the random effect being removed from the model. Despite being a quasi-likelihood approach, we show that regularized PQL is selection consistent, that is, it asymptotically selects the true set of fixed and random effects, in the setting where the cluster size grows with the number of clusters. Furthermore, we propose an information criterion for choosing the single tuning parameter and show that it facilitates selection consistency. Simulations demonstrate regularized PQL outperforms several currently employed methods for joint selection even if the cluster size is small compared to the number of clusters, while also offering dramatic reductions in computation time. Supplementary materials for this article are available online.

Original languageEnglish
Pages (from-to)1323-1333
Number of pages11
JournalJournal of the American Statistical Association
Issue number519
Publication statusPublished - 2017
Externally publishedYes


  • Fixed effects
  • Generalized linear mixed models
  • Lasso
  • Penalized likelihood
  • Quasi-likelihood
  • Variable selection


Dive into the research topics of 'Joint selection in mixed models using regularized PQL'. Together they form a unique fingerprint.

Cite this