Decoupling sparsity and smoothness in Dirichlet Belief Networks

Yaqiong Li, Xuhui Fan*, Ling Chen, Bin Li, Scott A. Sisson

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

Abstract

The Dirichlet Belief Network (DirBN) has been proposed as a promising deep generative model that uses Dirichlet distributions to form layer-wise connections and thereby construct a multi-stochastic layered deep architecture. However, the DirBN cannot simultaneously achieve both sparsity, whereby the generated latent distributions place weights on a subset of components, and smoothness, which requires that the posterior distribution should not be dominated by the data. To address this limitation we introduce the sparse and smooth Dirichlet Belief Network (ssDirBN) which can achieve both sparsity and smoothness simultaneously, thereby increasing modelling flexibility over the DirBN. This gain is achieved by introducing binary variables to indicate whether each entity’s latent distribution at each layer uses a particular component. As a result, each latent distribution may use only a subset of components in each layer, and smoothness is enforced on this subset. Extra efforts on modifying the models are also made to fix the issues which is caused by introducing these binary variables. Extensive experimental results on real-world data show significant performance improvements of ssDirBN over state-of-the-art models in terms of both enhanced model predictions and reduced model complexity.

Original languageEnglish
Title of host publicationMachine learning and knowledge discovery in databases. Research track
Subtitle of host publicationEuropean Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, proceedings, part II
EditorsNuria Oliver, Fernando Pérez-Cruz, Stefan Kramer, Jesse Read, Jose A. Lozano
Place of PublicationCham
PublisherSpringer, Springer Nature
Pages148-163
Number of pages16
ISBN (Electronic)9783030865207
ISBN (Print)9783030865191
DOIs
Publication statusPublished - 2021
Externally publishedYes
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2021 - Virtual, Online
Duration: 13 Sept 202117 Sept 2021

Publication series

NameLecture Notes in Artificial Intelligence
PublisherSpringer
Volume12976
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2021
CityVirtual, Online
Period13/09/2117/09/21

Keywords

  • Dirichlet belief networks
  • Markov chain Monte Carlo
  • Sparsity

Fingerprint

Dive into the research topics of 'Decoupling sparsity and smoothness in Dirichlet Belief Networks'. Together they form a unique fingerprint.

Cite this