Decoupling sparsity and smoothness in Dirichlet Belief Networks

Yaqiong Li, Xuhui Fan*, Ling Chen, Bin Li, Scott A. Sisson

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review


The Dirichlet Belief Network (DirBN) has been proposed as a promising deep generative model that uses Dirichlet distributions to form layer-wise connections and thereby construct a multi-stochastic layered deep architecture. However, the DirBN cannot simultaneously achieve both sparsity, whereby the generated latent distributions place weights on a subset of components, and smoothness, which requires that the posterior distribution should not be dominated by the data. To address this limitation we introduce the sparse and smooth Dirichlet Belief Network (ssDirBN) which can achieve both sparsity and smoothness simultaneously, thereby increasing modelling flexibility over the DirBN. This gain is achieved by introducing binary variables to indicate whether each entity’s latent distribution at each layer uses a particular component. As a result, each latent distribution may use only a subset of components in each layer, and smoothness is enforced on this subset. Extra efforts on modifying the models are also made to fix the issues which is caused by introducing these binary variables. Extensive experimental results on real-world data show significant performance improvements of ssDirBN over state-of-the-art models in terms of both enhanced model predictions and reduced model complexity.

Original languageEnglish
Title of host publicationMachine learning and knowledge discovery in databases. Research track
Subtitle of host publicationEuropean Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, proceedings, part II
EditorsNuria Oliver, Fernando Pérez-Cruz, Stefan Kramer, Jesse Read, Jose A. Lozano
Place of PublicationCham
PublisherSpringer, Springer Nature
Number of pages16
ISBN (Electronic)9783030865207
ISBN (Print)9783030865191
Publication statusPublished - 2021
Externally publishedYes
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2021 - Virtual, Online
Duration: 13 Sept 202117 Sept 2021

Publication series

NameLecture Notes in Artificial Intelligence
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2021
CityVirtual, Online


  • Dirichlet belief networks
  • Markov chain Monte Carlo
  • Sparsity


Dive into the research topics of 'Decoupling sparsity and smoothness in Dirichlet Belief Networks'. Together they form a unique fingerprint.

Cite this