Sequential Latent Dirichlet Allocation

Discover underlying topic structures within a document

Lan Du*, Wray Buntine, Huidong Jinr

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contribution

33 Citations (Scopus)

Abstract

Understanding how topics within a document evolve over its structure is an interesting and important problem. In this paper, we address this problem by presenting a novel variant of Latent Dirichlet Allocation (LDA): Sequential LDA (SeqLDA). This variant directly considers the underlying sequential structure, i.e., a document consists of multiple segments (e.g., chapters, paragraphs), each of which is correlated to its previous and subsequent segments. In our model, a document and its segments are modelled as random mixtures of the same set of latent topics, each of which is a distribution over words; and the topic distribution of each segment depends on that of its previous segment, the one for first segment will depend on the document topic distribution. The progressive dependency is captured by using the nested two-parameter Poisson Dirichlet process (PDP). We develop an efficient collapsed Gibbs sampling algorithm to sample from the posterior of the PDP. Our experimental results on patent documents show that by taking into account the sequential structure within a document, our SeqLDA model has a higher fidelity over LDA in terms of perplexity (a standard measure of dictionary-based compressibility). The SeqLDA model also yields a nicer sequential topic structure than LDA, as we show in experiments on books such as Melville's "The Whale".

Original languageEnglish
Title of host publicationProceedings - 10th IEEE International Conference on Data Mining, ICDM 2010
Pages148-157
Number of pages10
DOIs
Publication statusPublished - 2010
Event10th IEEE International Conference on Data Mining, ICDM 2010 - Sydney, NSW, Australia
Duration: 14 Dec 201017 Dec 2010

Other

Other10th IEEE International Conference on Data Mining, ICDM 2010
CountryAustralia
CitySydney, NSW
Period14/12/1017/12/10

Keywords

  • Collapsed Gibbs sampler
  • Document structure
  • Latent Dirichlet Allocation
  • Poisson-Dirichlet process

Fingerprint Dive into the research topics of 'Sequential Latent Dirichlet Allocation: Discover underlying topic structures within a document'. Together they form a unique fingerprint.

Cite this