Interpolating between types and tokens by estimating power-law generators

Sharon Goldwater; Thomas L. Griffiths; Mark Johnson

Interpolating between types and tokens by estimating power-law generators

Sharon Goldwater^*, Thomas L. Griffiths, Mark Johnson

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

87 Citations (Scopus)

Abstract

Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statistical models that generically produce power-laws, augmenting standard generative models with an adaptor that produces the appropriate pattern of token frequencies. We show that taking a particular stochastic process-the Pitman-Yor process-as an adaptor justifies the appearance of type frequencies in formal analyses of natural language, and improves the performance of a model for unsupervised learning of morphology.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 18 - Proceedings of the 2005 Conference
Editors	Yair Weiss, Bernhard Schalkopf, John Platt
Place of Publication	Cambridge, MA
Publisher	MIT Press
Pages	459-466
Number of pages	8
ISBN (Print)	9780262232531
Publication status	Published - 2005
Externally published	Yes
Event	2005 Annual Conference on Neural Information Processing Systems, NIPS - 2005 - Vancouver, Canada Duration: 5 Dec 2005 → 8 Dec 2005

Other

Other	2005 Annual Conference on Neural Information Processing Systems, NIPS - 2005
Country/Territory	Canada
City	Vancouver
Period	5/12/05 → 8/12/05

Access to Document

http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.129.7005

Cite this

@inproceedings{29bc8a00cef0420bbab29098991e88d5,

title = "Interpolating between types and tokens by estimating power-law generators",

abstract = "Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statistical models that generically produce power-laws, augmenting standard generative models with an adaptor that produces the appropriate pattern of token frequencies. We show that taking a particular stochastic process-the Pitman-Yor process-as an adaptor justifies the appearance of type frequencies in formal analyses of natural language, and improves the performance of a model for unsupervised learning of morphology.",

author = "Sharon Goldwater and Griffiths, {Thomas L.} and Mark Johnson",

year = "2005",

language = "English",

isbn = "9780262232531",

pages = "459--466",

editor = "Yair Weiss and Bernhard Schalkopf and John Platt",

booktitle = "Advances in Neural Information Processing Systems 18 - Proceedings of the 2005 Conference",

publisher = "MIT Press",

note = "2005 Annual Conference on Neural Information Processing Systems, NIPS - 2005 ; Conference date: 05-12-2005 Through 08-12-2005",

}

Goldwater, S, Griffiths, TL & Johnson, M 2005, Interpolating between types and tokens by estimating power-law generators. in Y Weiss, B Schalkopf & J Platt (eds), Advances in Neural Information Processing Systems 18 - Proceedings of the 2005 Conference. MIT Press, Cambridge, MA, pp. 459-466, 2005 Annual Conference on Neural Information Processing Systems, NIPS - 2005, Vancouver, Canada, 5/12/05. <http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.129.7005>

Interpolating between types and tokens by estimating power-law generators. / Goldwater, Sharon; Griffiths, Thomas L.; Johnson, Mark.
Advances in Neural Information Processing Systems 18 - Proceedings of the 2005 Conference. ed. / Yair Weiss; Bernhard Schalkopf; John Platt. Cambridge, MA: MIT Press, 2005. p. 459-466.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Interpolating between types and tokens by estimating power-law generators

AU - Goldwater, Sharon

AU - Griffiths, Thomas L.

AU - Johnson, Mark

PY - 2005

Y1 - 2005

N2 - Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statistical models that generically produce power-laws, augmenting standard generative models with an adaptor that produces the appropriate pattern of token frequencies. We show that taking a particular stochastic process-the Pitman-Yor process-as an adaptor justifies the appearance of type frequencies in formal analyses of natural language, and improves the performance of a model for unsupervised learning of morphology.

AB - Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statistical models that generically produce power-laws, augmenting standard generative models with an adaptor that produces the appropriate pattern of token frequencies. We show that taking a particular stochastic process-the Pitman-Yor process-as an adaptor justifies the appearance of type frequencies in formal analyses of natural language, and improves the performance of a model for unsupervised learning of morphology.

UR - http://www.scopus.com/inward/record.url?scp=33749240868&partnerID=8YFLogxK

M3 - Conference proceeding contribution

AN - SCOPUS:33749240868

SN - 9780262232531

SP - 459

EP - 466

BT - Advances in Neural Information Processing Systems 18 - Proceedings of the 2005 Conference

A2 - Weiss, Yair

A2 - Schalkopf, Bernhard

A2 - Platt, John

PB - MIT Press

CY - Cambridge, MA

T2 - 2005 Annual Conference on Neural Information Processing Systems, NIPS - 2005

Y2 - 5 December 2005 through 8 December 2005

ER -

Interpolating between types and tokens by estimating power-law generators

Abstract

Other

Access to Document

Other files and links

Fingerprint

Cite this