Representational bias in unsupervised learning of syllable structure

Sharon Goldwater*, Mark Johnson

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contribution

7 Citations (Scopus)


Unsupervised learning algorithms based on Expectation Maximization (EM) are often straightforward to implement and provably converge on a local likelihood maximum. However, these algorithms often do not perform well in practice. Common wisdom holds that they yield poor results because they are overly sensitive to initial parameter values and easily get stuck in local (but not global) maxima. We present a series of experiments indicating that for the task of learning syllable structure, the initial parameter weights are not crucial. Rather, it is the choice of model class itself that makes the difference between successful and unsuccessful learning. We use a language-universal rule-based algorithm to find a good set of parameters, and then train the parameter weights using EM. We achieve word accuracy of 95.9% on German and 97.1% on English, as compared to 97.4% and 98.1% respectively for supervised training.

Original languageEnglish
Title of host publicationProceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005)
Place of PublicationNew Brunswick, NJ
PublisherAssociation for Computational Linguistics (ACL)
Number of pages8
Publication statusPublished - Jun 2005
Externally publishedYes
Event9th Conference on Computational Natural Language Learning, CoNLL - 2005 - Ann Arbor, United States
Duration: 29 Jun 200530 Jun 2005


Other9th Conference on Computational Natural Language Learning, CoNLL - 2005
CountryUnited States
CityAnn Arbor

Fingerprint Dive into the research topics of 'Representational bias in unsupervised learning of syllable structure'. Together they form a unique fingerprint.

Cite this