Unsupervised word segmentation in context

Gabriel Synnaeve; Isabelle Dautriche; Benjamin Borschinger; Mark Johnson; Emmanuel Dupoux

Unsupervised word segmentation in context

Gabriel Synnaeve, Isabelle Dautriche, Benjamin Borschinger, Mark Johnson, Emmanuel Dupoux

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

5 Citations (Scopus)

63 Downloads (Pure)

Abstract

This paper extends existing word segmentation models to take non-linguistic context into account. It improves the token F-score of a top performing segmentation models by 2.5% on a 27k utterances dataset. We posit that word segmentation is easier in-context because the learner is not trying to access irrelevant lexical items. We use topics from a Latent Dirichlet Allocation model as a proxy for "activities" contexts, to label the Providence corpus. We present Adaptor Grammar models that use these context labels, and we study their performance with and without context annotations at test time.

Original language	English
Title of host publication	COLING 2014 - 25th International Conference on Computational Linguistics, Proceedings of COLING 2014: Technical Papers
Place of Publication	Stroudsburg, PA
Publisher	Association for Computational Linguistics, ACL Anthology
Pages	2326-2334
Number of pages	9
ISBN (Electronic)	9781941643266
Publication status	Published - 2014
Event	25th International Conference on Computational Linguistics, COLING 2014 - Dublin, Ireland Duration: 23 Aug 2014 → 29 Aug 2014

Other

Other	25th International Conference on Computational Linguistics, COLING 2014
Country/Territory	Ireland
City	Dublin
Period	23/08/14 → 29/08/14

Bibliographical note

Copyright the Author(s) 2014. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

Access to Document

Publisher version (open access).pdfFinal published version, 247 KB

Cite this

@inproceedings{c40ee351bb0140bd8ad3270cce89fd94,

title = "Unsupervised word segmentation in context",

abstract = "This paper extends existing word segmentation models to take non-linguistic context into account. It improves the token F-score of a top performing segmentation models by 2.5\% on a 27k utterances dataset. We posit that word segmentation is easier in-context because the learner is not trying to access irrelevant lexical items. We use topics from a Latent Dirichlet Allocation model as a proxy for {"}activities{"} contexts, to label the Providence corpus. We present Adaptor Grammar models that use these context labels, and we study their performance with and without context annotations at test time.",

author = "Gabriel Synnaeve and Isabelle Dautriche and Benjamin Borschinger and Mark Johnson and Emmanuel Dupoux",

note = "Copyright the Author(s) 2014. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.; 25th International Conference on Computational Linguistics, COLING 2014 ; Conference date: 23-08-2014 Through 29-08-2014",

year = "2014",

language = "English",

pages = "2326--2334",

booktitle = "COLING 2014 - 25th International Conference on Computational Linguistics, Proceedings of COLING 2014: Technical Papers",

publisher = "Association for Computational Linguistics, ACL Anthology",

}

Synnaeve, G, Dautriche, I, Borschinger, B, Johnson, M & Dupoux, E 2014, Unsupervised word segmentation in context. in COLING 2014 - 25th International Conference on Computational Linguistics, Proceedings of COLING 2014: Technical Papers. Association for Computational Linguistics, ACL Anthology, Stroudsburg, PA, pp. 2326-2334, 25th International Conference on Computational Linguistics, COLING 2014, Dublin, Ireland, 23/08/14.

Unsupervised word segmentation in context. / Synnaeve, Gabriel; Dautriche, Isabelle; Borschinger, Benjamin et al.
COLING 2014 - 25th International Conference on Computational Linguistics, Proceedings of COLING 2014: Technical Papers. Stroudsburg, PA: Association for Computational Linguistics, ACL Anthology, 2014. p. 2326-2334.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Unsupervised word segmentation in context

AU - Synnaeve, Gabriel

AU - Dautriche, Isabelle

AU - Borschinger, Benjamin

AU - Johnson, Mark

AU - Dupoux, Emmanuel

N1 - Copyright the Author(s) 2014. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

PY - 2014

Y1 - 2014

N2 - This paper extends existing word segmentation models to take non-linguistic context into account. It improves the token F-score of a top performing segmentation models by 2.5% on a 27k utterances dataset. We posit that word segmentation is easier in-context because the learner is not trying to access irrelevant lexical items. We use topics from a Latent Dirichlet Allocation model as a proxy for "activities" contexts, to label the Providence corpus. We present Adaptor Grammar models that use these context labels, and we study their performance with and without context annotations at test time.

AB - This paper extends existing word segmentation models to take non-linguistic context into account. It improves the token F-score of a top performing segmentation models by 2.5% on a 27k utterances dataset. We posit that word segmentation is easier in-context because the learner is not trying to access irrelevant lexical items. We use topics from a Latent Dirichlet Allocation model as a proxy for "activities" contexts, to label the Providence corpus. We present Adaptor Grammar models that use these context labels, and we study their performance with and without context annotations at test time.

UR - https://www.scopus.com/pages/publications/84959918809

M3 - Conference proceeding contribution

AN - SCOPUS:84959918809

SP - 2326

EP - 2334

BT - COLING 2014 - 25th International Conference on Computational Linguistics, Proceedings of COLING 2014: Technical Papers

PB - Association for Computational Linguistics, ACL Anthology

CY - Stroudsburg, PA

T2 - 25th International Conference on Computational Linguistics, COLING 2014

Y2 - 23 August 2014 through 29 August 2014

ER -

Unsupervised word segmentation in context

Abstract

Other

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this