We propose a novel approach to Vietnamese word segmentation. Our approach is based on the Single Classification Ripple Down Rules methodology (Compton and Jansen, 1990), where rules are stored in an exception structure and new rules are only added to correct segmentation errors given by existing rules. Experimental results on the benchmark Vietnamese treebank show that our approach outperforms previous state-of-the-art approaches JVnSegmenter, vnTokenizer, DongDu and UETsegmenter in terms of both accuracy and performance speed. Our code is open-source and available at: https://github.com/datquocnguyen/RDRsegmenter.
|Title of host publication||LREC 2018 - 11th International Conference on Language Resources and Evaluation|
|Editors||Nicoletta Calzolari, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Koiti Hasida, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis, Takenobu Tokunaga|
|Place of Publication||Luxemburg|
|Publisher||European Language Resources Association (ELRA)|
|Number of pages||6|
|Publication status||Published - 1 Jan 2019|
|Event||11th International Conference on Language Resources and Evaluation, LREC 2018 - Miyazaki, Japan|
Duration: 7 May 2018 → 12 May 2018
|Conference||11th International Conference on Language Resources and Evaluation, LREC 2018|
|Period||7/05/18 → 12/05/18|
Bibliographical noteVersion archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.
- Single classification ripple down rules
- Word segmentation