End-to-end speech recognition and disfluency removal

Paria Jamshid Lou; Mark Johnson

End-to-end speech recognition and disfluency removal

Paria Jamshid Lou, Mark Johnson

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

19 Citations (Scopus)

162 Downloads (Pure)

Abstract

Disfluency detection is usually an intermediate step between an automatic speech recognition (ASR) system and a downstream task. By contrast, this paper aims to investigate the task of end-to-end speech recognition and disfluency removal. We specifically explore whether it is possible to train an ASR model to directly map disfluent speech into fluent transcripts, without relying on a separate disfluency detection model. We show that end-to-end models do learn to directly generate fluent transcripts; however, their performance is slightly worse than a baseline pipeline approach consisting of an ASR system and a specialized disfluency detection model. We also propose two new metrics for evaluating integrated ASR and disfluency removal models. The findings of this paper can serve as a benchmark for further research on the task of end-to-end speech recognition and disfluency removal in the future.

Original language	English
Title of host publication	Findings of the Association for Computational Linguistics
Subtitle of host publication	Findings of ACL EMNLP 2020
Place of Publication	Stroudsburg, PA
Publisher	Association for Computational Linguistics (ACL)
Pages	2051-2061
Number of pages	11
ISBN (Electronic)	9781952148903
Publication status	Published - 2020
Event	Findings of the Association for Computational Linguistics, ACL 2020: EMNLP 2020 - Virtual, Online Duration: 16 Nov 2020 → 20 Nov 2020

Publication series

Name	Findings of the Association for Computational Linguistics Findings of ACL: EMNLP 2020

Conference

Conference	Findings of the Association for Computational Linguistics, ACL 2020: EMNLP 2020
City	Virtual, Online
Period	16/11/20 → 20/11/20

Bibliographical note

Copyright the Publisher 2021. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

Access to Document

Publisher version (open access)Final published version, 429 KBLicence: CC BY

https://aclanthology.org/2020.findings-emnlp.186

Cite this

Jamshid Lou, P., & Johnson, M. (2020). End-to-end speech recognition and disfluency removal. In Findings of the Association for Computational Linguistics : Findings of ACL EMNLP 2020 (pp. 2051-2061). (Findings of the Association for Computational Linguistics Findings of ACL: EMNLP 2020). Association for Computational Linguistics (ACL). https://aclanthology.org/2020.findings-emnlp.186

@inproceedings{b795636266154d6297489cd0cafef2c8,

title = "End-to-end speech recognition and disfluency removal",

abstract = "Disfluency detection is usually an intermediate step between an automatic speech recognition (ASR) system and a downstream task. By contrast, this paper aims to investigate the task of end-to-end speech recognition and disfluency removal. We specifically explore whether it is possible to train an ASR model to directly map disfluent speech into fluent transcripts, without relying on a separate disfluency detection model. We show that end-to-end models do learn to directly generate fluent transcripts; however, their performance is slightly worse than a baseline pipeline approach consisting of an ASR system and a specialized disfluency detection model. We also propose two new metrics for evaluating integrated ASR and disfluency removal models. The findings of this paper can serve as a benchmark for further research on the task of end-to-end speech recognition and disfluency removal in the future.",

author = "{Jamshid Lou}, Paria and Mark Johnson",

note = "Copyright the Publisher 2021. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.; Findings of the Association for Computational Linguistics, ACL 2020: EMNLP 2020 ; Conference date: 16-11-2020 Through 20-11-2020",

year = "2020",

language = "English",

series = "Findings of the Association for Computational Linguistics Findings of ACL: EMNLP 2020",

publisher = "Association for Computational Linguistics (ACL)",

pages = "2051--2061",

booktitle = "Findings of the Association for Computational Linguistics",

address = "United States",

}

Jamshid Lou, P & Johnson, M 2020, End-to-end speech recognition and disfluency removal. in Findings of the Association for Computational Linguistics : Findings of ACL EMNLP 2020. Findings of the Association for Computational Linguistics Findings of ACL: EMNLP 2020, Association for Computational Linguistics (ACL), Stroudsburg, PA, pp. 2051-2061, Findings of the Association for Computational Linguistics, ACL 2020: EMNLP 2020, Virtual, Online, 16/11/20. <https://aclanthology.org/2020.findings-emnlp.186>

End-to-end speech recognition and disfluency removal. / Jamshid Lou, Paria; Johnson, Mark.
Findings of the Association for Computational Linguistics : Findings of ACL EMNLP 2020. Stroudsburg, PA: Association for Computational Linguistics (ACL), 2020. p. 2051-2061 (Findings of the Association for Computational Linguistics Findings of ACL: EMNLP 2020).

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - End-to-end speech recognition and disfluency removal

AU - Jamshid Lou, Paria

AU - Johnson, Mark

N1 - Copyright the Publisher 2021. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

PY - 2020

Y1 - 2020

N2 - Disfluency detection is usually an intermediate step between an automatic speech recognition (ASR) system and a downstream task. By contrast, this paper aims to investigate the task of end-to-end speech recognition and disfluency removal. We specifically explore whether it is possible to train an ASR model to directly map disfluent speech into fluent transcripts, without relying on a separate disfluency detection model. We show that end-to-end models do learn to directly generate fluent transcripts; however, their performance is slightly worse than a baseline pipeline approach consisting of an ASR system and a specialized disfluency detection model. We also propose two new metrics for evaluating integrated ASR and disfluency removal models. The findings of this paper can serve as a benchmark for further research on the task of end-to-end speech recognition and disfluency removal in the future.

AB - Disfluency detection is usually an intermediate step between an automatic speech recognition (ASR) system and a downstream task. By contrast, this paper aims to investigate the task of end-to-end speech recognition and disfluency removal. We specifically explore whether it is possible to train an ASR model to directly map disfluent speech into fluent transcripts, without relying on a separate disfluency detection model. We show that end-to-end models do learn to directly generate fluent transcripts; however, their performance is slightly worse than a baseline pipeline approach consisting of an ASR system and a specialized disfluency detection model. We also propose two new metrics for evaluating integrated ASR and disfluency removal models. The findings of this paper can serve as a benchmark for further research on the task of end-to-end speech recognition and disfluency removal in the future.

UR - http://www.scopus.com/inward/record.url?scp=85107276520&partnerID=8YFLogxK

UR - http://purl.org/au-research/grants/arc/DP160102156

M3 - Conference proceeding contribution

AN - SCOPUS:85107276520

T3 - Findings of the Association for Computational Linguistics Findings of ACL: EMNLP 2020

SP - 2051

EP - 2061

BT - Findings of the Association for Computational Linguistics

PB - Association for Computational Linguistics (ACL)

CY - Stroudsburg, PA

T2 - Findings of the Association for Computational Linguistics, ACL 2020: EMNLP 2020

Y2 - 16 November 2020 through 20 November 2020

ER -

End-to-end speech recognition and disfluency removal

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Improved syntactic and semantic analysis for natural language processing

Cite this

End-to-end speech recognition and disfluency removal

Abstract

Publication series

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Projects

Improved syntactic and semantic analysis for natural language processing

Cite this