Unsupervised pre-training with sequence reconstruction loss for deep relation extraction models

Zhuang Li; Lizhen Qu; Qiongkai Xu; Mark Johnson

Unsupervised pre-training with sequence reconstruction loss for deep relation extraction models

Zhuang Li, Lizhen Qu, Qiongkai Xu, Mark Johnson

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

10 Citations (Scopus)

Abstract

Relation extraction models based on deep learning have been attracting a lot of attention recently. Little research is carried out to reduce their need of labeled training data. In this work, we propose an unsupervised pre-training method based on the sequence-to-sequence model for deep relation extraction models. The pre-trained models need only half or even less training data to achieve equivalent performance as the same models without pre-training.

Original language	English
Title of host publication	Proceedings of the Australasian Language Technology Association Workshop 2016
Editors	Trevor Cohn
Place of Publication	Melbourne
Publisher	Australasian Language Technology Association
Pages	54-64
Number of pages	11
Publication status	Published - 2016
Event	2016 Australasian Language Technology Association Workshop, ALTA 2016 - Caulfield, Australia Duration: 5 Dec 2016 → 7 Dec 2016

Conference

Conference	2016 Australasian Language Technology Association Workshop, ALTA 2016
Country/Territory	Australia
City	Caulfield
Period	5/12/16 → 7/12/16

Access to Document

https://aclanthology.org/U16-1006Licence: Unspecified

Cite this

@inproceedings{8b71453a8cdd4de0a48fe4da44dddebc,

title = "Unsupervised pre-training with sequence reconstruction loss for deep relation extraction models",

abstract = "Relation extraction models based on deep learning have been attracting a lot of attention recently. Little research is carried out to reduce their need of labeled training data. In this work, we propose an unsupervised pre-training method based on the sequence-to-sequence model for deep relation extraction models. The pre-trained models need only half or even less training data to achieve equivalent performance as the same models without pre-training.",

author = "Zhuang Li and Lizhen Qu and Qiongkai Xu and Mark Johnson",

year = "2016",

language = "English",

pages = "54--64",

editor = "Trevor Cohn",

booktitle = "Proceedings of the Australasian Language Technology Association Workshop 2016",

publisher = "Australasian Language Technology Association",

note = "2016 Australasian Language Technology Association Workshop, ALTA 2016 ; Conference date: 05-12-2016 Through 07-12-2016",

}

Li, Z, Qu, L, Xu, Q & Johnson, M 2016, Unsupervised pre-training with sequence reconstruction loss for deep relation extraction models. in T Cohn (ed.), Proceedings of the Australasian Language Technology Association Workshop 2016. Australasian Language Technology Association, Melbourne, pp. 54-64, 2016 Australasian Language Technology Association Workshop, ALTA 2016, Caulfield, Australia, 5/12/16. <https://aclanthology.org/U16-1006>

Unsupervised pre-training with sequence reconstruction loss for deep relation extraction models. / Li, Zhuang; Qu, Lizhen; Xu, Qiongkai et al.
Proceedings of the Australasian Language Technology Association Workshop 2016. ed. / Trevor Cohn. Melbourne: Australasian Language Technology Association, 2016. p. 54-64.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Unsupervised pre-training with sequence reconstruction loss for deep relation extraction models

AU - Li, Zhuang

AU - Qu, Lizhen

AU - Xu, Qiongkai

AU - Johnson, Mark

PY - 2016

Y1 - 2016

N2 - Relation extraction models based on deep learning have been attracting a lot of attention recently. Little research is carried out to reduce their need of labeled training data. In this work, we propose an unsupervised pre-training method based on the sequence-to-sequence model for deep relation extraction models. The pre-trained models need only half or even less training data to achieve equivalent performance as the same models without pre-training.

AB - Relation extraction models based on deep learning have been attracting a lot of attention recently. Little research is carried out to reduce their need of labeled training data. In this work, we propose an unsupervised pre-training method based on the sequence-to-sequence model for deep relation extraction models. The pre-trained models need only half or even less training data to achieve equivalent performance as the same models without pre-training.

UR - http://www.scopus.com/inward/record.url?scp=85050617672&partnerID=8YFLogxK

M3 - Conference proceeding contribution

SP - 54

EP - 64

BT - Proceedings of the Australasian Language Technology Association Workshop 2016

A2 - Cohn, Trevor

PB - Australasian Language Technology Association

CY - Melbourne

T2 - 2016 Australasian Language Technology Association Workshop, ALTA 2016

Y2 - 5 December 2016 through 7 December 2016

ER -

Unsupervised pre-training with sequence reconstruction loss for deep relation extraction models

Abstract

Conference

Access to Document

Other files and links

Fingerprint

Cite this