DuCL: dual-stage contrastive learning framework for Chinese semantic textual matching

Chinese semantic textual matching is a fundamental yet challenging task in natural language processing (NLP). How to accurately capture the features in a single piece of text and the interactive features between pieces of text is the core problem of the task. Although pretrained language models (PLMs) and contrastive learning (CL) have been applied to address the problem to some extent, the existing works usually just utilize contrastive learning to finetune the PLMs on one single perspective, such as the sentence or pair level, which neglects to capture the semantic features from the other perspective, leading to inefficient learning and suboptimal performance. To tackle the problem, we propose a novel dual-stage contrastive learning framework (DuCL) for Chinese semantic textual matching. Specifically, DuCL consists of two stages sequentially, i.e., CL on the sentence level and CL on the pair level, each of which is responsible to finetune PLMs from the corresponding perspective. Besides, DuCL introduces a block-enhanced interaction module to integrate token-level and block-level interactive features to generate a semantic matching representation for two pieces of text. Extensive experimental results on two real-world public datasets demonstrate that our method can achieve better performance than the representative and state-of-the-art methods.