Projects per year
Abstract
Currently, multimodal sentiment analysis (MSA) for personalized users under uncertain modalities missing has become a new challenging problem. To address this issue, we propose a two-step idea. First, we propose an effective MSA model under uncertain modalities missing and train it with some public datasets, thus to enable the model to possess better preliminary MSA ability. Then, we make the pretrained model to continuously learn user's personalized characteristics with online learning methods, thereby enable the model grow into a robust model for personalized MSA. Based on this idea, we propose a Personalized MSA model under uncertain modalities missing via Pretraining and Online Learning (termed as PMSAPO). For Personalized MSA under uncertain modalities missing, PMSAPO firstly generates the fused modality and allocate weights for each modality with a Fully Connected Neural Network Evaluation Module. Then, PMSAPO completes the final sentiment classification based on the fusion modality with a Joint feature optimization module. For the pretrained PMSAPO, we make it autonomously learn the personalized users via our proposed online learning techniques, including an online meta-learning method, a learning rate adaptive adjustment strategy, and a dynamic weight assignment strategy for sample data. Finally, based on three public benchmark datasets (IEMOCAP, MELD and CMU-MOSI), we conduct extensive experiments and prove that PMSAPO completely outperforms the Twelve state-of-the-art baseline models. (Code is available at https://github.com/SHX-AI/PMSAPO.)
| Original language | English |
|---|---|
| Article number | 114287 |
| Pages (from-to) | 1-19 |
| Number of pages | 19 |
| Journal | Knowledge-Based Systems |
| Volume | 329 |
| Issue number | Part A |
| DOIs | |
| Publication status | Published - 4 Nov 2025 |
Keywords
- Multimodal sentiment analysis
- Uncertain modalities missing
- Pretraining
- Online learning
Fingerprint
Dive into the research topics of 'Personalized multimodal sentiment analysis under uncertain modalities missing via pretraining and online learning'. Together they form a unique fingerprint.-
Efficient Management of Things for the Future World Wide Web
Sheng, M. (Primary Chief Investigator) & Mans, B. (Other)
1/01/17 → …
Project: Research
-
What Can You Trust in the Large and Noisy Web?
Sheng, M. (Primary Chief Investigator), Yang, J. (Chief Investigator), Zhang, W. (Chief Investigator) & Dustdar, S. (Partner Investigator)
1/05/20 → 30/04/23
Project: Research
Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver