Abstract
The creative nature of memes has made it possible for harmful content to spread quickly and widely on the internet. Harmful memes can range from spreading hate speech promoting violence, and causing emotional distress to individuals or communities. These memes are often designed to be misleading, manipulative, and controversial, making it challenging to detect and remove them from online platforms. Previous studies focused on how to fuse visual and language modalities to capture contextual information. However, meme analysis still severely suffers from data deficiency, resulting in insufficient learning of fusion modules. Further, using conventional pretrained encoders for text and images exhibits a greater semantic gap in feature spaces and leads to low performance. To address these gaps, this paper reformulates a harmful meme analysis as an auto-filling and presents a prompt-based approach to identify harmful memes. Specifically, we first transform multimodal data to a single (i.e., textual) modality by generating the captions and attributes of the visual data and then prepend the textual data in the prompt-based pre-trained language model. Experimental results on two benchmark harmful memes datasets demonstrate that our method outperformed state-of-the-art methods. We conclude with the transferability and robustness of our approach to identify creative harmful memes.
Original language | English |
---|---|
Title of host publication | The ACM Web Conference 2023 |
Subtitle of host publication | proceedings of the World Wide Web Conference WWW 2023 |
Place of Publication | New York |
Publisher | Association for Computing Machinery, Inc |
Pages | 3868-3872 |
Number of pages | 5 |
ISBN (Electronic) | 9781450394161 |
DOIs | |
Publication status | Published - 2023 |
Externally published | Yes |
Event | 2023 World Wide Web Conference, WWW 2023 - Austin, United States Duration: 30 Apr 2023 → 4 May 2023 |
Conference
Conference | 2023 World Wide Web Conference, WWW 2023 |
---|---|
Country/Territory | United States |
City | Austin |
Period | 30/04/23 → 4/05/23 |