Extended Multimodal Hate Speech Event Detection during Russia-Ukraine crisis - shared task at CASE 2024

Surendrabikram Thapa; Kritesh Rauniyar; Farhan Ahmad Jafri; Hariram Veeramani; Raghav Jain; Sandesh Jain; Francielle Vargas; Ali Hürriyetoğlu; Usman Naseem

Extended Multimodal Hate Speech Event Detection during Russia-Ukraine crisis - shared task at CASE 2024

Surendrabikram Thapa, Kritesh Rauniyar, Farhan Ahmad Jafri, Hariram Veeramani, Raghav Jain, Sandesh Jain, Francielle Vargas, Ali Hürriyetoğlu, Usman Naseem

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

6 Citations (Scopus)

66 Downloads (Pure)

Abstract

Addressing the need for effective hate speech moderation in contemporary digital discourse, the Multimodal Hate Speech Event Detection Shared Task made its debut at CASE 2023, co-located with RANLP 2023. Building upon its success, an extended version of the shared task was organized at the CASE workshop in EACL 2024. Similar to the earlier iteration, in this shared task, participants address hate speech detection through two subtasks. Subtask A is a binary classification problem, assessing whether text-embedded images contain hate speech. Subtask B goes further, demanding the identification of hate speech targets, such as individuals, communities, and organizations within text-embedded images. Performance is evaluated using the macro F1-score metric in both subtasks. With a total of 73 registered participants, the shared task witnessed remarkable achievements, with the best F1-scores in Subtask A and Subtask B reaching 87.27% and 80.05%, respectively, surpassing the leaderboard of the previous CASE 2023 shared task. This paper provides a comprehensive overview of the performance of seven teams that submitted results for Subtask A and five teams for Subtask B.

Original language	English
Title of host publication	Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024)
Place of Publication	Stroudsburg, PA
Publisher	Association for Computational Linguistics
Pages	221-228
Number of pages	8
ISBN (Electronic)	9798891760707
Publication status	Published - 2024
Event	7th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events from Text, CASE 2024 - St. Julian's, Malta Duration: 22 Mar 2024 → 22 Mar 2024

Conference

Conference	7th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events from Text, CASE 2024
Country/Territory	Malta
City	St. Julian's
Period	22/03/24 → 22/03/24

Bibliographical note

Copyright the Publisher 2024. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

Access to Document

Publisher version (open access)Final published version, 114 KBLicence: CC BY

https://aclanthology.org/2024.case-1.31Licence: CC BY

Cite this

Thapa, S., Rauniyar, K., Jafri, F. A., Veeramani, H., Jain, R., Jain, S., Vargas, F., Hürriyetoğlu, A., & Naseem, U. (2024). Extended Multimodal Hate Speech Event Detection during Russia-Ukraine crisis - shared task at CASE 2024. In Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024) (pp. 221-228). Association for Computational Linguistics. https://aclanthology.org/2024.case-1.31

Thapa, Surendrabikram ; Rauniyar, Kritesh ; Jafri, Farhan Ahmad et al. / Extended Multimodal Hate Speech Event Detection during Russia-Ukraine crisis - shared task at CASE 2024. Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024). Stroudsburg, PA : Association for Computational Linguistics, 2024. pp. 221-228

@inproceedings{18e6a0f7036a4e808342830f88749702,

title = "Extended Multimodal Hate Speech Event Detection during Russia-Ukraine crisis - shared task at CASE 2024",

abstract = "Addressing the need for effective hate speech moderation in contemporary digital discourse, the Multimodal Hate Speech Event Detection Shared Task made its debut at CASE 2023, co-located with RANLP 2023. Building upon its success, an extended version of the shared task was organized at the CASE workshop in EACL 2024. Similar to the earlier iteration, in this shared task, participants address hate speech detection through two subtasks. Subtask A is a binary classification problem, assessing whether text-embedded images contain hate speech. Subtask B goes further, demanding the identification of hate speech targets, such as individuals, communities, and organizations within text-embedded images. Performance is evaluated using the macro F1-score metric in both subtasks. With a total of 73 registered participants, the shared task witnessed remarkable achievements, with the best F1-scores in Subtask A and Subtask B reaching 87.27% and 80.05%, respectively, surpassing the leaderboard of the previous CASE 2023 shared task. This paper provides a comprehensive overview of the performance of seven teams that submitted results for Subtask A and five teams for Subtask B.",

author = "Surendrabikram Thapa and Kritesh Rauniyar and Jafri, {Farhan Ahmad} and Hariram Veeramani and Raghav Jain and Sandesh Jain and Francielle Vargas and Ali H{\"u}rriyetoğlu and Usman Naseem",

note = "Copyright the Publisher 2024. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.; 7th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events from Text, CASE 2024 ; Conference date: 22-03-2024 Through 22-03-2024",

year = "2024",

language = "English",

pages = "221--228",

booktitle = "Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024)",

publisher = "Association for Computational Linguistics",

}

Thapa, S, Rauniyar, K, Jafri, FA, Veeramani, H, Jain, R, Jain, S, Vargas, F, Hürriyetoğlu, A & Naseem, U 2024, Extended Multimodal Hate Speech Event Detection during Russia-Ukraine crisis - shared task at CASE 2024. in Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024). Association for Computational Linguistics, Stroudsburg, PA, pp. 221-228, 7th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events from Text, CASE 2024, St. Julian's, Malta, 22/03/24. <https://aclanthology.org/2024.case-1.31>

Extended Multimodal Hate Speech Event Detection during Russia-Ukraine crisis - shared task at CASE 2024. / Thapa, Surendrabikram; Rauniyar, Kritesh; Jafri, Farhan Ahmad et al.
Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024). Stroudsburg, PA: Association for Computational Linguistics, 2024. p. 221-228.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Extended Multimodal Hate Speech Event Detection during Russia-Ukraine crisis - shared task at CASE 2024

AU - Thapa, Surendrabikram

AU - Rauniyar, Kritesh

AU - Jafri, Farhan Ahmad

AU - Veeramani, Hariram

AU - Jain, Raghav

AU - Jain, Sandesh

AU - Vargas, Francielle

AU - Hürriyetoğlu, Ali

AU - Naseem, Usman

N1 - Copyright the Publisher 2024. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

PY - 2024

Y1 - 2024

N2 - Addressing the need for effective hate speech moderation in contemporary digital discourse, the Multimodal Hate Speech Event Detection Shared Task made its debut at CASE 2023, co-located with RANLP 2023. Building upon its success, an extended version of the shared task was organized at the CASE workshop in EACL 2024. Similar to the earlier iteration, in this shared task, participants address hate speech detection through two subtasks. Subtask A is a binary classification problem, assessing whether text-embedded images contain hate speech. Subtask B goes further, demanding the identification of hate speech targets, such as individuals, communities, and organizations within text-embedded images. Performance is evaluated using the macro F1-score metric in both subtasks. With a total of 73 registered participants, the shared task witnessed remarkable achievements, with the best F1-scores in Subtask A and Subtask B reaching 87.27% and 80.05%, respectively, surpassing the leaderboard of the previous CASE 2023 shared task. This paper provides a comprehensive overview of the performance of seven teams that submitted results for Subtask A and five teams for Subtask B.

AB - Addressing the need for effective hate speech moderation in contemporary digital discourse, the Multimodal Hate Speech Event Detection Shared Task made its debut at CASE 2023, co-located with RANLP 2023. Building upon its success, an extended version of the shared task was organized at the CASE workshop in EACL 2024. Similar to the earlier iteration, in this shared task, participants address hate speech detection through two subtasks. Subtask A is a binary classification problem, assessing whether text-embedded images contain hate speech. Subtask B goes further, demanding the identification of hate speech targets, such as individuals, communities, and organizations within text-embedded images. Performance is evaluated using the macro F1-score metric in both subtasks. With a total of 73 registered participants, the shared task witnessed remarkable achievements, with the best F1-scores in Subtask A and Subtask B reaching 87.27% and 80.05%, respectively, surpassing the leaderboard of the previous CASE 2023 shared task. This paper provides a comprehensive overview of the performance of seven teams that submitted results for Subtask A and five teams for Subtask B.

UR - http://www.scopus.com/inward/record.url?scp=85190283204&partnerID=8YFLogxK

M3 - Conference proceeding contribution

AN - SCOPUS:85190283204

SP - 221

EP - 228

BT - Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024)

PB - Association for Computational Linguistics

CY - Stroudsburg, PA

T2 - 7th Workshop on Challenges and Applications of Automated Extraction of Socio-Political Events from Text, CASE 2024

Y2 - 22 March 2024 through 22 March 2024

ER -

Thapa S, Rauniyar K, Jafri FA, Veeramani H, Jain R, Jain S et al. Extended Multimodal Hate Speech Event Detection during Russia-Ukraine crisis - shared task at CASE 2024. In Proceedings of the 7th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2024). Stroudsburg, PA: Association for Computational Linguistics. 2024. p. 221-228

Extended Multimodal Hate Speech Event Detection during Russia-Ukraine crisis - shared task at CASE 2024

Abstract

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this