AIMSafe: EEG-based driver behavior understanding via attention and incremental learning mechanisms

Landu Jiang; Cheng Luo; Tao Gu; Kezhong Lu; Dian Zhang

doi:10.1007/978-3-031-63992-0_16

AIMSafe: EEG-based driver behavior understanding via attention and incremental learning mechanisms

Landu Jiang
, Cheng Luo
, Tao Gu
, Kezhong Lu
, Dian Zhang^*

^*Corresponding author for this work

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

Abstract

In this paper, we propose AIMSafe, an electroencephalographic (EEG) based system that studies driver in-vehicle behaviors leveraging Attention networks and Incremental learning Mechanism for road Safety. Instead of using predefined classes, we categorize driver in-vehicle activities into different risk levels - a stronger motion may have a higher chance of unsafe driving. More specifically, we first employ a CNN based model to distinguish two basic activities - 1. normal driving and 2. unsafe driving. Moreover, AIMSafe also leverages smartphone IMU sensors generating soft hints that helps automatically label EEG data on road. We then adopt class-incremental learning to rank other Out-of-Distribution (OOD) driver activities (safe to unsafe) based on the Mahalanobis distance. A modified Squeeze-and-Excitation (SE) block is also used to adaptively select effective EEG electrodes for improving the system efficiency. Evaluation results (involving 11 males and 4 females) show that AIMSafe could achieve a detection accuracy over 95% on unsafe driving activities using only 4 electrodes.

Original language	English
Title of host publication	Mobile and ubiquitous systems: computing, networking and services
Subtitle of host publication	20th EAI International Conference, MobiQuitous 2023, proceedings, part II
Editors	Arkady Zaslavsky, Zhaolong Ning, Vana Kalogeraki, Dimitrios Georgakopoulos, Panos K. Chrysanthis
Place of Publication	Switzerland
Publisher	Springer, Springer Nature
Pages	244-255
Number of pages	12
ISBN (Electronic)	9783031639920
ISBN (Print)	9783031639913
DOIs	https://doi.org/10.1007/978-3-031-63992-0_16
Publication status	Published - 2024
Event	EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (20th : 2023) - Melbourne, Australia Duration: 14 Nov 2023 → 17 Nov 2023

Publication series

Name	Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST
Volume	594 LNICST
ISSN (Print)	1867-8211
ISSN (Electronic)	1867-822X

Conference

Conference	EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (20th : 2023)
Abbreviated title	MobiQuitous 2023
Country/Territory	Australia
City	Melbourne
Period	14/11/23 → 17/11/23

Keywords

driver education
driving safety
EEG
smart computing
wearable sensing

Access to Document

10.1007/978-3-031-63992-0_16

Cite this

Jiang, L., Luo, C., Gu, T., Lu, K., & Zhang, D. (2024). AIMSafe: EEG-based driver behavior understanding via attention and incremental learning mechanisms. In A. Zaslavsky, Z. Ning, V. Kalogeraki, D. Georgakopoulos, & P. K. Chrysanthis (Eds.), Mobile and ubiquitous systems: computing, networking and services: 20th EAI International Conference, MobiQuitous 2023, proceedings, part II (pp. 244-255). (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST; Vol. 594 LNICST). Springer, Springer Nature. https://doi.org/10.1007/978-3-031-63992-0_16

Jiang, Landu ; Luo, Cheng ; Gu, Tao et al. / AIMSafe : EEG-based driver behavior understanding via attention and incremental learning mechanisms. Mobile and ubiquitous systems: computing, networking and services: 20th EAI International Conference, MobiQuitous 2023, proceedings, part II. editor / Arkady Zaslavsky ; Zhaolong Ning ; Vana Kalogeraki ; Dimitrios Georgakopoulos ; Panos K. Chrysanthis. Switzerland : Springer, Springer Nature, 2024. pp. 244-255 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST).

@inproceedings{f4bcf778fe734675a45bb4a3fece9ca8,

title = "AIMSafe: EEG-based driver behavior understanding via attention and incremental learning mechanisms",

abstract = "In this paper, we propose AIMSafe, an electroencephalographic (EEG) based system that studies driver in-vehicle behaviors leveraging Attention networks and Incremental learning Mechanism for road Safety. Instead of using predefined classes, we categorize driver in-vehicle activities into different risk levels - a stronger motion may have a higher chance of unsafe driving. More specifically, we first employ a CNN based model to distinguish two basic activities - 1. normal driving and 2. unsafe driving. Moreover, AIMSafe also leverages smartphone IMU sensors generating soft hints that helps automatically label EEG data on road. We then adopt class-incremental learning to rank other Out-of-Distribution (OOD) driver activities (safe to unsafe) based on the Mahalanobis distance. A modified Squeeze-and-Excitation (SE) block is also used to adaptively select effective EEG electrodes for improving the system efficiency. Evaluation results (involving 11 males and 4 females) show that AIMSafe could achieve a detection accuracy over 95\% on unsafe driving activities using only 4 electrodes.",

keywords = "driver education, driving safety, EEG, smart computing, wearable sensing",

author = "Landu Jiang and Cheng Luo and Tao Gu and Kezhong Lu and Dian Zhang",

year = "2024",

doi = "10.1007/978-3-031-63992-0\_16",

language = "English",

isbn = "9783031639913",

series = "Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST",

publisher = "Springer, Springer Nature",

pages = "244--255",

editor = "Arkady Zaslavsky and Zhaolong Ning and Vana Kalogeraki and Dimitrios Georgakopoulos and Chrysanthis, \{Panos K.\}",

booktitle = "Mobile and ubiquitous systems: computing, networking and services",

address = "United States",

note = "EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (20th : 2023), MobiQuitous 2023 ; Conference date: 14-11-2023 Through 17-11-2023",

}

Jiang, L, Luo, C, Gu, T, Lu, K & Zhang, D 2024, AIMSafe: EEG-based driver behavior understanding via attention and incremental learning mechanisms. in A Zaslavsky, Z Ning, V Kalogeraki, D Georgakopoulos & PK Chrysanthis (eds), Mobile and ubiquitous systems: computing, networking and services: 20th EAI International Conference, MobiQuitous 2023, proceedings, part II. Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, vol. 594 LNICST, Springer, Springer Nature, Switzerland, pp. 244-255, EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (20th : 2023), Melbourne, Victoria, Australia, 14/11/23. https://doi.org/10.1007/978-3-031-63992-0_16

AIMSafe: EEG-based driver behavior understanding via attention and incremental learning mechanisms. / Jiang, Landu; Luo, Cheng; Gu, Tao et al.
Mobile and ubiquitous systems: computing, networking and services: 20th EAI International Conference, MobiQuitous 2023, proceedings, part II. ed. / Arkady Zaslavsky; Zhaolong Ning; Vana Kalogeraki; Dimitrios Georgakopoulos; Panos K. Chrysanthis. Switzerland: Springer, Springer Nature, 2024. p. 244-255 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST; Vol. 594 LNICST).

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - AIMSafe

T2 - EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (20th : 2023)

AU - Jiang, Landu

AU - Luo, Cheng

AU - Gu, Tao

AU - Lu, Kezhong

AU - Zhang, Dian

PY - 2024

Y1 - 2024

N2 - In this paper, we propose AIMSafe, an electroencephalographic (EEG) based system that studies driver in-vehicle behaviors leveraging Attention networks and Incremental learning Mechanism for road Safety. Instead of using predefined classes, we categorize driver in-vehicle activities into different risk levels - a stronger motion may have a higher chance of unsafe driving. More specifically, we first employ a CNN based model to distinguish two basic activities - 1. normal driving and 2. unsafe driving. Moreover, AIMSafe also leverages smartphone IMU sensors generating soft hints that helps automatically label EEG data on road. We then adopt class-incremental learning to rank other Out-of-Distribution (OOD) driver activities (safe to unsafe) based on the Mahalanobis distance. A modified Squeeze-and-Excitation (SE) block is also used to adaptively select effective EEG electrodes for improving the system efficiency. Evaluation results (involving 11 males and 4 females) show that AIMSafe could achieve a detection accuracy over 95% on unsafe driving activities using only 4 electrodes.

AB - In this paper, we propose AIMSafe, an electroencephalographic (EEG) based system that studies driver in-vehicle behaviors leveraging Attention networks and Incremental learning Mechanism for road Safety. Instead of using predefined classes, we categorize driver in-vehicle activities into different risk levels - a stronger motion may have a higher chance of unsafe driving. More specifically, we first employ a CNN based model to distinguish two basic activities - 1. normal driving and 2. unsafe driving. Moreover, AIMSafe also leverages smartphone IMU sensors generating soft hints that helps automatically label EEG data on road. We then adopt class-incremental learning to rank other Out-of-Distribution (OOD) driver activities (safe to unsafe) based on the Mahalanobis distance. A modified Squeeze-and-Excitation (SE) block is also used to adaptively select effective EEG electrodes for improving the system efficiency. Evaluation results (involving 11 males and 4 females) show that AIMSafe could achieve a detection accuracy over 95% on unsafe driving activities using only 4 electrodes.

KW - driver education

KW - driving safety

KW - EEG

KW - smart computing

KW - wearable sensing

UR - https://www.scopus.com/pages/publications/85200493531

U2 - 10.1007/978-3-031-63992-0_16

DO - 10.1007/978-3-031-63992-0_16

M3 - Conference proceeding contribution

AN - SCOPUS:85200493531

SN - 9783031639913

T3 - Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST

SP - 244

EP - 255

BT - Mobile and ubiquitous systems: computing, networking and services

A2 - Zaslavsky, Arkady

A2 - Ning, Zhaolong

A2 - Kalogeraki, Vana

A2 - Georgakopoulos, Dimitrios

A2 - Chrysanthis, Panos K.

PB - Springer, Springer Nature

CY - Switzerland

Y2 - 14 November 2023 through 17 November 2023

ER -

Jiang L, Luo C, Gu T, Lu K, Zhang D. AIMSafe: EEG-based driver behavior understanding via attention and incremental learning mechanisms. In Zaslavsky A, Ning Z, Kalogeraki V, Georgakopoulos D, Chrysanthis PK, editors, Mobile and ubiquitous systems: computing, networking and services: 20th EAI International Conference, MobiQuitous 2023, proceedings, part II. Switzerland: Springer, Springer Nature. 2024. p. 244-255. (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST). doi: 10.1007/978-3-031-63992-0_16