GazeRevealer: inferring password using smartphone front camera

Yao Wang, Wandong Cai, Tao Gu, Wei Shao, Ibrahim Khalil, Xianghua Xu

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

Abstract

The widespread use of smartphones has brought great convenience to our daily lives, while at the same time we have been increasingly exposed to security threats. Keystroke security is an essential element in user privacy protection. In this paper,we present GazeRevealer, a novel side-channel based keystroke inference framework to infer sensitive inputs on smartphone from video recordings of victim’s eye patterns captured from smartphone front camera. We observe that eye movements typically follow the keystrokes typing on the number-only soft keyboard during password input. By exploiting eye patterns, we are able to infer the passwords being entered. We propose a novel algorithm to extract sensitive eye pattern images from video streams, and classify different eye patterns with Support Vector Classification. We also propose a novel enhanced method to boost the inference accuracy. Compared with prior keystroke detection approaches, GazeRevealer does not require any external auxiliary devices, and it relies only on smartphone front camera. We evaluate the performance of GazeRevealer with three different types of smartphones, and the result shows that GazeRevealer achieves 77.43% detection accuracy for a single key number and 83.33% inference rate for the 6-digit password in the ideal case.

Original languageEnglish
Title of host publicationProceedings of the 15th EAI International Conference on Mobile and Ubiquitous Systems
Subtitle of host publicationComputing, Networking and Services, Mobiquitous 2018
Place of PublicationNew York Ny
PublisherAssociation for Computing Machinery (ACM)
Pages254-263
Number of pages10
ISBN (Electronic)9781450360937
DOIs
Publication statusPublished - 2018
Externally publishedYes
Event15th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, Mobiquitous 2018 - New York, United States
Duration: 5 Nov 20187 Nov 2018

Publication series

NameACM International Conference Proceeding Series

Conference

Conference15th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, Mobiquitous 2018
CountryUnited States
CityNew York
Period5/11/187/11/18

Keywords

  • Gaze estimation
  • Mobile security
  • Password inference

Fingerprint

Dive into the research topics of 'GazeRevealer: inferring password using smartphone front camera'. Together they form a unique fingerprint.

Cite this