Your eyes reveal your secrets: an eye movement based password inference on smartphone

Yao Wang, Wandong Cai, Tao Gu, Wei Shao

Research output: Contribution to journalArticlepeer-review

Abstract

The widespread use of smartphones has brought great convenience to our daily lives, while at the same time we have been increasingly exposed to security threats. Keystroke security is essential to user privacy protection. In this paper, we present GazeRevealer, a novel side-channel based keystroke inference framework to infer sensitive inputs on smartphone from video recordings of victim's eye patterns captured from smartphone front camera. We observe that eye movements typically follow the keystrokes typing on the number-only soft keyboard during password input. By exploiting eye movement patterns, we are able to infer the passwords being entered. We propose a novel algorithm to extract sensitive eye images from video streams, and classify these images with Support Vector Classification. We also propose a novel classification enhancement algorithm to further improve classification accuracy. Compared with prior keystroke detection approaches, GazeRevealer does not require any external auxiliary devices, and it only relies on smartphone front camera. We evaluate the performance of GazeRevealer on several smartphones under different real-life usage scenarios. The results show that GazeRevealer achieves an inference rate of 77.89 percent for single key number and an inference rate of 84.38 percent for 6-digit password in the ideal case.
Original languageEnglish
Pages (from-to)2714 - 2730
Number of pages17
JournalIEEE Transactions on Mobile Computing
Volume19
Issue number11
DOIs
Publication statusPublished - Nov 2019
Externally publishedYes

Keywords

  • Keystroke inference
  • gaze estimation
  • mobile security

Fingerprint Dive into the research topics of 'Your eyes reveal your secrets: an eye movement based password inference on smartphone'. Together they form a unique fingerprint.

Cite this