Capturing and predicting user frustration to support a smart operating system

Sepideh Goodarzy, Eric Keller, Maziyar Nazari, Eric Rozner, Richard Han, Mark Dras, Young Choon Lee, Deborah Richards

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review


This paper presents an IRB-approved human study to capture data to build models for human frustration prediction of computer users. First, an application was developed that ran in the user's computer/laptop/VM with Linux 20.04. Then, the application collected a variety of data from their computers, including: mouse clicks, movements and scrolls; the pattern of keyboard keys clicks; user audio features; and head movements through the user video; System-wide information such as computation, memory usage, network bandwidth, and input/output bandwidth of the running applications in the computer and user frustrations. Finally, the application sent the data to the cloud. After two weeks of data collection, supervised and semi-supervised models were developed offline to predict user frustration with the computer using the collected data. A semi-supervised model using a generative adversarial network (GAN) resulted in the highest accuracy of 90%.

Original languageEnglish
Title of host publicationIUI '23 Companion
Subtitle of host publicationcompanion proceedings of the 28th International Conference on Intelligent User Interfaces
Place of PublicationNew York
PublisherAssociation for Computing Machinery (ACM)
Number of pages4
ISBN (Electronic)9798400701078
Publication statusPublished - 2023
EventInternational Conference on Intelligent User Interfaces (28th : 2023) - Sydney, Australia
Duration: 27 Mar 202331 Mar 2023


ConferenceInternational Conference on Intelligent User Interfaces (28th : 2023)


  • Computer-based User Frustration
  • Intelligent Operating System


Dive into the research topics of 'Capturing and predicting user frustration to support a smart operating system'. Together they form a unique fingerprint.

Cite this