MDLdroidLite: a release-and-inhibit control approach to resource-efficient deep neural networks on mobile devices

Yu Zhang, Tao Gu, Xi Zhang

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

Abstract

Mobile Deep Learning (MDL) has emerged as a privacy-preserving learning paradigm for mobile devices. This paradigm offers unique features such as privacy preservation, continual learning and low-latency inference to the building of personal mobile sensing applications. However, squeezing Deep Learning to mobile devices is extremely challenging due to resource constraint. Traditional Deep Neural Networks (DNNs) are usually over-parametered, hence incurring huge resource overhead for on-device learning. In this paper, we present a novel on-device deep learning framework named MDLdroidLite that transforms traditional DNNs into resource-efficient model structures for on-device learning. To minimize resource overhead, we propose a novel Release-and-Inhibit Control (RIC) approach based on Model Predictive Control theory to efficiently grow DNNs from tiny to backbone. We also design a gate-based fast adaptation mechanism for channel-level knowledge transformation to quickly adapt new-born neurons with existing neurons, enabling safe parameter adaptation and fast convergence for on-device training. Our evaluations show that MDLdroidLite boosts on-device training on various PMS datasets with 28x to 50x less model parameters, 4x to 10x less floating number operations than the state-of-the-art model structures while keeping the same accuracy level.

Original languageEnglish
Title of host publicationSenSys 2020
Subtitle of host publicationProceedings of the 2020 18th ACM Conference on Embedded Networked Sensor Systems
Place of PublicationNew York, NY
PublisherAssociation for Computing Machinery (ACM)
Pages463-475
Number of pages13
ISBN (Electronic)9781450375900
DOIs
Publication statusPublished - Nov 2020
Event18th ACM Conference on Embedded Networked Sensor Systems, SenSys 2020 - Virtual, Online, Japan
Duration: 16 Nov 202019 Nov 2020

Publication series

NameSenSys 2020 - Proceedings of the 2020 18th ACM Conference on Embedded Networked Sensor Systems

Conference

Conference18th ACM Conference on Embedded Networked Sensor Systems, SenSys 2020
CountryJapan
CityVirtual, Online
Period16/11/2019/11/20

Keywords

  • deep neural networks
  • dynamic optimization control
  • mobile deep learning
  • resource constraint

Fingerprint

Dive into the research topics of 'MDLdroidLite: a release-and-inhibit control approach to resource-efficient deep neural networks on mobile devices'. Together they form a unique fingerprint.

Cite this