Perceptual-centric image super-resolution using heterogeneous processors on mobile devices

Kai Huang; Xiangyu Yin; Tao Gu; Wei Gao

doi:10.1145/3636534.3690698

Perceptual-centric image super-resolution using heterogeneous processors on mobile devices

Kai Huang, Xiangyu Yin, Tao Gu, Wei Gao^*

^*Corresponding author for this work

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

5 Citations (Scopus)

119 Downloads (Pure)

Abstract

Image super-resolution (SR) is widely used on mobile devices to enhance user experience. However, neural networks used for SR are computationally expensive, posing challenges for mobile devices with limited computing power. A viable solution is to use heterogeneous processors on mobile devices, especially the specialized hardware AI accelerators, for SR computations, but the reduced arithmetic precision on AI accelerators can lead to degraded perceptual quality in upscaled images. To address this limitation, in this paper we present SR For Your Eyes (FYE-SR), a novel image SR technique that enhances the perceptual quality of upscaled images when using heterogeneous processors for SR computations. FYE-SR strategically splits the SR model and dispatches different layers to heterogeneous processors, to meet the time constraint of SR computations while minimizing the impact of AI accelerators on image quality. Experiment results show that FYE-SR outperforms the best baselines, improving perceptual image quality by up to 2×, or reducing SR computing latency by up to 5.6× with on-par image quality.

Original language	English
Title of host publication	ACM MobiCom 2024
Subtitle of host publication	proceedings of the 30th International Conference on Mobile Computing and Networking
Place of Publication	New York, NY
Publisher	Association for Computing Machinery (ACM)
Pages	1361-1376
Number of pages	16
ISBN (Electronic)	9798400704895
DOIs	https://doi.org/10.1145/3636534.3690698
Publication status	Published - 2024
Event	Annual International Conference on Mobile Computing and Networking (30th : 2024) - Washington, United States Duration: 18 Nov 2024 → 22 Nov 2024 Conference number: 30th

Conference

Conference	Annual International Conference on Mobile Computing and Networking (30th : 2024)
Abbreviated title	ACM MobiCom '24
Country/Territory	United States
City	Washington
Period	18/11/24 → 22/11/24

Bibliographical note

Copyright the Author(s) 2024. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.

Keywords

heterogeneous computing
image super-resolution
mobile devices
neural networks
perceptual quality

Access to Document

10.1145/3636534.3690698Licence: CC BY

Publisher version (open access)Final published version, 1.41 MBLicence: CC BY

Cite this

@inproceedings{2576be4a1bb8475796a28402bdbfd6a7,

title = "Perceptual-centric image super-resolution using heterogeneous processors on mobile devices",

abstract = "Image super-resolution (SR) is widely used on mobile devices to enhance user experience. However, neural networks used for SR are computationally expensive, posing challenges for mobile devices with limited computing power. A viable solution is to use heterogeneous processors on mobile devices, especially the specialized hardware AI accelerators, for SR computations, but the reduced arithmetic precision on AI accelerators can lead to degraded perceptual quality in upscaled images. To address this limitation, in this paper we present SR For Your Eyes (FYE-SR), a novel image SR technique that enhances the perceptual quality of upscaled images when using heterogeneous processors for SR computations. FYE-SR strategically splits the SR model and dispatches different layers to heterogeneous processors, to meet the time constraint of SR computations while minimizing the impact of AI accelerators on image quality. Experiment results show that FYE-SR outperforms the best baselines, improving perceptual image quality by up to 2×, or reducing SR computing latency by up to 5.6× with on-par image quality.",

keywords = "heterogeneous computing, image super-resolution, mobile devices, neural networks, perceptual quality",

author = "Kai Huang and Xiangyu Yin and Tao Gu and Wei Gao",

note = "Copyright the Author(s) 2024. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.; Annual International Conference on Mobile Computing and Networking (30th : 2024), ACM MobiCom '24 ; Conference date: 18-11-2024 Through 22-11-2024",

year = "2024",

doi = "10.1145/3636534.3690698",

language = "English",

pages = "1361--1376",

booktitle = "ACM MobiCom 2024",

publisher = "Association for Computing Machinery (ACM)",

address = "United States",

}

Huang, K, Yin, X, Gu, T & Gao, W 2024, Perceptual-centric image super-resolution using heterogeneous processors on mobile devices. in ACM MobiCom 2024: proceedings of the 30th International Conference on Mobile Computing and Networking. Association for Computing Machinery (ACM), New York, NY, pp. 1361-1376, Annual International Conference on Mobile Computing and Networking (30th : 2024), Washington, United States, 18/11/24. https://doi.org/10.1145/3636534.3690698

Perceptual-centric image super-resolution using heterogeneous processors on mobile devices. / Huang, Kai; Yin, Xiangyu; Gu, Tao et al.
ACM MobiCom 2024: proceedings of the 30th International Conference on Mobile Computing and Networking. New York, NY: Association for Computing Machinery (ACM), 2024. p. 1361-1376.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Perceptual-centric image super-resolution using heterogeneous processors on mobile devices

AU - Huang, Kai

AU - Yin, Xiangyu

AU - Gu, Tao

AU - Gao, Wei

N1 - Conference code: 30th

PY - 2024

Y1 - 2024

N2 - Image super-resolution (SR) is widely used on mobile devices to enhance user experience. However, neural networks used for SR are computationally expensive, posing challenges for mobile devices with limited computing power. A viable solution is to use heterogeneous processors on mobile devices, especially the specialized hardware AI accelerators, for SR computations, but the reduced arithmetic precision on AI accelerators can lead to degraded perceptual quality in upscaled images. To address this limitation, in this paper we present SR For Your Eyes (FYE-SR), a novel image SR technique that enhances the perceptual quality of upscaled images when using heterogeneous processors for SR computations. FYE-SR strategically splits the SR model and dispatches different layers to heterogeneous processors, to meet the time constraint of SR computations while minimizing the impact of AI accelerators on image quality. Experiment results show that FYE-SR outperforms the best baselines, improving perceptual image quality by up to 2×, or reducing SR computing latency by up to 5.6× with on-par image quality.

AB - Image super-resolution (SR) is widely used on mobile devices to enhance user experience. However, neural networks used for SR are computationally expensive, posing challenges for mobile devices with limited computing power. A viable solution is to use heterogeneous processors on mobile devices, especially the specialized hardware AI accelerators, for SR computations, but the reduced arithmetic precision on AI accelerators can lead to degraded perceptual quality in upscaled images. To address this limitation, in this paper we present SR For Your Eyes (FYE-SR), a novel image SR technique that enhances the perceptual quality of upscaled images when using heterogeneous processors for SR computations. FYE-SR strategically splits the SR model and dispatches different layers to heterogeneous processors, to meet the time constraint of SR computations while minimizing the impact of AI accelerators on image quality. Experiment results show that FYE-SR outperforms the best baselines, improving perceptual image quality by up to 2×, or reducing SR computing latency by up to 5.6× with on-par image quality.

KW - heterogeneous computing

KW - image super-resolution

KW - mobile devices

KW - neural networks

KW - perceptual quality

UR - https://www.scopus.com/pages/publications/105002379883

U2 - 10.1145/3636534.3690698

DO - 10.1145/3636534.3690698

M3 - Conference proceeding contribution

C2 - 40129861

AN - SCOPUS:105002379883

SP - 1361

EP - 1376

BT - ACM MobiCom 2024

PB - Association for Computing Machinery (ACM)

CY - New York, NY

T2 - Annual International Conference on Mobile Computing and Networking (30th : 2024)

Y2 - 18 November 2024 through 22 November 2024

ER -

Perceptual-centric image super-resolution using heterogeneous processors on mobile devices

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this