An empirical study on model pruning and quantization

Yuzhe Tian; Tom H. Luan; Xi Zheng

doi:10.1007/978-3-031-40467-2_7

An empirical study on model pruning and quantization

Yuzhe Tian, Tom H. Luan, Xi Zheng^*

^*Corresponding author for this work

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

3 Citations (Scopus)

Abstract

In machine learning, model compression is vital for resource-constrained Internet of Things (IoT) devices, such as unmanned aerial vehicles (UAVs) and smart phones. Currently there are some state-of-the-art (SOTA) compression methods, but little study is conducted to evaluate these techniques across different models and datasets. In this paper, we present an in-depth study on two SOTA model compression methods, pruning and quantization. We apply these methods on AlexNet, ResNet18, VGG16BN and VGG19BN, with three well known datasets, Fashion-MNIST, CIFAR-10, and UCI-HAR. Through our study, we draw the conclusion that, applying pruning and retraining could keep the performance (average less than degrade) while reducing the model size (at compression rate) on spatial domain datasets (e.g. pictures); the performance on temporal domain datasets (e.g. motion sensors data) degrades more (average about degrade); the performance of quantization is related with the pruning rate and the network architecture. We also compare different clustering methods and reveal the impact on model accuracy and quantization ratio. Finally, we provide some interesting directions for future research.

Original language	English
Title of host publication	Broadband communications, networks, and systems
Subtitle of host publication	13th EAI International Conference, BROADNETS 2022, Virtual Event, March 12-13, 2023 proceedings
Editors	Wei Wang, Jun Wu
Place of Publication	Cham
Publisher	Springer, Springer Nature
Pages	111-125
Number of pages	15
ISBN (Electronic)	9783031404672
ISBN (Print)	9783031404665
DOIs	https://doi.org/10.1007/978-3-031-40467-2_7
Publication status	Published - 2023
Event	13th EAI International Conference on Broadband Communications, Networks, and Systems - Virtual, Online Duration: 12 Mar 2023 → 13 Mar 2023

Publication series

Name	Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
Publisher	Springer
Volume	511
ISSN (Print)	1867-8211
ISSN (Electronic)	1867-822X

Conference

Conference	13th EAI International Conference on Broadband Communications, Networks, and Systems
Abbreviated title	BROADNETS 2022
City	Virtual, Online
Period	12/03/23 → 13/03/23

Keywords

Model compression
Deep neural network
Edge computing

Access to Document

10.1007/978-3-031-40467-2_7

Cite this

Tian, Y., Luan, T. H., & Zheng, X. (2023). An empirical study on model pruning and quantization. In W. Wang, & J. Wu (Eds.), Broadband communications, networks, and systems: 13th EAI International Conference, BROADNETS 2022, Virtual Event, March 12-13, 2023 proceedings (pp. 111-125). (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering; Vol. 511). Springer, Springer Nature. https://doi.org/10.1007/978-3-031-40467-2_7

Tian, Yuzhe ; Luan, Tom H. ; Zheng, Xi. / An empirical study on model pruning and quantization. Broadband communications, networks, and systems: 13th EAI International Conference, BROADNETS 2022, Virtual Event, March 12-13, 2023 proceedings. editor / Wei Wang ; Jun Wu. Cham : Springer, Springer Nature, 2023. pp. 111-125 (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering).

@inproceedings{ac4c095267664802a8487907c76e9870,

title = "An empirical study on model pruning and quantization",

abstract = "In machine learning, model compression is vital for resource-constrained Internet of Things (IoT) devices, such as unmanned aerial vehicles (UAVs) and smart phones. Currently there are some state-of-the-art (SOTA) compression methods, but little study is conducted to evaluate these techniques across different models and datasets. In this paper, we present an in-depth study on two SOTA model compression methods, pruning and quantization. We apply these methods on AlexNet, ResNet18, VGG16BN and VGG19BN, with three well known datasets, Fashion-MNIST, CIFAR-10, and UCI-HAR. Through our study, we draw the conclusion that, applying pruning and retraining could keep the performance (average less than degrade) while reducing the model size (at compression rate) on spatial domain datasets (e.g. pictures); the performance on temporal domain datasets (e.g. motion sensors data) degrades more (average about degrade); the performance of quantization is related with the pruning rate and the network architecture. We also compare different clustering methods and reveal the impact on model accuracy and quantization ratio. Finally, we provide some interesting directions for future research.",

keywords = "Model compression, Deep neural network, Edge computing",

author = "Yuzhe Tian and Luan, \{Tom H.\} and Xi Zheng",

year = "2023",

doi = "10.1007/978-3-031-40467-2\_7",

language = "English",

isbn = "9783031404665",

series = "Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering",

publisher = "Springer, Springer Nature",

pages = "111--125",

editor = "Wei Wang and Jun Wu",

booktitle = "Broadband communications, networks, and systems",

address = "United States",

note = "13th EAI International Conference on Broadband Communications, Networks, and Systems, BROADNETS 2022 ; Conference date: 12-03-2023 Through 13-03-2023",

}

Tian, Y, Luan, TH & Zheng, X 2023, An empirical study on model pruning and quantization. in W Wang & J Wu (eds), Broadband communications, networks, and systems: 13th EAI International Conference, BROADNETS 2022, Virtual Event, March 12-13, 2023 proceedings. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol. 511, Springer, Springer Nature, Cham, pp. 111-125, 13th EAI International Conference on Broadband Communications, Networks, and Systems, Virtual, Online, 12/03/23. https://doi.org/10.1007/978-3-031-40467-2_7

An empirical study on model pruning and quantization. / Tian, Yuzhe; Luan, Tom H.; Zheng, Xi.
Broadband communications, networks, and systems: 13th EAI International Conference, BROADNETS 2022, Virtual Event, March 12-13, 2023 proceedings. ed. / Wei Wang; Jun Wu. Cham: Springer, Springer Nature, 2023. p. 111-125 (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering; Vol. 511).

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - An empirical study on model pruning and quantization

AU - Tian, Yuzhe

AU - Luan, Tom H.

AU - Zheng, Xi

PY - 2023

Y1 - 2023

N2 - In machine learning, model compression is vital for resource-constrained Internet of Things (IoT) devices, such as unmanned aerial vehicles (UAVs) and smart phones. Currently there are some state-of-the-art (SOTA) compression methods, but little study is conducted to evaluate these techniques across different models and datasets. In this paper, we present an in-depth study on two SOTA model compression methods, pruning and quantization. We apply these methods on AlexNet, ResNet18, VGG16BN and VGG19BN, with three well known datasets, Fashion-MNIST, CIFAR-10, and UCI-HAR. Through our study, we draw the conclusion that, applying pruning and retraining could keep the performance (average less than degrade) while reducing the model size (at compression rate) on spatial domain datasets (e.g. pictures); the performance on temporal domain datasets (e.g. motion sensors data) degrades more (average about degrade); the performance of quantization is related with the pruning rate and the network architecture. We also compare different clustering methods and reveal the impact on model accuracy and quantization ratio. Finally, we provide some interesting directions for future research.

AB - In machine learning, model compression is vital for resource-constrained Internet of Things (IoT) devices, such as unmanned aerial vehicles (UAVs) and smart phones. Currently there are some state-of-the-art (SOTA) compression methods, but little study is conducted to evaluate these techniques across different models and datasets. In this paper, we present an in-depth study on two SOTA model compression methods, pruning and quantization. We apply these methods on AlexNet, ResNet18, VGG16BN and VGG19BN, with three well known datasets, Fashion-MNIST, CIFAR-10, and UCI-HAR. Through our study, we draw the conclusion that, applying pruning and retraining could keep the performance (average less than degrade) while reducing the model size (at compression rate) on spatial domain datasets (e.g. pictures); the performance on temporal domain datasets (e.g. motion sensors data) degrades more (average about degrade); the performance of quantization is related with the pruning rate and the network architecture. We also compare different clustering methods and reveal the impact on model accuracy and quantization ratio. Finally, we provide some interesting directions for future research.

KW - Model compression

KW - Deep neural network

KW - Edge computing

UR - https://www.scopus.com/pages/publications/85172691657

UR - http://purl.org/au-research/grants/arc/DP210102447

UR - http://purl.org/au-research/grants/arc/LP190100676

U2 - 10.1007/978-3-031-40467-2_7

DO - 10.1007/978-3-031-40467-2_7

M3 - Conference proceeding contribution

AN - SCOPUS:85172691657

SN - 9783031404665

T3 - Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

SP - 111

EP - 125

BT - Broadband communications, networks, and systems

A2 - Wang, Wei

A2 - Wu, Jun

PB - Springer, Springer Nature

CY - Cham

T2 - 13th EAI International Conference on Broadband Communications, Networks, and Systems

Y2 - 12 March 2023 through 13 March 2023

ER -

Tian Y, Luan TH, Zheng X. An empirical study on model pruning and quantization. In Wang W, Wu J, editors, Broadband communications, networks, and systems: 13th EAI International Conference, BROADNETS 2022, Virtual Event, March 12-13, 2023 proceedings. Cham: Springer, Springer Nature. 2023. p. 111-125. (Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering). doi: 10.1007/978-3-031-40467-2_7

An empirical study on model pruning and quantization

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

SUT led : Context-aware verification and validation framework for autonomous driving

Cite this

An empirical study on model pruning and quantization

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Projects

SUT led : Context-aware verification and validation framework for autonomous driving

Cite this