Boosting accuracy of differentially private federated learning in industrial IoT with sparse responses

Laizhong Cui, Jiating Ma, Yipeng Zhou, Shui Yu

Research output: Contribution to journalArticlepeer-review

16 Citations (Scopus)

Abstract

Empowered by 5G, it has been extensively explored by existing works on the deployment of differentially private federated learning (DPFL) in the Industrial Internet of Things (IIoT). Through federated learning, decentralized IIoT devices can collaboratively train a machine learning model by merely exchanging model gradients with a parameter server (PS) for multiple global iterations. Differentially private (DP) mechanisms will be incorporated by IIoT devices (also called clients) to prevent the leakage of privacy due to the exposure of gradients because original gradients will be distorted DP noises. Yet, learning with distorted gradients can seriously deteriorate model accuracy, making DPFL unusable in reality. To address this problem, we propose a novel DPFL with sparse responses (DPFL-SR) algorithm, which applies the sparse vector technique to reduce the privacy budget consumption in each global iteration. Specifically, DPFL-SR evaluates the value of each gradient, and only distorts and uploads significant gradients to the PS because significant gradients are more essential for model training. Since insignificant gradients are not disclosed, the reserved privacy budget can be used to return significant gradients for more iterations so that DPFL-SR can achieve higher model accuracy without lowering the privacy protection level. Extensive experiments are conducted with the MNIST and Fashion-MNIST datasets to demonstrate the practicability and superiority of DPFL-SR in IIoT systems.

Original languageEnglish
Pages (from-to)910-920
Number of pages11
JournalIEEE Transactions on Industrial Informatics
Volume19
Issue number1
DOIs
Publication statusPublished - Jan 2023

Fingerprint

Dive into the research topics of 'Boosting accuracy of differentially private federated learning in industrial IoT with sparse responses'. Together they form a unique fingerprint.

Cite this