TY - JOUR
T1 - Boosting accuracy of differentially private federated learning in industrial IoT with sparse responses
AU - Cui, Laizhong
AU - Ma, Jiating
AU - Zhou, Yipeng
AU - Yu, Shui
PY - 2023/1
Y1 - 2023/1
N2 - Empowered by 5G, it has been extensively explored by existing works on the deployment of differentially private federated learning (DPFL) in the Industrial Internet of Things (IIoT). Through federated learning, decentralized IIoT devices can collaboratively train a machine learning model by merely exchanging model gradients with a parameter server (PS) for multiple global iterations. Differentially private (DP) mechanisms will be incorporated by IIoT devices (also called clients) to prevent the leakage of privacy due to the exposure of gradients because original gradients will be distorted DP noises. Yet, learning with distorted gradients can seriously deteriorate model accuracy, making DPFL unusable in reality. To address this problem, we propose a novel DPFL with sparse responses (DPFL-SR) algorithm, which applies the sparse vector technique to reduce the privacy budget consumption in each global iteration. Specifically, DPFL-SR evaluates the value of each gradient, and only distorts and uploads significant gradients to the PS because significant gradients are more essential for model training. Since insignificant gradients are not disclosed, the reserved privacy budget can be used to return significant gradients for more iterations so that DPFL-SR can achieve higher model accuracy without lowering the privacy protection level. Extensive experiments are conducted with the MNIST and Fashion-MNIST datasets to demonstrate the practicability and superiority of DPFL-SR in IIoT systems.
AB - Empowered by 5G, it has been extensively explored by existing works on the deployment of differentially private federated learning (DPFL) in the Industrial Internet of Things (IIoT). Through federated learning, decentralized IIoT devices can collaboratively train a machine learning model by merely exchanging model gradients with a parameter server (PS) for multiple global iterations. Differentially private (DP) mechanisms will be incorporated by IIoT devices (also called clients) to prevent the leakage of privacy due to the exposure of gradients because original gradients will be distorted DP noises. Yet, learning with distorted gradients can seriously deteriorate model accuracy, making DPFL unusable in reality. To address this problem, we propose a novel DPFL with sparse responses (DPFL-SR) algorithm, which applies the sparse vector technique to reduce the privacy budget consumption in each global iteration. Specifically, DPFL-SR evaluates the value of each gradient, and only distorts and uploads significant gradients to the PS because significant gradients are more essential for model training. Since insignificant gradients are not disclosed, the reserved privacy budget can be used to return significant gradients for more iterations so that DPFL-SR can achieve higher model accuracy without lowering the privacy protection level. Extensive experiments are conducted with the MNIST and Fashion-MNIST datasets to demonstrate the practicability and superiority of DPFL-SR in IIoT systems.
UR - http://www.scopus.com/inward/record.url?scp=85127478762&partnerID=8YFLogxK
U2 - 10.1109/TII.2022.3161517
DO - 10.1109/TII.2022.3161517
M3 - Article
AN - SCOPUS:85127478762
SN - 1551-3203
VL - 19
SP - 910
EP - 920
JO - IEEE Transactions on Industrial Informatics
JF - IEEE Transactions on Industrial Informatics
IS - 1
ER -