No Cover Image

Journal article 793 views 24 downloads

GRNN: Generative Regression Neural Network - A Data Leakage Attack for Federated Learning

Hans Ren, Jingjing Deng, Xianghua Xie Orcid Logo

ACM Transactions on Intelligent Systems and Technology, Volume: 13, Issue: 4

Swansea University Authors: Hans Ren, Jingjing Deng, Xianghua Xie Orcid Logo

Check full text

DOI (Published version): 10.1145/3510032

Abstract

Data privacy has become an increasingly important issue in Machine Learning (ML), where many approaches have been developed to tackle this challenge, e.g. cryptography (Homomorphic Encryption (HE), Differential Privacy (DP), etc.) and collaborative training (Secure Multi-Party Computation (MPC), Dis...

Full description

Published in: ACM Transactions on Intelligent Systems and Technology
ISSN: 2157-6904 2157-6912
Published: Association for Computing Machinery (ACM) 2022
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa59093
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2022-01-18T10:23:50Z
last_indexed 2023-01-11T14:40:07Z
id cronfa59093
recordtype SURis
fullrecord <?xml version="1.0" encoding="utf-8"?><rfc1807 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><bib-version>v2</bib-version><id>59093</id><entry>2022-01-05</entry><title>GRNN: Generative Regression Neural Network - A Data Leakage Attack for Federated Learning</title><swanseaauthors><author><sid>9e043b899a2b786672a28ed4f864ffcc</sid><firstname>Hans</firstname><surname>Ren</surname><name>Hans Ren</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>6f6d01d585363d6dc1622640bb4fcb3f</sid><firstname>Jingjing</firstname><surname>Deng</surname><name>Jingjing Deng</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>b334d40963c7a2f435f06d2c26c74e11</sid><ORCID>0000-0002-2701-8660</ORCID><firstname>Xianghua</firstname><surname>Xie</surname><name>Xianghua Xie</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2022-01-05</date><deptcode>MACS</deptcode><abstract>Data privacy has become an increasingly important issue in Machine Learning (ML), where many approaches have been developed to tackle this challenge, e.g. cryptography (Homomorphic Encryption (HE), Differential Privacy (DP), etc.) and collaborative training (Secure Multi-Party Computation (MPC), Distributed Learning and Federated Learning (FL)). These techniques have a particular focus on data encryption or secure local computation. They transfer the intermediate information to the third party to compute the final result. Gradient exchanging is commonly considered to be a secure way of training a robust model collaboratively in Deep Learning (DL). However, recent researches have demonstrated that sensitive information can be recovered from the shared gradient. Generative Adversarial Network (GAN), in particular, has shown to be effective in recovering such information. However, GAN based techniques require additional information, such as class labels which are generally unavailable for privacy-preserved learning. In this paper, we show that, in the FL system, image-based privacy data can be easily recovered in full from the shared gradient only via our proposed Generative Regression Neural Network (GRNN). We formulate the attack to be a regression problem and optimize two branches of the generative model by minimizing the distance between gradients. We evaluate our method on several image classification tasks. The results illustrate that our proposed GRNN outperforms state-of-the-art methods with better stability, stronger robustness, and higher accuracy. It also has no convergence requirement to the global FL model. Moreover, we demonstrate information leakage using face re-identification. Some defense strategies are also discussed in this work.</abstract><type>Journal Article</type><journal>ACM Transactions on Intelligent Systems and Technology</journal><volume>13</volume><journalNumber>4</journalNumber><paginationStart/><paginationEnd/><publisher>Association for Computing Machinery (ACM)</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint>2157-6904</issnPrint><issnElectronic>2157-6912</issnElectronic><keywords/><publishedDay>4</publishedDay><publishedMonth>2</publishedMonth><publishedYear>2022</publishedYear><publishedDate>2022-02-04</publishedDate><doi>10.1145/3510032</doi><url/><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm/><funders>EPSRC, EP/N028139/1</funders><projectreference/><lastEdited>2024-07-25T15:55:53.7040554</lastEdited><Created>2022-01-05T19:26:55.6015441</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Hans</firstname><surname>Ren</surname><order>1</order></author><author><firstname>Jingjing</firstname><surname>Deng</surname><order>2</order></author><author><firstname>Xianghua</firstname><surname>Xie</surname><orcid>0000-0002-2701-8660</orcid><order>3</order></author></authors><documents><document><filename>59093__22048__e7077b93967142c4969d9f5942416511.pdf</filename><originalFilename>plain.pdf</originalFilename><uploaded>2022-01-06T11:00:07.1980242</uploaded><type>Output</type><contentLength>7434019</contentLength><contentType>application/pdf</contentType><version>Accepted Manuscript</version><cronfaStatus>true</cronfaStatus><embargoDate>2024-01-01T00:00:00.0000000</embargoDate><copyrightCorrect>true</copyrightCorrect><language>eng</language></document></documents><OutputDurs/></rfc1807>
spelling v2 59093 2022-01-05 GRNN: Generative Regression Neural Network - A Data Leakage Attack for Federated Learning 9e043b899a2b786672a28ed4f864ffcc Hans Ren Hans Ren true false 6f6d01d585363d6dc1622640bb4fcb3f Jingjing Deng Jingjing Deng true false b334d40963c7a2f435f06d2c26c74e11 0000-0002-2701-8660 Xianghua Xie Xianghua Xie true false 2022-01-05 MACS Data privacy has become an increasingly important issue in Machine Learning (ML), where many approaches have been developed to tackle this challenge, e.g. cryptography (Homomorphic Encryption (HE), Differential Privacy (DP), etc.) and collaborative training (Secure Multi-Party Computation (MPC), Distributed Learning and Federated Learning (FL)). These techniques have a particular focus on data encryption or secure local computation. They transfer the intermediate information to the third party to compute the final result. Gradient exchanging is commonly considered to be a secure way of training a robust model collaboratively in Deep Learning (DL). However, recent researches have demonstrated that sensitive information can be recovered from the shared gradient. Generative Adversarial Network (GAN), in particular, has shown to be effective in recovering such information. However, GAN based techniques require additional information, such as class labels which are generally unavailable for privacy-preserved learning. In this paper, we show that, in the FL system, image-based privacy data can be easily recovered in full from the shared gradient only via our proposed Generative Regression Neural Network (GRNN). We formulate the attack to be a regression problem and optimize two branches of the generative model by minimizing the distance between gradients. We evaluate our method on several image classification tasks. The results illustrate that our proposed GRNN outperforms state-of-the-art methods with better stability, stronger robustness, and higher accuracy. It also has no convergence requirement to the global FL model. Moreover, we demonstrate information leakage using face re-identification. Some defense strategies are also discussed in this work. Journal Article ACM Transactions on Intelligent Systems and Technology 13 4 Association for Computing Machinery (ACM) 2157-6904 2157-6912 4 2 2022 2022-02-04 10.1145/3510032 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University EPSRC, EP/N028139/1 2024-07-25T15:55:53.7040554 2022-01-05T19:26:55.6015441 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Hans Ren 1 Jingjing Deng 2 Xianghua Xie 0000-0002-2701-8660 3 59093__22048__e7077b93967142c4969d9f5942416511.pdf plain.pdf 2022-01-06T11:00:07.1980242 Output 7434019 application/pdf Accepted Manuscript true 2024-01-01T00:00:00.0000000 true eng
title GRNN: Generative Regression Neural Network - A Data Leakage Attack for Federated Learning
spellingShingle GRNN: Generative Regression Neural Network - A Data Leakage Attack for Federated Learning
Hans Ren
Jingjing Deng
Xianghua Xie
title_short GRNN: Generative Regression Neural Network - A Data Leakage Attack for Federated Learning
title_full GRNN: Generative Regression Neural Network - A Data Leakage Attack for Federated Learning
title_fullStr GRNN: Generative Regression Neural Network - A Data Leakage Attack for Federated Learning
title_full_unstemmed GRNN: Generative Regression Neural Network - A Data Leakage Attack for Federated Learning
title_sort GRNN: Generative Regression Neural Network - A Data Leakage Attack for Federated Learning
author_id_str_mv 9e043b899a2b786672a28ed4f864ffcc
6f6d01d585363d6dc1622640bb4fcb3f
b334d40963c7a2f435f06d2c26c74e11
author_id_fullname_str_mv 9e043b899a2b786672a28ed4f864ffcc_***_Hans Ren
6f6d01d585363d6dc1622640bb4fcb3f_***_Jingjing Deng
b334d40963c7a2f435f06d2c26c74e11_***_Xianghua Xie
author Hans Ren
Jingjing Deng
Xianghua Xie
author2 Hans Ren
Jingjing Deng
Xianghua Xie
format Journal article
container_title ACM Transactions on Intelligent Systems and Technology
container_volume 13
container_issue 4
publishDate 2022
institution Swansea University
issn 2157-6904
2157-6912
doi_str_mv 10.1145/3510032
publisher Association for Computing Machinery (ACM)
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description Data privacy has become an increasingly important issue in Machine Learning (ML), where many approaches have been developed to tackle this challenge, e.g. cryptography (Homomorphic Encryption (HE), Differential Privacy (DP), etc.) and collaborative training (Secure Multi-Party Computation (MPC), Distributed Learning and Federated Learning (FL)). These techniques have a particular focus on data encryption or secure local computation. They transfer the intermediate information to the third party to compute the final result. Gradient exchanging is commonly considered to be a secure way of training a robust model collaboratively in Deep Learning (DL). However, recent researches have demonstrated that sensitive information can be recovered from the shared gradient. Generative Adversarial Network (GAN), in particular, has shown to be effective in recovering such information. However, GAN based techniques require additional information, such as class labels which are generally unavailable for privacy-preserved learning. In this paper, we show that, in the FL system, image-based privacy data can be easily recovered in full from the shared gradient only via our proposed Generative Regression Neural Network (GRNN). We formulate the attack to be a regression problem and optimize two branches of the generative model by minimizing the distance between gradients. We evaluate our method on several image classification tasks. The results illustrate that our proposed GRNN outperforms state-of-the-art methods with better stability, stronger robustness, and higher accuracy. It also has no convergence requirement to the global FL model. Moreover, we demonstrate information leakage using face re-identification. Some defense strategies are also discussed in this work.
published_date 2022-02-04T15:55:53Z
_version_ 1805563307550048256
score 11.030252

Similar Items