簡易檢索 / 詳目顯示

回結果列表
研究生： 丁于庭
Ting, Yu-Ting
論文名稱： 具有閾值安全聚合的聯邦學習之研究
A Study of Federated Learning with Threshold Secure Aggregation
指導教授： 紀博文
Chi, Po-Wen
口試委員： 紀博文
Chi, Po-Wen
王銘宏
Wang, Ming-Hung
莊允心
Chuang, Yun-Hsin
口試日期： 2024/07/25
學位類別： 碩士
Master
系所名稱： 資訊工程學系
Department of Computer Science and Information Engineering
論文出版年： 2024
畢業學年度： 112
語文別： 英文
論文頁數： 58
中文關鍵詞： 聯邦學習 、秘密共享
英文關鍵詞： Federated Learning, Secret Sharing
研究方法: 實驗設計法
DOI URL： http://doi.org/10.6345/NTNU202401353
論文種類： 學術論文
相關次數： 點閱：169  下載：1 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 聯邦學習是一種去中心化的且允許多個客戶端一起參與協作的隱私保護機制，讓客戶端之間無需交換資料集，透過上傳自身的梯度即可共同訓練模型。但近期的研究表示攻擊者透過客戶端的梯度就可以還原原始的訓練資料，聯邦學習也變得不再安全。因此開始有越來越多的研究使用不同的技術來保護梯度。常見的技術之一就是秘密分享，但以往使用秘密分享保護梯度隱私的研究，只要有一個份額遺失或是一台伺服器毀損便無法還原原始的梯度，聯邦學習的運作就會因此中斷。
    在本篇論文中，我們針對聯邦學習梯度聚合提出一種結合加法秘密共享的方法，讓攻擊者無法輕易地取得客戶端的原始梯度。此外，我們提出的方法也確保在一定的機率下，即便有伺服器毀損或部分梯度份額遺失也不會對聯邦學習運作造成任何影響。我們還額外加上了會員等級制度，讓不同等級的會員在最終會拿到不同準確度的模型。

    Federated learning is a decentralized privacy-preserving mechanism that allows multiple clients to collaborate without exchanging their datasets. Instead, they jointly train a model by uploading their own gradients. However, recent research has shown that attackers can use clients' gradients to reconstruct the original training data, compromising the security of federated learning. Therefore, there has been an increasing number of studies using different techniques to protect gradients. One common technique is secret sharing. However, in previous research on using secret sharing to protect gradient privacy, as long as one share is lost or a server is damaged, the original gradient cannot be reconstructed, causing federated learning to be interrupted.
    In this paper, we propose an approach that combines additive secret sharing for federated learning gradient aggregation, making it difficult for attackers to easily access clients' original gradients. Additionally, our proposed method ensures that with a certain probability, that even in the event of server damage or the loss of some gradient shares, it will not have any impact on the federated learning operation. We also added a membership level system, allowing members of varying levels to ultimately obtain models with different levels of accuracy.

    Chapter 1 Introduction 1 1.1 Motivation 4 1.2 Contributions 6 1.3 Organization 6 Chapter 2 Related Works 8 2.1 Federated Learning 8 2.2 Gradients Leakage Attack 12 2.3 Privacy-Preserving Federated Learning 13 2.3.1 Homomorphic Encryption 14 2.3.2 Differential Privacy 15 2.3.3 Secure Multi-Party Computation 16 Chapter 3 Federated Learning with Threshold Secure Aggregation 19 3.1 Requirements 19 3.1.1 Overview 19 3.2 Threshold Additive Secret Sharing 21 3.2.1 2-out-of-2 Additive Secret Sharing 21 3.2.2 n-out-of-n Additive Secret Sharing 23 3.2.3 Build 2-out-of-n Additive Secret Sharing from 2-out-of-2 Additive Secret Sharing 24 3.2.4 Build t-out-of-n Additive Secret Sharing from t-out-of-t Additive Secret Sharing 27 3.3 Proposed Approach 30 3.4 Quality of Service in Federated Learning 34 Chapter 4 Evaluation 40 4.1 Learning Performance and Computational Cost 40 4.1.1 Experimental Environment 40 4.1.2 Accuracy Comparison for Independent and Identically Distributed Data(IID) 41 4.1.3 Accuracy Comparison for Non-independent and Non-identically DistributedData(Non-IID) 42 4.1.4 Computational Cost for IID and Non-IID Data 44 4.2 The Recovery Probability for Server Errors 46 4.3 The Successful Attack Probability for Compromised Servers 48 4.4 Quality of Federated Learning Experiment 49 Chapter 5 Conclusions and Future Works 52 References 54

    H. Fazli Khojir, D. Alhadidi, S. Rouhani, and N. Mohammed, “Fedshare: Secure aggregation based on additive secret sharing in federated learning,” in Proceedings of the 27th International Database Engineered Applications Symposium, IDEAS ’23, (New York, NY, USA), p. 25–33, Association for Computing Machinery, 2023.
    H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” 2023.
    L. Deng, “The mnist database of handwritten digit images for machine learning research [best of the web],” IEEE signal processing magazine, vol. 29, no. 6, pp. 141– 142, 2012.
    H. Xiao, K. Rasul, and R. Vollgraf,“Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747, 2017.
    H. B. McMahan, E. Moore, D. Ramage, and B. A. y Arcas, “Federated learning of deep networks using model averaging,” CoRR, vol. abs/1602.05629, 2016.
    L. Zhu, Z. Liu, and S. Han, “Deep leakage from gradients,” 2019.
    Y. More, P. Ramachandran, P. Panda, A. Mondal, H. Virk, and D. Gupta, “Scotch: An efficient secure computation framework for secure aggregation,” 2022.
    Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Trans. Intell. Syst. Technol., vol. 10, jan 2019.
    D. -J. Han, M. Choi, J. Park, and J. Moon, “Fedmes: Speeding up federated learning with multiple edge servers,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 12, pp. 3870–3885, 2021.
    Z. Qu, X. Li, J. Xu, B. Tang, Z. Lu, and Y. Liu, “On the convergence of multi-server federated learning with overlapping area,” IEEE Transactions on Mobile Computing, vol. 22, no. 11, pp. 6647–6662, 2023.
    W. Wei, L. Liu, M. Loper, K.-H. Chow, M. E. Gursoy, S. Truex, and Y. Wu, “A framework for evaluating gradient leakage attacks in federated learning,” 2020.
    J. Geiping, H. Bauermeister, H. Dröge, and M. Moeller, “Inverting gradients - how easy is it to break privacy in federated learning?,” in Advances in Neural Information Processing Systems (H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds.), vol. 33, pp. 16937–16947, Curran Associates, Inc., 2020.
    B. Zhao, K. R. Mopuri, and H. Bilen, “idlg: Improved deep leakage from gradients,” 2020.
    C. Dwork, “Differential privacy,” in Automata, Languages and Programming (M. Bugliesi, B. Preneel, V. Sassone, and I. Wegener, eds.), (Berlin, Heidelberg), pp. 1–12, Springer Berlin Heidelberg, 2006.
    A. C. Yao, “Protocols for secure computations,” in 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982), pp. 160–164, 1982.
    J. Park and H. Lim, “Privacy-preserving federated learning using homomorphic encryption,” Applied Sciences, vol. 12, no. 2, 2022.
    H. Fang and Q. Qian, “Privacy preserving machine learning with homomorphic encryption and federated learning,” Future Internet, vol. 13, no. 4, 2021.
    C. Zhang, S. Li, J. Xia, W. Wang, F. Yan, and Y. Liu, “{BatchCrypt}: Efficient homomorphic encryption for {Cross-Silo} federated learning,” in 2020 USENIX annual technical conference (USENIX ATC 20), pp. 493–506, 2020.
    J. Ma, S. -A. Naas, S. Sigg, and X. Lyu,“Privacy-preserving federated learning based on multi-key homomorphic encryption,” International Journal of Intelligent Systems, vol. 37, no. 9, pp. 5880–5901, 2022.
    H. Ku, W. Susilo, Y. Zhang, W. Liu, and M. Zhang, “Privacy-preserving federated learning in medical diagnosis with homomorphic re-encryption,” Computer Standards & Interfaces, vol. 80, p. 103583, 2022.
    Z. Chuanxin, S. Yi, and W. Degang, “Federated learning with gaussian differential privacy,” in Proceedings of the 2020 2nd International Conference on Robotics, Intelligent Control and Artificial Intelligence, RICAI ’20, (New York, NY, USA), p. 296–301, Association for Computing Machinery, 2020.
    K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. S. Quek, and H. Vincent Poor, “Federated learning with differential privacy: Algorithms and performance analysis,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 3454–3469, 2020.
    A. Triastcyn and B. Faltings,“Federated learning with bayesian differential privacy,” in 2019 IEEE International Conference on Big Data (Big Data), pp. 2587–2596, 2019.
    L. Sun, J. Qian, and X. Chen, “Ldp-fl: Practical private aggregation in federated learning with local differential privacy,” 2021.
    A. Shamir,“How to sharea secret,”Commun. ACM, vol. 22, p. 612–613, nov 1979.
    J. Duan, J. Zhou, and Y. Li, “Privacy-preserving distributed deep learning based on secret sharing,” Information Sciences, vol. 527, pp. 108–127, 2020.
    Y. Li, Y. Zhou, A. Jolfaei, D. Yu, G. Xu, and X. Zheng,“Privacy-preserving federated learning framework based on chained secure multiparty computing,” IEEE Internet of Things Journal, vol. 8, no. 8, pp. 6178–6186, 2021.
    R. Xu, N. Baracaldo, Y. Zhou, A. Anwar, and H. Ludwig, “Hybridalpha: An efficient approach for privacy-preserving federated learning,” in Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security, AISec’19, (New York, NY, USA), p. 13–23, Association for Computing Machinery, 2019.
    C. S. at Columbia University,“Coms w4261: Introduction to cryptography.”https://www.cs.columbia.edu/~tal/4261/F19/secretsharingf19.pdf, 2019.
    E. Bisong and E. Bisong, “Google colaboratory,” Building machine learning and deep learning models on google cloud platform: a comprehensive guide for beginners, pp. 59–64, 2019.
    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al., “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems, vol. 32, 2019.

    下載圖示
    QR CODE