Awesome-Split-Learning

A incomplete survey for Split Learning (comprehensive enough) and Federated Learning (only most representative works).

This list covers most Split Learning works. We also include some Federated Learning works for a easy comparison with SL - are just a sub-list of https://github.com/chaoyanghe/Awesome-Federated-Learning.

*** Updated at 2022/3/7 ***

Category

• Split Learning Schemes

  • Sequentual/Original Split Learning

  • Split Federated Learning

  • Parallel Split Learning

  • Two-party Split Learning

• Evaluation Work

• Communication Reduction

• Privacy Attack

  • HBC Server

  • Malicious Server

• Privacy Attack Mitigation

  • HBC Server

  • Malicious Server

*** Updated at 2022/1/26 ***

Split Learning

Split Learning Schemes

Sequential Split Learning (Original)

1. Distributed learning of deep neural network over multiple agents

2. Split learning for health: Distributed deep learning without sharing raw patient data

3. Split Learning for collaborative deep learning in healthcare

Split Federated Learning

1. SplitFed: When Federated Learning Meets Split Learning

2. (NeurIPS '21) Federated Split Task-Agnostic Vision Transformer for COVID-19 CXR Diagnosis

3. Combining split and federated architectures for efficiency and privacy in deep learning

4. Privacy-Sensitive Parallel Split Learning

Parallel Split Learning (SFL without "F")

1. Comparison of Privacy-Preserving Distributed Deep Learning Methods in Healthcare

2. Spatio-Temporal Split Learning

3. Server-Side Local Gradient Averaging and Learning Rate Acceleration for Scalable Split Learning

4. Efficient Privacy Preserving Edge Intelligent Computing Framework for Image Classification in IoT

5. Flexible Parallel Learning in Edge Scenarios: Communication, Computational and Energy Cost

6. Splitfed learning without client-side synchronization: Analyzing client-side split network portion size to overall performance

Two-party Split Learning

1. (ICLR '21 workshop) Pyvertical: A vertical federated learning framework for multi-headed splitnn

2. (ICLR '22) Label Leakage and Protection in Two-party Split Learning

3. Gradient Inversion Attack: Leaking Private Labels in Two-Party Split Learning

4. FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data

Other Split Learning Variants

1. (NeurIPS '20) Group knowledge transfer: Federated learning of large cnns at the edge

Evaluation Work

1. End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things

2. Detailed comparison of communication efficiency of split learning and federated learning

3. Advancements of Federated Learning Towards Privacy Preservation: From Federated Learning to Split Learning

4. Triad of Split Learning: Privacy, Accuracy, and Performance

5. Computational Privacy with Split Learning: Benchmarking of Algorithmic Defenses against Reconstruction Attacks

Communication Reduction

1. Communication and Computation Reduction for Split Learning using Asynchronous Training

2. Communication-Efficient Multimodal Split Learning for mmWave Received Power Prediction

3. Communication-Efficient Split Learning Based on Analog Communication and Over the Air Aggregation

4. A Federated Learning Framework for Healthcare IoT devices

5. FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients

6. Improving the Communication and Computation Efficiency of Split Learning for IoT Applications

Privacy Attack

HBC Server

1. (CCS '15) Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures

2. (Asia-CCS '20) Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?

3. (ACSAC '19) Model inversion attacks against collaborative inference

4. UnSplit: Data-Oblivious Model Inversion, Model Stealing, and Label Inference Attacks Against Split Learning

Malicious Server

1. (CCS '21) Unleashing the Tiger: Inference Attacks on Split Learning

2. (PPAI '22 workshop) Feature Space Hijacking Attacks against Differentially Private Split Learning

Privacy Attack Mitigation

HBC Server

1. (Asia-CCS '20, increasing depth and differential privacy) Can We Use Split Learning on 1D CNN Models for Privacy Preserving Training?

2. NoPeek: Information leakage reduction to share activations in distributed deep learning

3. NoPeek-Infer: Preventing face reconstruction attacks in distributed inference after on-premise training

4. Practical Defences Against Model Inversion Attacks for Split Neural Networks

5. Get your Foes Fooled: Proximal Gradient Split Learning for Defense against Model Inversion Attacks on IoMT data

Malicious Server

1. SplitGuard: Detecting and Mitigating Training-Hijacking Attacks in Split Learning

Other Applications

Hardware Demo

1. SplitEasy: A Practical Approach for Training ML models on Mobile Devices

2. Split learning on FPGAs

Medical SL

1. (NeurIPS '21) Federated Split Task-Agnostic Vision Transformer for COVID-19 CXR Diagnosis

2. Feasibility study of multi-site split learning for privacy-preserving medical systems under data imbalance constraints in COVID-19, X-ray, and cholesterol dataset

NLP + SL

1. FedBERT: When Federated Learning Meets Pre-Training

Heterogeneous SL

1. FedAdapt: Adaptive Offloading for IoT Devices in Federated Learning

2. AdaSplit: Adaptive Trade-offs for Resource-constrained Distributed Deep Learning

Non-i.i.d. SL

1. Handling Data Heterogeneity with Generative Replay in Collaborative Learning for Medical Imaging

Device Mobility

1. FedFly: Towards Migration in Edge-based Distributed Federated Learning

Binarization SL

1.Efficient binarizing split learning based deep models for mobile applications

Split Learning + HE/MPC

1. PrivColl: Practical Privacy-Preserving Collaborative Machine Learning

2. Split HE: Fast Secure Inference Combining Split Learning and Homomorphic Encryption

Heteromodel SL

1. Distributed Heteromodal Split Learning for Vision Aided mmWave Received Power Prediction

RNN SL

1. FedSL: Federated Split Learning on Distributed Sequential Data in Recurrent Neural Networks

2. LSTMSPLIT: Effective SPLIT Learning based LSTM on Sequential Time-Series Data

Graph NN + SL

1. Towards Representation Identical Privacy-Preserving Graph Neural Network via Split Learning

Vulnerability in SFL

1. Vulnerability Due to Training Order in Split Learning

Federated Learning

Feature Inference Attack

1. Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning

2. Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning

Inversion Attack

1. Inverting Gradients -- How easy is it to break privacy in federated learning?

2. Deep Leakage from Gradients

3. See Through Gradients: Image Batch Recovery via GradInversion

4. Gradient Inversion with Generative Image Prior

Inversion Attack Mitigation

1. Soteria: Provable Defense Against Privacy Leakage in Federated Learning From Representation Perspective