Skip to content
/ EfficientPaperList Public
forked from hustzxd/PaperListTemplate

Paper about Pruning, Quantization, and Efficient-inference/training.

hustzxd.github.io/paperlisttemplate/
3 stars 1 fork Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

rhhc/EfficientPaperList

BranchesTags
 
 

Repository files navigation

Paper List Template

This template makes it easy for you to manage papers.

Table of Contents

Getting Started

  1. Add paper information by ./add_paper_info.sh or ./add_paper_info.sh <name>
  2. Run ./refresh_readme.sh
sparsegpt.prototxt 

paper {
  title: "SparseGPT: Massive Language Models Can be Accurately Pruned in one-shot."
  abbr: "SparseGPT"
  url: "https://arxiv.org/pdf/2301.00774.pdf"
  authors: "Elias Frantar"
  authors: "Dan Alistarh"
  institutions: "IST Austria"
  institutions: "Neural Magic"
}
pub {
  where: "arXiv"
  year: 2023
}
code {
  type: "Pytorch"
  url: "https://github.com/IST-DASLab/sparsegpt"
}
note {
  url: "SparseGPT.md"
}
keyword {
  words: "sparsity"
}

Paper List

keyword

Distributed

meta title publication year code note cover
0 Decentralized_FM Decentralized_FM_alpha Github 2022 Pytorch
Quantization

meta title publication year code note cover
0 RPTQ RPTQ: Reorder-based Post-training Quantization for Large Language Models arXiv 2023 PyTorch
Sparse/Pruning

meta title publication year code note cover
0 Deep Compression Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding ICLR 2016
1 OpenVINO Post-training deep neural network pruning via layer-wise calibration ICCV workshop 2021
2 abbr DFPC: Data flow driven pruning of coupled channels without data ICLR 2023
3 abbr Holistic Adversarially Robust Pruning ICLR 2023
4 MVUE Minimum Variance Unbiased N:M Sparsity for the Neural Gradients ICLR 2023
5 abbr Pruning Deep Neural Networks from a Sparsity Perspective ICLR 2023
6 abbr Rethinking Graph Lottery Tickets: Graph Sparsity Matters ICLR 2023
7 SMC Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! ICLR 2023 SMC-Bench
8 SparseGPT SparseGPT: Massive Language Models Can be Accurately Pruned in one-shot. arXiv 2023 Pytorch note image

year

2016

meta title publication year code note cover
0 Deep Compression Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding ICLR 2016
2021

meta title publication year code note cover
0 OpenVINO Post-training deep neural network pruning via layer-wise calibration ICCV workshop 2021
2022

meta title publication year code note cover
0 Decentralized_FM Decentralized_FM_alpha Github 2022 Pytorch
2023

meta title publication year code note cover
0 abbr A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis ICLR 2023
1 abbr A Unified Framework for Soft Threshold Pruning ICLR 2023
2 abbr Bit-Pruning: A Sparse Multiplication-Less Dot-Product ICLR 2023
3 abbr CrAM: A Compression-Aware Minimizer ICLR 2023
4 abbr DFPC: Data flow driven pruning of coupled channels without data ICLR 2023
5 abbr DepthFL: Depthwise Federated Learning for Heterogeneous Clients ICLR 2023
6 abbr Diffusion Models for Causal Discovery via Topological Ordering ICLR 2023
7 abbr Holistic Adversarially Robust Pruning ICLR 2023
8 abbr HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers ICLR 2023
9 abbr How I Learned to Stop Worrying and Love Retraining ICLR 2023
10 abbr Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks ICLR 2023
11 abbr MECTA: Memory-Economic Continual Test-Time Model Adaptation ICLR 2023
12 MVUE Minimum Variance Unbiased N:M Sparsity for the Neural Gradients ICLR 2023
13 NTK-SAP NTK-SAP: Improving neural network pruning by aligning training dynamics ICLR 2023
14 OTOv2 OTOv2: Automatic, Generic, User-Friendly ICLR 2023 Pytorch
15 abbr Over-parameterized Model Optimization with Polyak-Lojasiewicz Condition ICLR 2023
16 abbr Pruning Deep Neural Networks from a Sparsity Perspective ICLR 2023
17 abbr Rethinking Graph Lottery Tickets: Graph Sparsity Matters ICLR 2023
18 m Revisiting Pruning at Initialization Through the Lens of Ramanujan Graph ICLR 2023
19 abbr Searching Lottery Tickets in Graph Neural Networks: A Dual Perspective ICLR 2023
20 SMC Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! ICLR 2023 SMC-Bench
21 m Symmetric Pruning in Quantum Neural Networks ICLR 2023
22 abbr TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning ICLR 2023
23 m Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? ICLR 2023
24 RPTQ RPTQ: Reorder-based Post-training Quantization for Large Language Models arXiv 2023 PyTorch
25 SparseGPT SparseGPT: Massive Language Models Can be Accurately Pruned in one-shot. arXiv 2023 Pytorch note image

publication

Github

meta title publication year code note cover
0 Decentralized_FM Decentralized_FM_alpha Github 2022 Pytorch
ICCV workshop

meta title publication year code note cover
0 OpenVINO Post-training deep neural network pruning via layer-wise calibration ICCV workshop 2021
ICLR

meta title publication year code note cover
0 Deep Compression Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding ICLR 2016
1 abbr A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis ICLR 2023
2 abbr A Unified Framework for Soft Threshold Pruning ICLR 2023
3 abbr Bit-Pruning: A Sparse Multiplication-Less Dot-Product ICLR 2023
4 abbr CrAM: A Compression-Aware Minimizer ICLR 2023
5 abbr DFPC: Data flow driven pruning of coupled channels without data ICLR 2023
6 abbr DepthFL: Depthwise Federated Learning for Heterogeneous Clients ICLR 2023
7 abbr Diffusion Models for Causal Discovery via Topological Ordering ICLR 2023
8 abbr Holistic Adversarially Robust Pruning ICLR 2023
9 abbr HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers ICLR 2023
10 abbr How I Learned to Stop Worrying and Love Retraining ICLR 2023
11 abbr Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks ICLR 2023
12 abbr MECTA: Memory-Economic Continual Test-Time Model Adaptation ICLR 2023
13 MVUE Minimum Variance Unbiased N:M Sparsity for the Neural Gradients ICLR 2023
14 NTK-SAP NTK-SAP: Improving neural network pruning by aligning training dynamics ICLR 2023
15 OTOv2 OTOv2: Automatic, Generic, User-Friendly ICLR 2023 Pytorch
16 abbr Over-parameterized Model Optimization with Polyak-Lojasiewicz Condition ICLR 2023
17 abbr Pruning Deep Neural Networks from a Sparsity Perspective ICLR 2023
18 abbr Rethinking Graph Lottery Tickets: Graph Sparsity Matters ICLR 2023
19 m Revisiting Pruning at Initialization Through the Lens of Ramanujan Graph ICLR 2023
20 abbr Searching Lottery Tickets in Graph Neural Networks: A Dual Perspective ICLR 2023
21 SMC Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! ICLR 2023 SMC-Bench
22 m Symmetric Pruning in Quantum Neural Networks ICLR 2023
23 abbr TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning ICLR 2023
24 m Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? ICLR 2023
arXiv

meta title publication year code note cover
0 RPTQ RPTQ: Reorder-based Post-training Quantization for Large Language Models arXiv 2023 PyTorch
1 SparseGPT SparseGPT: Massive Language Models Can be Accurately Pruned in one-shot. arXiv 2023 Pytorch note image

instution

DS3Lab

meta title publication year code note cover
0 Decentralized_FM Decentralized_FM_alpha Github 2022 Pytorch
Eindhoven University of Technology

meta title publication year code note cover
0 SMC Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! ICLR 2023 SMC-Bench
Habana Labs

meta title publication year code note cover
0 MVUE Minimum Variance Unbiased N:M Sparsity for the Neural Gradients ICLR 2023
Houmo AI

meta title publication year code note cover
0 RPTQ RPTQ: Reorder-based Post-training Quantization for Large Language Models arXiv 2023 PyTorch
IST Austria

meta title publication year code note cover
0 SparseGPT SparseGPT: Massive Language Models Can be Accurately Pruned in one-shot. arXiv 2023 Pytorch note image
Intel Corporation

meta title publication year code note cover
0 OpenVINO Post-training deep neural network pruning via layer-wise calibration ICCV workshop 2021
Neural Magic

meta title publication year code note cover
0 SparseGPT SparseGPT: Massive Language Models Can be Accurately Pruned in one-shot. arXiv 2023 Pytorch note image
Stanford University

meta title publication year code note cover
0 Deep Compression Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding ICLR 2016
Tencent AI Lab

meta title publication year code note cover
0 RPTQ RPTQ: Reorder-based Post-training Quantization for Large Language Models arXiv 2023 PyTorch
University of Texas at Austin

meta title publication year code note cover
0 SMC Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! ICLR 2023 SMC-Bench
inst1

meta title publication year code note cover
0 abbr A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis ICLR 2023
1 abbr A Unified Framework for Soft Threshold Pruning ICLR 2023
2 abbr Bit-Pruning: A Sparse Multiplication-Less Dot-Product ICLR 2023
3 abbr CrAM: A Compression-Aware Minimizer ICLR 2023
4 abbr DFPC: Data flow driven pruning of coupled channels without data ICLR 2023
5 abbr DepthFL: Depthwise Federated Learning for Heterogeneous Clients ICLR 2023
6 abbr Diffusion Models for Causal Discovery via Topological Ordering ICLR 2023
7 abbr Holistic Adversarially Robust Pruning ICLR 2023
8 abbr HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers ICLR 2023
9 abbr How I Learned to Stop Worrying and Love Retraining ICLR 2023
10 abbr Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks ICLR 2023
11 abbr MECTA: Memory-Economic Continual Test-Time Model Adaptation ICLR 2023
12 NTK-SAP NTK-SAP: Improving neural network pruning by aligning training dynamics ICLR 2023
13 OTOv2 OTOv2: Automatic, Generic, User-Friendly ICLR 2023 Pytorch
14 abbr Over-parameterized Model Optimization with Polyak-Lojasiewicz Condition ICLR 2023
15 abbr Pruning Deep Neural Networks from a Sparsity Perspective ICLR 2023
16 abbr Rethinking Graph Lottery Tickets: Graph Sparsity Matters ICLR 2023
17 m Revisiting Pruning at Initialization Through the Lens of Ramanujan Graph ICLR 2023
18 abbr Searching Lottery Tickets in Graph Neural Networks: A Dual Perspective ICLR 2023
19 m Symmetric Pruning in Quantum Neural Networks ICLR 2023
20 abbr TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning ICLR 2023
21 m Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? ICLR 2023
inst2

meta title publication year code note cover
0 abbr A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis ICLR 2023
1 abbr A Unified Framework for Soft Threshold Pruning ICLR 2023
2 abbr Bit-Pruning: A Sparse Multiplication-Less Dot-Product ICLR 2023
3 abbr CrAM: A Compression-Aware Minimizer ICLR 2023
4 abbr DFPC: Data flow driven pruning of coupled channels without data ICLR 2023
5 abbr DepthFL: Depthwise Federated Learning for Heterogeneous Clients ICLR 2023
6 abbr Diffusion Models for Causal Discovery via Topological Ordering ICLR 2023
7 abbr Holistic Adversarially Robust Pruning ICLR 2023
8 abbr HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers ICLR 2023
9 abbr How I Learned to Stop Worrying and Love Retraining ICLR 2023
10 abbr Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks ICLR 2023
11 abbr MECTA: Memory-Economic Continual Test-Time Model Adaptation ICLR 2023
12 NTK-SAP NTK-SAP: Improving neural network pruning by aligning training dynamics ICLR 2023
13 OTOv2 OTOv2: Automatic, Generic, User-Friendly ICLR 2023 Pytorch
14 abbr Over-parameterized Model Optimization with Polyak-Lojasiewicz Condition ICLR 2023
15 abbr Pruning Deep Neural Networks from a Sparsity Perspective ICLR 2023
16 abbr Rethinking Graph Lottery Tickets: Graph Sparsity Matters ICLR 2023
17 m Revisiting Pruning at Initialization Through the Lens of Ramanujan Graph ICLR 2023
18 abbr Searching Lottery Tickets in Graph Neural Networks: A Dual Perspective ICLR 2023
19 m Symmetric Pruning in Quantum Neural Networks ICLR 2023
20 abbr TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning ICLR 2023
21 m Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? ICLR 2023

author

Bingzhe Wu

meta title publication year code note cover
0 RPTQ RPTQ: Reorder-based Post-training Quantization for Large Language Models arXiv 2023 PyTorch
Brian Chmiel

meta title publication year code note cover
0 MVUE Minimum Variance Unbiased N:M Sparsity for the Neural Gradients ICLR 2023
Dan Alistarh

meta title publication year code note cover
0 SparseGPT SparseGPT: Massive Language Models Can be Accurately Pruned in one-shot. arXiv 2023 Pytorch note image
Daniel Soudry

meta title publication year code note cover
0 MVUE Minimum Variance Unbiased N:M Sparsity for the Neural Gradients ICLR 2023
Elias Frantar

meta title publication year code note cover
0 SparseGPT SparseGPT: Massive Language Models Can be Accurately Pruned in one-shot. arXiv 2023 Pytorch note image
Ivan Lazarevich

meta title publication year code note cover
0 OpenVINO Post-training deep neural network pruning via layer-wise calibration ICCV workshop 2021
Jue Wang

meta title publication year code note cover
0 Decentralized_FM Decentralized_FM_alpha Github 2022 Pytorch
Name1

meta title publication year code note cover
0 abbr A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis ICLR 2023
1 abbr A Unified Framework for Soft Threshold Pruning ICLR 2023
2 abbr Bit-Pruning: A Sparse Multiplication-Less Dot-Product ICLR 2023
3 abbr CrAM: A Compression-Aware Minimizer ICLR 2023
4 abbr DFPC: Data flow driven pruning of coupled channels without data ICLR 2023
5 abbr DepthFL: Depthwise Federated Learning for Heterogeneous Clients ICLR 2023
6 abbr Diffusion Models for Causal Discovery via Topological Ordering ICLR 2023
7 abbr Holistic Adversarially Robust Pruning ICLR 2023
8 abbr HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers ICLR 2023
9 abbr How I Learned to Stop Worrying and Love Retraining ICLR 2023
10 abbr Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks ICLR 2023
11 abbr MECTA: Memory-Economic Continual Test-Time Model Adaptation ICLR 2023
12 NTK-SAP NTK-SAP: Improving neural network pruning by aligning training dynamics ICLR 2023
13 OTOv2 OTOv2: Automatic, Generic, User-Friendly ICLR 2023 Pytorch
14 abbr Over-parameterized Model Optimization with Polyak-Lojasiewicz Condition ICLR 2023
15 abbr Pruning Deep Neural Networks from a Sparsity Perspective ICLR 2023
16 abbr Rethinking Graph Lottery Tickets: Graph Sparsity Matters ICLR 2023
17 m Revisiting Pruning at Initialization Through the Lens of Ramanujan Graph ICLR 2023
18 abbr Searching Lottery Tickets in Graph Neural Networks: A Dual Perspective ICLR 2023
19 m Symmetric Pruning in Quantum Neural Networks ICLR 2023
20 abbr TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning ICLR 2023
21 m Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? ICLR 2023
Name2

meta title publication year code note cover
0 abbr A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis ICLR 2023
1 abbr A Unified Framework for Soft Threshold Pruning ICLR 2023
2 abbr Bit-Pruning: A Sparse Multiplication-Less Dot-Product ICLR 2023
3 abbr CrAM: A Compression-Aware Minimizer ICLR 2023
4 abbr DFPC: Data flow driven pruning of coupled channels without data ICLR 2023
5 abbr DepthFL: Depthwise Federated Learning for Heterogeneous Clients ICLR 2023
6 abbr Diffusion Models for Causal Discovery via Topological Ordering ICLR 2023
7 abbr Holistic Adversarially Robust Pruning ICLR 2023
8 abbr HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers ICLR 2023
9 abbr How I Learned to Stop Worrying and Love Retraining ICLR 2023
10 abbr Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks ICLR 2023
11 abbr MECTA: Memory-Economic Continual Test-Time Model Adaptation ICLR 2023
12 NTK-SAP NTK-SAP: Improving neural network pruning by aligning training dynamics ICLR 2023
13 OTOv2 OTOv2: Automatic, Generic, User-Friendly ICLR 2023 Pytorch
14 abbr Over-parameterized Model Optimization with Polyak-Lojasiewicz Condition ICLR 2023
15 abbr Pruning Deep Neural Networks from a Sparsity Perspective ICLR 2023
16 abbr Rethinking Graph Lottery Tickets: Graph Sparsity Matters ICLR 2023
17 m Revisiting Pruning at Initialization Through the Lens of Ramanujan Graph ICLR 2023
18 abbr Searching Lottery Tickets in Graph Neural Networks: A Dual Perspective ICLR 2023
19 m Symmetric Pruning in Quantum Neural Networks ICLR 2023
20 abbr TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning ICLR 2023
21 m Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? ICLR 2023
Nikita Malinin

meta title publication year code note cover
0 OpenVINO Post-training deep neural network pruning via layer-wise calibration ICCV workshop 2021
Shiwei Liu

meta title publication year code note cover
0 SMC Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! ICLR 2023 SMC-Bench
Song Han

meta title publication year code note cover
0 Deep Compression Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding ICLR 2016
Zhangyang Wang

meta title publication year code note cover
0 SMC Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! ICLR 2023 SMC-Bench
Zhihang Yuan

meta title publication year code note cover
0 RPTQ RPTQ: Reorder-based Post-training Quantization for Large Language Models arXiv 2023 PyTorch

References

  1. https://github.com/he-y/Awesome-Pruning
  2. https://github.com/htqin/awesome-model-quantization
  3. https://github.com/csyhhu/Awesome-Deep-Neural-Network-Compression/tree/master
  4. https://github.com/AojunZhou/Efficient-Deep-Learning
  5. https://github.com/chester256/Model-Compression-Papers

About

Paper about Pruning, Quantization, and Efficient-inference/training.

hustzxd.github.io/PaperListTemplate/

Resources

Readme
Activity
Custom properties

Stars

3 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • Python 92.2%
  • Shell 7.8%