EfficientPaper

Pruning, Quantization and efficient-inference/training paper list.

Table of Contents

• EfficientPaper
• Getting Started
• Paper List
  • keyword
  • year
  • publication
  • institution
  • author
• References

Getting Started

git clone https://github.com/hustzxd/EfficientPaper
pip install protobuf==5.27.2 pandas arxiv 

1. Add paper information by ./add_paper_info.sh
2. Run ./refresh_readme.sh

paper {
  title: "EfficientPaper: manage your research papers in an efficient way."
  abbr: "EfficientPaper"
  url: "https://github.com/hustzxd/EfficientPaper"
  authors: "hustzxd"
}
pub {
  where: "GitHub"
  year: 2023
}
code {
  type: "Pytorch"
  url: "https://github.com/hustzxd/EfficientPaper"
}
note {
  url: "EfficientPaper.md"
}
keyword {
  words: efficient_paper
}

Recent Changes

Paper List

year

2026

1. FlashOverlap: A Lightweight Design for Efficiently Overlapping Communication and Computation []

2025

1. AdaSkip: Adaptive Sublayer Skipping for Accelerating Long-Context LLM Inference []
2. Pruning Large Language Models with Semi-Structural Adaptive Sparse Training []
3. QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overhead []
4. Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention []
5. Training LLMs with MXFP4 []
6. COMET: Towards Partical W4A4KV4 LLMs Serving []
7. POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference []
8. vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention []
9. BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity []
10. KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs []
11. Enhancing One-shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse Mechanism []
12. Tree of Agents: Improving Long-Context Capabilities of Large Language Models through Multi-Perspective Reasoning []
13. CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion []
14. FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference []
15. Forgetting Transformer: Softmax Attention with a Forget Gate []
16. R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference []
17. ReAttention: Training-Free Infinite Context with Finite Attention Scope []
18. Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA []
19. TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attention []
20. Training-Free Activation Sparsity in Large Language Models []
21. AdaSplash: Adaptive Sparse Flash Attention []
22. BaWA: Automatic Optimizing Pruning Metric for Large Language Models with Balanced Weight and Activation []
23. CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration []
24. Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarity []
25. HashAttention: Semantic Sparsity for Faster Inference []
26. La RoSA: Enhancing LLM Efficiency via Layerwise Rotated Sparse Activation []
27. MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention []
28. ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference []
29. SlimLLM: Accurate Structured Pruning for Large Language Models []
30. SpargeAttn: Accurate Sparse Attention Accelerating Any Model Inference []
31. Sparsing Law: Towards Large Language Models with Greater Activation Sparsity []
32. Star Attention: Efficient LLM Inference over Long Sequences []
33. XAttention: Block Sparse Attention with Antidiagonal Scoring []
34. TorchAO: PyTorch-Native Training-to-Serving Model Optimization []
35. AMALI: An Analytical Model for Accurately Modeling LLM Inference on Modern GPUs []
36. SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting []
37. Unifying Mixture of Experts and Multi-Head Latent Attention for Efficient Language Models []
38. Rethinking Key-Value Cache Compression Techniques for Large Language Model Serving []
39. Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correction []
40. MoBA: Mixture of Block Attention for Long-Context LLMs []
41. SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training []
42. Týr-the-Pruner: Unlocking Accurate 50% Structural Pruning for LLMs via Global Sparsity Distribution Optimization []
43. NanoFlow: Towards Optimal Large Language Model Serving Throughput []
44. Acc-SpMM: Accelerating General-purpose Sparse Matrix-Matrix Multiplication with GPU Tensor Cores []
45. PQCache: Product Quantization-based KVCache for Long Context LLM Inference []
46. A Simple Linear Patch Revives Layer-Pruned Large Language Models []
47. Accelerating LLM Inference Throughput via Asynchronous KV Cache Prefetching []
48. Accelerating Prefilling for Long-Context LLMs via Sparse Pattern Sharing []
49. Adaptive Computation Pruning for the Forgetting Transformer []
50. Adaptive Layer-skipping in Pre-trained LLMs []
51. AhaKV: Adaptive Holistic Attention-Driven KV Cache Eviction for Efficient Inference of Large Language Models []
52. Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models []
53. AttentionPredictor: Temporal Pattern Matters for Efficient LLM Inference []
54. Binary Quantization For LLMs Through Dynamic Grouping []
55. CCQ: Convolutional Code for Extreme Low-bit Quantization in LLMs []
56. Characterizing Communication Patterns in Distributed Large Language Model Inference []
57. Characterizing Compute-Communication Overlap in GPU-Accelerated Distributed Deep Learning: Performance and Power Implications []
58. ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference []
59. Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts []
60. DBudgetKV: Dynamic Budget in KV Cache Compression for Ensuring Optimal Performance []
61. DReSS: Data-driven Regularized Structured Streamlining for Large Language Models []
62. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning []
63. DeltaLLM: A Training-Free Framework Exploiting Temporal Sparsity for Efficient Edge LLM Inference []
64. Efficient LLM Inference: Bandwidth, Compute, Synchronization, and Capacity are all you need []
65. Efficient Long-Decoding Inference with Reasoning-Aware Attention Sparsity []
66. EvolKV: Evolutionary KV Cache Compression for LLM Inference []
67. Exploiting Sparsity for Long Context Inference: Million Token Contexts on Commodity GPUs []
68. Fast and Simplex: 2-Simplicial Attention in Triton []
69. FastKV: KV Cache Compression for Fast Long-Context Processing with Token-Selective Propagation []
70. Faster VGGT with Block-Sparse Global Attention []
71. Flash Sparse Attention: An Alternative Efficient Implementation of Native Sparse Attention Kernel []
72. FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving []
73. FreqKV: Frequency Domain Key-Value Compression for Efficient Context Window Extension []
74. GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models []
75. HATA: Trainable and Hardware-Efficient Hash-Aware Top-k Attention for Scalable Large Model Inference []
76. HCAttention: Extreme KV Cache Compression via Heterogeneous Attention Computing for LLMs []
77. Hardware-Efficient Attention for Fast Decoding []
78. Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding []
79. Injecting Adrenaline into LLM Serving: Boosting Resource Utilization and Throughput via Attention Disaggregation []
80. Instruction-Following Pruning for Large Language Models []
81. KV Cache Compression for Inference Efficiency in LLMs: A Review []
82. KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse []
83. KVmix: Gradient-Based Layer Importance-Aware Mixed-Precision Quantization for KV Cache []
84. KeepKV: Eliminating Output Perturbation in KV Cache Compression for Efficient LLMs Inference []
85. LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation []
86. LServe: Efficient Long-sequence LLM Serving with Unified Sparse Attention []
87. LeanK: Learnable K Cache Channel Pruning for Efficient Decoding []
88. MIRAGE: KV Cache Optimization through Parameter Remapping for Multi-tenant LLM Serving []
89. MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production []
90. MiniCPM4: Ultra-Efficient LLMs on End Devices []
91. MiniMax-01: Scaling Foundation Models with Lightning Attention []
92. MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention []
93. Mixture of Experts in Large Language Models []
94. Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing []
95. Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation []
96. MoPEQ: Mixture of Mixed Precision Quantized Experts []
97. Mosaic: Composite Projection Pruning for Resource-efficient LLMs []
98. PagedEviction: Structured Block-wise KV Cache Pruning for Efficient Large Language Model Inference []
99. Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend NPUs []
100. PowerAttention: Exponentially Scaling of Receptive Fields for Effective Sparse Attention []
101. Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving []
102. Pruning General Large Language Models into Customized Expert Models []
103. QuickSilver -- Speeding up LLM Inference through Dynamic Token Halting, KV Skipping, Contextual Token Fusion, and Adaptive Matryoshka Quantization []
104. Qwen3 Technical Report []
105. R-KV: Redundancy-aware KV Cache Compression for Training-Free Reasoning Models Acceleration []
106. Radial Attention: $O(n\log n)$ Sparse Attention with Energy Decay for Long Video Generation []
107. Rectified Sparse Attention []
108. Retrospective Sparse Attention for Efficient Long-Context Generation []
109. RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations []
110. SALE : Low-bit Estimation for Efficient Sparse Attention in Long-context LLM Prefilling []
111. SEAP: Training-free Sparse Expert Activation Pruning Unlock the Brainpower of Large Language Models []
112. SeerAttention-R: Sparse Attention Adaptation for Long Reasoning []
113. Seesaw: High-throughput LLM Inference via Model Re-sharding []
114. SlimInfer: Accelerating Long-Context LLM Inference via Dynamic Token Pruning []
115. Speed Always Wins: A Survey on Efficient Architectures for Large Language Models []
116. SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layers []
117. Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding []
118. Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads []
119. The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs []
120. TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitives []
121. TokenWeave: Efficient Compute-Communication Overlap for Distributed LLM Inference []
122. Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler []
123. Unveiling Super Experts in Mixture-of-Experts Large Language Models []
124. Attention-Gym: Triton-Based Sparse and Quantization Attention []
125. DeepEP: an efficient expert-parallel communication library []
126. Unified KV Cache Compression Methods for Auto-Regressive Models []
127. kvpress: LLM KV cache compression made easy []

2024

1. Fluctuation-based Adaptive Structured Pruning for Large Language Models []
2. $\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens []
3. ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition []
4. LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding []
5. Centauri: Enabling Efficient Scheduling for Communication-Computation Overlap in Large Model Training via Communication Partitioning []
6. T3: Transparent Tracking & Triggering for Fine-grained Overlap of Compute & Collectives []
7. Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention []
8. A novel CUTLASS-based implementation of Tensor Parallelism for NVLink-enabled systems []
9. [Distributed w/ TorchTitan] Introducing Async Tensor Parallelism in PyTorch []
10. CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models []
11. Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption []
12. SparseInfer: Training-free Prediction of Activation Sparsity for Fast LLM Inference []
13. Post-Training Statistical Calibration for Higher Activation Sparsity []
14. A Simple and Effective Pruning Approach for Large Language Models []
15. Compressing LLMs: The Truth is Rarely Pure and Never Simple []
16. Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs []
17. Efficient Streaming Language Models with Attention Sinks []
18. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning []
19. Plug-and-Play: An Efficient Post-training Pruning Method for Large Language Models []
20. QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models []
21. SAS: Structured Activation Spasification []
22. SliceGPT: Compress Large Language Models by Deleting Rows and Columns []
23. ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models []
24. Accelerating Transformer Pre-training with 2:4 Sparsity []
25. EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty []
26. FrameQuant: Flexible Low-Bit Quantization for Transformers []
27. KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache []
28. LoRA+: Efficient Low Rank Adaptation of Large Models []
29. OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization []
30. Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity []
31. Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference []
32. SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models []
33. SparQ Attention: Bandwidth-Efficient LLM Inference []
34. Sparse is Enough in Fine-tuning Pre-trained Large Language Models []
35. Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency []
36. SqueezeLLM: Dense-and-Sparse Quantization []
37. TinyTrain: Resource-Aware Task-Adaptive Sparse Training of DNNs at the Data-Scarce Edge []
38. Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Experts []
39. Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention []
40. Splitwise: Efficient generative LLM inference using phase splitting []
41. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration []
42. Vidur: A Large-Scale Simulation Framework For LLM Inference []
43. KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization []
44. MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention []
45. MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models []
46. SGLang: Efficient Execution of Structured Language Model Programs []
47. SlimGPT: Layer-wise Structured Pruning for Large Language Models []
48. SparseLLM: Towards Global Pruning for Pre-trained Language Models []
49. ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification []
50. Fast and Effective Weight Update for Pruned Large Language Models []
51. Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity []
52. A Survey on Efficient Inference for Large Language Models []
53. A Survey on Inference Optimization Techniques for Mixture of Experts Models []
54. A Survey on Large Language Model Acceleration based on KV Cache Management []
55. APEX: An Extensible and Dynamism-Aware Simulator for Automated Parallel Execution in LLM Serving []
56. AVSS: Layer Importance Evaluation in Large Language Models via Activation Variance-Sparsity Analysis []
57. Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference []
58. Beyond 2:4: exploring V:N:M sparsity for efficient transformer inference on GPUs []
59. Beyond KV Caching: Shared Attention for Efficient LLMs []
60. Compact Language Models via Pruning and Knowledge Distillation []
61. CoreInfer: Accelerating Large Language Model Inference with Semantics-Inspired Adaptive Sparse Activation []
62. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model []
63. DeepSeek-V3 Technical Report []
64. DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models []
65. Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlapping []
66. DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads []
67. Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment []
68. Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes []
69. FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion []
70. FlashMask: Efficient and Rich Mask Extension of FlashAttention []
71. GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM []
72. Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache []
73. L4Q: Parameter Efficient Quantization-Aware Training on Large Language Models via LoRA-wise LSQ []
74. LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning []
75. LLM Inference Serving: Survey of Recent Advances and Opportunities []
76. LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference []
77. Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models []
78. Massive Activations in Large Language Models []
79. MiniCache: KV Cache Compression in Depth Dimension for Large Language Models []
80. MiniKV: Pushing the Limits of LLM Inference via 2-Bit Layer-Discriminative KV Cache []
81. Mixture-of-Depths: Dynamically allocating compute in transformer-based language models []
82. MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression []
83. Multi-matrix Factorization Attention []
84. No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization []
85. Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsification [] Pytorch
86. Post-Training Sparse Attention with Double Sparsity []
87. PowerInfer-2: Fast Large Language Model Inference on a Smartphone [] Website
88. PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization []
89. ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models []
90. Q-Sparse: All Large Language Models can be Fully Sparsely-Activated []
91. QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving [] Pytorch
92. ReLU2 Wins: Discovering Efficient Activation Functions for Sparse LLMs []
93. ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing []
94. Recycled Attention: Efficient inference for long-context language models []
95. Reducing Transformer Key-Value Cache Size with Cross-Layer Attention []
96. Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark []
97. SCBench: A KV Cache-Centric Analysis of Long-Context Methods []
98. SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantization []
99. SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration []
100. SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention []
101. SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs []
102. ShadowLLM: Predictor-based Contextual Sparsity for Large Language Models []
103. SnapKV: LLM Knows What You are Looking for Before Generation []
104. Transformers are Multi-State RNNs []
105. Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters [] Pytorch
106. XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models []
107. ZigZagkv: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty []
108. ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification []

2023

1. Diffuser: Efficient Transformers with Multi-hop Attention Diffusion for Long Sequences []
2. Gradient-based Intra-attention Pruning on Pre-trained Language Models []
3. Pruning Pre-trained Language Models Without Fine-Tuning []
4. Pruning Pre-trained Language Models with Principled Importance and Self-regularization []
5. Structured Pruning for Efficient Generative Pre-trained Language Models []
6. Overlap Communication with Dependent Computation via Decomposition in Large Deep Learning Models []
7. Structural Pruning of Large Language Models via Neural Architecture Search []
8. SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer []
9. TorchSparse++: Efficient Point Cloud Engine []
10. AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning []
11. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers []
12. Minimum Variance Unbiased N:M Sparsity for the Neural Gradients []
13. The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers []
14. Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time []
15. SparseGPT: Massive Language Models Can be Accurately Pruned in one-shot. []
16. Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation []
17. Efficient GPU Kernels for N:M-Sparse Weights in Deep Learning []
18. ZipLM: Inference-Aware Structured Pruning of Language Models []
19. VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores []
20. Efficient Memory Management for Large Language Model Serving with PagedAttention []
21. Efficient Methods for Natural Language Processing: A Survey []
22. SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models []
23. A Survey on Evaluation of Large Language Models []
24. A Survey on Model Compression for Large Language Models []
25. Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models []
26. CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X []
27. Compresso: Structured Pruning with Collaborative Prompting Learns Compact Large Language Models []
28. Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers []
29. Efficient Guided Generation for Large Language Models []
30. Fine-Tuning Language Models with Just Forward Passes []
31. Flash-Decoding for long-context inference []
32. Gradient-Free Structured Pruning with Unlabeled Data []
33. H$_2$O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models []
34. Knowledge-preserving Pruning for Pre-trained Language Models without Retraining []
35. LLM in a flash: Efficient Large Language Model Inference with Limited Memory []
36. LLM-Pruner: On the Structural Pruning of Large Language Models []
37. LoRAShear: Efficient Large Language Model Structured Pruning and Knowledge Recovery []
38. LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models []
39. OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models []
40. Post-training Quantization for Neural Networks with Provable Guarantees []
41. PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU []
42. Pruning Large Language Models via Accuracy Predictor []
43. QLoRA: Efficient Finetuning of Quantized LLMs []
44. QuIP: Quantization with Incoherence Processing []
45. RPTQ: Reorder-based Post-training Quantization for Large Language Models []
46. Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning []
47. SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression []
48. Sparse Fine-tuning for Inference Acceleration of Large Language Models []
49. Sparse Iso-FLOP Transformations for Maximizing Training Efficiency []
50. Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging []
51. Ten Lessons We Have Learned in the New Sparseland: A Short Handbook for Sparse Neural Network Researchers []
52. The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter []
53. Training Transformers with 4-bit Integers []
54. Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering []
55. ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation []
56. FasterTransformer []

2022

1. Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm []
2. TextPruner: A Model Pruning Toolkit for Pre-Trained Language Models []
3. Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads []
4. Creating Sparse GPT-3 Models with Iterative Pruning []
5. LoRA: Low-rank adaptation of large language models []
6. SPDY: Accurate Pruning with Speedup Guarantees []
7. Sparse Attention Acceleration with Synergistic In-Memory Pruning and On-Chip Recomputation []
8. A Fast Post-Training Pruning Framework for Transformers []
9. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness []
10. Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning []
11. ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers []
12. Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networks []
13. Transformer Acceleration with Dynamic Sparse Attention []
14. An Algorithm-Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformers []
15. The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models []

2021

1. Post-training deep neural network pruning via layer-wise calibration []
2. BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction []
3. Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch []
4. A Greedy Algorithm for Quantizing Neural Networks []
5. Channel Permutations for N:M Sparsity []
6. Accelerating Sparse Deep Neural Networks []
7. Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks []

2020

1. Fast Sparse ConvNets []
2. Inducing and Exploiting Activation Sparsity for Fast Neural Network Inference []
3. Movement Pruning: Adaptive Sparsity by Fine-Tuning []
4. GPU Kernels for Block-Sparse Weights []

2019

1. ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training []

2018

1. A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers []

2017

1. DSD: Dense-Sparse-Dense Training for Deep Neural Networks []
2. Attention Is All You Need []
3. Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon []

2016

1. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding []

1993

1. Optimal Brain Surgeon and general network pruning []

1989

1. Optimal Brain Damage []

References

1. https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling []

2. https://github.com/weigao266/Awesome-Efficient-Arch []

3. https://github.com/horseee/Awesome-Efficient-LLM []

4. https://github.com/DefTruth/Awesome-Diffusion-Inference []

5. https://github.com/DefTruth/Awesome-LLM-Inference []

6. https://github.com/AmberLJC/LLMSys-PaperList []

7. https://github.com/Hannibal046/Awesome-LLM []

8. https://github.com/AmadeusChan/Awesome-LLM-System-Papers []

9. https://github.com/KnowingNothing/compiler-and-arch []

10. https://papercopilot.com/paper-list

11. https://github.com/TreeAI-Lab/Awesome-KV-Cache-Management []

12. https://github.com/October2001/Awesome-KV-Cache-Compression []

13. https://github.com/he-y/Awesome-Pruning []

14. https://github.com/htqin/awesome-model-quantization []

15. https://github.com/csyhhu/Awesome-Deep-Neural-Network-Compression []

16. https://github.com/AojunZhou/Efficient-Deep-Learning []

17. https://github.com/chester256/Model-Compression-Papers []