Commits

Branch selector

 main

User selector

All users

Datepicker

All time

Commit History

Commits on Mar 15, 2024

• Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference(@NVIDIA etc)

  DefTruth

  authored
  3f7ddd0

  Copy full SHA for 3f7ddd0

Commits on Mar 10, 2024

• Add 📖Long Context Attention/KV Cache Optimization🔥🔥

  DefTruth

  authored
  1af0c91

  Copy full SHA for 1af0c91

• add 📖Long Context Attention/KV Cache Optimization🔥🔥

  DefTruth

  authored
  f3cc70e

  Copy full SHA for f3cc70e

Commits on Mar 8, 2024

• 🔥[QAQ] Quality Adaptive Quantization for LLM KV Cache(@@smail.nju.edu.cn)

  DefTruth

  authored
  ad7f8d5

  Copy full SHA for ad7f8d5

Commits on Mar 2, 2024

• 🔥[LightningAttention-2] A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models(@OpenNLPLab)

  DefTruth

  authored
  26b8229

  Copy full SHA for 26b8229

Commits on Mar 1, 2024

• [LLM-Viewer] LLM Inference Unveiled: Survey and Roofline Model Insights(@zhihang Yuan etc)

  DefTruth

  authored
  5b88965

  Copy full SHA for 5b88965

Commits on Feb 25, 2024

• Update README.md

  DefTruth

  authored
  034b490

  Copy full SHA for 034b490

Commits on Feb 23, 2024

• RelayAttention for Efficient Large Language Model Serving with Long System Prompts

  DefTruth

  authored
  667d7b7

  Copy full SHA for 667d7b7

Commits on Feb 17, 2024

• [Tensor Parallel] TP-AWARE DEQUANTIZATION(@IBM T.J. Watson Research Center)

  DefTruth

  authored
  0e8cfd4

  Copy full SHA for 0e8cfd4

• [Tensor Parallel] TP-AWARE DEQUANTIZATION(@IBM T.J. Watson Research Center)

  DefTruth

  authored
  f5d7460

  Copy full SHA for f5d7460

• QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference(@SqueezeBits Inc)

  DefTruth

  authored
  1a2d6bc

  Copy full SHA for 1a2d6bc

Commits on Feb 7, 2024

• Update README.md

  DefTruth

  authored
  2a1807b

  Copy full SHA for 2a1807b

• Decoding Speculative Decoding(@cs.wisc.edu)

  DefTruth

  authored
  1b21505

  Copy full SHA for 1b21505

• 🔥[LookaheadDecoding] Break the Sequential Dependency of LLM Inference Using LOOKAHEAD DECODING(@UCSD&Google&UC Berkeley)

  DefTruth

  authored
  51f43ee

  Copy full SHA for 51f43ee

Commits on Feb 6, 2024

• 🔥[Prompt Caching] Efficient Prompt Caching via Embedding Similarity

  DefTruth

  authored
  608a16e

  Copy full SHA for 608a16e

Commits on Feb 2, 2024

• KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization(@uc Berkeley)

  DefTruth

  authored
  5b2e225

  Copy full SHA for 5b2e225

Commits on Jan 29, 2024

• FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design(@microsoft etc)

  DefTruth

  authored
  d635065

  Copy full SHA for d635065

Commits on Jan 25, 2024

• INFERFLOW: AN EFFICIENT AND HIGHLY CONFIGURABLE INFERENCE ENGINE FOR LARGE LANGUAGE MODELS(@Tencent AI Lab)

  DefTruth

  authored
  f20ca62

  Copy full SHA for f20ca62

• Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

  DefTruth

  authored
  9784df8

  Copy full SHA for 9784df8

Commits on Jan 20, 2024

• DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

  DefTruth

  authored
  7c514ef

  Copy full SHA for 7c514ef

Commits on Jan 13, 2024

• FFSplit: Split Feed-Forward Network For Optimizing Accuracy-Efficiency Trade-off in Language Model Inference

  DefTruth

  authored
  a39bf9f

  Copy full SHA for a39bf9f

• MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts(@uw.edu.pl)

  DefTruth

  authored
  30fb302

  Copy full SHA for 30fb302

Commits on Jan 12, 2024

• Understanding LLMs: A Comprehensive Overview from Training to Inference

  DefTruth

  authored
  a95033d

  Copy full SHA for a95033d

Commits on Jan 8, 2024

• 🔥🔥[DistKV-LLM] Infinite-LLM: Efficient LLM Service for Long Context with DistAttention and Distributed KVCache(@alibaba etc)

  DefTruth

  authored
  5f3ecba

  Copy full SHA for 5f3ecba

• Fast and Optimal Weight Update for Pruned Large Language Models(@fmph.uniba.sk)

  DefTruth

  authored
  0f7e888

  Copy full SHA for 0f7e888

• Update README.md

  DefTruth

  authored
  eb4ff2b

  Copy full SHA for eb4ff2b

Commits on Jan 5, 2024

• Merge pull request DefTruth#2 from HuangLianghong/main

  Show description for fd700c9
  DefTruth

  authored
  fd700c9

  Copy full SHA for fd700c9

• Update README.md

  Show description for 1cbab90
  HuangLianghong

  authored
  1cbab90

  Copy full SHA for 1cbab90

Commits on Jan 2, 2024

• Update README.md

  DefTruth

  authored
  b6e0c50

  Copy full SHA for b6e0c50

• 🔥[LASER] The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction(@mit.edu)

  DefTruth

  authored
  e165ff7

  Copy full SHA for e165ff7

• [LASER] The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction(@mit.edu)

  DefTruth

  authored
  e5da5a8

  Copy full SHA for e5da5a8

• [LASER] The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction(@mit.edu)

  DefTruth

  authored
  8c692eb

  Copy full SHA for 8c692eb

Commits on Dec 31, 2023

• Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems

  DefTruth

  authored
  94502ab

  Copy full SHA for 94502ab

• 🔥 [Mixtral Offloading] Fast Inference of Mixture-of-Experts Language Models with Offloading

  DefTruth

  authored
  fa32b4b

  Copy full SHA for fa32b4b

Commits on Dec 29, 2023

• PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU(@sjtu)

  DefTruth

  authored
  3e16ddb

  Copy full SHA for 3e16ddb

Pagination

Previous

Next