Code

Exllama v2#1211

Code

Merged

Narsil merged 11 commits intomainhuggingface/text-generation-inference:mainfrom tmp_exllamav2huggingface/text-generation-inference:tmp_exllamav2Copy head branch name to clipboardNov 25, 2023

+3,144-20

Commits

Commits on Nov 25, 2023

• draft exllamav2, first commit

  

  flozi00

  authored and

  Narsil

  committed
  0ae4523

  Copy full SHA for 0ae4523

• build in docker

  

  flozi00

  authored and

  Narsil

  committed
  a02f683

  Copy full SHA for a02f683

• use exllamav2QuantLinear instead of exllama1

  

  flozi00

  authored and

  Narsil

  committed
  f96d997

  Copy full SHA for f96d997

• Update exllamav2 (illegal address issued)

  Narsil

  committed
  024bdb0

  Copy full SHA for 024bdb0

• Adding scratch space.

  Narsil

  committed
  fb64ce1

  Copy full SHA for fb64ce1

• Fixing exllamav2.

  

  Ubuntu

  authored and

  Narsil

  committed
  a61f432

  Copy full SHA for a61f432

• Exllamav2 functional.

  Narsil

  committed
  ff51589

  Copy full SHA for ff51589

• Fix imports.

  Narsil

  committed
  b041bf1

  Copy full SHA for b041bf1

• Deactivating v2 for sharded.

  Show description for a7ed31c
  Narsil

  committed
  a7ed31c

  Copy full SHA for a7ed31c

• Rebased.

  Narsil

  committed
  d9dffb5

  Copy full SHA for d9dffb5

• Keep exllamav1 for sharded flows.

  Narsil

  committed
  86009e2

  Copy full SHA for 86009e2