Code

fix(l4): fix fp8 logic on l4#2277

Code

Merged

OlivierDehaene merged 3 commits intomainhuggingface/text-generation-inference:mainfrom fix/l4_fp8huggingface/text-generation-inference:fix/l4_fp8Copy head branch name to clipboardJul 23, 2024

+11-6

Commits

Commits on Jul 22, 2024

• fix(l4): fix fp8 logic on l4

  OlivierDehaene

  committed
  3d0c7b8

• also quant weights with single scale

  OlivierDehaene

  committed
  473f968

Commits on Jul 23, 2024

• use marlin even on 89

  OlivierDehaene

  committed
  025f80d