Code

Mamba + Tensor Parallel Support#1184

Code

Merged

Quentin-Anthony merged 9 commits intomainEleutherAI/gpt-neox:mainfrom tp-mamba-neoxEleutherAI/gpt-neox:tp-mamba-neoxCopy head branch name to clipboardMar 15, 2024

+84-75

Commits

Commits on Mar 12, 2024

• TP works!

  haileyschoelkopf

  committed
  c95b6e2

• merge TP mamba changes with most current MambaLayer

  haileyschoelkopf

  committed
  28bd76d

• cleanup TP, confirmed working still

  haileyschoelkopf

  committed
  e669681

• make shapes with TP>1 work with conversion

  haileyschoelkopf

  committed
  f544092

• tested and PP works, so no need for assert blocking it in arguments

  haileyschoelkopf

  committed
  f82db1c

• update comment

  haileyschoelkopf

  committed
  696454f

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  4bc39c3

Commits on Mar 15, 2024

• Merge branch 'main' into tp-mamba-neox

  Quentin-Anthony

  authored
  f70c54d

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  417f885