Error when converting "state-spaces/mamba2-130m" weights to huggingface-compatible format #32496
Closed
2 of 4 tasks
Labels
bug
Good Second Issue
Issues that are more difficult to do than "Good First" issues - give it a try if you want!
System Info
Who can help?
@molbap @ArthurZucker
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
I tried to load https://huggingface.co/state-spaces/mamba2-130m into HF-compatible Mamba-2 (#32080), using the convert_mamba2_ssm_checkpoint_to_pytorch.py script. But the script assumes model weights to be in safetensors format:
transformers/src/transformers/models/mamba2/convert_mamba2_ssm_checkpoint_to_pytorch.py
Lines 32 to 35 in 984bc11
but the weight file is is in torch bin format and cannot be opened in this way.
Also, the script requires a tokenizer path:
transformers/src/transformers/models/mamba2/convert_mamba2_ssm_checkpoint_to_pytorch.py
Lines 55 to 61 in 984bc11
but
state-spaces/mamba2-130m
reusesEleutherAI/gpt-neox-20b
tokenizer instead of having its own.Expected behavior
convert_mamba2_ssm_checkpoint_to_pytorch.py
should be able to convert those Mamba-2 weights:The text was updated successfully, but these errors were encountered: