[BUG] ModuleNotFoundError: No module named 'bitsandbytes.mlu_setup'

[CHaunceyshen]

transformers==4.52.4
accelerate==1.7.0
deepspeed==0.15.2

[rank6]: Traceback (most recent call last):
[rank6]: File "/srv/langshi/data/code/ddst/train.py", line 68, in
[rank6]: main()
[rank6]: File "/srv/langshi/data/code/ddst/train.py", line 64, in main
[rank6]: trainer.train()
[rank6]: File "/torch/venv3/pytorch_infer/lib/python3.10/site-packages/transformers/trainer.py", line 2240, in train
[rank6]: return inner_training_loop(
[rank6]: File "/torch/venv3/pytorch_infer/lib/python3.10/site-packages/transformers/trainer.py", line 2369, in _inner_training_loop
[rank6]: model, self.optimizer = self.accelerator.prepare(self.model, self.optimizer)
[rank6]: File "/torch/venv3/pytorch_infer/lib/python3.10/site-packages/accelerate/accelerator.py", line 1433, in prepare
[rank6]: result = self._prepare_deepspeed(*args)
[rank6]: File "/torch/venv3/pytorch_infer/lib/python3.10/site-packages/accelerate/accelerator.py", line 2018, in _prepare_deepspeed
[rank6]: optimizer = map_pytorch_optim_to_deepspeed(optimizer)
[rank6]: File "/torch/venv3/pytorch_infer/lib/python3.10/site-packages/accelerate/utils/deepspeed.py", line 71, in map_pytorch_optim_to_deepspeed
[rank6]: import bitsandbytes.optim as bnb_opt
[rank6]: File "/torch/venv3/pytorch_infer/lib/python3.10/site-packages/bitsandbytes/init.py", line 6, in
[rank6]: from . import mlu_setup, utils, research
[rank6]: File "/torch/venv3/pytorch_infer/lib/python3.10/site-packages/bitsandbytes/research/init.py", line 2, in
[rank6]: from .autograd._functions import (
[rank6]: File "/torch/venv3/pytorch_infer/lib/python3.10/site-packages/bitsandbytes/research/autograd/_functions.py", line 9, in
[rank6]: import bitsandbytes.functional as F
[rank6]: File "/torch/venv3/pytorch_infer/lib/python3.10/site-packages/bitsandbytes/functional.py", line 20, in
[rank6]: from .cextension import COMPILED_WITH_CUDA, lib
[rank6]: File "/torch/venv3/pytorch_infer/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 9, in
[rank6]: from bitsandbytes.mlu_setup.main import CUDASetup
[rank6]: ModuleNotFoundError: No module named 'bitsandbytes.mlu_setup'

[Flink-ddd]

Hi @CHaunceyshen ,

Thanks for the report. The traceback points to an issue with your bitsandbytes installation. It seems to be incorrectly searching for an MLU-specific module. Please try the following steps:

Perform a clean reinstall:
pip uninstall bitsandbytes -y && pip install bitsandbytes

If the issue continues, pin to a stable version:
pip install bitsandbytes==0.41.1

Let us know if this resolves the problem.

[loadams]

Thanks @Flink-ddd - I'll close this for now as your solution looks correct and there has been no follow up. @CHaunceyshen let us know if this doesn't resolve the issue for you.