Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bitsandbytes can't handle multiple path locations in LD_LIBRARY_PATH #1112

Open
GuardMoony opened this issue Mar 6, 2024 · 2 comments
Open

bitsandbytes can't handle multiple path locations in LD_LIBRARY_PATH #1112

GuardMoony opened this issue Mar 6, 2024 · 2 comments

Comments

@GuardMoony
Copy link

GuardMoony commented Mar 6, 2024 •
edited
Loading

System Info

System Info
Ubuntu 20.04.6 LTS
CUDA Version: 12
Docker: dustynv/l4t-pytorch:r36.2.0
Python 3.10
Hardware:
Jetson orin 16GB (arm64)

Reproduction

Default pip install of package. Both 0.42.0 and 0.41.1 gave the same results:

`root@4b38f71b25b9:/models# python3 -m bitsandbytes
False

===================================BUG REPORT===================================
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: Welcome to bitsandbytes. For bug reports, please run

python -m bitsandbytes

warn(msg)

/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:166: UserWarning: /usr/local/cuda/compat:/usr/local/cuda/lib64: did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
The following directories listed in your path were found to be non-existent: {PosixPath('/data/models/torch')}
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
DEBUG: Possible options found for libcudart.so: {PosixPath('/usr/local/cuda/lib64/libcudart.so')}
CUDA SETUP: PyTorch settings found: CUDA_VERSION=122, Highest Compute Capability: 8.7.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
CUDA SETUP: Loading binary /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda122.so...
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda122.so: cannot open shared object file: No such file or directory
CUDA SETUP: Something unexpected happened. Please compile from source:
git clone https://github.com/TimDettmers/bitsandbytes.git
cd bitsandbytes
CUDA_VERSION=122
python setup.py install
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/lib/python3.10/runpy.py", line 146, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "/usr/lib/python3.10/runpy.py", line 110, in _get_module_details
import(pkg_name)
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/init.py", line 6, in
from . import cuda_setup, utils, research
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/research/init.py", line 1, in
from . import nn
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/research/nn/init.py", line 1, in
from .modules import LinearFP8Mixed, LinearFP8Global
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/research/nn/modules.py", line 8, in
from bitsandbytes.optim import GlobalOptimManager
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/optim/init.py", line 6, in
from bitsandbytes.cextension import COMPILED_WITH_CUDA
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cextension.py", line 20, in
raise RuntimeError('''
RuntimeError:
CUDA Setup failed despite GPU being available. Please run the following command to get more information:

    python -m bitsandbytes

    Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
    to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
    and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

root@4b38f71b25b9:/models# echo $LD_LIBRARY_PATH
/usr/local/cuda/compat:/usr/local/cuda/lib64:

root@4b38f71b25b9:/models# ls -al /usr/local/cuda/lib64
lrwxrwxrwx 1 root root 25 Aug 16 2023 /usr/local/cuda/lib64 -> targets/aarch64-linux/lib
root@4b38f71b25b9:/models# ls -al /usr/local/cuda/lib64/
total 4088260
drwxr-xr-x 4 root root 4096 Nov 10 01:45 .
drwxr-xr-x 5 root root 4096 Nov 10 01:45 ..
drwxr-xr-x 6 root root 4096 Nov 10 01:43 cmake
-rw-r--r-- 1 root root 1092664 Aug 17 2023 libcheckpoint.so
lrwxrwxrwx 1 root root 15 Aug 16 2023 libcublas.so -> libcublas.so.12
lrwxrwxrwx 1 root root 21 Aug 16 2023 libcublas.so.12 -> libcublas.so.12.2.5.6
-rw-r--r-- 1 root root 122347448 Aug 16 2023 libcublas.so.12.2.5.6
lrwxrwxrwx 1 root root 17 Aug 16 2023 libcublasLt.so -> libcublasLt.so.12
lrwxrwxrwx 1 root root 23 Aug 16 2023 libcublasLt.so.12 -> libcublasLt.so.12.2.5.6
-rw-r--r-- 1 root root 454665808 Aug 16 2023 libcublasLt.so.12.2.5.6
-rw-r--r-- 1 root root 662876174 Aug 16 2023 libcublasLt_static.a
-rw-r--r-- 1 root root 206596410 Aug 16 2023 libcublas_static.a
-rw-r--r-- 1 root root 1794146 Aug 16 2023 libcudadevrt.a
lrwxrwxrwx 1 root root 15 Aug 16 2023 libcudart.so -> libcudart.so.12
lrwxrwxrwx 1 root root 21 Aug 16 2023 libcudart.so.12 -> libcudart.so.12.2.140
-rw-r--r-- 1 root root 747424 Aug 16 2023 libcudart.so.12.2.140
-rw-r--r-- 1 root root 1403952 Aug 16 2023 libcudart_static.a
lrwxrwxrwx 1 root root 13 Aug 16 2023 libcudla.so -> libcudla.so.1
lrwxrwxrwx 1 root root 17 Aug 16 2023 libcudla.so.1 -> libcudla.so.1.0.0
-rw-r--r-- 1 root root 80832 Aug 16 2023 libcudla.so.1.0.0
lrwxrwxrwx 1 root root 14 Aug 16 2023 libcufft.so -> libcufft.so.11
lrwxrwxrwx 1 root root 22 Aug 16 2023 libcufft.so.11 -> libcufft.so.11.0.8.103
-rw-r--r-- 1 root root 178138720 Aug 16 2023 libcufft.so.11.0.8.103
-rw-r--r-- 1 root root 197350426 Aug 16 2023 libcufft_static.a
-rw-r--r-- 1 root root 197226444 Aug 16 2023 libcufft_static_nocallback.a
lrwxrwxrwx 1 root root 15 Aug 16 2023 libcufftw.so -> libcufftw.so.11
lrwxrwxrwx 1 root root 23 Aug 16 2023 libcufftw.so.11 -> libcufftw.so.11.0.8.103
-rw-r--r-- 1 root root 1390896 Aug 16 2023 libcufftw.so.11.0.8.103
-rw-r--r-- 1 root root 34662 Aug 16 2023 libcufftw_static.a
lrwxrwxrwx 1 root root 14 Aug 15 2023 libcufile.so -> libcufile.so.0
lrwxrwxrwx 1 root root 18 Aug 15 2023 libcufile.so.0 -> libcufile.so.1.7.2
-rw-r--r-- 1 root root 2818824 Aug 15 2023 libcufile.so.1.7.2
-rw-r--r-- 1 root root 18068490 Aug 15 2023 libcufile_static.a
-rw-r--r-- 1 root root 1290832 Aug 16 2023 libcufilt.a
-rw-r--r-- 1 root root 36666 Aug 16 2023 libculibos.a
lrwxrwxrwx 1 root root 14 Aug 17 2023 libcupti.so -> libcupti.so.12
lrwxrwxrwx 1 root root 20 Aug 17 2023 libcupti.so.12 -> libcupti.so.2023.2.2
-rw-r--r-- 1 root root 6539528 Aug 17 2023 libcupti.so.2023.2.2
lrwxrwxrwx 1 root root 15 Aug 16 2023 libcurand.so -> libcurand.so.10
lrwxrwxrwx 1 root root 23 Aug 16 2023 libcurand.so.10 -> libcurand.so.10.3.3.141
-rw-r--r-- 1 root root 109807304 Aug 16 2023 libcurand.so.10.3.3.141
-rw-r--r-- 1 root root 109957836 Aug 16 2023 libcurand_static.a
lrwxrwxrwx 1 root root 17 Aug 16 2023 libcusolver.so -> libcusolver.so.11
lrwxrwxrwx 1 root root 25 Aug 16 2023 libcusolver.so.11 -> libcusolver.so.11.5.2.141
-rw-r--r-- 1 root root 136591680 Aug 16 2023 libcusolver.so.11.5.2.141
lrwxrwxrwx 1 root root 19 Aug 16 2023 libcusolverMg.so -> libcusolverMg.so.11
lrwxrwxrwx 1 root root 27 Aug 16 2023 libcusolverMg.so.11 -> libcusolverMg.so.11.5.2.141
-rw-r--r-- 1 root root 99392712 Aug 16 2023 libcusolverMg.so.11.5.2.141
-rw-r--r-- 1 root root 19976762 Aug 16 2023 libcusolver_lapack_static.a
-rw-r--r-- 1 root root 964202 Aug 16 2023 libcusolver_metis_static.a
-rw-r--r-- 1 root root 153694962 Aug 16 2023 libcusolver_static.a
lrwxrwxrwx 1 root root 17 Aug 16 2023 libcusparse.so -> libcusparse.so.12
lrwxrwxrwx 1 root root 25 Aug 16 2023 libcusparse.so.12 -> libcusparse.so.12.1.2.141
-rw-r--r-- 1 root root 288405656 Aug 16 2023 libcusparse.so.12.1.2.141
-rw-r--r-- 1 root root 328978168 Aug 16 2023 libcusparse_static.a
-rw-r--r-- 1 root root 964202 Aug 16 2023 libmetis_static.a
lrwxrwxrwx 1 root root 13 Aug 16 2023 libnppc.so -> libnppc.so.12
lrwxrwxrwx 1 root root 19 Aug 16 2023 libnppc.so.12 -> libnppc.so.12.2.1.4
-rw-r--r-- 1 root root 1390936 Aug 16 2023 libnppc.so.12.2.1.4
-rw-r--r-- 1 root root 29142 Aug 16 2023 libnppc_static.a
lrwxrwxrwx 1 root root 15 Aug 16 2023 libnppial.so -> libnppial.so.12
lrwxrwxrwx 1 root root 21 Aug 16 2023 libnppial.so.12 -> libnppial.so.12.2.1.4
-rw-r--r-- 1 root root 18180800 Aug 16 2023 libnppial.so.12.2.1.4
-rw-r--r-- 1 root root 21128862 Aug 16 2023 libnppial_static.a
lrwxrwxrwx 1 root root 15 Aug 16 2023 libnppicc.so -> libnppicc.so.12
lrwxrwxrwx 1 root root 21 Aug 16 2023 libnppicc.so.12 -> libnppicc.so.12.2.1.4
-rw-r--r-- 1 root root 7543512 Aug 16 2023 libnppicc.so.12.2.1.4
-rw-r--r-- 1 root root 7700974 Aug 16 2023 libnppicc_static.a
lrwxrwxrwx 1 root root 16 Aug 16 2023 libnppidei.so -> libnppidei.so.12
lrwxrwxrwx 1 root root 22 Aug 16 2023 libnppidei.so.12 -> libnppidei.so.12.2.1.4
-rw-r--r-- 1 root root 11852840 Aug 16 2023 libnppidei.so.12.2.1.4
-rw-r--r-- 1 root root 14214298 Aug 16 2023 libnppidei_static.a
lrwxrwxrwx 1 root root 14 Aug 16 2023 libnppif.so -> libnppif.so.12
lrwxrwxrwx 1 root root 20 Aug 16 2023 libnppif.so.12 -> libnppif.so.12.2.1.4
-rw-r--r-- 1 root root 111792064 Aug 16 2023 libnppif.so.12.2.1.4
-rw-r--r-- 1 root root 116179074 Aug 16 2023 libnppif_static.a
lrwxrwxrwx 1 root root 14 Aug 16 2023 libnppig.so -> libnppig.so.12
lrwxrwxrwx 1 root root 20 Aug 16 2023 libnppig.so.12 -> libnppig.so.12.2.1.4
-rw-r--r-- 1 root root 44706592 Aug 16 2023 libnppig.so.12.2.1.4
-rw-r--r-- 1 root root 46975612 Aug 16 2023 libnppig_static.a
lrwxrwxrwx 1 root root 14 Aug 16 2023 libnppim.so -> libnppim.so.12
lrwxrwxrwx 1 root root 20 Aug 16 2023 libnppim.so.12 -> libnppim.so.12.2.1.4
-rw-r--r-- 1 root root 11016920 Aug 16 2023 libnppim.so.12.2.1.4
-rw-r--r-- 1 root root 10520554 Aug 16 2023 libnppim_static.a
lrwxrwxrwx 1 root root 15 Aug 16 2023 libnppist.so -> libnppist.so.12
lrwxrwxrwx 1 root root 21 Aug 16 2023 libnppist.so.12 -> libnppist.so.12.2.1.4
-rw-r--r-- 1 root root 43740128 Aug 16 2023 libnppist.so.12.2.1.4
-rw-r--r-- 1 root root 46249544 Aug 16 2023 libnppist_static.a
lrwxrwxrwx 1 root root 15 Aug 16 2023 libnppisu.so -> libnppisu.so.12
lrwxrwxrwx 1 root root 21 Aug 16 2023 libnppisu.so.12 -> libnppisu.so.12.2.1.4
-rw-r--r-- 1 root root 731040 Aug 16 2023 libnppisu.so.12.2.1.4
-rw-r--r-- 1 root root 16066 Aug 16 2023 libnppisu_static.a
lrwxrwxrwx 1 root root 15 Aug 16 2023 libnppitc.so -> libnppitc.so.12
lrwxrwxrwx 1 root root 21 Aug 16 2023 libnppitc.so.12 -> libnppitc.so.12.2.1.4
-rw-r--r-- 1 root root 5609984 Aug 16 2023 libnppitc.so.12.2.1.4
-rw-r--r-- 1 root root 5045656 Aug 16 2023 libnppitc_static.a
lrwxrwxrwx 1 root root 13 Aug 16 2023 libnpps.so -> libnpps.so.12
lrwxrwxrwx 1 root root 19 Aug 16 2023 libnpps.so.12 -> libnpps.so.12.2.1.4
-rw-r--r-- 1 root root 22359176 Aug 16 2023 libnpps.so.12.2.1.4
-rw-r--r-- 1 root root 22911758 Aug 16 2023 libnpps_static.a
lrwxrwxrwx 1 root root 18 Aug 16 2023 libnvJitLink.so -> libnvJitLink.so.12
lrwxrwxrwx 1 root root 24 Aug 16 2023 libnvJitLink.so.12 -> libnvJitLink.so.12.2.140
-rw-r--r-- 1 root root 46806856 Aug 16 2023 libnvJitLink.so.12.2.140
-rw-r--r-- 1 root root 60658502 Aug 16 2023 libnvJitLink_static.a
lrwxrwxrwx 1 root root 18 Aug 16 2023 libnvToolsExt.so -> libnvToolsExt.so.1
lrwxrwxrwx 1 root root 22 Aug 16 2023 libnvToolsExt.so.1 -> libnvToolsExt.so.1.0.0
-rw-r--r-- 1 root root 72576 Aug 16 2023 libnvToolsExt.so.1.0.0
lrwxrwxrwx 1 root root 15 Aug 16 2023 libnvblas.so -> libnvblas.so.12
lrwxrwxrwx 1 root root 21 Aug 16 2023 libnvblas.so.12 -> libnvblas.so.12.2.5.6
-rw-r--r-- 1 root root 780280 Aug 16 2023 libnvblas.so.12.2.5.6
-rw-r--r-- 1 root root 27067776 Aug 17 2023 libnvperf_host.so
-rw-r--r-- 1 root root 4462120 Aug 17 2023 libnvperf_target.so
-rw-r--r-- 1 root root 40360388 Aug 16 2023 libnvptxcompiler_static.a
lrwxrwxrwx 1 root root 25 Aug 16 2023 libnvrtc-builtins.so -> libnvrtc-builtins.so.12.2
lrwxrwxrwx 1 root root 29 Aug 16 2023 libnvrtc-builtins.so.12.2 -> libnvrtc-builtins.so.12.2.140
-rw-r--r-- 1 root root 2434768 Aug 16 2023 libnvrtc-builtins.so.12.2.140
-rw-r--r-- 1 root root 2458292 Aug 16 2023 libnvrtc-builtins_static.a
lrwxrwxrwx 1 root root 14 Aug 16 2023 libnvrtc.so -> libnvrtc.so.12
lrwxrwxrwx 1 root root 20 Aug 16 2023 libnvrtc.so.12 -> libnvrtc.so.12.2.140
-rw-r--r-- 1 root root 55086000 Aug 16 2023 libnvrtc.so.12.2.140
-rw-r--r-- 1 root root 72214234 Aug 16 2023 libnvrtc_static.a
-rw-r--r-- 1 root root 698784 Aug 17 2023 libpcsamplingutil.so
drwxr-xr-x 2 root root 4096 Nov 10 01:45 stubs`

When setting only 1 path location:

`root@4b38f71b25b9:/models# LD_LIBRARY_PATH=/usr/local/cuda/lib64 python3 -m bitsandbytes
/usr/local/lib/python3.10/dist-packages/torch/cuda/init.py:138: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 11040). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at /tmp/pytorch/c10/cuda/CUDAFunctions.cpp:108.)
return torch._C._cuda_getDeviceCount() > 0
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
warn("The installed version of bitsandbytes was compiled without GPU support. "
'NoneType' object has no attribute 'cadam32bit_grad_fp32'
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++ BUG REPORT INFORMATION ++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

++++++++++++++++++ /usr/local CUDA PATHS +++++++++++++++++++
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda115_nocublaslt.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda114.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda117.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda117_nocublaslt.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda114_nocublaslt.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda115.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda120_nocublaslt.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda118_nocublaslt.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda122.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda122_nocublaslt.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda111_nocublaslt.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda110_nocublaslt.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda121_nocublaslt.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda120.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda121.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda110.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda111.so
/usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cuda_linalg.so
/usr/local/lib/python3.10/dist-packages/torch/lib/libc10_cuda.so
/usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cuda.so
/usr/local/cuda-12.2/compat/libcuda.so
/usr/local/cuda-12.2/targets/aarch64-linux/lib/libcudart.so
/usr/local/cuda-12.2/targets/aarch64-linux/lib/stubs/libcuda.so

+++++++++++++++ WORKING DIRECTORY CUDA PATHS +++++++++++++++

++++++++++++++++++ LD_LIBRARY CUDA PATHS +++++++++++++++++++
+++++++++++++ /usr/local/cuda/lib64 CUDA PATHS +++++++++++++

++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
COMPILED_WITH_CUDA = False
COMPUTE_CAPABILITIES_PER_GPU = []
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Running a quick check that:
+ library is importable
+ CUDA function is callable

WARNING: Please be sure to sanitize sensible info from any such env vars!

The NVIDIA driver on your system is too old (found version 11040). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver.

Above we output some debug information. Please provide this info when creating an issue via https://github.com/TimDettmers/bitsandbytes/issues/new/choose ...`

Expected behavior

By default the application should run through the list of paths to find the required files.
@akx
Copy link
Contributor

akx commented Mar 7, 2024

This will be likely alleviated by #1041 some, but it should be noted that no version of bitsandbytes will actually even try to add any of the directories it might find libcudart in into your search path by itself.

@GuardMoony
Copy link
Author

Re-ran test with pull request code:

`root@4b38f71b25b9:/opt/bitsandbytes# python3 -m bitsandbytes
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++ BUG REPORT INFORMATION ++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
CUDA specs: CUDASpecs(highest_compute_capability=(8, 7), cuda_version_string='122', cuda_version_tuple=(12, 2))
PyTorch settings found: CUDA_VERSION=122, Highest Compute Capability: (8, 7).
To manually override the PyTorch CUDA version please see: https://github.com/TimDettmers/bitsandbytes/blob/main/docs/source/nonpytorchcuda.mdx
The directory listed in your path is found to be non-existent: /data/models/torch
Found duplicate CUDA runtime files (see below).

We select the PyTorch default CUDA runtime, which is 12.2,
but this might mismatch with the CUDA version that is needed for bitsandbytes.
To override this behavior set the BNB_CUDA_VERSION=<version string, e.g. 122> environmental variable.

For example, if you want to use the CUDA version 122,
BNB_CUDA_VERSION=122 python ...

OR set the environmental variable in your .bashrc:
export BNB_CUDA_VERSION=122

In the case of a manual override, make sure you set LD_LIBRARY_PATH, e.g.
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2,

  • Found CUDA runtime at: /usr/local/cuda/lib64/libcudart.so.12
  • Found CUDA runtime at: /usr/local/cuda/lib64/libcudart.so
  • Found CUDA runtime at: /usr/local/cuda/lib64/libcudart.so.12.2.140
    ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    ++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
    ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    Checking that the library is importable and CUDA is callable...
    SUCCESS!
    Installation was successful!`

This does indeed seem to fix the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@akx @GuardMoony