Skip to content

Commit

Permalink
[Fix] Export TORCH_CUDA_ARCH_LIST in install.sh (#133)
Browse files Browse the repository at this point in the history
* chore: update TORCH_CUDA_ARCH_LIST in install.sh

* chore: remove importlib from requirements.txt

* chore: Fix multi-gpu environment check in bitblas_target_detector.py

* chore: Update logger.info message in qlinear_bitblas.py

* Update install.sh

We like to not use PTX compilation for now.

---------

Co-authored-by: Qubitium-ModelCloud <qubitium@modelcloud.ai>
  • Loading branch information
LeiWang1999 and Qubitium authored Jul 1, 2024
1 parent e86f44c commit ac666be
Show file tree
Hide file tree
Showing 4 changed files with 3 additions and 4 deletions.
2 changes: 1 addition & 1 deletion gptqmodel/nn_modules/qlinear/bitblas_target_detector.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ def get_gpu_model_from_nvidia_smi(gpu_id: int = 0):

# for multiple cpus, CUDA_DEVICE_ORDER=PCI_BUS_ID must be set to match nvidia-smi or else gpu_id is
# most likely incorrect and the wrong gpu
if len(gpus) > 0 and os.environ.get("CUDA_DEVICE_ORDER") != "PCI_BUS_ID":
if len(gpus) > 1 and os.environ.get("CUDA_DEVICE_ORDER") != "PCI_BUS_ID":
raise EnvironmentError("Multi-gpu environment must set `CUDA_DEVICE_ORDER=PCI_BUS_ID`.")

if gpu_id >= len(gpus) or gpu_id < 0:
Expand Down
2 changes: 1 addition & 1 deletion gptqmodel/nn_modules/qlinear/qlinear_bitblas.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ def import_bitblas():

bitblas.auto_detect_nvidia_target = patched_auto_detect_nvidia_target
BITBLAS_TARGET = bitblas.auto_detect_nvidia_target(int(os.environ.get("CUDA_VISIBLE_DEVICES", "0")))
logger.info("BITBLAS_TARGET", BITBLAS_TARGET)
logger.info(f"BITBLAS_TARGET {BITBLAS_TARGET}")

if BITBLAS_DATABASE_PATH is None:
from bitblas.cache import get_database_path
Expand Down
2 changes: 1 addition & 1 deletion install.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash

export TORCH_CUDA_ARCH_LIST="6.0 6.1 7.0 7.5 8.0 8.6 8.9 9.0"
INSTALL_COMMAND="pip install -vvv --no-build-isolation ."

check_uv_version() {
Expand Down
1 change: 0 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,3 @@ threadpoolctl>=3.5.0
packaging>=24.1
ninja>=1.11.1.1
bitblas>=0.0.1.dev12
importlib-metadata

0 comments on commit ac666be

Please sign in to comment.