[Installation]: how to install 0.8.5 with cuda11.8

[zh794390558]

how to build vllm and xformers for cuda11.8, and lower pytorch version?

[reidliu41]

For vllm, you try this:
https://docs.vllm.ai/en/latest/getting_started/installation/gpu.html#pre-built-wheels

[zh794390558]

VLLM_VERSION=0.6.1.post1 need for cuda11.8，but I want to run Qwen3 which need 0.8.5 version ? How can i build 0.8.5 with cuda11.8? I always encounter xformers building error!

[zh794390558]

I have build vllm with local cuda11.7 and gcc 12, but encounter this problem, how to fix it?

Python 3.10.17

ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /mnt/vllm-env/lib/python3.10/site-packages/vllm/_C.so)

[zh794390558]

@reidliu41 maybe like this #2072 #14608.

Why not provide pre-build vllm for CUDA11.8? And why update pytorch deps to latest version?

[p208p2002]

I have build vllm with local cuda11.7 and gcc 12, but encounter this problem, how to fix it?

Python 3.10.17

ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /mnt/vllm-env/lib/python3.10/site-packages/vllm/_C.so)

Try update the Linux distribution (e.g. ubuntu 20.04 -> 22.04)

[zh794390558]

I have build vllm with local cuda11.7 and gcc 12, but encounter this problem, how to fix it?
Python 3.10.17

ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /mnt/vllm-env/lib/python3.10/site-packages/vllm/_C.so)

Try update the Linux distribution (e.g. ubuntu 20.04 -> 22.04)

I using Ubuntu 18.6, and I can not change my environment, or I will update to CUDA12.6.

[zh794390558]

Does vllm can officially relax restrictions of CUDA and Torch version? vllm deps xformers which using different compiler version.

[reidliu41]

VLLM_VERSION=0.6.1.post1 need for cuda11.8，but I want to run Qwen3 which need 0.8.5 version ? How can i build 0.8.5 with cuda11.8? I always encounter xformers building error!

Yes, https://github.com/vllm-project/vllm/releases/tag/v0.8.5 just support Qwen3 in 0.8.5.

Install vLLM with CUDA 11.8.
export VLLM_VERSION=0.6.1.post1

Just see the version for 11.8, not sure other version support, you can test or wait for more inputs

[zh794390558]

Using default gcc 8.2.0, I can not install vllm 0.8.5. But using gcc 10.1.0 it is succeed.

Xformers has many warning:

WARNING[XFORMERS]: Need to compile C++ extensions to use all xFormers features.
    Please install xformers properly (see https://github.com/facebookresearch/xformers#installing-xformers)
  Memory-efficient attention, SwiGLU, sparse and more won't be available.
  Set XFORMERS_MORE_DETAILS=1 for more details
xFormers 0.0.0
memory_efficient_attention.ckF:                    unavailable
memory_efficient_attention.ckB:                    unavailable
memory_efficient_attention.ck_decoderF:            unavailable
memory_efficient_attention.ck_splitKF:             unavailable
memory_efficient_attention.cutlassF-pt:            available
memory_efficient_attention.cutlassB-pt:            available
memory_efficient_attention.fa2F@v2.5.7-pt:         available
memory_efficient_attention.fa2B@v2.5.7-pt:         available
memory_efficient_attention.fa3F@0.0.0:             unavailable
memory_efficient_attention.fa3B@0.0.0:             unavailable
memory_efficient_attention.triton_splitKF:         available
indexing.scaled_index_addF:                        available
indexing.scaled_index_addB:                        available
indexing.index_select:                             available
sequence_parallel_fused.write_values:              unavailable
sequence_parallel_fused.wait_values:               unavailable
sequence_parallel_fused.cuda_memset_32b_async:     unavailable
sp24.sparse24_sparsify_both_ways:                  unavailable
sp24.sparse24_apply:                               unavailable
sp24.sparse24_apply_dense_output:                  unavailable
sp24._sparse24_gemm:                               unavailable
sp24._cslt_sparse_mm_search@0.4.0:                 available
sp24._cslt_sparse_mm@0.4.0:                        available
swiglu.dual_gemm_silu:                             unavailable
swiglu.gemm_fused_operand_sum:                     unavailable
swiglu.fused.p.cpp:                                not built
is_triton_available:                               True
pytorch.version:                                   2.6.0+cu118
pytorch.cuda:                                      available
gpu.compute_capability:                            8.0
gpu.name:                                          NVIDIA A100-SXM4-40GB
dcgm_profiler:                                     unavailable
build.info:                                        none
source.privacy:                                    open source

but xformers 0.0.27.post2 has no warning:

xFormers 0.0.27.post2
memory_efficient_attention.ckF:                    unavailable
memory_efficient_attention.ckB:                    unavailable
memory_efficient_attention.ck_decoderF:            unavailable
memory_efficient_attention.ck_splitKF:             unavailable
memory_efficient_attention.cutlassF-pt:            available
memory_efficient_attention.cutlassB-pt:            available
memory_efficient_attention.decoderF:               available
memory_efficient_attention.flshattF@2.5.6-pt:      available
memory_efficient_attention.flshattB@2.5.6-pt:      available
memory_efficient_attention.smallkF:                available
memory_efficient_attention.smallkB:                available
memory_efficient_attention.triton_splitKF:         available
indexing.scaled_index_addF:                        available
indexing.scaled_index_addB:                        available
indexing.index_select:                             available
sequence_parallel_fused.write_values:              available
sequence_parallel_fused.wait_values:               available
sequence_parallel_fused.cuda_memset_32b_async:     available
sp24.sparse24_sparsify_both_ways:                  available
sp24.sparse24_apply:                               available
sp24.sparse24_apply_dense_output:                  available
sp24._sparse24_gemm:                               available
sp24._cslt_sparse_mm@0.5.2:                        available
swiglu.dual_gemm_silu:                             available
swiglu.gemm_fused_operand_sum:                     available
swiglu.fused.p.cpp:                                available
is_triton_available:                               True
pytorch.version:                                   2.4.0+cu121
pytorch.cuda:                                      available
gpu.compute_capability:                            8.0
gpu.name:                                          NVIDIA A100-SXM4-40GB
dcgm_profiler:                                     unavailable
build.info:                                        available
build.cuda_version:                                1201
build.hip_version:                                 None
build.python_version:                              3.11.9
build.torch_version:                               2.4.0+cu121
build.env.TORCH_CUDA_ARCH_LIST:                    6.0+PTX 7.0 7.5 8.0+PTX
build.env.PYTORCH_ROCM_ARCH:                       None
build.env.XFORMERS_BUILD_TYPE:                     Release
build.env.XFORMERS_ENABLE_DEBUG_ASSERTIONS:        None
build.env.NVCC_FLAGS:                              None
build.env.XFORMERS_PACKAGE_FROM:                   wheel-v0.0.27.post2
build.nvcc_version:                                12.1.66
source.privacy:                                    open source

Why cuda12.1 not support ?

Can I using flash-atten instead of xformers?

[datalee]

VLLM_VERSION=0.6.1.post1 need for cuda11.8，but I want to run Qwen3 which need 0.8.5 version ? How can i build 0.8.5 with cuda11.8? I always encounter xformers building error!

搞定没？