Skip to content

Commit 88483a6

Browse files
hongxiayangjimpang
authored andcommitted
[ROCm] Fix build problem resulted from previous commit related to FP8 kv-cache support (vllm-project#2790)
1 parent 6958f2f commit 88483a6

File tree

2 files changed

+16
-0
lines changed

2 files changed

+16
-0
lines changed

Dockerfile.rocm

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ RUN cd /app \
7676
&& cd vllm \
7777
&& pip install -U -r requirements-rocm.txt \
7878
&& bash patch_xformers.rocm.sh \
79+
&& patch /opt/rocm/include/hip/amd_detail/amd_hip_bf16.h /app/vllm/rocm_patch/rocm_bf16.patch \
7980
&& python3 setup.py install \
8081
&& cd ..
8182

rocm_patch/rocm_bf16.patch

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
--- amd_hip_bf16.h 2024-02-06 18:28:58.268699142 +0000
2+
+++ amd_hip_bf16.h.new 2024-02-06 18:28:31.988647133 +0000
3+
@@ -90,10 +90,10 @@
4+
#include "math_fwd.h" // ocml device functions
5+
6+
#if defined(__HIPCC_RTC__)
7+
-#define __HOST_DEVICE__ __device__
8+
+#define __HOST_DEVICE__ __device__ static
9+
#else
10+
#include <climits>
11+
-#define __HOST_DEVICE__ __host__ __device__
12+
+#define __HOST_DEVICE__ __host__ __device__ static inline
13+
#endif
14+
15+
// Since we are using unsigned short to represent data in bfloat16, it can be of different sizes on

0 commit comments

Comments
 (0)