[Inference] FP8 dual gemm auto-tune and support compile parallelizati…

…on (#9151) * fp8 * check * check * check * check * cutlass fp8 * fp8 chech * check * ffn1 tune * delete * check * change file path * top_p_sampling_reject.cu
PaddlePaddle · Sep 20, 2024 · c4f7acf · c4f7acf
1 parent 25a5b4f
commit c4f7acf
Show file tree

Hide file tree

Showing 22 changed files with 1,000 additions and 1,010 deletions.
diff --git a/csrc/README.md b/csrc/README.md
@@ -10,9 +10,11 @@ pip install -r requirements.txt
 
 ## 编译 Cuda 算子
 
-生成 FP8的 cutlass 算子(编译耗时较长)
+生成 FP8的 cutlass 算子
 ```shell
-python generate_code_gemm_fused_kernels.py
+python utils/auto_gen_fp8_fp8_gemm_fused_kernels.py
+
+python utils/auto_gen_fp8_fp8_dual_gemm_fused_kernels.py
 ```
 
 编译

diff --git a/csrc/gpu/cutlass_kernels/fp8_gemm_fused/dispatch_dual_gemm_scale_bias_swiglu.h b/csrc/gpu/cutlass_kernels/fp8_gemm_fused/dispatch_dual_gemm_scale_bias_swiglu.h
diff --git a/csrc/gpu/cutlass_kernels/fp8_gemm_fused/dual_gemm_scale_bias_swiglu_16_32_64_stages3.h b/csrc/gpu/cutlass_kernels/fp8_gemm_fused/dual_gemm_scale_bias_swiglu_16_32_64_stages3.h