Add ROCM backend

[Ldpe2G]

可供参考分支：

#5410

编译流程：

登陆服务器，进入容器之后

编译环境配置

下载 cond 安装，然后按照指示配置 环境: https://github.com/Oneflow-Inc/conda-env#dev-env-for-oneflow

编译

进入到oneflow build目录，切到当前分支

source /home/env.sh

cmake .. -DBUILD_CUDA=OFF -DBUILD_ROCM=ON -DONEFLOW=ON -DUSE_CLANG_FORMAT=OFF -DCMAKE_BUILD_TYPE=Release -DTHIRD_PARTY=ON -DTREAT_WARNINGS_AS_ERRORS=OFF -DTHIRD_PARTY_MIRROR=aliyun -DBUILD_HWLOC=OFF -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++

make -j16

env.sh 的内容

#!/bin/bash

export ROCM_PATH=/opt/rocm
echo $ROCM_PATH
export HIP_PATH=${ROCM_PATH}/hip
export AMDGPU_TARGETS="gfx900;gfx906"
export PATH=${ROCM_PATH}/bin:${ROCM_PATH}/llvm/bin:${ROCM_PATH}/hcc/bin:${ROCM_PATH}/hip/bin:$PATH

export LD_LIBRARY_PATH=${ROCM_PATH}/lib:${ROCM_PATH}/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=${ROCM_PATH}/hip/lib:${ROCM_PATH}/llvm/lib:${ROCM_PATH}/opencl/lib/x86_64:$LD_LIBRARY_PATH
export C_INCLUDE_PATH=${ROCM_PATH}/include:${ROCM_PATH}/llvm/include:/opencl/include:${C_INCLUDE_PATH}
export CPLUS_INCLUDE_PATH=${ROCM_PATH}/include:${ROCM_PATH}/llvm/include:/opencl/include:${CPLUS_INCLUDE_PATH}
export PATH=${ROCM_PATH}/miopen/bin:${ROCM_PATH}/rocblas/bin:${ROCM_PATH}/hipsparse/bin:$PATH
export LD_LIBRARY_PATH=${ROCM_PATH}/miopen/lib:${ROCM_PATH}/rocblas/lib:$LD_LIBRARY_PATH
export MIOPEN_SYSTEM_DB_PATH=${ROCM_PATH}/miopen/share/miopen/db/
export LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH
export LIBRARY_PATH=/usr/lib64:$LIBRARY_PATH

export RCCL_PATH=$ROCM_PATH/rccl
export NCCL_PATH=$ROCM_PATH/rccl
export LD_LIBRARY_PATH=$RCCL_PATH/lib:$LD_LIBRARY_PATH

export MIOPEN_FIND_MODE=3

/opt/rocm/hip/samples/1_Utils/hipInfo/hipInfo
rocm-smi

[Ldpe2G]

cmake BUILID_TEST=ON 的编译步骤

进入容器后不要用 conda 环境，如果已经编译过了，先删除 build 目录，重新创建

source /home/env.sh

export CC=/opt/rocm/llvm/bin/clang
export CXX=/opt/rocm/llvm/bin/clang++

cmake .. -DBUILD_CUDA=OFF -DBUILD_ROCM=ON -DONEFLOW=ON -DUSE_CLANG_FORMAT=OFF -DCMAKE_BUILD_TYPE=Release -DTHIRD_PARTY=ON -DTREAT_WARNINGS_AS_ERRORS=OFF -DTHIRD_PARTY_MIRROR=aliyun -DBUILD_HWLOC=OFF -DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang -DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++ -DBUILD_TESTING=ON -DOF_SOFTMAX_USE_FAST_MATH=OFF

make -j32

注：

1、 如果编译中遇到 glog test 相关 clang 的编译报错，则手动修改 build 目录下的 _deps/glog-src/CMakeLists.txt 文件，在 640 行下加入 set(BUILD_TESTING OFF) 然后重新删除 CMakeCache.txt 再重新cmake，编译

2、 编译前先升级到 Python3.7 ，升级方式见 https://gist.github.com/wpupru/deda1cd96ea242d9a790e50cd0c97e9f

3、如果 cmake 报 blas 相关的错误，则要

yum install -y openblas openblas-devel