Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CUDA 12 branch to 23.08. #100

Merged
merged 8 commits into from
May 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/checks.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
uses: actions/checkout@v3
- name: Get PR Info
id: get-pr-info
uses: rapidsai/shared-action-workflows/get-pr-info@branch-23.06
uses: rapidsai/shared-action-workflows/get-pr-info@branch-23.08
- name: Run rapids-size-checker
if: ${{ inputs.enable_check_size }}
run: rapids-size-checker
Expand All @@ -60,7 +60,7 @@ jobs:
fetch-depth: 0
- name: Get PR Info
id: get-pr-info
uses: rapidsai/shared-action-workflows/get-pr-info@branch-23.06
uses: rapidsai/shared-action-workflows/get-pr-info@branch-23.08
- uses: actions/cache@v3
with:
path: ~/.cache/pre-commit
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/conda-cpp-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ jobs:
RAPIDS_BUILD_TYPE: ${{ inputs.build_type }}
PARALLEL_LEVEL: ${{ env.PARALLEL_LEVEL }}
steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/conda-cpp-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ jobs:
PARALLEL_LEVEL: ${{ env.PARALLEL_LEVEL }}
NVIDIA_VISIBLE_DEVICES: ${{ env.NVIDIA_VISIBLE_DEVICES }}
steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/conda-python-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ jobs:
RAPIDS_BUILD_TYPE: ${{ inputs.build_type }}
PARALLEL_LEVEL: ${{ env.PARALLEL_LEVEL }}
steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/conda-python-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ jobs:
PARALLEL_LEVEL: ${{ env.PARALLEL_LEVEL }}
NVIDIA_VISIBLE_DEVICES: ${{ env.NVIDIA_VISIBLE_DEVICES }}
steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/conda-upload-packages.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
env:
RAPIDS_BUILD_TYPE: ${{ inputs.build_type }}
steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/custom-job.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
PARALLEL_LEVEL: ${{ env.PARALLEL_LEVEL }}
NVIDIA_VISIBLE_DEVICES: ${{ env.NVIDIA_VISIBLE_DEVICES }}
steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand All @@ -69,7 +69,7 @@ jobs:
- name: Get PR Info
if: startsWith(github.ref_name, 'pull-request/')
id: get-pr-info
uses: rapidsai/shared-action-workflows/get-pr-info@branch-23.06
uses: rapidsai/shared-action-workflows/get-pr-info@branch-23.08
- name: Add PR Info
if: startsWith(github.ref_name, 'pull-request/')
run: |
Expand Down
47 changes: 14 additions & 33 deletions .github/workflows/wheels-manylinux-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ jobs:
RAPIDS_PY_VERSION: ${{ matrix.python }}

steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand All @@ -158,7 +158,7 @@ jobs:
persist-credentials: false

- name: Standardize repository information
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.06
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.08
with:
repo: ${{ inputs.repo }}
branch: ${{ inputs.branch }}
Expand All @@ -182,47 +182,30 @@ jobs:
ssh-key: ${{ secrets[inputs.extra-repo-deploy-key] }}
persist-credentials: false

- name: Set CTK-related vars from input CTK versions
uses: rapidsai/shared-action-workflows/wheel-ctk-name-gen@branch-23.06
with:
ctk: ${{ matrix.ctk }}
package-name: ${{ inputs.package-name }}

- name: Apply wheel modifications
- name: Build and repair the wheel
run: |
# Use gha-tools rapids-pip-wheel-version to generate wheel version then
# update the necessary files
versioneer_override="$(rapids-pip-wheel-version ${{ needs.wheel-epoch-timestamp.outputs.rapids_epoch_timestamp }})"

bash ci/release/apply_wheel_modifications.sh ${versioneer_override} ${{ env.RAPIDS_PY_WHEEL_CUDA_SUFFIX }}
PIP_CU_VERSION="$(rapids-wheel-ctk-name-gen ${{ matrix.ctk }})"

bash ci/release/apply_wheel_modifications.sh ${versioneer_override} "-${PIP_CU_VERSION}"
echo "The package name and/or version was modified in the package source. The git diff is:"
git diff

- name: Build and repair the wheel
run: |
pyenv global ${{ matrix.python }}

# Run before-wheel command
# This variable needs to be used as an un-evaluated string by downstream libraries
# The variable $PIP_CU_VERSION needs to be used as an unevaluated string by downstream libraries
# to support pulling wheels built from a previous workflow for different
# CUDA versions
PIP_CU_VERSION=$(echo ${{ env.RAPIDS_PY_WHEEL_CUDA_SUFFIX }} | sed "s/-//")
${{ inputs.before-wheel }}

cd "${{ inputs.package-dir }}"

# sccache configuration
export SCCACHE_S3_KEY_PREFIX=gha-cibw
export SCCACHE_REGION=us-east-2
export SCCACHE_IDLE_TIMEOUT=32768
export SCCACHE_BUCKET=rapids-sccache-east
export SCCACHE_S3_USE_SSL=true
export AWS_ACCESS_KEY_ID=${{ env.AWS_ACCESS_KEY_ID }}
export AWS_SECRET_ACCESS_KEY=${{ env.AWS_SECRET_ACCESS_KEY }}
export AWS_SESSION_TOKEN=${{ env.AWS_SESSION_TOKEN }}

# Set up skbuild options. Enable sccache in skbuild config options
export SKBUILD_CONFIGURE_OPTIONS="${{ inputs.skbuild-configure-options}} -DCMAKE_C_COMPILER_LAUNCHER=/usr/bin/sccache -DCMAKE_CXX_COMPILER_LAUNCHER=/usr/bin/sccache -DCMAKE_CUDA_COMPILER_LAUNCHER=/usr/bin/sccache"
export SKBUILD_CONFIGURE_OPTIONS="${{ inputs.skbuild-configure-options}}"
export SKBUILD_BUILD_OPTIONS="${{ inputs.skbuild-build-options }}"

# Set up for pip installation of dependencies from the nightly index
Expand All @@ -237,11 +220,7 @@ jobs:

# Repair the wheel
cd dist
if [[ "${{ matrix.arch }}" == "amd64" ]]; then
python -m auditwheel repair -w . ${{ inputs.package-name }}* --plat manylinux_2_17_x86_64
else
python -m auditwheel repair -w . ${{ inputs.package-name }}* --plat manylinux_2_31_aarch64
fi
python -m auditwheel repair -w . ${{ inputs.package-name }}*

if [[ "${{ matrix.arch }}" == "arm64" ]]; then
${{ inputs.post-repair-arm64 }}
Expand All @@ -254,8 +233,10 @@ jobs:
cd ..
mkdir final_dist
mv dist/${{ inputs.package-name }}*manylinux* final_dist

# rapids-upload-wheels-to-s3 uses rapids-package-name which implicitly relies
# on this variable being set
export RAPIDS_PY_WHEEL_NAME="${{ inputs.package-name }}_${PIP_CU_VERSION}"
rapids-upload-wheels-to-s3 final_dist
# Use a shell that loads the rc file so that we get the compiler settings
shell: bash -leo pipefail {0}

- name: Upload wheels to downloads.rapids.ai
run: rapids-upload-wheels-to-s3 "${{ inputs.package-dir }}"/final_dist
4 changes: 2 additions & 2 deletions .github/workflows/wheels-manylinux-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ jobs:
TWINE_PASSWORD: ${{ secrets.RAPIDSAI_PYPI_CI_PASSWORD }}
TWINE_REPOSITORY_URL: "https://pypi.k8s.rapids.ai/simple/"
steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand All @@ -64,7 +64,7 @@ jobs:
persist-credentials: false

- name: Standardize repository information
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.06
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.08
with:
repo: ${{ inputs.repo }}
branch: ${{ inputs.branch }}
Expand Down
18 changes: 8 additions & 10 deletions .github/workflows/wheels-manylinux-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ jobs:
CIBW_TEST_EXTRAS: "test"
CIBW_TEST_COMMAND: ${{ matrix.test-command }}
steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand All @@ -148,27 +148,25 @@ jobs:
persist-credentials: false

- name: Standardize repository information
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.06
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.08
with:
repo: ${{ inputs.repo }}
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}

- name: Set CTK-related vars from input CTK versions
uses: rapidsai/shared-action-workflows/wheel-ctk-name-gen@branch-23.06
with:
ctk: ${{ matrix.ctk }}
package-name: ${{ inputs.package-name }}

- name: Run citestwheel
run: |
# This variable needs to be used as an un-evaluated string by downstream libraries
PIP_CU_VERSION="$(rapids-wheel-ctk-name-gen ${{ matrix.ctk }})"

# The variable PIP_CU_VERSION needs to be used as an unevaluated string by downstream libraries
# to support pulling wheels built from a previous workflow for different
# CUDA versions
PIP_CU_VERSION=$(echo ${{ env.RAPIDS_PY_WHEEL_CUDA_SUFFIX }} | sed "s/-//")

export RAPIDS_BEFORE_TEST_COMMANDS_AMD64="${{ inputs.test-before-amd64 }}"
export RAPIDS_BEFORE_TEST_COMMANDS_ARM64="${{ inputs.test-before-arm64 }}"

# citestwheel.sh implicitly uses this to download previously built wheels
export RAPIDS_PY_WHEEL_NAME="${{ inputs.package-name }}_${PIP_CU_VERSION}"

/citestwheel.sh
27 changes: 11 additions & 16 deletions .github/workflows/wheels-pure-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ jobs:
RAPIDS_PY_VERSION: "3.10"

steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand All @@ -89,37 +89,32 @@ jobs:
persist-credentials: false

- name: Standardize repository information
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.06
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.08
with:
repo: ${{ inputs.repo }}
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}

- name: Set CTK-related vars from input CTK versions
uses: rapidsai/shared-action-workflows/wheel-ctk-name-gen@branch-23.06
with:
ctk: ${{ matrix.ctk }}
package-name: ${{ inputs.package-name }}

- name: Run before-wheel command
- name: Build pure python wheel with pip wheel
run: |
# This variable needs to be used as an un-evaluated string by downstream libraries
PIP_CU_VERSION="$(rapids-wheel-ctk-name-gen ${{ matrix.ctk }})"

# The variable PIP_CU_VERSION needs to be used as an unevaluated string by downstream libraries
# to support pulling wheels built from a previous workflow for different
# CUDA versions
PIP_CU_VERSION=$(echo ${RAPIDS_PY_WHEEL_CUDA_SUFFIX} | sed "s/-//")

${{ inputs.before-wheel }}

- name: Build pure python wheel with pip wheel
run: |
versioneer_override="$(rapids-pip-wheel-version ${{ needs.wheel-epoch-timestamp.outputs.rapids_epoch_timestamp }})"

bash ci/release/apply_wheel_modifications.sh ${versioneer_override} ${RAPIDS_PY_WHEEL_CUDA_SUFFIX}
bash ci/release/apply_wheel_modifications.sh ${versioneer_override} "-${PIP_CU_VERSION}"
echo "The package name and/or version was modified in the package source. The git diff is:"
git diff

python -m pip wheel -w ./dist ${{ inputs.package-dir }} --no-deps -vvv

- name: Upload wheels to downloads.rapids.ai
run: rapids-upload-wheels-to-s3 dist
# rapids-upload-wheels-to-s3 uses rapids-package-name which implicitly relies
# on this variable being set
export RAPIDS_PY_WHEEL_NAME="${{ inputs.package-name }}_${PIP_CU_VERSION}"
rapids-upload-wheels-to-s3 dist
4 changes: 2 additions & 2 deletions .github/workflows/wheels-pure-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ jobs:
TWINE_PASSWORD: ${{ secrets.RAPIDSAI_PYPI_CI_PASSWORD }}
TWINE_REPOSITORY_URL: "https://pypi.k8s.rapids.ai/simple/"
steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand All @@ -68,7 +68,7 @@ jobs:
persist-credentials: false

- name: Standardize repository information
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.06
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.08
with:
repo: ${{ inputs.repo }}
branch: ${{ inputs.branch }}
Expand Down
20 changes: 9 additions & 11 deletions .github/workflows/wheels-pure-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ jobs:
CIBW_TEST_COMMAND: ${{ inputs.test-unittest }}
PIP_EXTRA_INDEX_URL: "https://pypi.k8s.rapids.ai/simple"
steps:
- uses: aws-actions/configure-aws-credentials@v1-node16
- uses: aws-actions/configure-aws-credentials@v2
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
Expand All @@ -85,26 +85,24 @@ jobs:
persist-credentials: false

- name: Standardize repository information
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.06
uses: rapidsai/shared-action-workflows/rapids-github-info@branch-23.08
with:
repo: ${{ inputs.repo }}
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}

- name: Set CTK-related vars from input CTK versions
uses: rapidsai/shared-action-workflows/wheel-ctk-name-gen@branch-23.06
with:
ctk: ${{ matrix.ctk }}
package-name: ${{ inputs.package-name }}

- name: Run citestwheel
run: |
# This variable needs to be used as an un-evaluated string by downstream libraries
PIP_CU_VERSION="$(rapids-wheel-ctk-name-gen ${{ matrix.ctk }})"

# The variable PIP_CU_VERSION needs to be used as an unevaluated string by downstream libraries
# to support pulling wheels built from a previous workflow for different
# CUDA versions
PIP_CU_VERSION=$(echo ${RAPIDS_PY_WHEEL_CUDA_SUFFIX} | sed "s/-//")


export RAPIDS_BEFORE_TEST_COMMANDS_AMD64="${{ inputs.test-before }}"

# citestwheel.sh implicitly uses this to download previously built wheels
export RAPIDS_PY_WHEEL_NAME="${{ inputs.package-name }}_${PIP_CU_VERSION}"

/citestwheel.sh
7 changes: 0 additions & 7 deletions wheel-ctk-name-gen/README.md

This file was deleted.

26 changes: 0 additions & 26 deletions wheel-ctk-name-gen/action.yml

This file was deleted.