Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Define CUDA versions for workflows centrally #1052

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Define CUDA versions for workflows centrally #1052

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
04a3a05
Define CUDA version in dockerfile to allow Dependabot to pick it up
rickardp Feb 7, 2024
1acc07e
Added dependabot section
rickardp Feb 7, 2024
efcd78f
Use Python for parsing Dockerfile
rickardp Feb 27, 2024
530676c
Merge branch 'main' into dockerfile
rickardp Feb 27, 2024
2889393
Update get_docker_info.py
rickardp Mar 29, 2024
1243718
Update python-package.yml
rickardp Mar 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Define CUDA version in dockerfile to allow Dependabot to pick it up
  • Loading branch information
@rickardp
rickardp committed Feb 7, 2024
commit 04a3a05f0ca8d6cf63e5f5e3345a5fa7aa8bae42
15 changes: 11 additions & 4 deletions .github/workflows/python-package.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ jobs:
matrix:
os: [ubuntu-latest, windows-latest]
arch: [x86_64, aarch64]
cuda_version: ['12.1.0']
cuda_version: ['12']
exclude:
- os: windows-latest # This probably requires arm64 Windows agents
arch: aarch64
Expand All @@ -97,6 +97,13 @@ jobs:
# Check out code
- uses: actions/checkout@v4
# Linux: We use Docker to build cross platform Cuda (aarch64 is built in emulation)
- name: Set CUDA version
shell: bash
run: |
cuda_version=$(scripts/get_docker_info.sh docker/cuda.Dockerfile cuda${{ matrix.cuda_version }})
echo "CUDA version to use is $cuda_version"
echo "CUDA_VERSION=$cuda_version" >> $GITHUB_ENV
# On Linux we use CMake within Docker
- name: Set up Docker multiarch
if: startsWith(matrix.os, 'ubuntu')
uses: docker/setup-qemu-action@v2
Expand All @@ -111,7 +118,7 @@ jobs:
if: startsWith(matrix.os, 'windows')
id: cuda-toolkit
with:
cuda: ${{ matrix.cuda_version }}
cuda: ${{ vars.CUDA_VERSION }}
method: 'local'
# sub-packages: '["nvcc","cudart","nvrtc_dev","cublas_dev","cusparse_dev","visual_studio_integration"]'
- name: Add msbuild to PATH
Expand All @@ -133,7 +140,7 @@ jobs:
build_arch=${{ matrix.arch }}
for NO_CUBLASLT in ON OFF; do
if [ ${build_os:0:6} == ubuntu ]; then
image=nvidia/cuda:${{ matrix.cuda_version }}-devel-ubuntu22.04
image=nvidia/cuda:${{ vars.CUDA_VERSION }}-devel-ubuntu22.04
echo "Using image $image"
docker run --platform linux/$build_arch -i -w /src -v $PWD:/src $image sh -c \
"apt-get update \
Expand All @@ -150,7 +157,7 @@ jobs:
- name: Upload build artifact
uses: actions/upload-artifact@v4
with:
name: shared_library_cuda_${{ matrix.os }}_${{ matrix.arch }}_${{ matrix.cuda_version }}
name: shared_library_cuda_${{ matrix.os }}_${{ matrix.arch }}_${{ vars.CUDA_VERSION }}
path: output/*
retention-days: 7
build-wheels:
Expand Down
7 changes: 7 additions & 0 deletions docker/cuda.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Note: This Dockerfile is currently not used for building an image, but
# for extracting the current version of CUDA to use.
# By using a Dockerfile, it is possible to automatically upgrade CUDA
# patch versions through Dependabot.

FROM nvidia/cuda:11.8.0-devel-ubuntu22.04 AS cuda11
FROM nvidia/cuda:12.1.0-devel-ubuntu22.04 AS cuda12
31 changes: 31 additions & 0 deletions scripts/get_docker_info.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/bin/sh

version_only=
if [ "$1" = "-v" ]; then
version_only=1
shift 1
fi

dockerfile=$1
shift 1

if [ ! -f $dockerfile ]; then
echo "Dockerfile not found" >> /dev/stderr
exit 1
fi

if [ -z "$1" ]; then
echo "No target specified" >> /dev/stderr
exit 1
fi

tag=$(grep "AS $1\$" $dockerfile | sed -e 's/FROM *//' -e 's/ *AS .*//')
if [ -z "$tag" ]; then
echo "Target $1 not defined" >> /dev/stderr
exit 1
fi
if [ "$version_only" = "1" ]; then
echo $tag | sed -e 's/.*://'
else
echo $tag
fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😳 Could we write this in Python, pretty please? 😂

Something like

import argparse
import re

from_line_re = re.compile(r"FROM\s+(?P<image>\S+)\s+AS\s+(?P<target>\S+)")


def find_image_in_dockerfile(dockerfile, target):
    with open(dockerfile) as f:
        for line in f.readlines():
            if (m := from_line_re.match(line)) and m.group("target") == target:
                return m.group("image")
    raise ValueError(f"Target {target} not defined in {dockerfile}")


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("dockerfile", required=True)
    ap.add_argument("target", required=True)
    ap.add_argument("-v", "--version-only", action="store_true")
    args = ap.parse_args()
    image = find_image_in_dockerfile(args.dockerfile, args.target)
    if args.version_only:
        image = image.rpartition(":")[-1]
    print(image)

, I think...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No objection from me 😃 I just took this script off my library of useful stuff TBH