Skip to content

Commit

Permalink
CI script improvements (llvm#1547)
Browse files Browse the repository at this point in the history
* ci: update versions of external actions

Node.js 12 actions are deprecated and will eventually go away, so this
patch bumps the old actions to their latest versions that use Node.js
16.

* ci: replace deprecated action with bash commands

The llvm/actions/install-ninja action uses Node.js 12, which is
deprecated.  Since that action is not updated to work with Node.js 16,
this patch replaces that action with equivalent bash commands to install
Ninja.

* ci: use smaller ccache artifacts to reduce evictions

Over time, our ccache sizes have grown quite large (some as large as
1.3 GB), which results in us routinely exceeding GitHub's limits, thus
triggering frequent cache evictions.  As a result, cache downloads and
uploads take unnecessary long, in addition to fewer cache entries being
available.

Based on experiments on a clean cache state, it appears that we need
less than 300 MB of (compressed) ccache artifacts for each build type.
Anything larger than that will accrue changes from the past that aren't
needed.

To alleviate the cache burden, this patch sets the maximum ccache size
to be 300 MB.  This change should not affect the success or failure of
our builds.  I will monitor the build times to check whether this change
causes any performance degradation.

* ci: use consistent platform identifiers

Prior to this patch, some of our builds ran on `ubuntu-latest`, while
some others ran on `ubuntu-20.04` and others ran on `ubuntu-22.04`, with
similar situations for macOS and windows.  This patch instead sets all
Linux builds to run on `ubuntu-latest`, all macOS builds to run on
`macos-latest`, and all Windows builds to run on `windows-latest`, to
make debugging future CI failures a little easier.
  • Loading branch information
ashay authored Nov 3, 2022
1 parent 2162253 commit f847642
Show file tree
Hide file tree
Showing 6 changed files with 23 additions and 11 deletions.
20 changes: 16 additions & 4 deletions .github/actions/setup-build/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ runs:

steps:
- name: Set up Python
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: '3.10'

Expand All @@ -28,12 +28,24 @@ runs:
python -m pip install -r requirements.txt
shell: bash

- name: Install Ninja
uses: llvm/actions/install-ninja@55d844821959226fab4911f96f37071c1d4c3268
- name: Install Ninja (Linux)
if: ${{ runner.os == 'Linux' }}
run: sudo apt-get install -y ninja-build
shell: bash

- name: Install Ninja (macOS)
if: ${{ runner.os == 'macOS' }}
run: brew install ninja
shell: bash

- name: Install Ninja (Windows)
if: ${{ runner.os == 'Windows' }}
run: pip install ninja
shell: bash

- name: Ccache for C++ compilation
uses: hendrikmuhs/ccache-action@v1.2
with:
key: ${{ runner.os }}-torch_mlir_build_assets-${{ inputs.cache-suffix }}
max-size: 2G
max-size: 300M
verbose: 2
2 changes: 1 addition & 1 deletion .github/workflows/bazelBuildAndTest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ concurrency:
jobs:
ubuntu-build:
name: ubuntu-x86_64
runs-on: ubuntu-22.04
runs-on: ubuntu-latest

steps:
- name: Checkout torch-mlir
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/buildAndTest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,16 +43,16 @@ jobs:
include:
# Specify OS versions
- os-arch: ubuntu-x86_64
os: ubuntu-22.04
os: ubuntu-latest
- os-arch: macos-arm64
os: macos-12
os: macos-latest
- os-arch: windows-x86_64
os: windows-latest
runs-on: ${{ matrix.os }}

steps:
- name: Checkout torch-mlir
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
submodules: 'true'

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/buildRelease.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ jobs:

build_macos:
name: MacOS Build
runs-on: macos-12
runs-on: macos-latest
steps:
- name: Get torch-mlir
uses: actions/checkout@v2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/gh-pages-releases.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
jobs:
scrape_and_publish_releases:
name: "Scrape and publish releases"
runs-on: ubuntu-20.04
runs-on: ubuntu-latest

# Don't run this in everyone's forks.
if: github.repository == 'llvm/torch-mlir'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/releaseSnapshotPackage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ on:
jobs:
release_snapshot_package:
name: "Tag snapshot release"
runs-on: ubuntu-20.04
runs-on: ubuntu-latest
# Don't run this in everyone's forks.
if: github.repository == 'llvm/torch-mlir'
steps:
Expand Down

0 comments on commit f847642

Please sign in to comment.