Skip to content

Commit 031d127

Browse files
authored
ci: introduce read-only and read-write PyTorch build caches (#1546)
Until recently, we had to either risk feature branches creating PyTorch build caches (which were unusable by the main branch or other parallel feature branches because of GitHub's rules around sharing caches among branches) or we had to limit the PyTorch build caches to only the main branch, causing CI runs on feature branches to be terribly slow because they had to rebuild PyTorch each time. This patch enables the best of both worlds, by using a fork (github.com/ashay/cache) of the GitHub's cache action, where the fork adds an option (called `save`) which, when set, uploads a new cache entry. We thus set this `save` flag only when we're building PyTorch from source in Torch-MLIR's main branch, whereas all other builds set this `save` flag to `false`. The ability to conditionally update the cache has been an oft-requested feature on the original (github.com/actions/cache) repository and multiple unmerged PRs exist to allow conditional cache updates, so it is likely that using the fork is only a temporary solution.
1 parent 7987104 commit 031d127

File tree

2 files changed

+7
-6
lines changed

2 files changed

+7
-6
lines changed

.github/workflows/RollPyTorch.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ jobs:
7171
- name: Update PyTorch Build Cache (if running on main branch)
7272
if: github.ref_name == 'main'
7373
id: cache-pytorch
74-
uses: actions/cache@v3
74+
uses: ashay/cache@v1
7575
with:
7676
path: ${{ github.workspace }}/build_tools/python_deploy/wheelhouse
77-
key: ubuntu-x86_64-pytorch-${{ env.PT_HASH }}
77+
key: ${{ runner.os }}-pytorch-${{ env.PT_HASH }}

.github/workflows/buildAndTest.yml

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -73,13 +73,14 @@ jobs:
7373
with:
7474
arch: x64
7575

76-
- name: Cache PyTorch Build (if running on main branch)
77-
if: github.ref_name == 'main' && matrix.os-arch != 'windows-x86_64'
76+
- name: Cache PyTorch Build
77+
if: matrix.os-arch != 'windows-x86_64'
7878
id: cache-pytorch
79-
uses: actions/cache@v3
79+
uses: ashay/cache@v1
8080
with:
8181
path: ${{ github.workspace }}/build_tools/python_deploy/wheelhouse
82-
key: ${{ matrix.os-arch }}-pytorch-${{ env.PT_HASH }}
82+
key: ${{ runner.os }}-pytorch-${{ env.PT_HASH }}
83+
save: ${{ github.ref_name == 'main' && matrix.torch-binary == 'OFF' }}
8384

8485
- name: Build and Test os-arch='ubuntu-x86_64' llvm-build='${{ matrix.llvm-build }}' torch-binary='${{ matrix.torch-binary }}'
8586
if: ${{ matrix.os-arch == 'ubuntu-x86_64' }}

0 commit comments

Comments
 (0)