Skip to content

Commit

Permalink
Cache on action failure; extend docs
Browse files Browse the repository at this point in the history
  • Loading branch information
jonashaag committed Mar 10, 2022
1 parent 5fe88d3 commit 98bc874
Show file tree
Hide file tree
Showing 7 changed files with 69 additions and 6 deletions.
23 changes: 23 additions & 0 deletions .github/workflows/test_caching.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,3 +110,26 @@ jobs:
- name: test environment name
run: |
python -c "import os; env = os.path.basename(os.environ['CONDA_PREFIX']); assert env == 'testenv'"
test_env_fail:
name: Test environment creation failure
runs-on: ubuntu-latest
strategy:
matrix:
include:
- cache-downloads: true
cache-env: false
- cache-downloads: false
cache-env: true
steps:
- uses: actions/checkout@v2

- name: install mamba
uses: ./
with:
environment-file: false
environment-name: test-fail
extra-specs: |
micromamba <0.1
cache-downloads: ${{ matrix.cache-downloads }}
cache-env: ${{ matrix.cache-env }}
20 changes: 19 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ The environment.yml or .lock file for the conda environment. If 'false', no envi

### `cache-env-key`

(Optional) Custom environment cache key used with 'cache-env: true'. The default environment cache key will invalidate the cache whenever the contents of the 'environment-file' or 'extra-specs' change, plus once per day.
(Optional) Custom environment cache key used with 'cache-env: true'. With the default environment cache key, separate caches will be created for each operating system (eg., Linux) and platform (eg., x64) and day (eg., 2022-01-31), and the cache will be invalidated whenever the contents of 'environment-file' or 'extra-specs' change.

<!-- end generated -->

Expand Down Expand Up @@ -125,6 +125,24 @@ or `extra-specs` change, plus once per day. See the `cache-env-key` option for c
cache-env: true
```

## Notes on caching

### Branches have separate caches

Due to a [limitation of GitHub Actions](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache)
any download or environment caches created on a branch will not be available on the main/parent branch
after merging. This also applies to PRs.

See also [this thread](https://github.com/mamba-org/provision-with-micromamba/issues/42#issuecomment-1062007161).

### When to use download caching

Please see [this comment for now](https://github.com/mamba-org/provision-with-micromamba/pull/38#discussion_r808837618).

### When to use environment caching

Please see [this comment for now](https://github.com/mamba-org/provision-with-micromamba/pull/38#discussion_r808837618).

## More examples

More examples may be found in this repository's [tests](.github/workflows).
Expand Down
6 changes: 3 additions & 3 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,9 @@ inputs:
cache-env-key:
description: >-
Custom environment cache key used with 'cache-env: true'.
The default environment cache key will invalidate the cache whenever the contents of the
'environment-file' or 'extra-specs' change, plus once per day.
With the default environment cache key, separate caches will be created for each
operating system (eg., Linux) and platform (eg., x64) and day (eg., 2022-01-31),
and the cache will be invalidated whenever the contents of 'environment-file' or 'extra-specs' change.
required: false
# cache-env-always-update:
# description: >-
Expand All @@ -62,4 +63,3 @@ runs:
using: "node16"
main: "dist/main/index.js"
post: "dist/post/index.js"
post-if: success()
9 changes: 8 additions & 1 deletion dist/main/index.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions dist/post/index.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 8 additions & 1 deletion index.js
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,10 @@ function sha256 (s) {
return h.digest().hexSlice()
}

function sha256Short (s) {
return sha256(s).substr(0, 8)
}

function touch (filename) {
// https://remarkablemark.org/blog/2017/12/17/touch-file-nodejs/
const time = new Date()
Expand Down Expand Up @@ -266,7 +270,7 @@ channel_priority: strict

// Try to load the entire env from cache.
if (inputs.cacheEnv) {
const envHash = sha256(fs.readFileSync(envFilePath)) + '-' + sha256(JSON.stringify(inputs.extraSpecs))
const envHash = sha256Short(fs.readFileSync(envFilePath)) + '-' + sha256Short(JSON.stringify(inputs.extraSpecs))
const key = inputs.cacheEnvKey || `${MAMBA_PLATFORM}-${process.arch} ${today()} ${envHash}`
envCacheArgs = [path.join(PATHS.micromambaEnvs, envName), `micromamba-env ${key}`]
envCacheHit = await tryRestoreCache(...envCacheArgs)
Expand Down Expand Up @@ -321,6 +325,9 @@ Write-Host "Profile already exists and new content added"
await executeBash(`source ${PATHS.bashprofile} && micromamba info && micromamba list`)
}
core.endGroup()

// This must always be last in main().
core.saveState('mainRanSuccessfully', true)
}

async function run () {
Expand Down
4 changes: 4 additions & 0 deletions post.js
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ async function trimPkgsCacheFolder (cacheFolder) {
}

async function main () {
if (!core.getState('mainRanSuccessfully')) {
core.notice('Conda environment setup failed. Cache will not be saved.')
return
}
for (const [path, key, options] of JSON.parse(core.getState('postCacheArgs') || '[]')) {
if (key.startsWith('micromamba-pkgs ')) {
await trimPkgsCacheFolder(path)
Expand Down

0 comments on commit 98bc874

Please sign in to comment.