Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# SCM syntax highlighting & preventing 3-way merges
pixi.lock merge=binary linguist-language=YAML linguist-generated=true -diff
93 changes: 93 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
name: Test pixi-lock actions

on:
push:
branches: [main, test-me/*]
pull_request:
branches: [main]

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

env:
PIXI_VERSION: "v0.63.0"

jobs:
cache-pixi-lock:
runs-on: ubuntu-latest
outputs:
cache-key: ${{ steps.create-and-cache.outputs.cache-key }}
pixi-version: ${{ steps.create-and-cache.outputs.pixi-version }}
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
path: pixi-lock

- name: Copy test files to working directory
shell: bash
run: cp -r pixi-lock/ci/test/* .

- name: Run create-and-cache action
id: create-and-cache
uses: ./pixi-lock/create-and-cache
with:
pixi-version: ${{ env.PIXI_VERSION }}

- name: Verify pixi.lock exists
shell: bash
run: |
if [ -f "pixi.lock" ]; then
echo "pixi.lock exists"
else
echo "pixi.lock does not exist"
exit 1
fi

restore-and-install:
needs: cache-pixi-lock
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- name: Checkout repository
uses: actions/checkout@v6
with:
path: pixi-lock

- name: Copy test files to working directory
shell: bash
run: cp -r pixi-lock/ci/test/* .

- name: Restore pixi.lock from cache
uses: ./pixi-lock/restore
with:
cache-key: ${{ needs.cache-pixi-lock.outputs.cache-key }}

- name: Verify pixi.lock exists
shell: bash
run: |
if [ -f "pixi.lock" ]; then
echo "pixi.lock exists"
else
echo "pixi.lock does not exist"
exit 1
fi

- name: Setup pixi and install environment
uses: prefix-dev/setup-pixi@v0.9.3
with:
pixi-version: ${{ needs.cache-pixi-lock.outputs.pixi-version }}

- name: Verify environment installed
shell: bash
run: |
if [ -d ".pixi/envs/default" ]; then
echo "Environment installed successfully"
else
echo "Environment not installed"
exit 1
fi
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# pixi environments
.pixi/*
!.pixi/config.toml

pixi.lock
26 changes: 26 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-ast
- id: check-json
types: [text]
files: \.(json|ipynb)$
- repo: https://github.com/rbubley/mirrors-prettier # Update mirror as official mirror is deprecated
rev: v3.6.2
hooks:
- id: prettier
- repo: https://github.com/ComPWA/taplo-pre-commit
rev: v0.9.3
hooks:
- id: taplo-format
args:
[
"--option",
"array_auto_collapse=false",
"--option",
"align_comments=false",
]
87 changes: 87 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# pixi-lock

> [!NOTE]
> This repo is likely to be moved to https://github.com/xarray-contrib

This repo provides two GitHub Actions for managing `pixi.lock` files with caching:

- **`create-and-cache`**: Generates a `pixi.lock` file and caches it
- **`restore`**: Restores the cached `pixi.lock` file in downstream jobs

This two-action pattern is so that the lockfile can be omitted from the git
history, but still be generated in a performant manner (i.e., regenerated
and cached with a key that depends on `pixi.toml` and the date -
then shared across jobs).


## Usage

```yaml
jobs:
cache-pixi-lock:
runs-on: ubuntu-latest
outputs:
cache-key: ${{ steps.pixi-lock.outputs.cache-key }}
pixi-version: ${{ steps.pixi-lock.outputs.pixi-version }}
steps:
- uses: actions/checkout@v4
- uses: Parcels-code/pixi-lock/create-and-cache@v1
id: pixi-lock
with:
pixi-version: ... # TODO: update with your selected pixi version

ci:
needs: cache-pixi-lock
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- uses: actions/checkout@v4
- uses: Parcels-code/pixi-lock/restore@v1
with:
cache-key: ${{ needs.cache-pixi-lock.outputs.cache-key }}
- uses: prefix-dev/setup-pixi@v... # TODO: update with your selected setup-pixi version
with:
pixi-version: ${{ needs.cache-pixi-lock.outputs.pixi-version }}
# ... your CI steps
```

### Inputs & Outputs

#### `create-and-cache`

| Input | Description | Required | Default |
|-------|-------------|----------|---------|
| `pixi-version` | Version of pixi to use for generating the lock file | No | `latest` |

| Output | Description |
|--------|-------------|
| `pixi-version` | The pixi version used |
| `cache-key` | The cache key (includes today's date) |

#### `restore`

| Input | Description | Required |
|-------|-------------|----------|
| `cache-key` | The cache key from `create-and-cache` | Yes |

> [!NOTE]
> The cache key includes the current date, so the lock file is regenerated daily.
> The fallback key (yesterday's date) is calculated automatically to handle edge cases where the restore job runs just after midnight.

## Why not commit the lock file?

Committing your lock file is considered good practice when working on application code. Providing fixed package versions results in perfect reproducibility of environments between machines, and hence also reproducibility of results.

When developing and testing _library_ code, we don't want our environments to stay completely fixed - we want to test against environments covering a wide range of package versions to ensure compatability, including an environment includ the latest available versions of packages.

The easiest way to test against the latest versions of packages - and avoid the noisy commit history (and additional overhead) of regularly updating a lock file in git - is instead to ignore the lock file and rely on developers and CI to generate their own lock files. This forgoes perfect reprodubility between developer machines, and with CI machines.


## Dev notes

### Release checklist

- Update the `README.md` bumping the version of action
- Cut release
14 changes: 14 additions & 0 deletions ci/test/pixi.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
[workspace]
authors = ["Vecko <36369090+VeckoTheGecko@users.noreply.github.com>"]
channels = ["conda-forge"]
name = "test"
platforms = ["win-64", "linux-64", "osx-64", "osx-arm64", "linux-aarch64"]
version = "0.1.0"

[tasks]

[dependencies]
python = "*"
numpy = "*"
pandas = "*"
xarray = "*"
54 changes: 54 additions & 0 deletions create-and-cache/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Create and Cache Pixi Lock
description: Generate a pixi.lock file and cache it for use by other jobs.

inputs:
pixi-version:
description: "Version of pixi to use for generating the lock file"
required: false
default: "latest"

outputs:
pixi-version:
description: "The pixi version used (for passing to downstream jobs)"
value: ${{ inputs.pixi-version }}
cache-key:
description: "The cache key used"
value: ${{ steps.cache-key.outputs.key }}

runs:
using: "composite"
steps:
- name: Get cache key
id: cache-key
shell: bash
run: |
today=$(date +'%Y-%m-%d')
key="pixi-lock_${{ inputs.pixi-version }}_${{ hashFiles('pixi.toml') }}_${today}"
echo "key=${key}" >> "$GITHUB_OUTPUT"

- name: Restore pixi.lock from cache
uses: actions/cache/restore@v5
id: restore
with:
path: pixi.lock
key: ${{ steps.cache-key.outputs.key }}

- name: Setup pixi
uses: prefix-dev/setup-pixi@v0.9.3
if: ${{ !steps.restore.outputs.cache-hit }}
with:
pixi-version: ${{ inputs.pixi-version }}
run-install: false

- name: Run pixi lock
if: ${{ !steps.restore.outputs.cache-hit }}
shell: bash
run: pixi lock

- name: Save pixi.lock to cache
uses: actions/cache/save@v5
if: ${{ !steps.restore.outputs.cache-hit }}
with:
path: pixi.lock
key: ${{ steps.cache-key.outputs.key }}
enableCrossOsArchive: true
33 changes: 33 additions & 0 deletions restore/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Restore Pixi Lock
description: Restore a previously cached pixi.lock file.

inputs:
cache-key:
description: "The cache key from create-and-cache"
required: true

runs:
using: "composite"
steps:
- name: Calculate fallback key
id: fallback
shell: bash
run: |
cache_key="${{ inputs.cache-key }}"
# Extract base (everything before the last _) and date (after the last _)
base="${cache_key%_*}"
current_date="${cache_key##*_}"
# Calculate yesterday's date
yesterday=$(date -d "${current_date} - 1 day" +'%Y-%m-%d' 2>/dev/null || date -j -f '%Y-%m-%d' -v-1d "${current_date}" +'%Y-%m-%d')
echo "key=${base}_${yesterday}" >> "$GITHUB_OUTPUT"

- name: Restore pixi.lock from cache
uses: actions/cache/restore@v5
id: restore
with:
path: pixi.lock
key: ${{ inputs.cache-key }}
enableCrossOsArchive: true
restore-keys: |
${{ steps.fallback.outputs.key }}
fail-on-cache-miss: true