Skip to content

Conversation

kirkrodrigues
Copy link
Member

@kirkrodrigues kirkrodrigues commented Oct 16, 2025

  • Save cache entries using unique key per entry.
  • Restore latest entries using key prefix.
  • Avoid using outputs from optionally-run restore task.

Description

#815 added caching for the input and output of lint:check-cpp-lint-static-full, but two issues were preventing the cache from being updated correctly:

  1. I misunderstood the docs for the cache action, so the cache was not being updated on every push to main. Specifically, cache entries are immutable for a given key, so every new entry needs to have a new key. I mistakenly set the cache key to be the same for every entry, so no new entries could be written.
  2. In scheduled runs, the cache was not being saved since the key was empty. This is because the key was derived from the restore step, but scheduled runs don't run the restore step (intentionally).

This PR resolves the issues as follows:

  1. When saving a cache entry, we now append a hash (of the entry's contents) to the key. When restoring the cache, we use a prefix of that key, and actions/cache/restore will automatically select the newest entry.

Note

Although the action/cache docs imply that cache entries are stored in different namespaces determined by the OS used, running
gh api -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" /repos/y-scope/clp/actions/caches shows that cache entries for different OSes have the same version. So we still must use a different cache key prefix per OS and build configuration.

  1. For action/cache/save, we specify the cache key manually rather than trying to rely on the value from action/cache/restore.

This fix should reduce clp-core-build-macos workflow runtimes from 2-3 hours to ~20 minutes.

Checklist

  • The PR satisfies the contribution guidelines.
  • This is a breaking change and that has been indicated in the PR title, OR this isn't a
    breaking change.
  • Necessary docs have been updated, OR no docs need to be updated.

Validation performed

  • Merged the change to main in my fork.
  • Validated that the new cache entries were created successfully.
  • Re-ran the clp-build-artifact and clp-macos-build workflows and validated that they utilized the cache and ran much more quickly.
  • Changed a C++ file on this branch and validated that clang-tidy was only run on that file, while all others were cached.

Note

clang-tidy violations are not being detected correctly, so we can't test if violations will be detected by the workflow. This will be resolved in another PR.

Summary by CodeRabbit

  • Chores
    • Improved CI workflow caching: adjusted restore/save behavior so scheduled runs perform full lint checks, and introduced OS/config-specific cache keys with hash-based inputs to keep restore/save prefixes in sync — reducing missed lint issues and improving build consistency and reliability across environments.

(cherry picked from commit 5972e08)
(cherry picked from commit 8ab2a05)
This reverts commit 5972e08.

(cherry picked from commit 0d3b410)
(cherry picked from commit 98ce1de)
(cherry picked from commit ba3843d)
(cherry picked from commit 3f5c920)
This reverts commit 0d3b410.

(cherry picked from commit 623d7c3)
(cherry picked from commit d544455)
This reverts commit 623d7c3.

(cherry picked from commit 3b97707)
(cherry picked from commit 71fcbd1)
(cherry picked from commit a4d60d7)
(cherry picked from commit 4ed354a)
(cherry picked from commit 7e7a7dd)
This reverts commit 40e55ad.
@kirkrodrigues kirkrodrigues requested a review from a team as a code owner October 16, 2025 21:33
Copy link
Contributor

coderabbitai bot commented Oct 16, 2025

Walkthrough

GitHub Actions workflows updated to change lint cache restore/save behavior: scheduled runs no longer restore lint caches (forcing full lint), cache keys made OS- and configuration-specific, and save keys use hashFiles-based prefixes that must stay in sync with restore prefixes.

Changes

Cohort / File(s) Summary
Lint cache key synchronization & scheduled-run handling
/.github/workflows/clp-artifact-build.yaml, /.github/workflows/clp-core-build-macos.yaml
Reworked cache restore/save logic for lint:check-cpp-static-full: restore is skipped for scheduled runs, restore key prefixes are OS-/config-specific, and save keys now include hashFiles(...) of lint checksum and build outputs. Notes added requiring restore and save key prefixes to remain in-sync.
Environment and execution adjustments
/.github/workflows/clp-artifact-build.yaml
Ensured lint runs on a cleaned environment (explicit HOME and run-on-image) and updated cache key construction for ubuntu paths to match the new hash-based save key approach.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant Scheduler as "Scheduled run"
    participant CI as "GitHub Actions job"
    participant Cache as "actions/cache"
    participant Lint as "lint:check-cpp-static-full"
    rect #E8F0FE
      Note over Scheduler,CI: Scheduled run (cron)
    end
    Scheduler->>CI: trigger job
    CI->>Cache: restore keys (skip for scheduled run)
    Cache-->>CI: no cache restored
    CI->>Lint: run full lint on all files
    Lint-->>CI: lint results
    CI->>Cache: save with hashFiles-based key (cannot rely on restore output)
Loading
sequenceDiagram
    autonumber
    participant Push as "Push/PR run"
    participant CI as "GitHub Actions job"
    participant Cache as "actions/cache"
    participant Lint as "lint:check-cpp-static-full"
    Push->>CI: trigger job
    CI->>Cache: restore using OS/config prefix (may match saved hash-prefix)
    Cache-->>CI: cache restored (if keys match)
    alt cache hit
      CI->>Lint: run lint (incremental/fast)
    else cache miss
      CI->>Lint: run full lint
    end
    Lint-->>CI: lint results
    CI->>Cache: save using hashFiles-based key (must match restore prefix format)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related issues

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The pull request title "fix(ci): Correctly update and restore cache for lint:check-cpp-lint-static-full's generated files (fixes #1419)" directly reflects the core purpose of the changeset. The PR addresses caching issues in the GitHub Actions workflows by making the cache keys unique for save operations (appending a hash), using key prefixes for restore operations, and explicitly setting cache keys per OS and build configuration. The title accurately captures the main fix—correcting how cache is updated and restored for lint check generated files. The title is concise, specific, and clearly communicates the primary change without unnecessary details.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@kirkrodrigues kirkrodrigues changed the title fix(ci): Replace undefined cache key for actions/cache/save with explicit value (fixes #1419). fix(ci): Use unique cache key per cache entry to be saved and restore using key prefix (fixes #1419). Oct 16, 2025
@kirkrodrigues kirkrodrigues changed the title fix(ci): Use unique cache key per cache entry to be saved and restore using key prefix (fixes #1419). fix(ci): Save cache entries using unique key per entry, and restore entries using key prefix (fixes #1419). Oct 17, 2025
Copy link
Contributor

@Bill-hbrhbr Bill-hbrhbr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than we could update the pinned action sha to the latest v4.3.0 release, i don't have other comments. Logic looks correct but there's no way to test until we merge to main.

0057852bfaa89a56745cba8c7296529d2fc39830

@Bill-hbrhbr
Copy link
Contributor

For title, how about:

fix(ci): Enable proper cache updates and reuse for C++ static linting task checksums (fixes #1419).

@kirkrodrigues
Copy link
Member Author

For title, how about:

fix(ci): Enable proper cache updates and reuse for C++ static linting task checksums (fixes #1419).

How about a multi-line PR title & body:

fix(ci): Correctly update and restore cache for `lint:check-cpp-lint-static-full`'s generated files (fixes #1419):
- Save cache entries using unique key per entry.
- Restore entries using key prefix.
- Don't use outputs from optionally run `restore` task.

@kirkrodrigues
Copy link
Member Author

Other than we could update the pinned action sha to the latest v4.3.0 release, i don't have other comments. Logic looks correct but there's no way to test until we merge to main.

0057852bfaa89a56745cba8c7296529d2fc39830

I'd prefer to do this in another PR so that future readers don't think this was a necessary change for this PR.

@Bill-hbrhbr
Copy link
Contributor

For title, how about:

fix(ci): Enable proper cache updates and reuse for C++ static linting task checksums (fixes #1419).

How about a multi-line PR title & body:

fix(ci): Correctly update and restore cache for `lint:check-cpp-lint-static-full`'s generated files (fixes #1419):
- Save cache entries using unique key per entry.
- Restore entries using key prefix.
- Don't use outputs from optionally run `restore` task.

Restore latest entries...
Avoid using attributes/outputs

just some suggestions. otherwise lgtm

@kirkrodrigues kirkrodrigues changed the title fix(ci): Save cache entries using unique key per entry, and restore entries using key prefix (fixes #1419). fix(ci): Correctly update and restore cache for lint:check-cpp-lint-static-full's generated files (fixes #1419): Oct 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants