Improve CI workflow verification and caching#26
Merged
Conversation
The clip cache can make generated tile semantics depend on which ancestor clips happen to be present during parallel tile generation. That is useful for normal runtime, but it makes CI semantic checks sensitive to cache scheduling instead of the underlying output path being verified. Add a compile-time CMake switch that replaces ClipCache with always-miss no-op methods, then build a second CI binary with that switch enabled. Normal tile outputs are still generated and archive-validated with the cache enabled, while repeat and cross-runner semantic comparisons use only the no-cache artifacts. Co-authored-by: Codex <noreply@openai.com>
The CMake jobs listed the same vcpkg packages directly in the workflow, which made dependency changes harder to review and made the binary cache depend on unrelated workflow edits. Move the dependency list into vcpkg.json with a pinned builtin baseline, use manifest-mode installs in the Windows and Linux vcpkg jobs, and key the binary cache on the manifest files. Add Dependabot coverage for the vcpkg baseline so dependency refreshes are proposed as focused changes. Co-authored-by: Codex <noreply@openai.com>
Add compiler caching to the CMake and Makefile jobs so repeated CI runs can reuse object files in addition to the vcpkg binary cache. Move repeated tile generation commands into shared Bash and PowerShell helpers, then require uploaded artifacts to exist and avoid spending time recompressing generated tile outputs. Add a configure-only CMake smoke job and parse checks for PowerShell CI scripts so common workflow or build-setup mistakes fail before the full matrix starts. Co-authored-by: Codex <noreply@openai.com>
Hosted runner vcpkg checkouts can be too shallow to resolve the manifest builtin-baseline, which makes vcpkg install fail before CMake starts. Fetch the baseline named in vcpkg.json before dependency installation so Dependabot baseline updates remain usable across the Windows and Linux CMake jobs. Co-authored-by: Codex <noreply@openai.com>
Fetching the manifest baseline is not enough for hosted runner vcpkg checkouts because vcpkg also reads the working-tree versions database. Check out the baseline named in vcpkg.json before installing dependencies so the registry files and builtin-baseline agree on both Windows and Linux CMake jobs. Co-authored-by: Codex <noreply@openai.com>
The normal tile-generation path passes an empty suffix so that the generated files keep their historical names. PowerShell treats a mandatory string parameter as rejecting empty input unless it is explicitly allowed, which made the Windows CMake job fail before tilemaker ran. Allow only this parameter to accept an empty string. The non-empty no-clip-cache suffix path and the shell script behavior are unchanged. Co-authored-by: Codex <noreply@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI-generated PR.
This is a fork-only draft PR for validating CI changes before deciding whether to send anything upstream.
Summary
This updates the CI workflow to make generated tile checks more useful and repeated runs faster:
TILEMAKER_DISABLE_CLIP_CACHEbuild option for semantic verification buildsvcpkg.jsonClip Cache Verification Mode
The clip cache disable option is intended for CI verification, or for anyone who explicitly wants guaranteed repeatability over runtime performance.
Production behavior remains unchanged. The normal CI builds still compile and run tilemaker with the clip cache enabled. Only the semantic verification artifacts are generated with
TILEMAKER_DISABLE_CLIP_CACHE=ON, so normal output generation and packaged binaries continue to exercise the default production path.This avoids treating known clip-cache scheduling differences as semantic failures while still keeping the normal cache-enabled build covered by build and tile-generation checks.
Generated tile output is not expected to semantically match across all CI runners until systemed#889 is merged. The no-clip-cache verification path is meant to provide a stable CI comparison once the remaining semantic-output fix is present.
Dependency And Cache Changes
vcpkg.jsonnow records the Windows/Linux vcpkg dependency set directly in the repository. CI installs from the manifest instead of repeating the dependency list inline in the workflow. This gives Dependabot a file it can understand and keeps dependency drift easier to review.The workflow also adds:
vcpkg.jsonvcpkg.jsonbuiltin-baselinebefore manifest install, so hosted-runner vcpkg checkouts use the matching registry filesThese caches are best-effort. The build remains correct on cache miss; the benefit is reduced rebuild time on warmed runners and repeated CI runs.
Validation Changes
The workflow now runs cheap checks before starting dependency-heavy builds:
The changed-file gate also treats
.github/scripts/*,vcpkg.json, andvcpkg-configuration.jsonas runtime-impacting so changes to CI helpers or dependency manifests do not accidentally skip the matrix that needs to exercise them.Tile Generation Helpers
The repeated generated-tile commands were moved into:
.github/scripts/generate-tiles.sh.github/scripts/generate-tiles.ps1Both helpers check tilemaker's exit status and verify that each expected output file exists and is non-empty. This keeps the Windows, CMake, and Makefile jobs aligned and avoids maintaining the same error handling in several inline workflow blocks.
Artifact Handling
Generated tile and binary uploads now set
if-no-files-found: error, so missing output is reported at the upload step instead of silently producing incomplete artifacts.Generated tile artifact uploads also use
compression-level: 0. The outputs are already compressed formats, so this avoids spending CI time recompressing MBTiles/PMTiles artifacts.Local Checks
Ran locally:
actionlint .github/workflows/ci.ymlshellcheck .github/scripts/generate-tiles.sh.github/workflows/ci.ymland.github/dependabot.ymlvcpkg.json.github/scripts/generate-tiles.ps1git diff --cached --check.github/scripts/generate-tiles.shagainst the Liechtenstein fixture using the no-clip-cache binary