Skip to content

refactor(executor): simplify registry artifact selection - #3265

Open
daryllimyt wants to merge 8 commits into
mainfrom
agent/simplify-registry-artifact-selection
Open

refactor(executor): simplify registry artifact selection#3265
daryllimyt wants to merge 8 commits into
mainfrom
agent/simplify-registry-artifact-selection

Conversation

@daryllimyt

@daryllimyt daryllimyt commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Make registry artifact selection explicit and remove the dead gzip fallback for current SquashFS artifacts.

What changed

  • Select one artifact representation before materialization instead of carrying an ordered candidate list.
  • Prefer a published SquashFS sidecar for current tarball URIs.
  • Preserve genuine legacy tarball-only artifacts when no sidecar exists.
  • Stop deriving a sibling tarball URI from direct SquashFS artifacts; current publishing does not produce that fallback.
  • Propagate SquashFS materialization failures directly instead of masking them as unrelated object-store misses.

Validation

  • uv run pytest --confcutdir=tests/unit tests/unit/test_registry_artifacts.py -q — 133 passed
  • uv run pytest --confcutdir=tests/integration tests/integration/test_registry_artifact_cache_mount_lifecycle.py -q — 2 passed
  • uv run ruff check ...
  • uv run ruff format --check ...
  • uv run basedpyright ...

LOC breakdown

Category + -
Logic 59 102
Tests 55 71

Stack

PR 4 of 4. Depends on #3264.


Summary by cubic

Simplifies executor registry artifact selection by choosing one artifact up front and removing the unused gzip fallback for modern SquashFS. This makes selection explicit and surfaces real SquashFS failures instead of masking them.

  • Selection: For tarball URIs, prefer a published SquashFS sidecar; fall back to legacy gzip only when no sidecar exists. For direct SquashFS URIs, do not derive a sibling gzip; failures propagate.
  • Cache lookup: Recheck the selected artifact’s cached path after acquiring the lease; when selecting a sidecar for a tarball URI, also look for a cached legacy tarball path.
  • Internal API and logs: Replace candidate lists with _select_artifact and _materialize_artifact; introduce _build_artifact; remove _first_cached_path; reduce logs to one “Materializing selected registry artifact” line.
  • Tests and contention: Update tests to the new API and assert no gzip fallback is requested on SquashFS failure; keep lease contention/retry behavior from the merged parent unchanged.

Written for commit c604ea0. Summary will update on new commits.

Review in cubic

@daryllimyt daryllimyt added engine Improvements or additions to the workflow engine refactor Refactoring tests Changes to unit and integration tests labels Aug 18, 2026 — with ChatGPT Codex Connector
@daryllimyt
daryllimyt marked this pull request as ready for review August 18, 2026 14:54
@zeropath-ai

zeropath-ai Bot commented Aug 18, 2026

Copy link
Copy Markdown

No security or compliance issues detected. Reviewed everything up to c604ea0.

Security Overview
Detected Code Changes
Change Type Relevant files
Enhancement ► tests/unit/test_registry_artifacts.py
    Rename test_artifact_candidates_prefer_squashfs_sidecar to test_artifact_selection_prefers_squashfs_sidecar
► tests/unit/test_registry_artifacts.py
    Rename test_artifact_candidates_direct_squashfs_have_no_gzip_fallback to test_artifact_selection_direct_squashfs_has_no_gzip_fallback
► tests/unit/test_registry_artifacts.py
    Rename test_artifact_candidates_fall_back_to_gzip to test_artifact_selection_uses_legacy_gzip_without_sidecar
► tests/unit/test_registry_artifacts.py
    Rename test_artifact_candidates_skip_non_registry_tarballs to test_artifact_selection_skips_non_registry_tarballs
► tests/unit/test_registry_artifacts.py
    Rename test_legacy_tarball_uri_falls_back_after_squashfs_sidecar_fails to test_materialize_does_not_request_missing_gzip_fallback
► tests/unit/test_registry_artifacts.py
    Rename several internal mock/patch references from _artifact_candidates to _select_artifact and _materialize_candidates to _materialize_artifact
► tracecat/executor/registry_artifacts.py
    Replace _artifact_candidates with _select_artifact in _acquire_artifact and related logic
► tracecat/executor/registry_artifacts.py
    Replace _materialize_candidates with _materialize_artifact in materialization flow
► tracecat/executor/registry_artifacts.py
    Rename _first_cached_path usage and implement artifact-level caching via artifact.cached_path(ctx)
► tracecat/executor/registry_artifacts.py
    Rename _candidate_artifacts to _build_artifact and adjust return types to a single RegistryArtifact
► tracecat/executor/registry_artifacts.py
    Adjust _locally_cached_path to utilize artifact.cached_path(ctx) and fallback to legacy behavior when needed
► tracecat/executor/registry_artifacts.py
    Replace _artifact_candidates usage in multiple places with _select_artifact and adapt materialization calls to _materialize_artifact

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: de22908095

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1302 to 1303
if cached_paths := artifact.cached_path(ctx):
return cached_paths

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Recheck all local representations after artifact selection

When another cache instance finishes a legacy tarball extraction while this instance awaits _sidecar_exists(), this post-selection check considers only the selected SquashFS artifact. The valid tarball that appeared after the earlier _locally_cached_path() check is therefore ignored, causing a redundant SquashFS download/extraction and potentially a capacity or tool-availability failure despite an immediately reusable cache entry. Recheck _locally_cached_path(ctx, artifact_uri) here so the cross-instance race preserves the new legacy-cache compatibility behavior.

Useful? React with 👍 / 👎.

Base automatically changed from agent/retry-registry-cache-contention to main August 18, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engine Improvements or additions to the workflow engine refactor Refactoring tests Changes to unit and integration tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant