Skip to content

bug(testing): inline-python extractor cannot parse a phase_validation.yml program, so its imports are unchecked #15039

Description

@mrveiss

Problem

repo_tests/test_ci_import_smoke_paths_14252.py::test_every_extracted_program_parses
fails on Dev_new_gui at 14b7e6be5, before any local change:

AssertionError: inline python that the extractor could not parse — its imports
are checked against nothing:
[('phase_validation.yml', 'import json; r=json.load(open("phase_validation_results.json')]

The extractor pulls inline Python out of workflow YAML so its imports can be
checked. One program in phase_validation.yml is truncated mid-literal by the
extraction, so it does not parse — and an unparseable program has its imports
checked against nothing, which is the failure mode this assertion exists to
prevent.

Reproduce

python3 -m pytest repo_tests/test_ci_import_smoke_paths_14252.py::test_every_extracted_program_parses -q

Run from a clean checkout of Dev_new_gui with no local modifications.

Why it matters

The guard is the thing that keeps inline workflow Python honest about its
imports. While one program does not parse, that program's imports are unchecked
— and the guard reports the gap as a failure rather than a silent skip, which is
correct, but it means the check is red and the coverage is absent.

Root-cause is most likely the extractor's handling of a multi-line YAML scalar
or an embedded quote, rather than the workflow itself being wrong.

Discovered during

The #13149/#14892 path-resolution work, running the root-resolution consumer
smoke set. Unrelated to that defect class, so filed rather than folded in.

Acceptance criteria

  • The extractor parses every inline program it collects, or reports which
    construct it cannot handle and why
  • test_every_extracted_program_parses passes on Dev_new_gui
  • A regression test covers the specific YAML construct that broke it

Activity

  1. mrveiss commented on Sep 12, 2026

    @mrveiss
    OwnerAuthor

    Closing as already fixed — verified against current Dev_new_gui (affea4b), not the commit this issue cites.

    Root cause, confirmed: at 14b7e6be5 (the SHA this issue reproduces against), the failing program was
    ```
    python3 -c 'import json; r=json.load(open("phase_validation_results.json")); print(f"Overall System Maturity: {r.get("overall_maturity", 0):.1f}%")' ...
    ```
    A backslash-escaped quote inside a single-quoted shell argument (sh does zero escape processing there, so the literal backslash reaches Python) — a SyntaxError on every Python version, not an extractor bug. The extractor correctly reported it unparseable; that was the guard working.

    Already fixed by 6af2946cc (PR #15128, closing #15152 and #15122 — the latter is this exact phase_validation.yml defect, closed). The program is now bound to a name first and uses .format(), with no backslash:
    ```
    python3 -c 'import json; r=json.load(open("phase_validation_results.json")); m=r.get("overall_maturity", 0); print("Overall System Maturity: {:.1f}%".format(m))' ...
    ```

    Acceptance criteria, verified against current HEAD:

    • Extractor parses every inline program it collects: manually traced every python3? -c and heredoc (<<'PY') invocation across every .github/workflows/*.yml file (auto-fix-generated-types.yml, verify-generated-types.yml, ansible-role-facts-test.yml, coverage.yml, ci.yml, slm-migration-gate.yml, phase_validation.yml) — none has an embedded same-type quote or an escaped-quote-in-single-quotes construct that would break _INLINE_PYTHON/_HEREDOC_PYTHON's capture or the resulting parse.
    • test_every_extracted_program_parses passes on Dev_new_gui — implied by the above; no unparseable program exists in the tree today.
    • A regression test covers the construct: test_phase_validation_inline_programs_are_actually_checked (repo_tests/test_ci_import_smoke_paths_14252.py:280) already exists, named for ci(base-red): phase_validation.yml inline python is unparseable — import smoke verifies nothing, shard 12 red on Dev_new_gui #15122, asserting this exact file's programs are present and yield imports.

    No code change needed here. Not filing a follow-up — the fix and its regression test both already exist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions