Repository navigation
ci(guard): path filters skip python-suite for ansible-only changes, but repo_tests read ansible files #15713
Description
Activity
Superseded in practice by what #15712 turned up while fixing the instance, and worth recording before this thread goes cold.
The premise here was that path filters skip
python-suitefor ansible-only changes, so arepo_testreading ansible files can break base unseen. That is what happened with #15704. But the fix in #15712 changed the calculus: the two resolvers that broke are now the only ones that read role defaults, and both were broken for a reason deeper than filtering —_VARIABLEhad never understood a dotted reference at all, and neither resolver had inventory vars in scope. Widening a path filter would have caught the symptom one PR earlier; it would not have found that.Reopening this as written would buy a filter change whose acceptance criteria I can no longer state honestly: I do not know that the remaining
repo_testsreading non-Python trees are at risk, because I have not measured which ones resolve anything that can go stale. Filing a broad 'audit every filter' issue without that measurement is the kind of speculative work this backlog does not need.What survives is narrower and already tracked: #15714 asks for a sweep of exclusion/scoping shapes across
repo_tests,scriptsandpipeline-scripts, which is the same question asked where there is evidence of more than one instance.Closing rather than leaving an issue whose criteria I would not be able to verify. If an ansible-only change breaks base again, that is the second instance and this reopens with real evidence behind it.
Reopening — I closed this a moment ago and the reasoning was wrong.
I argued the real cause was the two resolvers rather than the filter, so a filter change would only have caught the symptom earlier. Both halves of that are true and neither justifies closing: the filter gap is demonstrated, not hypothetical. #15704 merged with a green board and broke
Dev_new_guifor every Python PR, and #15712 fixed the resolvers without touching the filter. The exact same sequence can happen tomorrow with a differentrepo_test."I cannot state the acceptance criteria" is a reason to narrow an issue, not to close it. So, narrowed to what the evidence actually supports:
Acceptance criteria
- Measure which
repo_tests/*.pyread files outside thepython-suitepath filter — derived from the tests' own path constants, not guessed. That number is the finding; if it is one or two, this is a smaller issue than it looks - For each, decide: widen the filter to cover the tree it reads, or record why a stale read there cannot break base
- The measurement is repeatable, so a new
repo_testsfile reading a new tree is visible rather than discovered by a red base - Proven by mutation: change only a file in one of those trees, confirm the suite that reads it now runs
Dropping the original "audit every filtered workflow" criterion — that was the speculative part, and it is what made the issue look unclosable.
Related
- tech-debt(ansible): derive every code_source_dir fallback from the SSOT (#15632, #15601) #15704 — merged green, broke base
- fix(test): render role defaults against the inventory globals they now use #15712 — fixed the two resolvers; deliberately did not change the filter
- bug(test): requirements walk excludes its own tree when the checkout lives under .worktrees #15714 — the sibling sweep, for exclusion/scoping shapes rather than filter reach
- Measure which
- added 4 commits that reference this issue
on Sep 5, 2026 Closing on all four criteria, verified against merged code on
Dev_new_gui(PR #15715).# Criterion Evidence 1 Measure which repo_testsread files outside the filter, derived not guessed178 guards parsed; 27 uncovered inputs, from the guards' own path constants 2 For each, widen the filter or record why a stale read cannot break base ansible tree covered by autobot-slm-backend/ansible/**; the other 27 recorded inrepo_tests/python_filter_uncovered_reads.py3 The measurement is repeatable, so a new guard reading a new tree is visible repo_tests/python_filter_covers_its_guards_test.py— a new uncovered read fails;MAX_UNCOVERED_READS = 27is asserted equal, not as a bound4 Proven by mutation contrast pairs on every detector half: covered/uncovered path, composed/unanchored chain, root file/bare word Why the ansible tree was covered wholesale
Eighteen
repo_testsfiles read it; the filter named exactly three.j2files from it, added one at a time by #14544 as each broke. #15704 then changed role defaults only, took the required-context shim's green, merged, and left base red on all twelve shards.The file's own comment already argued the case: narrow patterns cover 100% of the surface on the day they are written, and that is the property that decays silently. It decayed twice before this.
The number moved three times, and that is the finding
pass uncovered what the detector had been missing first 23 — composed paths 24 _REPO_ROOT / "a" / "b"— no literal contains a slash; 52 guards build paths that wayrepository-root files 27 a root file has no tree component, so pytest.iniwas dropped — a guard readingpytest.iniwas bypassable by editingpytest.iniown reads 27 the module excluded itself wholesale, exempting its real read of the filter file Every refinement found more, never fewer. That is what a reach failure looks like from the inside, and it is why the record carries the detector's limits beside the count rather than presenting the number as settled.
The 27 are recorded, not fixed, and deliberately
Covering the remainder wholesale would run twelve shards on nearly every pull request. The sharpest entries are the CI definitions themselves — editing
.github/workflows/code-quality.ymlis exactly when you want the guard that reads it to run — but that trade is worth making per tree, deliberately, and it is not a call to make inside a guard PR. The record is down-only and bidirectional: a new bypass fails, and an entry left behind after the filter grew to cover it fails just as loudly.Related
- tech-debt(ansible): derive every code_source_dir fallback from the SSOT (#15632, #15601) #15704 — merged green, broke base; the instance
- fix(test): render role defaults against the inventory globals they now use #15712 — fixed the two resolvers it broke
- bug(test): requirements walk excludes its own tree when the checkout lives under .worktrees #15714 — the sibling sweep, for exclusion/scoping shapes
- added 6 commits that reference this issue
on Sep 5, 2026
An ansible-only change merged green and broke
Dev_new_guifor every Python PR. The mechanism is worth fixing, not just the instance.What happened
#15704 changed only files under
autobot-slm-backend/ansible/. Thepython-suiteworkflow's path filters do not match that prefix, so all twelve shards skipped, and the PR merged with a green board.repo_tests/autobot_shared_symlink_relative_13539_test.pyreads those ansible files. It broke on the change and nothing ran it until the next unrelated Python PR (#15711) inherited a red base:Fixed in PR #15712. This issue is about why it was possible.
The general shape
Path filters gate on what a change touches. A guard gates on what it reads. When those differ, the filter is answering a question the guard did not ask.
repo_tests/is full of tests in exactly this position — they parse workflows, ansible roles, manifests, baselines and docs. Every one of them can be broken by a change that never touches a.pyfile, and every one of them is skipped by a filter keyed on Python paths. This is the #15667 family again: a check that reports on what it reaches, with nothing checking whether it was reached at all.Why "just run everything" is not the answer
python-suiteis twelve shards; running it on every docs typo is a real cost, and the path filters exist for a reason. The fix is to make the filter's inputs match the suite's actual reach, not to delete the filter.Acceptance criteria
python-suitepath filter includes the non-Python trees thatrepo_tests/reads — measured from the tests' own file references, not guessedrepo_testsfile reading a new tree does not silently fall outside the filter againrepo_tests/*.pyreferences a path prefix the filter does not cover — the check on the filter's reach that this issue is aboutpython-suiteRelated