You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bug(test): test_development_speedup swallows 3 of its 4 sub-test failures and still reports success #14931
test_development_speedup reports success when 3 of its 4 sub-tests fail
Found reviewing PR #14929. That PR converted the last of four swallowing handlers in this function into a raise; the other three still log and continue.
Duplicated verbatim in two files:
autobot-infrastructure/shared/scripts/analysis/test_npu_code_search.py (~lines 144, 164, 184 still swallowing; 212 converted)
autobot-npu-worker/npu_code_search_test.py (~lines 142, 162, 182 still swallowing; 210 converted)
Four such blocks: duplicate detection, pattern analysis, import analysis, comprehensive analysis. Only the fourth now raises. So if find_duplicate_code(), identify_code_patterns() or analyze_imports_and_dependencies() throws, the function still reaches its return True and the driver counts the test as passed.
That means the assertion #14929 added is narrower than it looks: it catches failure of the last sub-test only. Three quarters of what this test exercises can fail silently.
#14929's remit was the driver-consumption defect — nine tests whose return True was converted to a bare return, breaking the driver's verdict. Un-swallowing the other three handlers is a behaviour change to what those tests assert, which is a different question and deserves its own review. Recorded here rather than folded in.
Note on duplication
These two files are near-identical copies. Whatever fix lands must land in both, or they drift — that is the [union-merge] concern that produced #14873 and #14886. Consider whether one should import the other rather than being maintained twice.
Acceptance
A failure in any of the four sub-tests fails the test, not just the fourth
All four except Exception blocks in test_development_speedup now raise AssertionError(...) from e instead of swallowing the failure, applied
identically in both near-identical copies
(autobot-infrastructure/shared/scripts/analysis/test_npu_code_search.py and autobot-npu-worker/npu_code_search_test.py). Verified present in base: 4 raise AssertionError sites.
The acceptance criterion "driver still distinguishes pass from fail" is preserved: main() sums truthiness (result = await test_func(); if result: success_count += 1),
so the function still ends return True. The return contract was kept deliberately
rather than removed, because that driver reads it — unlike #14941's, which does not.
Activity
fix(test): three checks that could not report what they were written …
github-actions commentedon Aug 24, 2026
PR #14980 (merged to
Dev_new_gui) references this issue with a close keyword.If this issue is fully resolved, close it manually. If work remains, no action is needed.
mrveiss commentedon Aug 24, 2026
Delivered and verified in
Dev_new_gui.0256c96dd6e3891e85964494dc83f2447d443528git merge-base --is-ancestorWhat landed
All four
except Exceptionblocks intest_development_speedupnowraise AssertionError(...) from einstead of swallowing the failure, appliedidentically in both near-identical copies
(
autobot-infrastructure/shared/scripts/analysis/test_npu_code_search.pyandautobot-npu-worker/npu_code_search_test.py). Verified present in base: 4raise AssertionErrorsites.The acceptance criterion "driver still distinguishes pass from fail" is preserved:
main()sums truthiness (result = await test_func(); if result: success_count += 1),so the function still ends
return True. The return contract was kept deliberatelyrather than removed, because that driver reads it — unlike #14941's, which does not.
Closing.