Skip to content

test(coverage): 11 test functions whose bodies execute nothing, counted for the first time #15256

Closed

Description

@mrveiss

Measured for the first time by #15189's third sweep. The shape was never counted — not because detecting it was hard, but because nobody had asked.

What

11 test functions whose bodies execute nothing — every statement is pass, ..., a bare string, or a bare print(...) — while the function name asserts a behaviour:

File Lines
autobot-backend/config/timeout_configuration_test.py 241, 247
autobot-backend/services/rag_integration_test.py 536, 542, 548, 554, 560
autobot-backend/tests/api/test_knowledge_grounding.py 614, 620, 626, 632

All eleven are in autobot-backend; every other tree measures zero.

Why neither existing detector sees them

  • The returning-test guard looks for a return — these do not return.
  • The swallowed-assertion guard looks for an assert under a bare handler — these have no assert at all.

So a function with no body passes both guards and shows green. The known worst instance of this shape was test_migrated_files_import, which had a body of pass plus six print() calls announcing six successful imports — the import statements having been stripped at some point. It asserted six imports while performing none, and reported success for all six.

PR #15254 adds detection (74 lines, driven off the existing collection model rather than a second AST walk) and ratchets the population at 11, down-only. It deliberately excludes skip/skipif/xfail — a declared skip is honest — and anything that calls something, which is a smoke test rather than an empty one.

What remains

The eleven are now counted and pinned, not fixed. Each needs a real test written, and each needs something the repository does not currently have on hand: the rag_integration and knowledge_grounding ones need a live endpoint or a real KnowledgeBase.

That is the honest position — and it is why they were ratcheted rather than deleted. Deleting them would drop the record that someone intended coverage here; leaving them unmarked would keep eleven green tiles that assert nothing.

Acceptance criteria

  • For each of the eleven, decide: write the test, or delete it and record what was intended to be covered. A third option — leave it empty and green — is what this issue exists to end.
  • Any that stay pending get an explicit pytest.mark.skip with a reason and an issue reference, so the guard's honest-skip exclusion applies and the tile stops reading as a pass.
  • The ratchet goes down as each is resolved, never up.
  • For each written test: a contrast mutation showing it fails when the behaviour it names is broken.
  • Confirm no twelfth has appeared since — the guard now makes that a CI failure rather than a discovery.

Refs #15189, PR #15254.

Activity

  1. github-actions commented on Aug 31, 2026

    @github-actions
    Contributor

    PR #15351 (merged to Dev_new_gui) references this issue with a close keyword.

    fix(test): write the 11 vacuous tests for real and fix silent-failure helpers

    If this issue is fully resolved, close it manually. If work remains, no action is needed.

  2. mrveiss commented on Aug 31, 2026

    @mrveiss
    OwnerAuthor

    Closing — all criteria met. 8fa01bf0e3 (PR #15351), verified against merged Dev_new_gui.

    All 11 vacuous test functions were implemented, not deleted and not skipped:

    File Was Now
    config/timeout_configuration_test.py:241,247 empty body real asyncio.wait_for(timeout=kb_timeouts.document_add) runtime test via DocumentsMixin
    services/rag_integration_test.py:536-560 (5) empty body moved to rag_integration_api_test.py, hitting the real api.knowledge_rag router through TestClient
    tests/api/test_knowledge_grounding.py:614-632 (4) empty body hit the real api.knowledge_grounding router through TestClient, agent and Redis patched per test

    _EMPTY_BODIED on merged base is {} — a derived hard zero across all 13 trees, not a list of clean trees that can go stale.

    No second guard was written: the pre-existing #15254 sweep was extended instead.

    One defect found in review and fixed before merge, worth recording here. Draining _EMPTY_BODIED to {} silently disabled the guard's own anti-vacuity floor — _collapsed() derived entirely from that dict, so with it empty the check "the sweep no longer finds the tests it is supposed to be scanning" could never fire at either call site. Reaching zero is what removed the protection. A guard against tests that execute nothing had itself become passable by finding nothing.

    Replaced with floors that do not derive from the baseline, present on merged base at tests_that_execute_nothing_test.py:120-121:

    _TOTAL_FUNCTION_FLOOR = 25000     # measured 28708 across 13 trees
    _TREE_FLOOR = 10
    _sweep_shortfalls()               # collects every shortfall, does not abort on the first
    

    Proven to fire, not assumed: a collapsed sweep reports ['only 0 test functions collected, floor is 25000', 'only 0 trees reached, floor is 10']; a healthy one reports [].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions