perf(http): monitoring_integration test_connection sites leak _active_requests (pre-#12989 recipe)

Split out of #12979 while converting batch 3 (integrations/cicd_integration.py, integrations/version_control_integration.py).

Problem

The pilot conversion (PR #12982, merged as 8e2203c2a) predates #12989. Its two status-inspecting sites use the recipe that was current at the time:

autobot-backend/integrations/monitoring_integration.py:69 (DatadogIntegration.test_connection)
autobot-backend/integrations/monitoring_integration.py:278 (NewRelicIntegration.test_connection)

async with await get_http_client().get(
    f"{self.base_url}/validate",
    headers=self._get_headers(),
    timeout=aiohttp.ClientTimeout(total=10),
) as response:
    if response.status == 200:
        ...

That form releases the connection correctly — the async with is the release point, and the accompanying comment says so. But get() is request(), which increments _active_requests and delegates the success-path decrement to the caller. Neither site calls decrement_active(), so the counter only grows.

This is exactly the defect #12981 described and #12989 fixed by introducing tracked_request(), which owns the counter as well as the response block. The *_json() helpers in the same file were fixed by that change; these two raw-response sites were not, because they cannot use *_json() — they must inspect response.status and return a structured IntegrationHealth rather than raise.

Evidence

Both test_connection() methods driven five times each against a local aiohttp server:

requests issued        : 10
still-acquired         : 0
active_requests counter: 10   <-- should be 0

Connections released, counter leaked 1:1 with requests.

Why it matters

_active_requests is not cosmetic. It drives utilization = _active_requests / _current_pool_size in _adjust_pool_size(), and gates the _active_requests == 0 condition in decrement_active() that applies deferred session recreation. A monotonically rising counter pushes the pool toward _pool_max and makes deferred recreation unreachable.

Datadog and New Relic health checks are polled, so this accrues continuously wherever those integrations are enabled.

Fix

Swap both sites to tracked_request(), keeping the existing status-inspection logic unchanged:

async with get_http_client().tracked_request(
    "GET",
    f"{self.base_url}/validate",
    headers=self._get_headers(),
    timeout=aiohttp.ClientTimeout(total=10),
    suppress_error_log=True,
) as response:
    if response.status == 200:
        ...

Both sites already catch, log, and encode the failure as IntegrationStatus.UNHEALTHY, so suppress_error_log=True is appropriate — the manager's default ERROR log would duplicate a message the call site already emits.

Verification

Reuse the per-batch method from #12979: exercise both test_connection() paths (200 and non-2xx) against a local aiohttp server and assert connector._acquired_per_host still-acquired is 0 and _active_requests returns to 0. Today the second assertion fails.

Not fixed in the batch-3 PR because it is a different file and outside that batch's stated scope.

Refs #12979, #12981, #12989.

fix(http): track active requests in monitoring test_connection sites (#…

PR #12996 (merged to Dev_new_gui) references this issue with a close keyword.

fix(http): track active requests in monitoring test_connection sites (#12992)

If this issue is fully resolved, close it manually. If work remains, no action is needed.

Closed by PR #12996, merged to Dev_new_gui as ade45c09d.

Counter fix (the defect): get_json()/post_json() incremented _active_requests and never decremented it on the success path. Since utilization = _active_requests / _current_pool_size drives pool auto-sizing, and deferred session recreation is gated on if self._active_requests == 0, the counter drifting upward meant the pool would grow without cause and never recreate its session. Measured before the fix: 16 after 16 requests; after: 0.

Both now route through a tracked_request() async context manager that decrements in finally, so the release point is the same async with that releases the connection.

Convergence guard (why this issue needed more than the fix): autobot-backend/tests/test_raw_client_session_ceiling_12992.py asserts an AST-counted ceiling on raw aiohttp.ClientSession(...) constructions, because #12979 could not converge by subtraction — origin gained ~8 new constructions during two in-flight batches. Counting is AST-based, not grep: three textual matches are string literals (a docstring example, a print(), and a regex replacement template) and grep reads 3 high in both directions.

The guard also carries a vacuity check — test_walker_scans_a_nonempty_tree asserts the walk finds >1000 files and specifically reaches integrations/monitoring_integration.py, so a directory rename or a broadened exclusion cannot silently reduce the walk to zero and let the ceiling pass while checking nothing.

Verified on the merged tip (origin/Dev_new_gui): walker scans 2184 files, actual count 71.

Known follow-up, tracked here so it is not lost: the ceiling constant merged at 87, which was correct against its own base but is now 16 slack because #12999 (−16) landed on top of it. PR #13000 removes a further 15 (the two densest files, telegram_bot_service.py and tts_client.py). Rather than tighten twice, the ceiling will be lowered once to the re-measured count after #13000 merges. Batch briefs for #12979 now require the ceiling to be lowered in the same PR that removes the sessions, re-measured against a freshly-synced tree immediately before push — a count taken early in a worktree is a snapshot of that worktree.

fix(egress): four knowledge connectors go through the egress guard, a…

Navigation Menu

Repository navigation

perf(http): monitoring_integration test_connection sites leak _active_requests (pre-#12989 recipe) #12992

Problem

Evidence

Why it matters

Fix

Verification

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

perf(http): monitoring_integration test_connection sites leak _active_requests (pre-#12989 recipe) #12992

Description

Problem

Evidence

Why it matters

Fix

Verification

Activity

github-actions commented on Jul 30, 2026

mrveiss commented on Jul 30, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions