Skip to content

perf(http): route integrations/base+github+m365+notion, monitoring/service_monitor/env_analyzer through shared pool (#12979) - #13003

Merged
mrveiss merged 1 commit into
Dev_new_guifrom
issue-12979-batch8-integrations-api-codeanalysis
Jul 30, 2026
Merged

mrveiss merged 1 commit into
Dev_new_guifrom
issue-12979-batch8-integrations-api-codeanalysis

Conversation

@mrveiss

@mrveiss mrveiss commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Thinking Path

Batch 8 of #12979: the fragmented tail in integrations/ + api/ + code_analysis/. Measured the real remaining set with the ceiling test's own AST walker (never grep — grep over-counts string literals) rather than trusting the issue's snapshot list, per batch 7's finding that an unflagged carve-out can hide in "known" scope. The measured set matched the issue's list exactly (7 files, 9 sites) plus confirmed the 3 explicitly-excluded SSRF files stay untouched.

Read every site individually per the recipe (not grep-and-sweep): none of the 9 sites use raise_for_status() cleanly through to a *_json()-shaped read — all are integration/health-probe/proxy contracts that sentinel-encode non-2xx into a structured result, so all 9 convert to tracked_request().

One site (api/marketplace_sources.py::_fetch_catalog_document) turned out to be a fourth-category RAW carve-out not called out in the task's known-members list: a custom SSRF-pinned TCPConnector (_PinnedResolver + resolve_safe_ip_async), the same shape as knowledge/connectors/oauth_flow.py. Left raw with an in-code comment rather than converting — pooling it would silently drop the pin.

While touching api/monitoring.py, found a pre-existing #12981-class counter leak in two functions that already used the shared client but via the pre-#12989 async with await client.get() form (releases the connection, never balances _active_requests). Fixed alongside since it's the same file, same defect class the ceiling exists to prevent.

What Changed

Converted to tracked_request() (all 9 sites, status-inspecting/health-probe/proxy contracts — none raise on HTTP status):

  • integrations/base.py::_make_request — generic integration helper, arbitrary method, returns {"status_code", "body", "headers"}.
  • integrations/github_integration.py::_github_request — retry loop (403/429/5xx), never raises.
  • integrations/microsoft365_integration.py::_make_graph_request — handles 204 and non-JSON bodies explicitly.
  • integrations/notion_integration.py::_notion_request — reads json(content_type=None), never raises.
  • api/monitoring.py::_fetch_alertmanager_alerts — new conversion; suppress_error_log=True (failure already logged at WARNING).
  • api/monitoring.py::_query_prometheus_instant / _query_prometheus_range — pre-existing bug fix: switched from the leaked-counter async with await client.get() form to tracked_request().
  • api/service_monitor.py::_check_http_health — health probe; suppress_error_log=True; ssl=False forwards as a per-request kwarg (not a carve-out, per batch 7's finding).
  • code_analysis/src/env_analyzer.py::_check_ollama_health — health probe; suppress_error_log=True.
  • code_analysis/src/env_analyzer.py::_call_ollama_filter — dropped the session parameter (sole caller always passed session=None, so the reused-session branch was dead code).

RAW carve-out (documented in-code, not converted):

  • api/marketplace_sources.py::_fetch_catalog_document — custom SSRF-pinned TCPConnector; pooling it would drop the pin and reopen the DNS-rebinding hole it exists to close.

Test seams retargeted (repo-wide patch(.*ClientSession grep found exactly these 3, all the "mock aiohttp.ClientSession directly" variant):

  • integrations/github_integration_test.py, integrations/microsoft365_integration_test.py, integrations/notion_integration_test.py — all switched from patching aiohttp.ClientSession to patching <module>.get_http_client and returning a mock whose tracked_request() yields the mocked response.

Ceiling lowered: tests/test_raw_client_session_ceiling_12992.py MAX_RAW_CLIENT_SESSIONS: 53 → 30, re-measured by AST walker against a tree freshly synced to origin/Dev_new_gui immediately before push (confirmed not behind).

Verification

(a) Static checks — py_compile, flake8 --max-line-length=120, black --check --line-length=120 all clean on every touched file.

(b) Local aiohttp server exercise — all 9 converted call sites driven on success AND failure branches (200, 204, 403, 429, 500, connection-refused) against a local server, 22 checks, 0 unexpected:

still-acquired=0 active_requests=0

Connections released, counter balanced.

(c) Baseline diff (the only real evidence per the recipe) — cp-swapped the origin/Dev_new_gui versions of all touched source+test files in, ran the curated regression scope (all 3 retargeted test files + whatsapp_integration_test.py + integration_unknown_action_contract_test.py + api/marketplace_sources_routes_test.py + api/api_endpoint_migrations_test.py + tests/test_concurrency_global_locks.py), then restored the converted files and re-ran:

baseline: 65 passed, 1 failed, 2112 skipped
after:    65 passed, 1 failed, 2112 skipped   (identical failure: pre-existing #12993 CWD-sensitivity)

Identical set both states — no regressions, no seam silently stopped intercepting.

(d) Ceiling — AST walker measured 38 → 30 on this branch (confirmed against origin/Dev_new_gui immediately before push, not behind). Negative-case check confirmed the assertion trips at 29.

Model Used

Claude Opus 5 (claude-opus-5)

@github-actions

Copy link
Copy Markdown
Contributor

✅ SSOT Configuration Compliance: Passing

🎉 No hardcoded values detected that have SSOT config equivalents!

…ring+service_monitor+env_analyzer through the shared pool (#12979)

Batch 8 of the raw-aiohttp.ClientSession sweep: integrations/base.py,
github_integration.py, microsoft365_integration.py, notion_integration.py,
api/monitoring.py, api/service_monitor.py, code_analysis/src/env_analyzer.py
now route through autobot_shared.http_client.get_http_client()'s
tracked_request() instead of constructing per-request sessions. All 9 sites
use tracked_request() (none raise on HTTP status, so get_json()/post_json()
would change error-path contracts). api/marketplace_sources.py's
_fetch_catalog_document stays RAW: it pins an SSRF-guarded TCPConnector that
the pooled client cannot express per-request, the same carve-out category as
knowledge/connectors/oauth_flow.py.

Also fixed a pre-existing #12981-class counter leak found while touching
api/monitoring.py: _query_prometheus_instant/_query_prometheus_range already
used the shared client but via the pre-#12989 `async with await client.get()`
form, which releases the connection but never balances _active_requests.

Dropped env_analyzer.py's EnvironmentAnalyzer._call_ollama_filter session
parameter: its sole caller always passed session=None, so the
reused-session branch was dead code the pooled client now makes unnecessary.

Ceiling lowered 53 -> 30 in tests/test_raw_client_session_ceiling_12992.py,
re-measured by AST walker against a tree freshly synced to
origin/Dev_new_gui immediately before push.
@mrveiss
mrveiss force-pushed the issue-12979-batch8-integrations-api-codeanalysis branch from 6532803 to f5239cc Compare July 30, 2026 02:14
@mrveiss
mrveiss merged commit 842e3e8 into Dev_new_gui Jul 30, 2026
41 of 42 checks passed
@mrveiss
mrveiss deleted the issue-12979-batch8-integrations-api-codeanalysis branch July 30, 2026 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant