Skip to content

fix(agent-os): apply the require_tls gate to the configured server URL - #3512

Closed
LHMQ878 wants to merge 3 commits into
microsoft:mainfrom
LHMQ878:fix/mcp-auth-tls-gate-ignores-configured-url
Closed

LHMQ878 wants to merge 3 commits into
microsoft:mainfrom
LHMQ878:fix/mcp-auth-tls-gate-ignores-configured-url

Conversation

@LHMQ878

@LHMQ878 LHMQ878 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Fixes #3511

What was wrong

McpAuthPolicy.check enforced require_tls only against the URL the caller passed. That parameter defaults to "" and the gate was skipped entirely when it was falsy, so entry.url — written by both constructors and by from_yaml, documented on McpServerEntry and in docs/specs/MCP-SECURITY-GATEWAY-1.0.md §10.3 — was never read by any code path.

policy = McpAuthPolicy(servers=[McpServerEntry(
    name="finance", url="http://mcp.internal/finance",
    allowed_auth_methods=["mtls"], require_tls=True)])

policy.check("finance", "mtls").allowed                             # -> True
policy.check("finance", "mtls", url="http://mcp.internal/finance")  # -> False

Both calls describe the same connection to the same server under the same policy, and they disagree. Which one fires depended on whether the caller redundantly restated configuration the policy object already held.

The from_yaml path always took the permissive branch, since that is exactly the deployment shape where the URL lives in config and the call site has no reason to pass it again:

mcp_auth_policy:
  servers:
    - name: finance-tools
      url: http://mcp.internal/finance
      allowed_auth_methods: [mtls]
      require_tls: true
McpAuthPolicy.from_yaml(cfg).check("finance-tools", auth_method="mtls").allowed
# -> True

from_yaml parses url into the entry and nothing reads it back. A configuration that says require_tls: true and points at http:// is the misconfiguration this check exists to catch, and the check stayed silent.

What changed

effective_url = url or entry.url. The caller-supplied URL keeps priority, since it describes the connection actually being dialed and may legitimately differ from the registered pattern — a plaintext dial against an https-registered server is still blocked.

With no URL from either source, the gate stays skipped, exactly as before. Denying every server registered without a URL would be a separate policy decision about existing configurations, not part of fixing this one, so it is deliberately out of scope.

Scheme inspection moves into _tls_violation, and TLS_SCHEMES becomes a module constant. A non-empty URL with no scheme — the shape a bare host or the documented glob pattern takes (mcp.internal, *.internal/finance) — was already denied by "" not in {"https","wss"}, but only as a by-product of set membership, reported as scheme '' is not in the TLS allowlist. It is now denied deliberately, with URL 'mcp.internal' has no scheme, so TLS cannot be verified, and covered by a test. This matters more than it did before: the change makes that path reachable from configured URLs for the first time.

Compatibility

Strictly more denials, and only in the case the guard was written to reject: a server whose configured URL is not TLS while require_tls is True. Every combination that was allowed for a legitimate reason still is — require_tls=False, an https/wss configured URL, no URL anywhere, and unknown servers on the default path (which has no TLS gate at all, unchanged here).

min_tls_version is stored, documented, and specified (§10.6 step 5) but read by no code path. That is unimplemented functionality rather than a guard that misreports, so it is noted in #3511 and left for its own issue rather than folded in here.

Verification

  • tests/test_mcp_auth_enforcement.py: 31 passed. With the source file stashed: 7 failed / 24 passed — every new assertion fails on main.
  • Full tests/ suite (-p no:randomly --continue-on-collection-errors): 4478 passed / 257 failed against base 4464 / 257 — exactly +14, the new test count, no new failures.
  • tests/test_spec_mcp_gateway_conformance.py cannot be collected on main (pre-existing ModuleNotFoundError: agent_sre, one of three such modules), so its S10.1–S10.11 assertions were replayed by hand against the patched module — all still hold, including S10.7 (http:// caller URL denied) and S10.11 (add_server with no URL still allowed).
  • ruff check: 4 findings, identical to base (UP037 in the source, I001 + two F401 in the test file — all pre-existing). ruff format --diff: the remaining reformat suggestions are all on lines this PR does not touch, matching base exactly.
  • cspell on all 133 added lines: clean.

Tests added

TestTlsGateUsesTheConfiguredUrl — 8 methods / 14 cases:

  • plaintext configured URL denied with no caller URL, over http, ws, ftp, gopher, and two scheme-less forms
  • https and wss configured URLs allowed; scheme case ignored
  • a caller URL still overrides the configured one in the restrictive direction
  • no URL anywhere leaves the gate skipped (pins the deliberately-unchanged case)
  • require_tls=False still allows plaintext
  • an out-of-allowlist auth method is still reported as an allowlist failure, not masked as a TLS one — the ordering matters for the error a user sees
  • the from_yaml path, which is where this fails in practice

Checklist

  • Tests added that fail without the fix and pass with it
  • ruff check / ruff format state unchanged relative to base
  • No new failures in the full test suite

`McpAuthPolicy.check` enforced `require_tls` only against the URL the caller
passed. That parameter defaults to `""` and the gate was skipped entirely when
it was falsy, so `entry.url` -- written by both constructors and by
`from_yaml`, documented on `McpServerEntry` and in the gateway spec -- was
never read by any code path.

A server registered with a plaintext URL and `require_tls=True` was therefore
allowed by any caller that did not repeat the URL:

    policy = McpAuthPolicy(servers=[McpServerEntry(
        name="finance", url="http://mcp.internal/finance",
        allowed_auth_methods=["mtls"], require_tls=True)])

    policy.check("finance", "mtls").allowed                              # True
    policy.check("finance", "mtls", url="http://mcp.internal/finance")   # False

Both calls describe the same connection to the same server under the same
policy. Which one fires depended on whether the caller redundantly restated
configuration the policy object already held -- and the `from_yaml` path, where
the URL lives in config and the call site has no reason to pass it again, always
took the permissive branch.

The URL now falls back to `entry.url`. A caller-supplied URL keeps priority,
since it describes the connection actually being dialed and may legitimately
differ from the registered pattern. With no URL from either source the gate
stays skipped as before; denying every server registered without a URL is a
separate policy decision.

Scheme inspection moves to `_tls_violation`, which reports a scheme-less URL --
the shape a bare host or a glob pattern takes -- as "TLS cannot be verified"
rather than as `scheme '' is not in the allowlist`. That case was already denied
by set membership; it is now denied on purpose, with a reason that says why,
and covered by a test. This path becomes reachable from configured URLs for the
first time with this change.

Signed-off-by: LHMQ878 <LHMQ878@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 30, 2026 01:30
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions github-actions Bot added the tests label Jul 30, 2026
@github-actions

Copy link
Copy Markdown
🤖 AI Agent: contributor-guide — View details

AI-generated review output. Treat it as untrusted analysis and verify before acting.

Welcome, and thank you for your detailed contribution!

Your test coverage is excellent, ensuring the fix is well-verified. Before merging, please check:

  1. Ensure all new code adheres to the project's style guide, especially for docstring formatting.
  2. Confirm the updated logic aligns with the documented behavior in docs/specs/MCP-SECURITY-GATEWAY-1.0.md.

Refer to CONTRIBUTING.md for guidance.

@github-actions

Copy link
Copy Markdown
🤖 AI Agent: security-scanner — View details

AI-generated review output. Treat it as untrusted analysis and verify before acting.

No security issues found.

@github-actions

Copy link
Copy Markdown
🤖 AI Agent: code-reviewer — View details

AI-generated review output. Treat it as untrusted analysis and verify before acting.

TL;DR: 0 blockers, 0 warnings. The PR resolves a critical security gap in the require_tls policy gate and includes comprehensive tests.

# Sev Issue Where

No action items or warnings. Clean change.

@github-actions

Copy link
Copy Markdown
🤖 AI Agent: docs-sync-checker — Docs Sync

AI-generated review output. Treat it as untrusted analysis and verify before acting.

Docs Sync

  • McpAuthPolicy.check() in agent_os/mcp_auth_enforcement.py -- missing updated docstring to reflect the new behavior of effective_url = url or entry.url.
  • McpServerEntry in agent_os/mcp_auth_enforcement.py -- missing updated docstring to clarify the behavior of require_tls with respect to the url field.
  • README.md -- no evidence of updates to reflect changes in McpAuthPolicy.check() behavior or require_tls enforcement.
  • CHANGELOG -- missing entry for the behavioral change in McpAuthPolicy.check() regarding require_tls enforcement.

@github-actions

Copy link
Copy Markdown
🤖 AI Agent: breaking-change-detector — API Compatibility

AI-generated review output. Treat it as untrusted analysis and verify before acting.

API Compatibility

Severity Change Impact
High McpAuthPolicy.check now enforces require_tls against the configured server URL if the caller does not provide a URL. Existing configurations with require_tls=True and non-TLS URLs (e.g., http://) will now be denied, which may break existing setups that relied on the previous behavior.
Medium _tls_violation now explicitly rejects URLs without a scheme (e.g., mcp.internal) when require_tls=True. Configurations with scheme-less URLs and require_tls=True will now be denied, which may break existing setups.

Notes

  • The changes result in stricter enforcement of TLS requirements, which may cause previously allowed configurations to be denied.
  • The behavior for servers registered without a URL remains unchanged; the TLS gate is skipped in such cases.

@github-actions github-actions Bot added the size/M Medium PR (< 200 lines) label Jul 30, 2026
@github-actions

Copy link
Copy Markdown
🤖 AI Agent: test-generator — `agent-os/src/agent_os/mcp_auth_enforcement.py`

AI-generated review output. Treat it as untrusted analysis and verify before acting.

agent-os/src/agent_os/mcp_auth_enforcement.py

  • test_no_url_anywhere_leaves_the_gate_skipped -- Add a test for cases where both url and entry.url are empty, ensuring the gate is skipped as intended.
  • test_invalid_url_format_handling -- Test behavior when urlparse raises an exception due to an invalid URL format.
  • test_tls_violation_edge_cases -- Validate _tls_violation with edge cases like malformed URLs or unexpected input types.

agent-os/tests/test_mcp_auth_enforcement.py

  • test_invalid_scheme_error_message -- Verify that _tls_violation produces the correct error message for unsupported schemes like ftp or gopher.
  • test_empty_url_error_handling -- Ensure _tls_violation handles empty or None URLs gracefully.

@github-actions

Copy link
Copy Markdown

🔴 Contributor Check: HIGH

Check Result
Profile HIGH
Credential LOW
Overall HIGH

Automated check by AGT Contributor Check.

@github-actions github-actions Bot added the needs-review:HIGH Contributor reputation check flagged HIGH risk label Jul 30, 2026
@github-actions

Copy link
Copy Markdown

PR Review Summary

Check Status Details
🔍 Code Review ⚠️ Missing No current-run comment
🛡️ Security Scan ⚠️ Missing No current-run comment
🔄 Breaking Changes ⚠️ Missing No current-run comment
📝 Docs Sync ⚠️ Missing No current-run comment
🧪 Test Coverage ⚠️ Missing No current-run comment

Verdict: ⚠️ AI review incomplete; ready for human review

AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

TL;DR: 0 blockers, 2 warnings. Warnings are fine as follow-ups.

This PR fixes a correctness gap in agent_os MCP auth enforcement: require_tls is now enforced against the effective server URL (caller url if provided, otherwise the server’s configured entry.url), and the TLS scheme check is centralized in a helper. It also adds regression tests covering the previously fail-open configuration path (notably from_yaml).

Changes:

  • Enforce require_tls against effective_url = url or entry.url, so configured plaintext URLs are denied even when callers omit url.
  • Refactor scheme validation into _tls_violation() and introduce a TLS_SCHEMES module constant.
  • Add a dedicated test suite covering configured URL enforcement, schemeless URLs, override behavior, and YAML parsing.
# Sev Issue Where
1 Warn New tests use is True / is False boolean identity assertions; rest of file uses truthiness tests/test_mcp_auth_enforcement.py
2 Warn Inline comment inaccurately describes prior TLS logic as prefix-based / bypassed by non-http schemes src/agent_os/mcp_auth_enforcement.py

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
agent-governance-python/agent-os/src/agent_os/mcp_auth_enforcement.py Applies TLS gate to configured URL via effective_url and factors TLS scheme logic into _tls_violation() with TLS_SCHEMES.
agent-governance-python/agent-os/tests/test_mcp_auth_enforcement.py Adds regression tests ensuring require_tls applies to YAML/configured URLs and covers schemeless/non-TLS schemes.
Comments suppressed due to low confidence (6)

agent-governance-python/agent-os/tests/test_mcp_auth_enforcement.py:148

  • Avoid ... is True in assertions; prefer assert expr for consistency with the rest of the test module and to avoid boolean-identity comparisons.
    @pytest.mark.parametrize("url", ["https://mcp.internal/finance", "wss://mcp.internal/ws"])
    def test_tls_configured_url_is_allowed(self, url: str) -> None:
        assert self._policy(url).check("finance", auth_method="mtls").allowed is True

agent-governance-python/agent-os/tests/test_mcp_auth_enforcement.py:161

  • These assertions use is True/is False. In this file, other tests use truthiness (assert ... / assert not ...), which is clearer and avoids boolean-identity comparisons.
        assert policy.check("finance", auth_method="mtls").allowed is True
        assert (
            policy.check("finance", auth_method="mtls", url="http://mcp.internal/finance").allowed
            is False
        )

agent-governance-python/agent-os/tests/test_mcp_auth_enforcement.py:168

  • Avoid ... is True in assertions; prefer assert expr for consistency with the rest of the test module.
    def test_no_url_anywhere_leaves_the_gate_skipped(self) -> None:
        # Unchanged behaviour: with no URL from either source there is nothing
        # to evaluate. Denying this would be a separate policy decision about
        # every server registered without a URL.
        assert self._policy("").check("finance", auth_method="mtls").allowed is True

agent-governance-python/agent-os/tests/test_mcp_auth_enforcement.py:172

  • Avoid ... is True/... is False in assertions; prefer assert expr / assert not expr to match the style used elsewhere in this test file.
    def test_require_tls_false_allows_plaintext(self) -> None:
        policy = self._policy("http://mcp.internal/finance", require_tls=False)
        assert policy.check("finance", auth_method="mtls").allowed is True

agent-governance-python/agent-os/tests/test_mcp_auth_enforcement.py:178

  • Avoid is False when asserting on booleans; the rest of this test module uses assert not result.allowed for denials.
        result = self._policy("http://mcp.internal/finance").check("finance", auth_method="oauth2")
        assert result.allowed is False
        assert "not allowed for server" in result.reason

agent-governance-python/agent-os/tests/test_mcp_auth_enforcement.py:189

  • Avoid ... is False in assertions; prefer assert not ... for consistency with the rest of the tests.
        assert policy.check("finance-tools", auth_method="mtls").allowed is False

Comment on lines +141 to +144
def test_plaintext_configured_url_is_denied_without_a_caller_url(self, url: str) -> None:
result = self._policy(url).check("finance", auth_method="mtls")
assert result.allowed is False
assert "requires TLS" in result.reason

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 3e349183 — switched to truthiness throughout.

You're right that it's inconsistent: the other 9 assertions in this file already use
assert not result.allowed, and mine were the odd ones out.

One note on the suggested alternatives, since they aren't equivalent here. == False is
ruff E712 and this package enables it (select = ["E"] in agent-os/pyproject.toml), so
it would fail lint:

E712 Avoid equality comparisons to `False`; use `not result.allowed` for false checks

not ... is clean, so that's what I used — which is also what you led with and what the
file already does. 31 passed.

Comment on lines 181 to 185
@@ -152,20 +184,28 @@ def check(self, server_name: str, auth_method: str, url: str = "") -> AuthCheckR
# all bypassed the require_tls gate because none of
# them start with "http://" — only `https://` and

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, and thanks — that paragraph is wrong and I should have caught it when I
added mine underneath it.

To be precise about what happened: the prefix wording is pre-existing, not new in this PR
(it's at the merge base, 24d5725). But it was accurate about nothing even then — the base
code is if entry.require_tls and url: followed by urlparse(url).scheme, exactly as you
say. So a non-HTTP scheme never bypassed anything; the only bypass was the falsy url,
which is what this PR fixes.

Since my paragraph sits directly below it, leaving it would read as though this PR were
also changing prefix matching to scheme parsing. Narrowed to what the code does in
3e349183:

# TLS check: the scheme is parsed and matched against the
# allowlist below, so only `https://` and `wss://` count as
# TLS-secured transports.

The falsy-url explanation below it stays, since that part is what changes here.

@LHMQ878

LHMQ878 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

The Spell Check failure here is not about anything this branch wrote — it is the CI bug in #3530, and I can now point at this PR as a live instance of it.

scripts/ci/changed_lines.py runs git diff <base> (two-dot: base tip vs working tree), so every commit that landed on main after this branch was cut shows up, and the pre-existing side of each shows up as an added line. Reproduced locally with the exact job command (cspell@8.17.3, --config .cspell.json):

$ python scripts/ci/changed_lines.py --base origin/main --mode added-lines ...
160125 added lines          # this PR's own diff is 133 lines, 2 files

$ cspell ci-diff/spell-check-added-lines.txt --config .cspell.json
... Unknown word (SNOMED) / (huggingface) / (CBRN) / (Sarin) / (cryptominer) / (SPENDGUARD) ...
   reported at lines 155439-159896, in files this branch does not touch

With the base resolved to its merge base with HEAD, which is what #3530 changes:

$ python scripts/ci/changed_lines.py --base $(git merge-base HEAD origin/main) ...
133 added lines

$ cspell ci-diff/... --config .cspell.json --no-progress --no-summary
(no output, exit 0)

So this branch's added lines are clean; the check is reading 40 commits of main as this PR's work. Nothing in this PR can fix it — the only contributor-side workaround is a rebase, which is why it is worth fixing in the job rather than in each branch.

I would rather not rebase to go green, since that hides the instance. Happy to rebase if a maintainer prefers a green tick here over the evidence.

…x note

Two review points from microsoft#3512, both correct.

The new tests used `assert ... is False`, while the other 9 assertions in
this file use `assert not ...`. `== False` is not an option here (ruff
E712 is enabled via `select = ["E"]`), so the fix is truthiness, which
also matches the file's existing convention.

The "rather than testing for the literal 'http://' prefix" paragraph was
inherited from the pre-existing comment and describes a prefix check that
was never in the code -- the gate was skipped because `url` was falsy, and
a URL that *was* supplied already went through scheme parsing. Narrowed it
to what the code actually does; the part explaining the falsy-`url` skip,
which is what this PR changes, stays.
Copilot AI review requested due to automatic review settings August 2, 2026 20:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comment on lines +139 to +140
if not scheme:
return f"URL {url!r} has no scheme, so TLS cannot be verified"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 97d400d8, and this one is mine rather than pre-existing — worth being explicit
about that, since the previous two were inherited.

The base version only ever emitted the scheme (URL scheme {scheme!r} is not in the TLS allowlist). I added the no-scheme branch in this PR and echoed the URL verbatim in it, so
http://user:s3cret@mcp.internal or ?access_token=s3cret would have reached
agt.audit.reason. Confirmed the propagation path you cite:

# otel_audit_backend.py
ATTR_REASON = "agt.audit.reason"
...
if entry.reason:
    attributes[ATTR_REASON] = entry.reason

Note the schemed branch was never affected — it only ever interpolated scheme. It was
specifically the schemeless case, which is the one where the "URL" is most likely to be a
raw config string.

Kept the message actionable without the URL by naming the allowlist instead:

if not scheme:
    return f"the URL has no scheme, so TLS cannot be verified (need {allowlist})"

server_name is already on AuthCheckResult and is in the outer reason, so context isn't
lost. Added a parametrized regression test over userinfo, query-param, and schemeless forms
asserting both the secret and the raw URL are absent from reason — it fails on the previous
commit and passes now. 34 passed.

Third review point from microsoft#3512, and this one is mine: the no-scheme branch
echoed the URL verbatim.

`AuthCheckResult.reason` is promoted into audit attributes --
`otel_audit_backend` maps it to `agt.audit.reason` -- and a server URL can
carry a token in its userinfo or query string, so
`http://user:s3cret@host` or `?access_token=s3cret` would land in logs.
The base version only ever emitted the scheme, so this was a regression I
introduced, not pre-existing.

The scheme is all the message needs, and it is not secret. Added the
allowlist to the no-scheme text so the reason stays actionable without the
URL, and a parametrized test asserting the secret and the raw URL are both
absent from `reason`.
Copilot AI review requested due to automatic review settings August 2, 2026 21:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@liamcrumm

Copy link
Copy Markdown
Contributor

The gate fix is right, since require_tls was skipped whenever the caller passed no url, which is every from_yaml call site. Servers that are not in the allowlist fall through to the default policy path, which applies no TLS check at all, so an unregistered http:// server still passes on auth method alone. min_tls_version is parsed from YAML and stored on McpServerEntry but never read anywhere in the repo, so require_tls is all or nothing.

@LHMQ878

LHMQ878 commented Aug 5, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for confirming. I agree the configured server URL needs to go through the same TLS gate, and the current branch keeps that enforcement while avoiding exposing the URL in the denial reason.

@MohammadHaroonAbuomar

Copy link
Copy Markdown
Collaborator

Closing per maintainer decision; this repository is not accepting submissions from this account.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:HIGH Contributor reputation check flagged HIGH risk size/M Medium PR (< 200 lines) tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

McpAuthPolicy require_tls never checks the server's configured URL, so a plaintext MCP server passes

4 participants