Skip to content

fix: decode nested encodings symmetrically - #4123

Merged
Ricky Gummadi (Ricky-G) merged 2 commits into
mainfrom
ricky-g-normalize-nested-encoding-decode-order
Sep 24, 2026
Merged

Ricky Gummadi (Ricky-G) merged 2 commits into
mainfrom
ricky-g-normalize-nested-encoding-decode-order

Conversation

@Ricky-G

Copy link
Copy Markdown
Contributor

Summary

Restore symmetric nested-encoding normalization in Python and Rust. Percent-, Unicode-escape-, and HTML-entity-wrapped base64/hex payloads now decode fully and emit both transform tags.

Fixes #3502 and #3521.

Problem

The ambiguous outer-layer guard required an English-marker gain. When decoding revealed another encoded blob, both sides scored zero, leaving the six reported nesting forms verbatim with no transform metadata. PR #3503 was closed unmerged, so the defect remains on main.

Changes

File What changed
agent-governance-python/agent-os/src/agent_os/normalize.py Reuse the base64/hex decoder to accept an outer decode only when the revealed blob actually produces printable UTF-8 text.
agent-governance-python/agent-os/tests/test_normalize.py Cover both nesting orders for all six wrapper/blob pairs and reject a nested blob that decodes to nonprintable content.
agent-governance-rust/agentmesh/src/normalize.rs Mirror the guard and regression coverage in Rust.

Testing

  • python -m pytest agent-governance-python/agent-os/tests/test_normalize.py -q — 23 passed.
  • cargo test --manifest-path agent-governance-rust/agentmesh/Cargo.toml normalize — passed.
  • python -m ruff check --select E,F,W --ignore E501 ... — passed.
  • rustfmt --edition 2021 --check agent-governance-rust/agentmesh/src/normalize.rs — passed.
  • Crate-wide cargo fmt --check still reports a pre-existing formatting difference in agentmesh/src/prompt_injection_embedding.rs; the changed module is formatted.

Gate ambiguous outer-layer decoding on an English gain or a printable base64/hex decode benefit, keeping Python and Rust aligned.

Signed-off-by: Ricky Gummadi <ricky.gummadi@outlook.com>

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@github-actions github-actions Bot added the size/L Large PR (< 500 lines) label Sep 24, 2026
@github-actions

Copy link
Copy Markdown

PR Review Summary

Check Status Details
🔍 Code Review ⚠️ Missing No current-run comment
🛡️ Security Scan ⚠️ Missing No current-run comment
🔄 Breaking Changes ⚠️ Missing No current-run comment
📝 Docs Sync ⚠️ Missing No current-run comment
🧪 Test Coverage ⚠️ Missing No current-run comment

Verdict: ⚠️ AI review incomplete; ready for human review

AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims.

@MohammadHaroonAbuomar MohammadHaroonAbuomar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • agent-governance-rust/agentmesh/src/normalize.rs:1 Both license-header checks fail because this file has lacked the two-line Microsoft MIT header since #2991 and the checks are diff-scoped; add the header and they go green.

Comment thread agent-governance-python/agent-os/tests/test_normalize.py Outdated
Add the MIT header to the Rust normalization module and rename the non-printable regression tests for spelling checks.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Ricky Gummadi <ricky.gummadi@outlook.com>
@Ricky-G

Copy link
Copy Markdown
Contributor Author

Both license-header checks fail because this file has lacked the two-line Microsoft MIT header since #2991; add the header and they go green.

Added the requested copyright and license header to normalize.rs in 3fcf3241.

@MohammadHaroonAbuomar MohammadHaroonAbuomar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at 3fcf324. The only changes since the verified 0cc316c are the license header on normalize.rs and the non_printable rename in both test names; both license-header checks and the spell check are green, and the fuzzing job still running is not required. The decoder change was verified in the previous round: all six wrapper-outside nesting forms decode with both tags in Python and Rust with no divergence, the printable-UTF-8 gate refuses binary and control-heavy payloads even when wrapped, ordinary text with percent escapes, entities, hex-looking words and JSON escapes is untouched, and the depth and ratio caps hold on large nested input.

@Ricky-G
Ricky Gummadi (Ricky-G) merged commit 70ab89e into main Sep 24, 2026
135 checks passed
@Ricky-G
Ricky Gummadi (Ricky-G) deleted the ricky-g-normalize-nested-encoding-decode-order branch September 24, 2026 01:31
Yuvraj Singh (yuvrajsingh2428) pushed a commit to yuvrajsingh2428/agent-governance-toolkit that referenced this pull request Oct 1, 2026
* fix: decode nested encodings symmetrically

Gate ambiguous outer-layer decoding on an English gain or a printable base64/hex decode benefit, keeping Python and Rust aligned.

Signed-off-by: Ricky Gummadi <ricky.gummadi@outlook.com>

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix: address normalize review feedback

Add the MIT header to the Rust normalization module and rename the non-printable regression tests for spelling checks.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Ricky Gummadi <ricky.gummadi@outlook.com>

---------

Signed-off-by: Ricky Gummadi <ricky.gummadi@outlook.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Signed-off-by: yuvrajsingh2428 <offcyuvi2428@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Large PR (< 500 lines) tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

normalize: nested encodings decode in only one order; percent/escape/entity wrapping a base64 or hex blob is left verbatim with no transform tag

2 participants