Don't filter "<...>" in code paragraphs or inline code in markdown posted to issues, issue comments and PR descriptions #18829

Activity

1. dsyme commented on Feb 28, 2026

   

   dsyme

   on Feb 28, 2026

   CollaboratorAuthor

   More actions

   /scout Please investigate this and get back to me

   👀React with 👀1Reacted by github-actions
2. pelikhan commented on Feb 28, 2026

   

   pelikhan

   on Feb 28, 2026

   Collaborator

   More actions

   /scout Please investigate this and get back to me

   👀React with 👀1Reacted by github-actions
3. pelikhan commented on Feb 28, 2026

   

   pelikhan

   on Feb 28, 2026

   Collaborator

   More actions

   we can probably use unified
4. github-actions commented on Feb 28, 2026

   

   github-actionsBot

   on Feb 28, 2026 – with GitHub Actions

   Contributor

   More actions

   🔍 Scout Research Report

   Triggered by @pelikhan

   Executive Summary

   The issue is confirmed: sanitize_content_core.cjs applies XML/HTML filtering functions (removeXmlComments and convertXmlTags) as raw string operations across the entire markdown output — including code fences and inline code spans. This means generic type parameters like `List(String)` become `List(String)`, and `` style comments inside code blocks get silently removed. The fix is to parse markdown into an AST first and apply the aggressive filters only to prose text nodes, not code nodes. The unified/`remark` ecosystem (`@pelikhan`'s suggestion) is the right tool for this.

   Root Cause Analysis

   Current Sanitization Pipeline

   In actions/setup/js/sanitize_content_core.cjs, the main sanitizeContentCore() function runs at lines 776–803 in this order, all against the raw string:

   removeXmlComments(sanitized)   → removes  anywhere (including code blocks!)
   convertXmlTags(sanitized)      → converts (Tag) to (Tag) anywhere (including code blocks!)
   sanitizeUrlProtocols(sanitized)
   sanitizeUrlDomains(sanitized)
   neutralizeGitHubReferences(sanitized)
   neutralizeBotTriggers(sanitized)
   neutralizeTemplateDelimiters(sanitized)
   balanceCodeRegions(sanitized)
   

   The Two Problematic Functions

   removeXmlComments(s) (line 322): Uses `` in a code block disappears silently.

   convertXmlTags(s) (line 340): Uses /<(\/?[A-Za-z!][^>]*?)>/g globally — converts any (Tag) not on the allowed list to (Tag). Code blocks with generics like (T), (String), (int), (MyType) etc. are all hit. None of these are in the GFM allowlist.

   Both functions have no awareness of markdown code regions.

   Recommended Solution: AST-Aware Sanitization with unified/remark

   Approach

   Parse the markdown into an mdast AST, walk only the appropriate node types, apply filters selectively, then re-serialize.

   import { unified } from 'unified';
   import remarkParse from 'remark-parse';
   import remarkStringify from 'remark-stringify';
   import { visit, SKIP } from 'unist-util-visit';
   
   function sanitizeMarkdownAware(content, opts) {
     const processor = unified()
       .use(remarkParse)
       .use(() => (tree) => {
         // Apply aggressive XML/comment filters ONLY to prose text nodes
         visit(tree, 'text', (node) => {
           node.value = removeXmlComments(node.value);
           node.value = convertXmlTags(node.value);
           node.value = neutralizeTemplateDelimiters(node.value);
         });
   
         // URL filtering applies to link/image URLs (structural), not text
         visit(tree, ['link', 'image'], (node) => {
           node.url = sanitizeUrlProtocols(node.url);
           node.url = sanitizeUrlDomains(node.url, opts.allowedDomains);
         });
   
         // 'code' and 'inlineCode' nodes are intentionally skipped
       })
       .use(remarkStringify);
   
     return String(processor.processSync(content));
   }

   Node types to filter (prose text): text, heading, paragraph, blockquote, list, listItem

   Node types to skip (code content): code (fenced blocks), inlineCode (backtick spans), html

   Key Tools

   Package	Purpose
   unified + remark-parse	Parse markdown → mdast AST
   unist-util-visit	Walk the AST, visit specific node types
   remark-stringify / mdast-util-to-markdown	Re-serialize AST → markdown string

   All are already part of the unified ecosystem; unified is already referenced in the comments in this repo.

   Known Challenges to Address

   1. Round-trip fidelity: remark-stringify may normalize some markdown formatting (fence style, list markers, trailing spaces). The output is semantically identical but may look slightly different. Evaluate whether this is acceptable or if a custom stringifier is needed.

   2. neutralizeAllMentions and neutralizeBotTriggers: These currently operate on the raw string with lookaheads/lookbehinds to skip backtick-quoted mentions. In an AST approach, they would naturally only touch text nodes, which is cleaner — but the existing tests should be verified.

   3. Raw HTML in markdown: The html node type in mdast represents raw embedded HTML. This currently gets processed by convertXmlTags. A policy decision is needed: should raw HTML in markdown be filtered (current behavior) or preserved?

   4. URL filtering in text: URLs that appear as plain text (not as [text](url) links) may be parsed as text nodes rather than link nodes in some markdown flavors. The URL sanitizers need to continue handling these.

   Simpler Alternative: Code-Region-Aware String Splitting

   If AST round-tripping is deemed too risky for output fidelity, a lighter approach would be to split the input string into code and non-code regions first, apply filters only to non-code regions, and rejoin. The existing markdown_code_region_balancer.cjs already contains infrastructure for detecting code fences.

   This is more brittle than proper AST parsing (edge cases around nested fences, escaped backticks) but avoids the serialization fidelity concern. However, given that balanceCodeRegions already rewrites the code fence structure, using the full AST approach is likely cleaner and more maintainable.

   Recommendations

   1. Use unified/remark-parse + unist-util-visit to make the sanitization AST-aware, as suggested by @pelikhan. This is the cleanest fix.
   2. Apply removeXmlComments and convertXmlTags only to text nodes, skipping code and inlineCode node types entirely.
   3. Refactor URL filtering to operate on link/image node .url properties rather than raw string regex, while keeping a text-level URL scanner for bare URLs.
   4. Retain all current filters for non-code text — the security rationale for filtering (script) and similar tags in prose remains valid.
   5. Write regression tests covering: generic type parameters in inline code (`List(T)`), generic types in code fences, HTML comments in code blocks, and bare URLs in both code and prose.

   Key Sources

   • unist-util-visit — walk specific node types in the AST (supports SKIP to avoid descending into code nodes)
   • mdast-util-to-markdown — serialize mdast back to markdown
   • unified ecosystem — the core processing pipeline
   • [Debugging unified/remark ASTs]((swizec.com/redacted) — practical guide for working with the AST

   References: §22522220094

   🔭 Intelligence gathered by Scout
5. github-actions

   added

   scout

   on Feb 28, 2026
6. pelikhan

   added

   community

   safe-outputs

   on Feb 28, 2026
7. github-actions commented on Jul 31, 2026

   

   github-actionsBot

   on Jul 31, 2026 – with GitHub Actions

   Contributor

   More actions

   Warning

   threat detection engine error
   The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.

   Details

   The threat detection engine failed to produce results.

   Review the workflow run logs for details.

   Assigning to @pelikhan, the most active recent contributor to this repo, since this is the oldest open issue without an assignee.

   Generated by 🔧 Daily Assign Issue To User · auto · 18.9 AIC · ⊞ 5.4K · ◷
8. github-actions

   assigned

   pelikhan

   on Aug 1, 2026
9. github-actions commented on Aug 1, 2026

   

   github-actionsBot

   on Aug 1, 2026 – with GitHub Actions

   Contributor

   More actions

   Assigning to @pelikhan, the most recently active human contributor with merged PRs in this repo, as this is the oldest unassigned open issue.

   Generated by 🔧 Daily Assign Issue To User · auto · 16.5 AIC · ⌖ 11.4 AIC · ⊞ 5.4K · ◷
10. pelikhan commented on Oct 7, 2026

    

    pelikhan

    on Oct 7, 2026

    Collaborator

    More actions

    Status (2026-10-07; Needs progress check): The issue reports that safe-output sanitization changes angle-bracketed text inside Markdown code, and a maintainer suggested unified on February 28 as a possible approach. No fix is verified in this discussion; the next step is to check current behavior and assess a code-aware change while retaining necessary URL filtering.
11. pelikhan

    added

    bugSomething isn't workingSomething isn't working

    cluster:markdown-fidelityPreserve Markdown code syntax while applying safe-output sanitizationPreserve Markdown code syntax while applying safe-output sanitization

    on Oct 14, 2026
12. pelikhan commented on Oct 16, 2026

    

    pelikhan

    on Oct 16, 2026

    Collaborator

    More actions

    Current-state refresher (2026-10-16): apparently resolved for the reported angle-bracket corruption. #22005 merged on March 21. Current sanitization applies XML-comment removal and tag conversion only to non-code regions; fenced code and inline code retain angle brackets. Regression cases cover backtick/tilde fences, inline and multi-backtick spans, generic types, and XML comments inside versus outside code.

    I also executed a focused local check against the current core sanitizer: inline List<T>, fenced Array<string> and a code comment were preserved, while a prose <Abc> tag was neutralized and an untrusted URL was filtered. Thus preserving code did not simply disable the remaining filters. This was a targeted reproduction check, not the full test suite or a live issue-posting workflow.

    The October 7 refresher's lack of a verified fix is superseded by this implementation/merge evidence. Any remaining failure should be narrowed to a specific Markdown edge case.
13. pelikhan

    closed this as completedon Oct 16, 2026