Skip to content

SystemNotificationEvent content renders as ordinary assistant text instead of distinct system-notification style #958

Open

Description

@JoshuaRowePhantom

Summary

<system_notification> content from the Copilot SDK appears under an [assistant] label with the XML tags visible. The data model is correct end-to-end; the bug is entirely in the rendering layer and consists of two separate root causes that must both be present to produce the observed symptom.

This bug is CLI-only. The GUI WebView path correctly renders [system] because ChatHistoryItemViewModel uses item.Role.Value for the label. The CLI path ignores item.Role entirely.

Root Cause 1 — CLI OnTurnCompleted Always Labels Items "assistant"

File: Phantom.Workspaces.Agent.Cli/Program.cs, lines 355–376

private void OnTurnCompleted(object? sender, AgentChatHistoryItem item)
{
    var itemText = string.Concat(item.Contents.OfType<TextContent>().Select(static c => c.Text));
    ...
    if (!this.supportsInteractiveRendering)
    {
        Console.WriteLine($"assistant > {itemText}");  // ignores item.Role
        return;
    }
    this.FinalizeAssistantTurn();  // also always the assistant visual context
}

AgentChat.TurnCompleted fires for every AgentChatHistoryItem promoted to history — including {Role=System} items. OnTurnCompleted (subscribed at line 92) never inspects item.Role. A system-role item is rendered identically to an assistant message.

Fix: Branch on item.Role — for ChatRole.System, either render with a system > prefix, route to a separate display path, or suppress CLI output (system notifications are informational metadata, not assistant replies).

Root Cause 2 — StripSystemNotificationTags Exact-Match Failure

File: Phantom.Workspaces.Llm.Core/CopilotSdkStreamAdapter.cs, lines 272–285

private static string StripSystemNotificationTags(string content)
{
    const string openTag = "<system_notification>";
    const string closeTag = "</system_notification>";

    var trimmed = content.Trim();
    if (trimmed.StartsWith(openTag, StringComparison.Ordinal)
        && trimmed.EndsWith(closeTag, StringComparison.Ordinal))
    {
        return trimmed[openTag.Length..^closeTag.Length].Trim();
    }

    return content;   // raw XML returned if pattern doesn't match exactly
}

The SDK documentation states the notification content is "typically" wrapped in <system_notification> tags — not always. If the content has text outside the tags, uses different casing, or has no tags at all, StripSystemNotificationTags returns the content verbatim including the XML markup.

Fix: Use a regex or IndexOf-based approach to extract the inner content regardless of surrounding text, or accept that tag-less content is also valid and strip tags wherever they appear in the string.

Data Pipeline (verified correct)

The SystemNotificationEvent carries the correct ChatRole.System through every layer:

Step Location Role preserved
SystemNotificationEvent received CopilotSdkStreamAdapter.cs:168–178 ✅ ChatRole.System set
Routed to root channel CopilotSubAgentRouter.cs:83 ✅
Passed through middleware StreamingPersistenceMiddleware.cs ✅
Coalesced to history item AgentChat.cs CoalesceSegmentAsync:1285–1292 ✅ AgentChatHistoryItem { Role=System }
CLI rendering Program.cs:369 ❌ Role ignored — labeled assistant >
GUI rendering ChatHistoryItemViewModel.cs:19 ✅ Shows [system] correctly

Affected Files

File Root Cause What to change
Phantom.Workspaces.Agent.Cli/Program.cs #1 Inspect item.Role in OnTurnCompleted; render system items differently
Phantom.Workspaces.Llm.Core/CopilotSdkStreamAdapter.cs #2 Robustify StripSystemNotificationTags to handle non-exact-match content

Expected Tests

Test Name Class What It Verifies
OnTurnCompleted_SystemRoleItem_DoesNotRenderAsAssistant CliRenderingTests (new) A history item with Role=System is not output with the assistant > prefix
OnTurnCompleted_SystemRoleItem_RendersWithSystemPrefix CliRenderingTests (new) A history item with Role=System renders with system > prefix (or is suppressed)
StripSystemNotificationTags_ContentWithSurroundingText_StillStripped CopilotSdkStreamAdapterTests Tags are stripped even when non-tag text precedes or follows them
StripSystemNotificationTags_ContentWithNoTags_ReturnsContentUnchanged CopilotSdkStreamAdapterTests Content with no tags is returned as-is without throwing
SystemNotificationEvent_AppearsInHistory_WithSystemRole CopilotSdkStreamAdapterTests End-to-end: SystemNotificationEvent produces AgentChatHistoryItem with Role=System

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdiagnosedRoot cause identified

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions