Skip to content

Consider lifting "> Skills used: ..." trailer strip from cs-agent-runner to general agent runner #30

@Jianping-Xiong

Description

@Jianping-Xiong

Problem

In theory, promptMode: "none" means "do not inject the Skills Reporting instruction into the system prompt", so models shouldn't emit any Skills-used trailer in their reply. In practice, some models (observed on Qwen; likely affects others trained on similar corpora) carry this pattern as a training prior and still output a trailing line like:

Skills used: skill-name

This is meta-info leak that's irrelevant — or outright confusing — to end users receiving the reply.

Current state

Customer Service path does a targeted regex strip in src/customer-service/rag/cs-agent-runner.ts before returning the reply (see commit 0e56871 fix(cs-agent-runner): strip model-injected "Skills used" trailer).

Other callers that use promptMode: "none" (e.g. pi-embedded-runner and any future ones) still surface the trailer.

Suggestion (not a decided plan)

Lift the strip to the general agent runner so every promptMode: "none" caller benefits. Things to consider before doing so:

  • Should the strip apply to all promptModes or only "none"? A "none" caller explicitly opts out of Skills Reporting, so it's the cleanest scope; other modes may legitimately want the trailer.
  • Is the regex robust across locales / different "Skills used" phrasings models might invent?
  • Is there a better anchor than regex (e.g. structured parsing of model output, or a post-processing hook in the agent runner)?

Opening this as a discussion — not a decided refactor plan.

References

  • Commit: 0e56871
  • Affected file (current fix): src/customer-service/rag/cs-agent-runner.ts

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions