Problem
In theory, promptMode: "none" means "do not inject the Skills Reporting instruction into the system prompt", so models shouldn't emit any Skills-used trailer in their reply. In practice, some models (observed on Qwen; likely affects others trained on similar corpora) carry this pattern as a training prior and still output a trailing line like:
Skills used: skill-name
This is meta-info leak that's irrelevant — or outright confusing — to end users receiving the reply.
Current state
Customer Service path does a targeted regex strip in src/customer-service/rag/cs-agent-runner.ts before returning the reply (see commit 0e56871 fix(cs-agent-runner): strip model-injected "Skills used" trailer).
Other callers that use promptMode: "none" (e.g. pi-embedded-runner and any future ones) still surface the trailer.
Suggestion (not a decided plan)
Lift the strip to the general agent runner so every promptMode: "none" caller benefits. Things to consider before doing so:
- Should the strip apply to all promptModes or only
"none"? A "none" caller explicitly opts out of Skills Reporting, so it's the cleanest scope; other modes may legitimately want the trailer.
- Is the regex robust across locales / different "Skills used" phrasings models might invent?
- Is there a better anchor than regex (e.g. structured parsing of model output, or a post-processing hook in the agent runner)?
Opening this as a discussion — not a decided refactor plan.
References
- Commit: 0e56871
- Affected file (current fix):
src/customer-service/rag/cs-agent-runner.ts
Problem
In theory,
promptMode: "none"means "do not inject the Skills Reporting instruction into the system prompt", so models shouldn't emit any Skills-used trailer in their reply. In practice, some models (observed on Qwen; likely affects others trained on similar corpora) carry this pattern as a training prior and still output a trailing line like:This is meta-info leak that's irrelevant — or outright confusing — to end users receiving the reply.
Current state
Customer Service path does a targeted regex strip in
src/customer-service/rag/cs-agent-runner.tsbefore returning the reply (see commit 0e56871fix(cs-agent-runner): strip model-injected "Skills used" trailer).Other callers that use
promptMode: "none"(e.g.pi-embedded-runnerand any future ones) still surface the trailer.Suggestion (not a decided plan)
Lift the strip to the general agent runner so every
promptMode: "none"caller benefits. Things to consider before doing so:"none"? A"none"caller explicitly opts out of Skills Reporting, so it's the cleanest scope; other modes may legitimately want the trailer.Opening this as a discussion — not a decided refactor plan.
References
src/customer-service/rag/cs-agent-runner.ts