POST /v1/roleplay is a native Cloudflare Worker route for long-running,
streaming roleplay. OpenAI-compatible clients can use
POST /roleplay/v1/chat/completions or
POST /v1/roleplay/chat/completions. Session state and orchestration stay in
the Worker. Each session maps to one SQLite-backed Durable Object, so turns for
that session stay ordered while different sessions scale independently. When
an OpenCode model is selected, only the provider request uses Container egress
to avoid OpenCode's Worker-signature block.
The production model policy uses NanoGPT first and learns between:
- Kimi K2.6:
kimi-k2.6 - stable GLM-5.2:
zai-org/glm-5.2:thinking,glm-5.2, andglm-5.2-venice - opt-in GLM-5.3:
glm-5.3through OpenCode only
The generic GLM route is pinned to 5.2: NanoGPT's subscription route first,
then OpenCode glm-5.2, and NavyAI's glm-5.2-venice last. OpenCode
glm-5.3 is available only when the caller explicitly selects the 5.3 alias.
LinkAPI remains a Kimi-only tier and OpenRouter is not part of the production
roleplay chain. Provider order is strict; latency and reliability operate only
among each tier's eligible families.
NanoGPT accepts NANOGPT_API_KEY, numbered NANOGPT_API_KEY_N secrets, and
the compatibility NANO_GPT_KEY[_N] names. A definite 401, 403, or 429
advances to the next key. A 402 insufficient-balance rejection does the same.
The successful key identifier—not the secret—is kept
in that Durable Object's session state and is preferred on later turns.
Fresh sessions begin with NANOGPT_PREFERRED_KEY_INDEX when it is configured;
they do not delay generation with a separate catalog probe. Definite
generation rejections still rotate immediately, and the remembered key is
revalidated after NANOGPT_KEY_CHECK_EVERY_REQUESTS successful uses.
NanoGPT GLM requests use the catalog's exact
zai-org/glm-5.2:thinking ID and its documented max reasoning effort.
Authenticate with the dedicated ROLEPLAY_API_KEY. The bootstrap
ADMIN_API_KEY remains accepted for administrative clients:
curl "$PROXY_BASE_URL/v1/roleplay" \
-H "Authorization: Bearer $ROLEPLAY_API_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: story-42-turn-1" \
-d '{
"session_id": "story-42-main",
"input": "I close the library door and ask who followed us.",
"character": {
"name": "Mira",
"persona": "A guarded court mage who hides concern behind dry humor.",
"scenario": "A rival has infiltrated the moonlit palace library.",
"style": "Tense gothic fantasy, close third person."
},
"lore": [
{
"keys": ["library", "palace"],
"content": "The palace library has one sealed passage behind the west shelves."
}
],
"model_preference": "auto",
"response_length": "balanced",
"prompt_cache": true,
"memory": {"mode": "auto"},
"stream": true
}'session_id accepts 8-128 letters, digits, underscores, or hyphens. An
explicit body or X-Roleplay-Session-ID value always wins. Without one, the
Worker hashes a client conversation_id, chat_id, or thread_id when
available. For OpenAI-style full-history requests, it otherwise derives a
stable session from the opening messages and the roleplay credential. A
delta-only request with no stable identifier receives a new generated session.
The selected value and its source are returned in X-Roleplay-Session-ID and
X-Roleplay-Session-Source.
Use either:
input: one new user turn.messages: OpenAI-style text messages.
When both are present, input is appended as the newest user message.
An individual system, developer, user, or assistant message may use the
remaining bounded request body instead of being rejected at 128,000
characters. Before provider egress, long text is split losslessly into ordered
same-role fragments below the gateway field limit; Unicode code points are
never split and concatenating the fragments reproduces the caller's text.
Protected directives that exceed one Durable Object value are stored in
ordered shards and reconstructed on later turns. Tool-result messages retain
the 128,000-character limit because splitting one result would break its
one-to-one tool-call association.
history_mode controls how incoming messages combine with stored history:
auto(default): detects common full-history and delta-message client patterns and avoids duplicate turns.append: treats every supplied message as new.replace: replaces recent raw history for this request.
character fields are retained with session memory. Lore entries are
request-scoped; only entries marked always or whose keys match recent text
are injected, which keeps each request bounded.
model_preference accepts auto, speed, kimi, glm, glm-5.2, or
glm-5.3. OpenAI-compatible
clients can instead set model to roleplay:auto, roleplay:speed,
roleplay:kimi, roleplay:glm, roleplay:5.2, or roleplay:5.3.
roleplay:glm and roleplay:5.2 both stay on 5.2 while following provider
priority. roleplay:5.3 selects only OpenCode 5.3. The concrete kimi-k2.6,
glm-5.3, and glm-5.2 values are also accepted.
Every roleplay generation defaults to the strongest provider-compatible
reasoning mode. Callers can lower generation effort with reasoning_effort;
semantic max maps to the selected provider's real ceiling. Model-backed
memory compaction remains at maximum reasoning. NavyAI receives max;
NanoGPT receives max for GLM and xhigh for Kimi; LinkAPI receives high;
OpenRouter receives reasoning.effort set to
high for Kimi and xhigh for GLM 5.2; OpenCode GLM 5.3 and 5.2 receive
max.
OpenCode Kimi K2.6 keeps its fixed native thinking mode because that transport
does not expose a supported effort overlay for that model.
Prompt caching is automatic above PROMPT_CACHE_MIN_TOKENS (1,024 estimated
input tokens by default). NanoGPT receives its caching: true routing hint only
in NANOGPT_BILLING_MODE=standard; subscription mode omits that PAYG-triggering
field and reports nanogpt-subscription-only. Other roleplay providers keep an unchanged schema and can reuse
stable prefixes through their automatic caching. Set "prompt_cache": false
for one turn or PROMPT_CACHE_ENABLED=false for the deployment. The response
reports X-MultiLLM-Prompt-Cache, X-MultiLLM-Prompt-Cache-Mode, and the
estimated eligible token count. Only upstream prompt reuse is requested;
roleplay outputs are never response-cached or replayed.
Prompt caching is not conversation storage. The production Worker persists and resends exact raw dialogue until it exceeds 128,000 estimated tokens. Crossing that boundary forces continuity compaction and retains the newest 32 raw messages beside the structured digest. The 640,000-byte Durable Object message budget prevents storage pressure from silently truncating an active history.
response_length accepts compact, balanced, or immersive. It changes the
pacing instruction without shrinking the model's available output capacity.
Every mode discourages repeated recap and stagnant dialogue.
Streaming is on unless "stream": false is sent. If max_tokens is omitted,
the Worker requests the selected provider/model's largest output that still
fits beside the current input. Explicit positive integer values remain caller
ceilings for ordinary API clients. output_mode: "unlimited", a
max_tokens value of at least 1000000, or an authenticated request from the
JanitorAI browser removes that caller ceiling. Provider and combined-context
capacity remain the physical safety boundary; there is no proxy-wide 20k
ceiling.
When a protected caller directive marks an IMAGE PROMPT: block as mandatory
for story responses, the Worker adds a short final-output reminder immediately
before dialogue and reserves at least
ROLEPLAY_IMAGE_PROMPT_MIN_OUTPUT_TOKENS (2,048 by default). This does not
rewrite bounded provider output or make a second request for bounded callers.
An explicit no image command or OOC-prefixed turn bypasses both the reminder
and budget floor.
During a quiet streaming interval, the Worker emits a valid SSE comment every
10 seconds. These keepalives carry no model content and let clients distinguish
long Kimi/GLM thinking from a dead connection. The Worker imposes no active
stream idle deadline; it continues until the provider finishes or the client
disconnects. Streaming responses use Cache-Control: no-cache, no-transform
so intermediaries do not transform or coalesce the SSE body. Ordinary provider
data events are forwarded unchanged; only an
intermediate output-limit terminator is rewritten for continuation. A stream is
recorded as complete only after [DONE] or a non-length finish reason. In
unlimited mode, an intermediate finish_reason: "length" and its [DONE] are
withheld while the same assistant response continues from its exact partial
text. The same repair applies to finish_reason: "stop" when a protected
directive requires an IMAGE PROMPT: and one or more of its declared standard
fields are still missing. A complete contract is accepted without another
request. Unlimited streams also continue when an upstream connection ends
after emitting partial assistant text but before sending a terminal finish
event. Image-contract repair is independently limited and proceeds only while
the canonical missing-field set strictly decreases. Current labels such as
Camera, Primary subject, Setting, Lighting, and Composition map to the
same contract as their legacy aliases. No-progress commentary is discarded,
multiple image markers are reduced to one final block, and only the cleaned
assistant text is eligible for session persistence. The client receives one
terminal finish event and one [DONE]. Finite
requests still expose their provider finish reason directly. An unterminated
EOF that cannot be continued is never stored as completed assistant memory.
If a provider closes after reasoning but before any visible response, the
Worker discards that hidden leg and retries the same model once from a clean
response boundary. The failed reasoning is neither streamed nor persisted.
Create a proxy configuration with:
| Field | Value |
|---|---|
| Name | MultiLLM Roleplay |
| Proxy URL | https://multillm-proxy.cserules.workers.dev/roleplay/v1/chat/completions |
| API key | The value of ROLEPLAY_API_KEY from the local .env |
| Model | roleplay:auto |
| Custom prompt | None unless the character needs an extra instruction |
The proxy URL is already the full Chat Completions endpoint, so leave
Add /chat/completions disabled. Save the configuration and hard-refresh
JanitorAI before selecting it.
Use roleplay:glm or roleplay:5.2 for the stable GLM-5.2 chain. Use
roleplay:5.3 only when deliberately testing OpenCode GLM-5.3; it does not
fall back to a different model version.
JanitorAI's public help pages do not specify the JSON sentinel used by its
Unlimited control. MultiLLM does not depend on that implementation detail:
an authenticated browser request whose Origin is https://janitorai.com is
always normalized to unlimited output. Known max_tokens: 1000000 sentinel
requests and explicit output_mode: "unlimited" API requests receive the same
behavior. A smaller number sent by JanitorAI is not allowed to truncate the
story.
If JanitorAI's browser console reports a connect-src Content Security Policy
block for the Worker origin, the request never reaches MultiLLM. A Worker CORS
header cannot override JanitorAI's CSP; JanitorAI must allowlist the origin, or
the client must use an endpoint already allowed by that policy. Direct
https://api.navy/v1/chat/completions is an alternative when JanitorAI permits
api.navy, but it bypasses MultiLLM failover and Durable Object memory.
JanitorAI normally sends OpenAI-style message history. Repeated requests with the same opening land on the same Durable Object, so compacted continuity memory survives later turns and Worker deployments until the inactivity TTL expires. If a client supplies a conversation ID, that ID is preferred. Two branches with exactly the same opening and no client conversation ID share the derived session; use an explicit ID with API clients that need branch isolation.
Caller-supplied system and developer messages are protected control
directives. They are stored separately from dialogue and reinserted unchanged
on later delta-only requests. When present, they lead the upstream message
list and suppress the Worker's generic roleplay-writing prompt. Explicit
structured character fields remain available as labelled context without
being placed ahead of the caller's contract. In auto mode, a newly supplied
directive set replaces the retained set; a request without directives reuses
it. append adds new directives, replace uses exactly the supplied set (and
therefore clears it when none are supplied), and memory.mode: off uses only
directives present in that request.
Every session records attempts, successes, failures, EWMA time to first byte, and EWMA total time for each provider/model pair. New sessions explore both Kimi and GLM, then prefer the faster reliable model. Per-session learning avoids a global Durable Object bottleneck and keeps regional behavior local to the conversation.
Automatic fallback occurs only after a response that clearly rejected work:
400, 401, 402, 403, 404, 413, 415, 422, 429, or an explicit
503 service-unavailable response. Ordered model arrays are exhausted inside
one provider before the next provider tier is tried. Transport errors and
other 5xx responses remain ambiguous because a provider might have started
generation; the Worker stops instead of risking a duplicate request.
Idempotency-Key is optional but recommended. The session stores recent keys
before any model call. Reusing one returns 409 and does not start another
generation. This is a duplicate-execution guard, not a response replay cache.
Memory modes:
auto: below the local threshold, no extra model call. Above it, the selected model receives older dialogue and decides whether semantic compaction is useful.force: asks for compaction whenever there is older dialogue.off: neither reads nor writes conversation memory for the turn. Routing metrics still update.
The compaction model must return a bounded JSON continuity digest containing a summary, character facts, relationships, world state, unresolved threads, and style. The digest is reinserted as untrusted historical context. Recent turns remain raw.
Only user, assistant, and tool dialogue can enter model compaction, local
extractive compaction, or a compaction checkpoint. system and developer
directives never enter those paths and do not count toward the normal dialogue
compaction trigger. The total context, including protected directives, still
controls mandatory compaction and the hard limit. If protected directives and
the retained recent context cannot fit, the request is rejected with 413
instead of weakening or summarizing the directives.
Storage pressure always makes compaction mandatory. The raw recent-message
window automatically shrinks below ROLEPLAY_KEEP_RECENT_MESSAGES until the
retained history and expected reply fit the storage budget. Storage-only
compaction changes the durable checkpoint for later turns but does not shorten
the current model request while its complete history still fits an eligible
provider. If context capacity itself forces compaction, the current generation
uses the compacted continuity digest.
Each successful compaction also stores a versioned SHA-256 checkpoint over the
character name and exact compacted user/assistant/tool prefix. Protected
directives are excluded from both the hash and message count. When a later
full-history request has the same checkpoint, the Worker removes only that
already-summarized dialogue prefix before merging and sends the durable
continuity digest instead. A mismatch keeps the checkpoint optimization
disabled and falls back to the normal history merge and compaction rules;
approximate or similarity-based checkpoint matches are never accepted.
Roleplay uses the same safety contract as MultiLLM context optimization:
authoritative instructions and the newest raw turn remain unchanged, while
eligible older dialogue becomes terse continuity notes. Repeated full-history
requests reuse the exact checkpoint instead of paying to summarize the same
prefix again. Response headers expose X-MultiLLM-Estimated-Input-Before,
X-MultiLLM-Estimated-Input-After, and
X-MultiLLM-Messages-Summarized; session metrics accumulate estimated input
tokens saved. Final assistant output is never compressed.
By default, the hard limit is derived from the largest eligible provider/model
context and compaction starts at 90% of that limit. A nonzero
ROLEPLAY_HARD_INPUT_TOKENS or ROLEPLAY_COMPACT_TRIGGER_TOKENS explicitly
overrides the derived value. When the effective hard limit is reached,
compaction becomes mandatory. If that model request fails, returns malformed
JSON, or declines, the Worker creates a
bounded, role-labelled extractive memory from the prior digest and archived
dialogue. Omissions are marked, the same dialogue-only SHA-256 checkpoint is
stored, and final generation continues without retrying the summarizer.
X-Roleplay-Memory: local_compacted and local_compactions in session metrics
expose this degraded path. Model compaction is an additional provider request
and can be billable.
All model-compaction candidates share one eight-second budget. A failed compaction starts a per-session exponential backoff (one minute up to fifteen minutes), so later forced turns immediately use local compaction instead of repeating a slow or rate-limited auxiliary call. A later successful model compaction clears the failure count and backoff.
Token counts are fast UTF-8 byte estimates, not provider tokenizer or billing measurements.
Session storage expires after 30 days of inactivity by default. The Durable Object alarm atomically removes that session's memory and metrics.
Ordinary OpenAI-compatible JSON bodies and SSE data events stay unchanged. If
an upstream model emits separate reasoning fields or raw <think> markup, the
Worker reconciles them into exactly one visible block:
<think>[provider: opencode | model: glm-5.2]
...provider reasoning...</think>
...visible roleplay response...
Duplicate or malformed provider reasoning is removed, including nested tags, stray closing tags, and tags divided across SSE frames. The selected route is written inside the block so clients that do not expose HTTP response headers can still identify it. Only the visible roleplay response is retained in continuity memory; provider reasoning and route metadata are not fed back into later turns.
During streaming silence, the Worker may interleave the SSE keepalive comment described above. The Worker also adds:
X-Roleplay-Session-IDX-Roleplay-Session-SourceX-Roleplay-ProviderX-Roleplay-ModelX-Roleplay-SelectionX-Roleplay-MemoryX-Roleplay-Estimated-Input-TokensX-Roleplay-Max-Output-TokensX-Roleplay-Fallback-CountX-Roleplay-State-CacheX-Roleplay-Credential-CheckServer-Timing
These headers are exposed through CORS.
Server-Timing separates session queue time, Durable Object state loading,
credential revalidation, memory compaction, request preparation, upstream
header wait, and total time to response headers. This makes slow storage,
compaction, same-session serialization, and provider startup independently
visible without logging roleplay content. The session metrics endpoint keeps a
bounded 64-request window per provider/model and reports p50, p95, and p99 for
time to first byte and total generation time. Metrics reads do not wait for an
active generation to finish; pending_turns exposes the active and queued
turn count alongside the last completed snapshot.
Warm Durable Object instances retain their already-loaded state in memory. Normal turns therefore avoid repeated storage reads, update the inactivity alarm at most once per day, and write only changed state partitions after the completion. Durable storage remains authoritative across eviction and Worker deployments; the in-memory copy is only a hot-path read cache.
The request sent to the selected provider has this order (memory and lore are omitted when absent):
{
"model": "kimi-k2.6",
"messages": [
{
"role": "system",
"content": "<proxy roleplay policy and character profile>"
},
{
"role": "system",
"content": "<caller system directive, unchanged>"
},
{
"role": "developer",
"content": "<caller developer directive, unchanged>"
},
{
"role": "system",
"content": "[Untrusted roleplay continuity memory...]"
},
{
"role": "system",
"content": "[Relevant lore]..."
},
{
"role": "user",
"content": "<recent compactable dialogue>"
}
]
}Apart from reasoning normalization, the selected provider's Chat Completions
JSON or SSE events are returned without reshaping. Session metrics report
protected_directives and estimated_protected_directive_tokens without
returning directive content.
Inspect configured tiers:
curl "$PROXY_BASE_URL/v1/roleplay/models" \
-H "Authorization: Bearer $ROLEPLAY_API_KEY"Inspect one session's bounded routing metrics without returning its dialogue:
curl "$PROXY_BASE_URL/v1/roleplay/metrics?session_id=story-42-main" \
-H "Authorization: Bearer $ROLEPLAY_API_KEY"The Worker requires the ROLEPLAY_SESSION Durable Object binding configured in
wrangler.jsonc. Store a roleplay-only client credential separately from the
administrative key:
npx wrangler secret put ROLEPLAY_API_KEYNanoGPT is the preferred provider:
npx wrangler secret put NANOGPT_API_KEY
npx wrangler secret put NANOGPT_API_KEY_1Add later tiers only when used:
npx wrangler secret put OPENCODE_GO_API_KEY
npx wrangler secret put LINKAPI_KEY
npx wrangler secret put OPENROUTER_API_KEY
npx wrangler secret put NAVYAI_API_KEYNon-secret tuning variables:
| Variable | Default | Purpose |
|---|---|---|
ROLEPLAY_PROVIDER_ORDER |
nanogpt,opencode,linkapi,openrouter,navyai |
Strict provider tiers |
ROLEPLAY_KIMI_MODEL |
kimi-k2.6 |
Default Kimi model ID |
ROLEPLAY_GLM_MODEL |
glm-5.2 |
Default GLM model ID |
ROLEPLAY_PROVIDER_MODELS |
{} |
JSON provider-specific Kimi/GLM ID or ordered fallback IDs |
ROLEPLAY_PROVIDER_FAMILIES |
{} |
JSON provider-to-family allowlists; an empty list disables that provider |
ROLEPLAY_PROVIDER_LIMITS |
{} |
JSON provider/family context and output overrides |
NANOGPT_PREFERRED_KEY_INDEX |
unset | Numbered NanoGPT key attempted first for a fresh session |
NANOGPT_KEY_CHECK_EVERY_REQUESTS |
50 |
Successful uses before the remembered NanoGPT key is catalog-revalidated |
ROLEPLAY_COMPACT_TRIGGER_TOKENS |
128000 |
Raw-dialogue threshold that forces continuity compaction; 0 derives it from provider capacity |
ROLEPLAY_COMPACT_TRIGGER_PERCENT |
90 |
Derived compaction threshold percentage |
ROLEPLAY_HARD_INPUT_TOKENS |
0 |
0 derives the hard limit from eligible providers |
ROLEPLAY_MEMORY_TARGET_TOKENS |
1200 |
Terse continuity-memory target |
ROLEPLAY_KEEP_RECENT_MESSAGES |
32 |
Raw recent messages retained after compaction; shrinks under storage pressure |
ROLEPLAY_IMAGE_PROMPT_MIN_OUTPUT_TOKENS |
2048 |
Minimum story budget when protected directives require a final image-prompt block |
ROLEPLAY_CONTEXT_REPLY_RESERVE_TOKENS |
4096 |
Reply space retained when deriving input capacity |
ROLEPLAY_CONTEXT_SAFETY_TOKENS |
1024 |
Combined-context safety margin |
ROLEPLAY_MAX_REQUEST_BYTES |
8388608 |
Bounded JSON ingress |
ROLEPLAY_MAX_STORED_BYTES |
640000 |
Exact raw-message storage budget before forced compaction |
ROLEPLAY_COMPACTION_MAX_TOKENS |
1200 |
Digest output ceiling |
ROLEPLAY_COMPACTION_TIMEOUT_MS |
8000 |
Shared model-compaction budget before local fallback |
ROLEPLAY_UPSTREAM_HEADER_TIMEOUT_MS |
90000 |
Header wait before fail-closed abort |
ROLEPLAY_STREAM_HEARTBEAT_MS |
10000 |
SSE keepalive interval during upstream silence |
ROLEPLAY_MAX_AUTO_CONTINUATIONS |
8 |
Maximum hidden continuation legs after genuine provider output limits or truncated EOFs |
ROLEPLAY_MAX_OUTPUT_CONTRACT_REPAIRS |
1 |
Separate maximum repairs for a stopped response with an incomplete required image-prompt contract; no-progress repairs stop immediately |
ROLEPLAY_SESSION_TTL_SECONDS |
2592000 |
Inactivity retention |
Provider catalogs can use namespaced IDs or expose multiple generations of a model family. A string selects one ID; an array preserves priority among the candidates eligible for the requested version. Generic GLM selects only 5.2:
{
"opencode": {
"glm": ["glm-5.2", "glm-5.3"]
},
"navyai": {
"glm": "glm-5.2-venice"
}
}Store that compact JSON as ROLEPLAY_PROVIDER_MODELS. Secrets never belong in
this variable or wrangler.jsonc.
The roleplay model catalog also returns context_window,
max_output_tokens, and limits_source for every configured route. Limits
are provider-specific: NanoGPT Kimi, for example, is not assumed to have the
same capacity as NavyAI Kimi.
OpenAI-compatible clients may send a max_tokens ceiling larger than a
selected provider supports. The roleplay route clamps that ceiling to the
provider's available model capacity and returns the effective value in
X-Roleplay-Max-Output-Tokens; it does not reject the turn solely because the
client requested a larger ceiling. Unlimited requests ignore the caller value,
use that effective provider capacity for each leg, and transparently continue
streaming after a length finish. Set ROLEPLAY_MAX_AUTO_CONTINUATIONS=0 to
disable output-limit continuation stitching without changing provider-capacity
selection. Image-prompt contract repair has its own smaller bound,
ROLEPLAY_MAX_OUTPUT_CONTRACT_REPAIRS (one by default), and is disabled by
setting that value to zero.
Override a gateway only when its live catalog or an exercised request proves a different limit:
{
"navyai": {
"glm": {
"context_window": 1048576,
"max_output_tokens": 131072
}
}
}Store that compact JSON as ROLEPLAY_PROVIDER_LIMITS.