Skip to content

Commit e2948f1

Browse files
authored
feat(webapp): query boundary pinned end-to-end and a capped query retry (#4549)
## What & why The agent's query tool is read-only, but that was true by three separate facts and only one of them had a test. This proves the boundary holds by contract rather than by prompt, and stops a broken query from burning a whole agent turn. Two small guards, the rest is tests. [TRI-11165](https://linear.app/triggerdotdev/issue/TRI-11165). ## Stack Stacked on **#4548** (watch-mode keepalive). Merge that first. ## What's inside - **A route-level read-only test** (`apps/webapp/test/queryRouteReadOnly.test.ts`) that drives `api.v1.query` with a real signed environment JWT: a multi-statement write and a mutating statement are both refused before anything reaches ClickHouse, and a plain read passes so the seam stays live. - **`readonly=1` made non-overridable** in `queryService.server.ts` — caller `clickhouseSettings` were spread after the defaults and could clear it. - **A per-turn query-retry cap** in the agent's `run_query` tool (`internal-packages/dashboard-agent/src/tool-api.ts`): three consecutive failures returns a terminal "stop and answer with what you have". ## Key decisions - **The read-only guarantee is grammar-level, not filtered.** TRQL has no write statements — they don't parse — ClickHouse runs with `readonly=1`, and the org/project/env scoping is injected server-side from the credential. The request body can't widen scope or turn a read into a write. - **The deny test runs through the route, not the parser.** A parser-only test would stay green if a refactor routed agent SQL around the compiler; driving the real route with a signed JWT pins the boundary end-to-end, and the deliberate positive read keeps the assertion honest. - **The retry cap lives per turn, not in the prompt.** A failed query hands the model the database error to fix, and usually it does — but the only other limit was the turn's 10 steps, so one query the model couldn't fix could eat the whole turn and leave the user with no answer. The tool set is built per turn, so the counter caps consecutive failures; a success resets it. The retry instruction rides the error text, so the prompt prefix is unchanged. ## Testing - `queryRouteReadOnly.test.ts` — write statements → 400, ClickHouse never called; a read passes. - `tool-query-retry-cap.test.ts` — terminal at the third consecutive failure, counter resets on a success. - The load-bearing guards were control-broken first (readonly override re-enabled; cap removed) and the tests went red.
1 parent 550b717 commit e2948f1

41 files changed

Lines changed: 5293 additions & 64 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: feature
4+
---
5+
6+
The dashboard agent now comes with a monthly message allowance. A message that fails to send doesn't count against it.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: feature
4+
---
5+
6+
Watches now respect your plan's limits: free plans can run a limited number of watches at once and for a shorter window, with a prompt to upgrade for more.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: improvement
4+
---
5+
6+
Queries stay read-only, and the agent now stops after a few failed queries in a row and answers with what it found instead of spending the whole reply retrying.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: fix
4+
---
5+
6+
When queries are queued up and one is turned away, you now get a clear "try again shortly" instead of an error that looks like a problem with the query itself.

‎apps/webapp/app/components/dashboard-agent/AgentUpgradeGate.tsx‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
11
import { Link } from "@remix-run/react";
2+
import { AgentMonoLogo } from "~/components/primitives/AgentDotMatrix";
23
import { LinkButton } from "~/components/primitives/Buttons";
34
import { useOrganization } from "~/hooks/useOrganizations";
4-
import { cn } from "~/utils/cn";
55
import { v3BillingPath } from "~/utils/pathBuilder";
6-
import { AgentIcon, AGENT_ICON_ACCENT_CLASS, ASK_AGENT_LABEL } from "./agent-identity";
6+
import { ASK_AGENT_LABEL } from "./agent-identity";
7+
import { messageQuotaReachedCopy } from "./message-quota";
78

89
// Matches the composer's outer geometry so the replacement lands in the same place.
910
const SLOT = "flex shrink-0 flex-col bg-background-bright px-3 pb-3 pt-1";
@@ -22,14 +23,12 @@ export function AgentUpgradeBlock({
2223
{context}
2324
<div className="mt-1.5 flex flex-col gap-2 rounded-md border border-border-bright bg-background-dimmed p-3">
2425
<div className="flex items-center gap-1.5">
25-
<AgentIcon className={cn("size-4 shrink-0", AGENT_ICON_ACCENT_CLASS)} />
26+
<AgentMonoLogo size={16} decorative className="shrink-0" />
2627
<span className="text-sm font-medium text-text-bright">
2728
Upgrade to unlock {ASK_AGENT_LABEL}
2829
</span>
2930
</div>
30-
<p className="text-xs text-text-dimmed">
31-
You've used all {limit} messages included on the Free plan. Your chats stay here to read.
32-
</p>
31+
<p className="text-xs text-text-dimmed">{messageQuotaReachedCopy(limit)}</p>
3332
<LinkButton variant="primary/small" to={v3BillingPath(organization)} fullWidth>
3433
Upgrade
3534
</LinkButton>

‎apps/webapp/app/components/dashboard-agent/DashboardAgentChat.tsx‎

Lines changed: 27 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ import { DashboardAgentContextBanner } from "./DashboardAgentContextBanner";
1717
import { DashboardAgentHero } from "./DashboardAgentHero";
1818
import { DashboardAgentMessages, type TurnActivity } from "./DashboardAgentMessages";
1919
import { MESSAGE_TOO_LARGE_ERROR } from "./message-limits";
20+
import { FREE_PLAN_MESSAGE_LIMIT, parseQuotaReachedResponse } from "./message-quota";
2021
import { createTranscriptOrder, orderTranscript } from "./message-order";
2122
import { navigateDestination } from "./navigate-target";
2223
import { pendingNavigateIntents, pendingWatchIntents } from "./pending-intents";
@@ -102,6 +103,9 @@ export function DashboardAgentChat({
102103
onActivityChange?: (chatId: string, activity: TurnActivity | null) => void;
103104
}) {
104105
const [input, setInput] = useState("");
106+
// Set when the server refuses a send over the cap, so the block shows at once rather than
107+
// waiting for the next quota poll.
108+
const [quotaReached, setQuotaReached] = useState<{ limit: number } | null>(null);
105109
const navigate = useNavigate();
106110
const location = useLocation();
107111
const toast = useToast();
@@ -128,6 +132,18 @@ export function DashboardAgentChat({
128132
.catch(() => null)) as { error?: string } | null;
129133
throw new Error(data?.error ?? MESSAGE_TOO_LARGE_ERROR);
130134
}
135+
// Over the message cap: show the upgrade block instead of a generic turn error.
136+
if (res.status === 403) {
137+
const data = (await res
138+
.clone()
139+
.json()
140+
.catch(() => null)) as { error?: string; limit?: number } | null;
141+
const reached = parseQuotaReachedResponse(res.status, data);
142+
if (reached) {
143+
setQuotaReached(reached);
144+
throw new Error("You've reached your message limit.");
145+
}
146+
}
131147
return res;
132148
},
133149
clientData,
@@ -185,9 +201,12 @@ export function DashboardAgentChat({
185201
const orderRef = useRef(createTranscriptOrder(initialMessages));
186202
const messages = orderTranscript(rawMessages, orderRef.current);
187203

188-
// Counted here, not in the panel, so it includes the turn just sent.
189-
const quota = useAgentMessageQuota({ actionPath, chatId, messages });
190-
const atMessageCap = quota.kind === "reached";
204+
// Read here, not in the panel, so it re-reads as each turn settles.
205+
const quota = useAgentMessageQuota({ actionPath, chatId, status });
206+
// Either the poll saw the cap, or a send was just refused over it.
207+
const atMessageCap = quota.kind === "reached" || quotaReached !== null;
208+
const messageCapLimit =
209+
quotaReached?.limit ?? (quota.kind === "reached" ? quota.limit : FREE_PLAN_MESSAGE_LIMIT);
191210

192211
const isStreaming = status === "streaming";
193212
// From status, not the last part: the indicator must stay up through silent tool calls.
@@ -252,6 +271,8 @@ export function DashboardAgentChat({
252271
}, [sendRequest, submit, canSend]);
253272

254273
const retry = useCallback(() => {
274+
// Over the cap, a retry only earns another 403 — same guard as `submit`.
275+
if (atMessageCap) return;
255276
// A watch's consent record is a user message nobody typed, so retry never treats it as one.
256277
const action = retryAction(
257278
messages.filter((m) => !(m.role === "user" && isWatchRequestMessageId(m.id)))
@@ -264,7 +285,7 @@ export function DashboardAgentChat({
264285
return;
265286
}
266287
void sendMessage({ text: action.text, messageId: action.messageId });
267-
}, [messages, sendMessage, regenerate, clearError]);
288+
}, [messages, sendMessage, regenerate, clearError, atMessageCap]);
268289

269290
const resolveUri = useTriggerUriResolver(actionPath);
270291

@@ -414,9 +435,9 @@ export function DashboardAgentChat({
414435
/>
415436
)}
416437
{watchCard ? <div className="px-3 pb-2">{watchCard}</div> : null}
417-
{quota.kind === "reached" ? (
438+
{atMessageCap ? (
418439
<AgentUpgradeBlock
419-
limit={quota.limit}
440+
limit={messageCapLimit}
420441
context={
421442
<DashboardAgentContextBanner
422443
projectSlug={projectSlug}

‎apps/webapp/app/components/dashboard-agent/DashboardAgentDraft.tsx‎

Lines changed: 41 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import type { SuggestedPrompt } from "@internal/dashboard-agent-contracts";
22
import { useCallback, useMemo, useState } from "react";
3+
import { AgentUpgradeBlock } from "./AgentUpgradeGate";
34
import { DashboardAgentComposer } from "./DashboardAgentComposer";
45
import { DashboardAgentContextBanner } from "./DashboardAgentContextBanner";
56
import { DashboardAgentHero } from "./DashboardAgentHero";
@@ -16,6 +17,7 @@ export function DashboardAgentDraft({
1617
pageContext,
1718
promotedPrompt,
1819
watchCard,
20+
capReached,
1921
}: {
2022
onSubmit: (text: string) => void;
2123
projectSlug: string;
@@ -24,6 +26,7 @@ export function DashboardAgentDraft({
2426
pageContext?: AgentPageContext;
2527
promotedPrompt?: SuggestedPrompt;
2628
watchCard?: React.ReactNode;
29+
capReached?: { limit: number } | null;
2730
}) {
2831
const [input, setInput] = useState("");
2932

@@ -43,12 +46,14 @@ export function DashboardAgentDraft({
4346

4447
const submit = useCallback(
4548
(text: string) => {
49+
// Suggested prompts reach here via the hero, bypassing the composer's cap guard.
50+
if (capReached) return;
4651
const trimmed = text.trim();
4752
if (!trimmed) return;
4853
setInput("");
4954
onSubmit(trimmed);
5055
},
51-
[onSubmit]
56+
[onSubmit, capReached]
5257
);
5358

5459
return (
@@ -57,25 +62,41 @@ export function DashboardAgentDraft({
5762
pageContext={pageContext}
5863
promoted={promotedPrompt}
5964
composer={
60-
<div className="flex w-full flex-col gap-3">
61-
{watchCard}
62-
<DashboardAgentComposer
63-
layout="hero"
64-
value={input}
65-
onChange={setInput}
66-
onSubmit={() => submit(input)}
67-
onStop={() => {}}
68-
isStreaming={false}
69-
placeholderSuggestion={watchCard ? undefined : placeholderSuggestion}
70-
context={
71-
<DashboardAgentContextBanner
72-
projectSlug={projectSlug}
73-
environmentSlug={environmentSlug}
74-
currentPage={currentPage}
75-
/>
76-
}
77-
/>
78-
</div>
65+
capReached ? (
66+
<div className="flex w-full flex-col gap-3">
67+
{watchCard}
68+
<AgentUpgradeBlock
69+
limit={capReached.limit}
70+
context={
71+
<DashboardAgentContextBanner
72+
projectSlug={projectSlug}
73+
environmentSlug={environmentSlug}
74+
currentPage={currentPage}
75+
/>
76+
}
77+
/>
78+
</div>
79+
) : (
80+
<div className="flex w-full flex-col gap-3">
81+
{watchCard}
82+
<DashboardAgentComposer
83+
layout="hero"
84+
value={input}
85+
onChange={setInput}
86+
onSubmit={() => submit(input)}
87+
onStop={() => {}}
88+
isStreaming={false}
89+
placeholderSuggestion={watchCard ? undefined : placeholderSuggestion}
90+
context={
91+
<DashboardAgentContextBanner
92+
projectSlug={projectSlug}
93+
environmentSlug={environmentSlug}
94+
currentPage={currentPage}
95+
/>
96+
}
97+
/>
98+
</div>
99+
)
79100
}
80101
/>
81102
);

‎apps/webapp/app/components/dashboard-agent/DashboardAgentPanel.tsx‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ import {
2424
writeLastChat,
2525
} from "./last-chat-storage";
2626
import { DashboardAgentDraft } from "./DashboardAgentDraft";
27+
import { parseQuotaReachedResponse } from "./message-quota";
2728
import { WatchCard } from "./WatchCard";
2829
import { watchDraftFor } from "./watch-card";
2930
import { NO_WATCH_CARD, watchCardReducer } from "./watch-card-state";
@@ -114,6 +115,8 @@ export function DashboardAgentPanel({
114115
// Until the list has arrived, the page load's server count is the better answer.
115116
const [chatsLoaded, setChatsLoaded] = useState(false);
116117
const [active, setActive] = useState<ActiveChat | null>(null);
118+
// A refused `create` over the cap: the draft shows the upgrade block instead of a raw toast.
119+
const [capReached, setCapReached] = useState<{ limit: number } | null>(null);
117120
// Starts true so an `openWith` request waits for the restore instead of racing it.
118121
const [loading, setLoading] = useState(
119122
() => readLastChat(storageKey)?.path === location.pathname
@@ -260,14 +263,22 @@ export function DashboardAgentPanel({
260263
publicAccessToken?: string;
261264
headStarted?: boolean;
262265
error?: string;
266+
limit?: number;
263267
};
264268
if (seq !== openChatRequestSeq.current) return;
265269
if (!res.ok || !data.chatId || !data.publicAccessToken) {
270+
const reached = parseQuotaReachedResponse(res.status, data);
271+
if (reached) {
272+
setCapReached(reached);
273+
setActive(null);
274+
return;
275+
}
266276
console.error(`Dashboard agent: failed to create chat (${res.status})`, data.error);
267277
toast.error(data.error ?? "We couldn't start that chat. Try again in a moment.");
268278
setActive(null);
269279
return;
270280
}
281+
setCapReached(null);
271282
setActive({
272283
chatId: data.chatId,
273284
organizationId: organization.id,
@@ -309,6 +320,7 @@ export function DashboardAgentPanel({
309320
panelOrg.current = organization.id;
310321
claimChatSlot();
311322
setActive(null);
323+
setCapReached(null);
312324
setLoading(false);
313325
setChats([]);
314326
setChatsLoaded(false);
@@ -634,6 +646,7 @@ export function DashboardAgentPanel({
634646
pageContext={pageContext}
635647
promotedPrompt={promotedPrompt}
636648
watchCard={watchCardElement}
649+
capReached={capReached}
637650
/>
638651
)}
639652
</AgentPanelColumn>

‎apps/webapp/app/components/dashboard-agent/message-quota.test.ts‎

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,12 @@
11
import { describe, expect, it } from "vitest";
2-
import { countUserMessages, FREE_PLAN_MESSAGE_LIMIT, resolveMessageQuota } from "./message-quota";
2+
import {
3+
countUserMessages,
4+
FREE_PLAN_MESSAGE_LIMIT,
5+
MESSAGE_QUOTA_REACHED_ERROR,
6+
messageQuotaReachedCopy,
7+
parseQuotaReachedResponse,
8+
resolveMessageQuota,
9+
} from "./message-quota";
310

411
describe("resolveMessageQuota", () => {
512
it("caps a Free plan at the limit", () => {
@@ -42,6 +49,37 @@ describe("resolveMessageQuota", () => {
4249
});
4350
});
4451

52+
describe("parseQuotaReachedResponse", () => {
53+
it("maps a create/in 403 cap body to the limit", () => {
54+
// Both the create path and the `in` transport refuse with this exact body.
55+
expect(
56+
parseQuotaReachedResponse(403, { error: MESSAGE_QUOTA_REACHED_ERROR, limit: 20 })
57+
).toEqual({ limit: 20 });
58+
});
59+
60+
it("falls back to the free limit when the body omits it", () => {
61+
expect(parseQuotaReachedResponse(403, { error: MESSAGE_QUOTA_REACHED_ERROR })).toEqual({
62+
limit: FREE_PLAN_MESSAGE_LIMIT,
63+
});
64+
});
65+
66+
it("ignores other errors and non-403 statuses so they surface normally", () => {
67+
expect(parseQuotaReachedResponse(403, { error: "something_else" })).toBeNull();
68+
expect(parseQuotaReachedResponse(500, { error: MESSAGE_QUOTA_REACHED_ERROR })).toBeNull();
69+
expect(parseQuotaReachedResponse(403, null)).toBeNull();
70+
});
71+
});
72+
73+
describe("messageQuotaReachedCopy", () => {
74+
it("is a friendly sentence naming the limit, never the raw code", () => {
75+
const copy = messageQuotaReachedCopy(20);
76+
expect(copy).toContain("all 20 messages");
77+
expect(copy).toContain("Free plan");
78+
// Control break: if the mapping leaked the server code, this fails.
79+
expect(copy).not.toContain(MESSAGE_QUOTA_REACHED_ERROR);
80+
});
81+
});
82+
4583
describe("countUserMessages", () => {
4684
it("counts only what the user sent", () => {
4785
expect(

‎apps/webapp/app/components/dashboard-agent/message-quota.ts‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,28 @@ export function resolveMessageQuota({
2727
: { kind: "within", used, limit, remaining };
2828
}
2929

30+
// The server code both the create and `in` paths refuse with. The client owns the copy,
31+
// so this code must never reach the UI as text.
32+
export const MESSAGE_QUOTA_REACHED_ERROR = "message_quota_reached";
33+
34+
// Maps a 403 refusal body to the cap signal, or null for any other error. Both paths use
35+
// this so a `message_quota_reached` code routes to the upgrade block, never a raw toast.
36+
export function parseQuotaReachedResponse(
37+
status: number,
38+
data: { error?: string; limit?: number } | null | undefined
39+
): { limit: number } | null {
40+
if (status === 403 && data?.error === MESSAGE_QUOTA_REACHED_ERROR) {
41+
return { limit: data.limit ?? FREE_PLAN_MESSAGE_LIMIT };
42+
}
43+
return null;
44+
}
45+
46+
// The upgrade block's sentence. Pure so the copy is asserted directly, and so the raw
47+
// server code can never be what the user reads.
48+
export function messageQuotaReachedCopy(limit: number): string {
49+
return `You've used all ${limit} messages included on the Free plan. Your chats stay here to read.`;
50+
}
51+
3052
// A watch's consent record is a user message the person never typed, so it is
3153
// excluded here exactly as the stored count excludes it.
3254
export function countUserMessages(messages: { role: string; id?: string }[]): number {

0 commit comments

Comments
 (0)