[pull] main from CopilotKit:main - #2
Merged
Merged
Conversation
Co-authored-by: Ariel Weinberger <Weinberger.Ariel@gmail.com>
Co-authored-by: Ariel Weinberger <Weinberger.Ariel@gmail.com>
* exit prerelease mode * Release v1.3.2 * fix formatting
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Added the Hacktoberfest folder called demos containing a README.md & TEMPLATE.md
changed the folder name to community from demos and removed LIST.md
fixed the typo here in readme
|
|
pull Bot
pushed a commit
that referenced
this pull request
Apr 22, 2026
…opilotKit#4155) ## Summary Showcase demos (e.g. gen-ui-tool-based's "Traffic pie chart", Beautiful Chat's pie/bar-chart suggestions) were rendering nothing in prod because aimock's substring-match fixtures cross-fired across demos with different tool surfaces, returning tool names the target agent never registered. This PR narrows the fixture patterns to fix the immediate breakage and adds a static validator so the same class of drift fails CI before it reaches prod. ## What broke Aimock serves deterministic responses in prod for cost reasons. Fixtures substring-match the user message and return hardcoded tool calls. The `"pie chart"` pattern returned `query_data` (for Beautiful Chat's two-step flow where it has that tool), but the same substring also fires for gen-ui-tool-based's suggestions — which only registers `render_pie_chart`. The returned `query_data` call dangles, no UI renders. Beautiful Chat also looped on the same fixture: aimock has no conversation-turn awareness, so after `query_data` returned its tool result, the follow-up model call had the same user-message context and the fixture re-matched, emitting `query_data` again — infinite until LangGraph's iteration cap. Exposed by PR CopilotKit#4113 (showcase-ops) / the prod-mode migration cluster that started routing prod traffic through aimock. The fixtures themselves had been incrementally drifting for a while, but the routing change turned silent drift into user-visible breakage across many demos simultaneously. ## Fix #1 — narrow the fixture patterns `showcase/aimock/feature-parity.json`: - Replaced the generic `"pie chart"` / `"bar chart"` / `"show pie"` / `"show bar"` patterns with **6 per-suggestion specific-phrase matches**. gen-ui-tool-based and declarative-gen-ui get `render_pie_chart` / `render_bar_chart` directly; Beautiful Chat gets `pieChart` / `barChart` with real data (skipping the query_data loop). - Narrowed `"schedule"` + `"meeting"` into one match for Beautiful Chat's "30-minute meeting to learn about CopilotKit" → `scheduleTime`. - Narrowed `"flight"` / `"fly"` to `"flights from SFO to JFK"`. - Narrowed `"background"` to `"sunset-themed gradient"`. - Removed `"trip"`, `"sales"`, `"pipeline"`, `"todo"` — too generic, substring-false-firing across unrelated demos. Interrupt and A2UI demos with those prompts fall through to the real LLM proxy. ## Fix #2 — static drift guardrail New `showcase/scripts/validate-fixture-tool-surface.ts`: - Pure `validate()` function: for each fixture with tool-call responses, finds every demo whose suggestion prompt contains the fixture's match substring, then asserts the fixture's returned tool names are all registered by that demo's agent. - CLI walks `packages/*/` collecting: - suggestions from `page.tsx` + sibling `hooks/*.tsx` - frontend tools from `useComponent` / `useHumanInTheLoop` / `useFrontendTool` / `useRenderTool` / `useDefaultRenderTool` calls - backend tools via `api/copilotkit*/route.ts` → agentId→graphId map → `langgraph.json` graph→Python-file → `@tool` decorators + `tools=[...]` arrays - 7 vitest cases written TDD-first (watched fail, then implemented) covering: drift detection, content-only fixtures, no-matching-demo, case-insensitivity, multi-demo cross-check, multi-tool responses. Runs as `npx tsx showcase/scripts/validate-fixture-tool-surface.ts`. Exit 0 = clean; exit 1 = per-fixture drift report. Current state: **33 fixtures × 191 demos, no drift.** Counterfactual: reverting just the pie-chart fixture fix correctly flags `langgraph-python/gen-ui-tool-based` and `langgraph-python/declarative-gen-ui` — i.e. the exact demos that broke in prod. ## Also fixed — unrelated LangGraph Python Dockerfile bug `showcase/packages/langgraph-python/Dockerfile`: `WORKDIR /app` left `/app` owned by root; the explicit `--chown=app:app` on COPY lines chowned *contents* but not the directory itself, so the `app` user couldn't create `.langgraph_api` (the in-memory LangGraph runtime's cache dir) and the agent crashed on boot with `PermissionError: [Errno 13]`. Added a non-recursive `chown app:app /app` (preserves the perf intent of not doing a recursive chown). ## Known follow-ups (out of scope for this PR) - Interrupt demos (`gen-ui-interrupt`, `interrupt-headless`, `hitl-in-chat`) and A2UI demos (`declarative-gen-ui`, `a2ui-fixed-schema`, `mcp-apps`, Calculator App) currently fall through to the real LLM when unmatched. Faithful fakes would need per-agent fixture keying in the aimock engine — separate upstream change. - Python backend-tool parser uses regex; misses star-spread collections like `tools=[query_data, *todo_tools]`. Safe failure mode — missing tools surface as guardrail drift rather than silently pass. - Add this validator to `.github/workflows/showcase_*.yml` so CI runs it on every PR touching fixtures or demos. ## Test plan - [x] `npx vitest run showcase/scripts/__tests__/validate-fixture-tool-surface.test.ts` — 7/7 pass - [x] `npx vitest run showcase/scripts/__tests__/aimock-fixtures.test.ts` — 17/17 still pass (no regression) - [x] `npx tsx showcase/scripts/validate-fixture-tool-surface.ts` — 33 fixtures × 191 demos, 0 drift - [x] Counterfactual: revert fixture fix, validator flags gen-ui-tool-based + declarative-gen-ui exactly - [x] Direct aimock probe (localhost:4010) for all 8 fixed prompts → returns correct tool call - [x] End-to-end through local LangGraph agent (localhost:3100) → gen-ui-tool-based "Traffic pie chart" suggestion → `render_pie_chart` with real data - [ ] Merge → Railway redeploys aimock + langgraph-python → smoke cycle on Beautiful Chat + gen-ui-tool-based + declarative-gen-ui suggestions
pull Bot
pushed a commit
that referenced
this pull request
Apr 24, 2026
The multimodal demo failed three ways:
1. Try-with-sample-image rendered a broken thumbnail because the sample
PNG on disk is a Git LFS pointer stub when LFS is not pulled at build
time. The browser base64-encoded the 130-byte text stub, fed it into
the chat as a valid-looking image/png, and CopilotChat rendered it
as a broken <img>.
2. Try-with-sample-pdf attached in the composer but the agent said
"no image/document attached". Same broken-image pipeline also hid the
real symptom: the published @ag-ui/langgraph converter (0.0.x) only
understands the legacy { type: binary, mimeType, data | url }
AG-UI content-part shape. The modern { type: image | document,
source: {...} } parts CopilotChat emits are silently filtered out of
the LangChain message stream, so the LangGraph agent never saw them.
3. Manual drag-and-drop of an image failed for the same reason as #2.
Fixes:
- sample-attachment-buttons.tsx: reject Git LFS pointer stubs and
validate PNG/PDF magic bytes on the client before feeding them to the
attachment queue. Surfaces an actionable error pointing at
git-lfs-pull instead of silently producing a broken thumbnail.
- page.tsx: install an onRunInitialized subscriber on the active agent
that rewrites outgoing user-message image / document / audio / video
parts to the legacy binary shape the runtime converter preserves.
Everything else (CopilotChat UI, useAttachments upload pipeline,
paperclip + drag-drop + paste paths) is untouched; we only retarget
the wire format so the LangGraph side receives the attachment.
- multimodal_agent.py: after the rewrite, attachments arrive in the
agent as LangChain image_url content parts (data URLs) regardless of
upstream modality. Route on MIME instead of the AG-UI part type:
image/* forwards to GPT-4o natively, application/pdf is flattened to
text via pypdf. The modern document shape is still handled for
forward-compat when the runtime is upgraded.
Verified: classifier helpers pass inline unit checks; tsc --noEmit
introduces no new type errors relative to HEAD (module-resolution
errors for @copilotkit/shared and lucide-react are pre-existing on this
branch and unrelated to the multimodal demo).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )