Enable follow-up questions for resolution searches #433
Conversation
- Implement sanitization of message context by filtering out image parts before calling `querySuggestor` to ensure compatibility with text-based models. - Call `querySuggestor` and append the `FollowupPanel` UI in the `resolution_search` action. - Update `aiState` with contextually relevant follow-up questions and markers, using a consistent `groupeId` for all assistant messages. - Ensure TypeScript compatibility for the sanitized message array.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
Important Review skippedBot user detected. To trigger a single review, invoke the You can disable this status message by setting the Comment |
PR Compliance Guide 🔍Below is a summary of compliance checks for this PR:
Compliance status legend🟢 - Fully Compliant🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label |
|||||||||||||||||||||||
PR Code Suggestions ✨Explore these optional code suggestions:
|
|||||||||
![charliecreates[bot]](https://avatars.githubusercontent.com/in/1140780?s=60&v=4){kind=small}
There was a problem hiding this comment.
The sanitization approach is directionally correct, but it may produce empty messages after filtering image parts, which can negatively affect or break follow-up suggestion prompting. The current as CoreMessage assertion is avoidable and can hide future schema mismatches; prefer returning properly typed objects without casting and consider dropping/normalizing empty-content messages.
Additional notes (1)
- Compatibility |
app/actions.tsx:97-105
The sanitization currently only removestype === 'image'. If other non-text multimodal parts exist (e.g.,file,audio,video, or future extensions), they’ll still be forwarded to a text-only model. Consider whitelisting supported part types (e.g.,text) rather than blacklistingimage, to make this resilient to future additions.
Summary of changes
What changed
- Added a
sanitizedMessages: CoreMessage[]transformation that filters outimageparts from any message whosecontentis an array. - Updated the
resolution_searchpath to callquerySuggestor(uiStream, sanitizedMessages)instead of passing the rawmessages. - Goal: ensure follow-up question generation works even when conversation history contains multimodal (image) parts.
| const sanitizedMessages: CoreMessage[] = messages.map(m => { | ||
| if (Array.isArray(m.content)) { | ||
| return { | ||
| ...m, | ||
| content: m.content.filter(part => part.type !== 'image') | ||
| } as CoreMessage | ||
| } | ||
| return m | ||
| }) | ||
|
|
||
| const relatedQueries = await querySuggestor(uiStream, sanitizedMessages); |
There was a problem hiding this comment.
sanitizedMessages currently preserves non-image parts but can leave a message with content: [] after filtering. Depending on how querySuggestor builds prompts, empty assistant/user messages can degrade results or cause edge-case failures (e.g., models or prompt builders expecting non-empty content). It’s safer to drop messages that become empty after sanitization, or replace them with a minimal text placeholder.
Also, this uses an as CoreMessage assertion even though you already annotated the array as CoreMessage[]. Prefer avoiding assertions here by constructing a value that naturally satisfies the type, or by narrowing in a way that doesn’t require casting—casts can mask future regressions when message shapes evolve.
Suggestion
Consider filtering out messages that become empty after removing image parts, and avoid the type assertion.
Example:
const sanitizedMessages: CoreMessage[] = messages
.map(m => {
if (!Array.isArray(m.content)) return m;
const content = m.content.filter(part => part.type !== 'image');
return { ...m, content };
})
.filter(m => !Array.isArray(m.content) || m.content.length > 0);If you need to preserve message count, you could replace empty array content with a short text part instead.
Reply with "@CharlieHelps yes please" if you'd like me to add a commit with this change.
|
|
2 participants
User description
This change enables context-aware follow-up questions and a follow-up panel after a resolution search (satellite/map image analysis), mirroring the behavior of regular queries. It fixes an issue where follow-up questions were not being generated, likely due to multimodal content in the conversation history which is now sanitized before being passed to the
querySuggestoragent. It also ensures consistent AI state updates and proper TypeScript typing.PR created automatically by Jules for task 14197512294429231241 started by @ngoiyaeric
PR Type
Enhancement, Bug fix
Description
Enable follow-up questions for resolution search results
Sanitize message context by filtering image parts before calling
querySuggestorEnsure compatibility with text-based models and consistent AI state updates
Diagram Walkthrough
File Walkthrough
actions.tsx
Add message sanitization and follow-up panel for resolution searchesapp/actions.tsx
message content before passing to
querySuggestorsanitizedMessagestransformation that preserves messagestructure while removing image content
FollowupPanelUI component within a Section titled"Follow-up" after resolution search
CoreMessage[]type