Include Map Drawings in Resolution Search Context #466
Conversation
…updated the `HeaderSearchButton` to include `drawnFeatures` in requests and modified the server-side logic to ensure the agent receives and parses this data correctly. I also improved the `resolutionSearch` agent's prompt to incorporate user-drawn features and measurements, while adding robust error handling and a `DrawnFeature` interface for better type safety. Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
|
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (2)
WalkthroughParses user-drawn map features from form data and forwards them into the resolution search pipeline; the agent now accepts an optional DrawnFeature[] and includes those features in the system prompt for analysis. Changes
Sequence DiagramsequenceDiagram
participant User as User / UI
participant Button as HeaderSearchButton
participant Action as AppActions
participant Agent as ResolutionSearch
User->>Button: Trigger search (includes mapData.drawnFeatures)
Button->>Button: Serialize `drawnFeatures` to JSON
Button->>Action: Submit form (messages, timezone, drawnFeatures)
Action->>Action: Parse `drawnFeatures` (try/catch -> array)
Action->>Agent: Call resolutionSearch(messages, timezone, drawnFeatures)
Agent->>Agent: Build system prompt (enumerate drawn features)
Agent->>Agent: Run AI resolution search
Agent-->>Action: Return results
Action-->>User: Present results
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
ⓘ Your approaching your monthly quota for Qodo. Upgrade your plan PR Compliance Guide 🔍Below is a summary of compliance checks for this PR:
Compliance status legend🟢 - Fully Compliant🟡 - Partial Compliant 🔴 - Not Compliant ⚪ - Requires Further Human Verification 🏷️ - Compliance label |
||||||||||||||||||||||||||
![charliecreates[bot]](https://avatars.githubusercontent.com/in/1140780?s=60&v=4){kind=small}
There was a problem hiding this comment.
The main gaps are runtime validation/constraints on drawnFeatures (untrusted input) and the limited usefulness of the current prompt enrichment because geometry is not included in any actionable form. Additionally, geometry: any makes it easy for oversized/unsafe structures to leak through, increasing prompt size/cost and reducing predictability.
Additional notes (5)
- Security |
app/actions.tsx:46-56
drawnFeaturesis parsed from an untrusted string and then forwarded into the agent. The current code silently falls back to[]on parse errors, but it doesn’t validate the shape of the parsed value (e.g., object vs array, overly large payloads). This can lead to prompt injection/garbage context, unexpectedly large prompts, and higher token costs/latency.
Even a minimal runtime validation (array + whitelisting fields + length limits) would materially improve safety and predictability without fighting TypeScript (since FormData is runtime).
-
Maintainability |
app/actions.tsx:51-55
Catching parse errors is good, butconsole.errorhere may end up noisy and could log user-provided content details in server logs. Since this is non-fatal and you already degrade gracefully, a lower log level (or a structured log without the raw payload) is usually preferable. -
Maintainability |
lib/agents/resolution-search.tsx:36-36
The agent prompt only includestype+measurement, but not the actual geometry (coordinates). Given the PR goal (“focus its analysis on the specific areas or lengths identified by the user”), omitting geometry prevents the model from associating the drawing with specific locations in the image.
If you intentionally excluded geometry to control prompt size, you should at least include a compact representation (e.g., bounding box, centroid, length/area numeric value) so the context is actionable.
- Maintainability |
lib/agents/resolution-search.tsx:26-31
DrawnFeature.geometryis typed asany, which makes it easy for unsafe/oversized structures to flow into prompts/logging later. Since this is specifically GeoJSON-ish, prefer a narrower type (evenunknown+ runtime refinement), or a minimal in-house shape like{ type: 'Polygon' | 'LineString'; coordinates: ... }.
This is especially relevant here because geometry size directly impacts prompt size and cost.
- Security |
lib/agents/resolution-search.tsx:26-26
The prompt only includestypeandmeasurement, butmeasurementis inserted verbatim. Ifmeasurementoriginates from user-editable UI text, it can be used for prompt injection (“ignore previous instructions…”) or simply to inflate the prompt. Also, theDrawnFeatureincludesgeometry, but it’s not used—either include a safe summary (e.g., bbox/length/area already computed) or drop it from the interface here to avoid implying the agent uses it.
Summary of changes
What changed
Frontend (components/header-search-button.tsx)
- Appends user map drawings to the resolution search request:
- Adds
formData.append('drawnFeatures', JSON.stringify(mapData.drawnFeatures || [])).
- Adds
Backend action (app/actions.tsx)
- Reads
drawnFeaturesfromFormData, parses it viaJSON.parse, and logs a parse error if invalid. - Passes the parsed drawings into the agent call:
resolutionSearch(messages, timezone, drawnFeatures).
Agent prompt (lib/agents/resolution-search.tsx)
- Introduces a
DrawnFeatureinterface (withtype: 'Polygon' | 'LineString',measurement, andgeometry). - Extends
resolutionSearch(...)to accept optionaldrawnFeaturesand conditionally enriches the system prompt with a bullet list of user-drawn features and their measurements.
ⓘ Your approaching your monthly quota for Qodo. Upgrade your plan PR Code Suggestions ✨Explore these optional code suggestions:
|
|||||||||||||||
…8159538311285122724
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@app/actions.tsx`:
- Around line 49-55: The parsed drawnFeatures may not be an array or may be too
large; update the handling around drawnFeaturesString parsing to validate and
cap the payload before sending to the agent: after JSON.parse in the same block
(affecting drawnFeatures variable), ensure the result is an Array, filter to
only expected feature objects (e.g., objects with geometry and properties), drop
invalid entries, and enforce a hard cap (e.g., maxFeatures) and/or per-feature
size limit (truncate large coordinates or remove heavy properties) — if
truncated, log a warning. Keep drawnFeatures as a safe, bounded array that you
pass onward to the agent (instead of the raw parsed value).
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (3)
app/actions.tsxcomponents/header-search-button.tsxlib/agents/resolution-search.tsx
🧰 Additional context used
🧬 Code graph analysis (3)
components/header-search-button.tsx (2)
components/map/google-map.tsx (2)
features(38-52)df(39-46)components/map/map-data-context.tsx (1)
mapData(41-49)
lib/agents/resolution-search.tsx (4)
components/map/google-map.tsx (2)
features(38-52)df(39-46)lib/agents/tools/geospatial.tsx (1)
Location(18-23)components/map/map-data-context.tsx (1)
MapData(15-32)components/map/google-geojson-layer.tsx (1)
feature(27-44)
app/actions.tsx (1)
lib/agents/resolution-search.tsx (1)
resolutionSearch(33-82)
🔇 Additional comments (4)
components/header-search-button.tsx (1)
93-93: LGTM: drawnFeatures serialized with a safe default.
This keeps the payload consistent and avoids undefined values.app/actions.tsx (1)
99-99: LGTM: drawnFeatures passed through to the agent.
This wires the new payload into the resolution search flow as intended.lib/agents/resolution-search.tsx (2)
26-34: LGTM: DrawnFeature interface and optional parameter are cleanly introduced.
This keeps backward compatibility and matches the map data shape.
49-51: LGTM: conditional prompt enrichment is clear.
Assuming upstream sanitization/capping, this integrates user-drawn context cleanly.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
| const drawnFeaturesString = formData?.get('drawnFeatures') as string; | ||
| let drawnFeatures = []; | ||
| try { | ||
| drawnFeatures = drawnFeaturesString ? JSON.parse(drawnFeaturesString) : []; | ||
| } catch (e) { | ||
| console.error('Failed to parse drawnFeatures:', e); | ||
| } |
There was a problem hiding this comment.
Validate and cap drawnFeatures before passing to the agent.
JSON.parse can return non-arrays or oversized payloads; this can break downstream logic and inflate the prompt. Consider array guards, type filtering, and size limits.
Suggested hardening
- const drawnFeaturesString = formData?.get('drawnFeatures') as string;
- let drawnFeatures = [];
- try {
- drawnFeatures = drawnFeaturesString ? JSON.parse(drawnFeaturesString) : [];
- } catch (e) {
- console.error('Failed to parse drawnFeatures:', e);
- }
+ const drawnFeaturesRaw = formData?.get('drawnFeatures');
+ let drawnFeatures = [];
+ try {
+ const parsed =
+ typeof drawnFeaturesRaw === 'string' && drawnFeaturesRaw.length > 0
+ ? JSON.parse(drawnFeaturesRaw)
+ : [];
+ drawnFeatures = Array.isArray(parsed)
+ ? parsed
+ .filter(
+ df =>
+ df &&
+ (df.type === 'Polygon' || df.type === 'LineString') &&
+ typeof df.measurement === 'string'
+ )
+ .slice(0, 25)
+ .map(df => ({
+ ...df,
+ measurement: df.measurement
+ .replace(/[\r\n]+/g, ' ')
+ .slice(0, 64),
+ }))
+ : [];
+ } catch (e) {
+ console.error('Failed to parse drawnFeatures:', e);
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const drawnFeaturesString = formData?.get('drawnFeatures') as string; | |
| let drawnFeatures = []; | |
| try { | |
| drawnFeatures = drawnFeaturesString ? JSON.parse(drawnFeaturesString) : []; | |
| } catch (e) { | |
| console.error('Failed to parse drawnFeatures:', e); | |
| } | |
| const drawnFeaturesRaw = formData?.get('drawnFeatures'); | |
| let drawnFeatures = []; | |
| try { | |
| const parsed = | |
| typeof drawnFeaturesRaw === 'string' && drawnFeaturesRaw.length > 0 | |
| ? JSON.parse(drawnFeaturesRaw) | |
| : []; | |
| drawnFeatures = Array.isArray(parsed) | |
| ? parsed | |
| .filter( | |
| df => | |
| df && | |
| (df.type === 'Polygon' || df.type === 'LineString') && | |
| typeof df.measurement === 'string' | |
| ) | |
| .slice(0, 25) | |
| .map(df => ({ | |
| ...df, | |
| measurement: df.measurement | |
| .replace(/[\r\n]+/g, ' ') | |
| .slice(0, 64), | |
| })) | |
| : []; | |
| } catch (e) { | |
| console.error('Failed to parse drawnFeatures:', e); | |
| } |
🤖 Prompt for AI Agents
In `@app/actions.tsx` around lines 49 - 55, The parsed drawnFeatures may not be an
array or may be too large; update the handling around drawnFeaturesString
parsing to validate and cap the payload before sending to the agent: after
JSON.parse in the same block (affecting drawnFeatures variable), ensure the
result is an Array, filter to only expected feature objects (e.g., objects with
geometry and properties), drop invalid entries, and enforce a hard cap (e.g.,
maxFeatures) and/or per-feature size limit (truncate large coordinates or remove
heavy properties) — if truncated, log a warning. Keep drawnFeatures as a safe,
bounded array that you pass onward to the agent (instead of the raw parsed
value).
- Updated `HeaderSearchButton` to send `drawnFeatures` in resolution search requests. - Updated the request processing server action to parse and pass `drawnFeatures` to the agent. - Enhanced the `resolutionSearch` agent to include user-drawn features and their measurements in the system prompt. - Added `try-catch` and explicit typing for `drawnFeatures` to ensure a successful build. - Defined `DrawnFeature` interface for better type safety. Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
|
@coderabbitai the resolution search has no response |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 30616
The most likely causes for no response are:
Let's check the server logs first: Immediate debugging steps:
console.log('Starting resolutionSearch with:', {
messagesCount: messages.length,
timezone,
drawnFeaturesCount: drawnFeatures.length,
hasImage: messages.some(m => Array.isArray(m.content) && m.content.some(p => p.type === 'image'))
});
const analysisResult = await resolutionSearch(messages, timezone, drawnFeatures) as any;
console.log('ResolutionSearch result:', analysisResult);
export async function resolutionSearch(messages: CoreMessage[], timezone: string = 'UTC', drawnFeatures?: DrawnFeature[]) {
console.log('resolutionSearch called with:', { timezone, drawnFeaturesCount: drawnFeatures?.length });
// ... existing code ...
const { object } = await generateObject({
model: await getModel(hasImage),
system: systemPrompt,
messages: filteredMessages,
schema: resolutionSearchSchema,
})
console.log('resolutionSearch completed:', { hasSummary: !!object.summary, featuresCount: object.geoJson?.features?.length });
return object
}Can you:
🏁 Script executed: Length of output: 128 🏁 Script executed: Length of output: 3231 🏁 Script executed: Length of output: 5334
Root Cause AnalysisThe
If none of these API keys are configured, the model initialization will fail silently or throw an error, causing the resolution search to hang or fail without a response. Immediate Steps to Fix1. Check your server logsLook for errors like:
2. Verify API Key ConfigurationCheck that at least ONE of these environment variables is set: 3. Add Enhanced DebuggingAdd this logging to async function processResolutionSearch() {
try {
console.log('[DEBUG] Starting resolutionSearch with:', {
messageCount: messages.length,
timezone,
drawnFeaturesCount: drawnFeatures.length,
hasImage: messages.some(m => Array.isArray(m.content) && m.content.some(p => p.type === 'image'))
});
// Call the simplified agent, which now returns data directly.
const analysisResult = await resolutionSearch(messages, timezone, drawnFeatures) as any;
console.log('[DEBUG] ResolutionSearch completed:', {
hasSummary: !!analysisResult?.summary,
summaryLength: analysisResult?.summary?.length,
featureCount: analysisResult?.geoJson?.features?.length
});
// ... rest of codeAnd in export async function resolutionSearch(messages: CoreMessage[], timezone: string = 'UTC', drawnFeatures?: DrawnFeature[]) {
console.log('[DEBUG] resolutionSearch invoked:', { timezone, drawnFeaturesCount: drawnFeatures?.length });
try {
const localTime = new Date().toLocaleString('en-US', {
timeZone: timezone,
hour: '2-digit',
minute: '2-digit',
hour12: true,
weekday: 'long',
year: 'numeric',
month: 'long',
day: 'numeric'
});
// ... systemPrompt code ...
const filteredMessages = messages.filter(msg => msg.role !== 'system');
const hasImage = messages.some(message =>
Array.isArray(message.content) &&
message.content.some(part => part.type === 'image')
)
console.log('[DEBUG] About to call getModel with requireVision:', hasImage);
const model = await getModel(hasImage);
console.log('[DEBUG] Model retrieved successfully');
console.log('[DEBUG] Calling generateObject...');
const { object } = await generateObject({
model,
system: systemPrompt,
messages: filteredMessages,
schema: resolutionSearchSchema,
})
console.log('[DEBUG] generateObject completed:', { hasSummary: !!object.summary });
return object;
} catch (error) {
console.error('[ERROR] resolutionSearch failed:', error);
throw error;
}
}4. Quick QuestionsCan you tell me:
The most common issue is missing API key configuration - the code expects at least one provider to be set up. 🏁 Script executed: Length of output: 4976 Based on the README and package.json, I can see your project uses multiple AI providers. Here's what you need: Required Environment VariablesYour application needs at least ONE of these AI provider API keys configured: For resolution search with images, you MUST use a vision-capable model. Critical Questions to DebugPlease answer these to help identify the issue:
Without at least one properly configured AI provider API key, the resolution search will fail silently because Quick Test: Try adding an 🏁 Script executed: Length of output: 1589 🏁 Script executed: Length of output: 0 Also, I notice you tried to run my environment variable examples as a script - those were meant to be added to your 🏁 Script executed: Length of output: 461 🎯 Found the Issue!Root Cause: No AI provider API keys are configured in your Your ✅ Solution: Add an AI Provider API KeyYou need to add at least one of these to your Option 1: OpenAI (Recommended - Easiest to Get)# Add this to your .env file
OPENAI_API_KEY=sk-proj-...your_key_hereGet your key at: https://platform.openai.com/api-keys Option 2: xAI (Grok)# Add this to your .env file
XAI_API_KEY=xai-...your_key_hereGet your key at: https://console.x.ai/ Option 3: Google Gemini# Add this to your .env file
GEMINI_3_PRO_API_KEY=...your_key_here📝 Step-by-Step Fix
🔍 To Confirm the FixAfter adding an API key and restarting, check your server console. You should see the debug logs we discussed earlier if you added them, or if there are still errors, they should now appear in the console. Would you like me to help you with anything else once you've added an API key? For example, I can help you:
|
I've fixed the issue where the resolution search was not outputting tokens. I correctly switched to using |
- Switched to `streamObject` and `partialObjectStream` to fix the issue where resolution search was not outputting tokens. - Included user-drawn features and their measurements in the resolution search context. - Added robust error handling for parsing drawings and improved type safety. Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
This commit integrates several key features and fixes from the requested PRs: - Resolution Search: Added drawnFeatures support, fixed streaming logic, and improved vision model selection. - Mobile UI: Optimized chat input padding, fixed filename wrapping, and added mobile search portal. - Text Wrapping: Applied whitespace-normal and break-words to various components for better responsiveness. - Metadata: Updated LICENSE and Stripe checkout links. - Consistency: Maintained existing usage and history components as requested. Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
This commit provides a final consolidated update of the features requested from PRs #466, #469, #470, #471, and #473: - Resolution Search: Finalized streaming logic and spatial context integration. - UI: Consolidated Tailwind-based layout fixes for mobile responsiveness and text wrapping. - Stability: Removed initial values from StreamableValue instances in app/actions.tsx to ensure hydration consistency. - Metadata: Verified license and copyright information. - Compliance: Ensured Usage and History components remain intact as requested. Co-authored-by: ngoiyaeric <115367894+ngoiyaeric@users.noreply.github.com>
2 participants
User description
This change includes user-drawn map features (polygons, lines) and their corresponding measurements in the context provided to the resolution search agent. This allows the AI to focus its analysis on the specific areas or lengths identified by the user on the map.
Key changes:
components/header-search-button.tsxnow appendsmapData.drawnFeaturesto the searchFormData.app/actions.tsxextracts this data and passes it to the agent, with error handling for parsing.lib/agents/resolution-search.tsxuses the drawing data to enrich the system prompt, providing the AI with structured information about the user's focus areas.PR created automatically by Jules for task 18159538311285122724 started by @ngoiyaeric
PR Type
Enhancement
Description
Integrate map-drawn features into resolution search context
Pass drawn features from frontend to backend via FormData
Enrich agent system prompt with user-drawn measurements
Add DrawnFeature interface for type safety
Diagram Walkthrough
File Walkthrough
header-search-button.tsx
Add drawn features to search FormDatacomponents/header-search-button.tsx
drawnFeaturesfrommapDatato FormData as JSON stringactions.tsx
Extract and parse drawn features from requestapp/actions.tsx
drawnFeaturesstring from FormData in resolution_searchaction
array
drawnFeaturesarray toresolutionSearchagent functionresolution-search.tsx
Add drawn features to agent prompt contextlib/agents/resolution-search.tsx
DrawnFeatureinterface defining id, type, measurement, andgeometry
resolutionSearchfunction signature to accept optionaldrawnFeaturesparameterwhen present
Summary by CodeRabbit
New Features
Bug Fixes
✏️ Tip: You can customize this high-level summary in your review settings.