Skip to content

Develop#42

Merged
ssdeanx merged 2 commits intomainfrom
develop
Dec 12, 2025
Merged

Develop#42
ssdeanx merged 2 commits intomainfrom
develop

Conversation

@ssdeanx
Copy link
Owner

@ssdeanx ssdeanx commented Dec 12, 2025

No description provided.

…port

- Added ProgressEvent and WorkflowProgressEvent interfaces to manage progress events in workflows and networks.
- Implemented ProgressEventItem and WorkflowProgressPanel components to display progress events in the UI.
- Updated NetworkRoutingPanel to show recent progress events.
- Enhanced WorkflowNode to filter and display streaming events for running steps.
- Introduced WorkflowSuspendDialog for handling workflow approval during pauses.
- Modified WorkflowProvider to extract and manage progress events from messages.
- Updated various components to integrate progress event handling and improve user experience.
- Introduced BatchWebScraperTool component for batch scraping functionality.
- Added LinkExtractorTool component for extracting links from web pages.
- Implemented SiteMapExtractorTool component for generating site maps.
- Enhanced WebScraperTool component with improved UI and functionality.
- Updated index files to export new tools for easier access.
- Removed unused tool-part-transform helper file to clean up the codebase.
Copilot AI review requested due to automatic review settings December 12, 2025 15:26
@continue
Copy link

continue bot commented Dec 12, 2025

All Green - Keep your PRs mergeable

Learn more

All Green is an AI agent that automatically:

✅ Addresses code review comments

✅ Fixes failing CI checks

✅ Resolves merge conflicts

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @ssdeanx, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@github-actions
Copy link

🤖 Hi @ssdeanx, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.

@coderabbitai
Copy link

coderabbitai bot commented Dec 12, 2025

Summary by CodeRabbit

  • New Features

    • Added progress event tracking and real-time display for network and workflow operations
    • Introduced workflow suspension and approval workflow with dialog UI for pausing and resuming executions
    • Added new web scraping tool interfaces with UI components for displaying scraper results, batch operations, site maps, and extracted links
    • Displays streaming progress updates during workflow and agent execution
  • Style

    • Updated responsive design styling across workflow components

✏️ Tip: You can customize this high-level summary in your review settings.

Walkthrough

This PR introduces progress event tracking and suspend/resume capabilities for workflows and networks, adds new web-scraper tool UI components, and removes the tool-part-transform mapping utility. Progress events are extracted from message parts and tracked in context state, displayed via new UI panels. Workflow suspension and approval flows are enabled through new context methods and dialog components.

Changes

Cohort / File(s) Summary
Context/Provider Updates
app/workflows/providers/workflow-context.tsx, app/networks/providers/network-context.tsx
Added WorkflowProgressEvent, WorkflowSuspendPayload, and ProgressEvent types; extended context interfaces with progressEvents, suspendPayload, and new methods (approveWorkflow, updated resumeWorkflow); implemented extraction of progress events from message parts and state management for suspension/resume flows.
Workflow UI Components
app/workflows/components/workflow-progress-panel.tsx, app/workflows/components/workflow-suspend-dialog.tsx, app/workflows/components/workflow-node.tsx, app/workflows/components/workflow-canvas.tsx
Added WorkflowProgressPanel and WorkflowSuspendDialog components; integrated progress event rendering in canvas; extended WorkflowNode to display in-progress streaming events; imported new types and components.
Network UI Components
app/networks/components/network-routing-panel.tsx
Extended NetworkRoutingPanel to render progress events with new ProgressEventItem component, displaying latest events with status icons and color-coding.
Web-Scraper Tool Components
src/components/ai-elements/tools/web-scraper-tool.tsx, src/components/ai-elements/tools/batch-web-scraper-tool.tsx, src/components/ai-elements/tools/site-map-extractor-tool.tsx, src/components/ai-elements/tools/link-extractor-tool.tsx
Added four new client-side React components for rendering web-scraper tool outputs with error, loading, and success states, including extraction summaries, tabbed content, and per-URL results.
Tool Exports
src/components/ai-elements/tools/index.ts, src/mastra/tools/index.ts, src/mastra/tools/web-scraper-tool.ts
Added exports for new tool components and tool implementations; introduced LinkExtractorUITool type alias; expanded mastra tools public API with nine new tool exports.
Workflow Header & Input Styling
app/workflows/components/workflow-header.tsx, app/workflows/components/workflow-input-panel.tsx
Updated Tailwind width and height classes from pixel-based to scale-based tokens; modified resume button onClick handler to use inline arrow function.
Removed
app/chat/helpers/tool-part-transform.ts
Deleted utility that mapped Mastra data-tool parts to DynamicToolUIPart objects, including nested shape handling and state normalization logic.

Sequence Diagram

sequenceDiagram
    participant Assistant as Assistant<br/>(Message)
    participant Context as Workflow/Network<br/>Context
    participant Extractor as Progress Event<br/>Extractor
    participant State as Context State
    participant Panel as Progress<br/>Panel/Dialog

    Assistant->>Context: Receive message with<br/>data-tool-* parts
    Context->>Extractor: Extract progress events<br/>from message parts
    Extractor->>Extractor: Parse stage, status,<br/>message, agentId
    Extractor->>State: Update progressEvents<br/>and suspendPayload
    State->>Panel: Trigger re-render with<br/>new events
    Panel->>Panel: Group by stage &<br/>render event items
    
    Note over Panel: If suspendPayload exists<br/>& status="paused"
    Panel->>Panel: Show SuspendDialog<br/>with approve/reject
    Panel->>Context: User clicks Approve<br/>(calls approveWorkflow)
    Context->>Context: Clear suspendPayload<br/>Resume workflow
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

  • Deletion of tool-part-transform.ts: Review implications of removing Mastra data-tool mapping; verify no orphaned dependencies remain.
  • Progress event extraction logic: Complex nested shape handling in both workflow-context.tsx and network-context.tsx useEffect hooks; requires verification of field mapping (stage, status, message, agentId, data).
  • Suspend/resume flow: New approveWorkflow method and updated resumeWorkflow signature with optional resumeData; verify state clearing and payload handling across all workflow transitions (selectWorkflow, stopWorkflow, runWorkflow).
  • New UI components: Six new tool UI components with consistent patterns (error/loading/success states); verify data shape assumptions match tool output types.
  • State synchronization: Ensure progressEvents and suspendPayload are reset consistently when workflows change or network selections update.

Possibly related PRs

Suggested reviewers

  • sourcery-ai

🐰 Progress events bloom in context divine,
Workflows pause, await approval's sign,
Suspend and resume in harmony aligned,
With scrapers five and stages well-defined,
The canvas now shows hearts that intertwine!

Pre-merge checks and finishing touches

❌ Failed checks (1 warning, 2 inconclusive)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
Title check ❓ Inconclusive The title 'Develop' is generic and does not convey any meaningful information about the changeset, which spans workflow progress tracking, suspend/resume functionality, and new web-scraper tool components. Replace with a specific title describing the main change, such as 'Add workflow progress events and suspend/resume functionality' or 'Implement progress tracking for workflows and network operations'.
Description check ❓ Inconclusive No description was provided by the author, making it impossible to assess whether the changeset is adequately documented or explained. Add a description explaining the purpose and scope of changes, including the new progress event tracking, suspend/resume workflows, and web-scraper tool UI components.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch develop

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @ssdeanx, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces significant enhancements to both network and workflow execution by integrating detailed progress tracking and interactive elements. It adds dedicated UI components for various web scraping tools, allowing for richer visualization of their operations and results. A new workflow suspension and approval mechanism enables human-in-the-loop interaction for complex workflows, improving control and oversight. Additionally, the changes include refactoring of tool handling and a substantial expansion of available web scraping tools, providing more robust and informative user experiences.

Highlights

  • New UI Components for Web Scraping Tools: Dedicated React components (WebScraperTool, BatchWebScraperTool, SiteMapExtractorTool, LinkExtractorTool) have been added to provide rich, interactive user interfaces for various web scraping operations, enhancing visualization and user interaction with these tools.
  • Workflow Progress Panel: A new WorkflowProgressPanel component has been introduced to display real-time progress events during workflow execution, offering users a clearer view of ongoing processes.
  • Workflow Suspension and Approval Mechanism: The workflow context now supports pausing workflows and requiring user approval to resume, enabling human-in-the-loop interaction for critical steps, complemented by a new WorkflowSuspendDialog for user input.
  • Enhanced Network Routing Panel: The NetworkRoutingPanel has been updated to display ProgressEvents, providing more detailed and granular feedback on the status and activities within network operations.
  • Expanded Web Scraper Tools: Several new web scraping related tools, including batchWebScraperTool, siteMapExtractorTool, linkExtractorTool, htmlToMarkdownTool, listScrapedContentTool, contentCleanerTool, apiDataFetcherTool, scrapingSchedulerTool, and dataExporterTool, have been exported and defined, significantly broadening the web scraping capabilities.
  • Refactored Tool Handling: The app/chat/helpers/tool-part-transform.ts file has been removed, indicating a simplification or change in how dynamic tool parts are processed, alongside commented-out integration points for the new custom web scraping components in app/chat/components/agent-tools.tsx.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@github-actions
Copy link

🤖 I'm sorry @ssdeanx, but I was unable to process your request. Please see the logs for more details.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces significant new features for visualizing agent and workflow progress, including progress event tracking and a UI for workflow suspension and approval. It also adds several new UI components for rendering detailed results from web scraping tools. The changes are generally well-implemented, but I've identified a critical issue with unstable key generation in React components that could lead to UI bugs. I've also noted some commented-out code that should be removed for better maintainability. Addressing these points will improve the stability and clarity of the new features.

Comment on lines +297 to +333
for (const part of message.parts) {
// Handle agent and workflow progress events
if (part.type === "data-tool-agent" || part.type === "data-tool-workflow") {
const agentPart = part as { data: AgentDataPart }
const eventData = agentPart.data

if (eventData?.data?.text?.trim()) {
allProgressEvents.push({
id: `${message.id}-${part.type}-${Date.now()}`,
stage: part.type.replace("data-tool-", ""),
status: "in-progress",
message: eventData.data.text,
agentId: eventData.id,
timestamp: new Date(),
data: eventData,
})
}
}

// Handle custom progress events from tools
if (typeof part.type === "string" && part.type.startsWith("data-tool-progress")) {
const progressPart = part as { type: string; data?: { status?: string; message?: string; stage?: string; agentId?: string } }
const eventData = progressPart.data

if (eventData?.status && (eventData.status === "in-progress" || eventData.status === "done" || eventData.status === "error")) {
allProgressEvents.push({
id: `${message.id}-${part.type}-${Date.now()}`,
stage: eventData.stage ?? "progress",
status: eventData.status,
message: eventData.message ?? `${part.type} ${eventData.status}`,
agentId: eventData.agentId,
timestamp: new Date(),
data: eventData,
})
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The id for ProgressEvent is generated using Date.now(). Since allProgressEvents is recalculated on every render when messages changes, Date.now() will produce a new ID for the same logical event. This creates an unstable key for React components that use this ID, leading to unnecessary re-renders and potential loss of component state. A more stable ID should be generated using the message ID and the index of the part within the message.

        for (const [partIndex, part] of message.parts.entries()) {
          // Handle agent and workflow progress events
          if (part.type === "data-tool-agent" || part.type === "data-tool-workflow") {
            const agentPart = part as { data: AgentDataPart }
            const eventData = agentPart.data

            if (eventData?.data?.text?.trim()) {
              allProgressEvents.push({
                id: `${message.id}-${part.type}-${partIndex}`,
                stage: part.type.replace("data-tool-", ""),
                status: "in-progress",
                message: eventData.data.text,
                agentId: eventData.id,
                timestamp: new Date(),
                data: eventData,
              })
            }
          }

          // Handle custom progress events from tools
          if (typeof part.type === "string" && part.type.startsWith("data-tool-progress")) {
            const progressPart = part as { type: string; data?: { status?: string; message?: string; stage?: string; agentId?: string } }
            const eventData = progressPart.data

            if (eventData?.status && (eventData.status === "in-progress" || eventData.status === "done" || eventData.status === "error")) {
              allProgressEvents.push({
                id: `${message.id}-${part.type}-${partIndex}`,
                stage: eventData.stage ?? "progress",
                status: eventData.status,
                message: eventData.message ?? `${part.type} ${eventData.status}`,
                agentId: eventData.agentId,
                timestamp: new Date(),
                data: eventData,
              })
            }
          }
        }

Comment on lines +263 to +316
for (const part of message.parts) {
// Handle workflow progress events (data-workflow, data-tool-workflow)
if (part.type === "data-workflow" || part.type === "data-tool-workflow") {
const workflowPart = part as { data?: { text?: string; status?: string; stepId?: string } }
const eventData = workflowPart.data

if (eventData && eventData.text !== null && typeof eventData.text === "string" && eventData.text.trim()) {
allProgressEvents.push({
id: `${message.id}-${part.type}-${Date.now()}`,
stage: part.type.replace("data-", "").replace("-tool-", " "),
status: "in-progress",
message: eventData.text,
stepId: eventData.stepId,
timestamp: new Date(),
data: eventData,
})
}
}

// Handle suspend payloads (data-workflow-suspend)
if (part.type === "data-workflow-suspend") {
const suspendPart = part as { data?: { message?: string; requestId?: string; stepId?: string } }
const suspendData = suspendPart.data

if (suspendData?.message !== null && suspendData?.message !== undefined &&
suspendData?.stepId !== null && suspendData?.stepId !== undefined) {
detectedSuspendPayload = {
message: suspendData.message,
requestId: suspendData.requestId,
stepId: suspendData.stepId,
}
setWorkflowStatus("paused")
}
}

// Handle custom progress events from workflow steps
if (typeof part.type === "string" && part.type.startsWith("data-workflow-progress")) {
const progressPart = part as { type: string; data?: { status?: string; message?: string; stage?: string; stepId?: string } }
const eventData = progressPart.data

if (eventData && eventData.status !== null && typeof eventData.status === "string" &&
(eventData.status === "in-progress" || eventData.status === "done" || eventData.status === "error")) {
allProgressEvents.push({
id: `${message.id}-${part.type}-${Date.now()}`,
stage: eventData.stage ?? "workflow",
status: eventData.status,
message: eventData.message ?? `${part.type} ${eventData.status}`,
stepId: eventData.stepId,
timestamp: new Date(),
data: eventData,
})
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The id for WorkflowProgressEvent is generated using Date.now(). Since allProgressEvents is recalculated on every render when messages changes, Date.now() will produce a new ID for the same logical event. This creates an unstable key for React components that use this ID, leading to unnecessary re-renders and potential loss of component state. A more stable ID should be generated using the message ID and the index of the part within the message.

        for (const [partIndex, part] of message.parts.entries()) {
          // Handle workflow progress events (data-workflow, data-tool-workflow)
          if (part.type === "data-workflow" || part.type === "data-tool-workflow") {
            const workflowPart = part as { data?: { text?: string; status?: string; stepId?: string } }
            const eventData = workflowPart.data

            if (eventData && eventData.text !== null && typeof eventData.text === "string" && eventData.text.trim()) {
              allProgressEvents.push({
                id: `${message.id}-${part.type}-${partIndex}`,
                stage: part.type.replace("data-", "").replace("-tool-", " "),
                status: "in-progress",
                message: eventData.text,
                stepId: eventData.stepId,
                timestamp: new Date(),
                data: eventData,
              })
            }
          }

          // Handle suspend payloads (data-workflow-suspend)
          if (part.type === "data-workflow-suspend") {
            const suspendPart = part as { data?: { message?: string; requestId?: string; stepId?: string } }
            const suspendData = suspendPart.data

            if (suspendData?.message !== null && suspendData?.message !== undefined &&
                suspendData?.stepId !== null && suspendData?.stepId !== undefined) {
              detectedSuspendPayload = {
                message: suspendData.message,
                requestId: suspendData.requestId,
                stepId: suspendData.stepId,
              }
              setWorkflowStatus("paused")
            }
          }

          // Handle custom progress events from workflow steps
          if (typeof part.type === "string" && part.type.startsWith("data-workflow-progress")) {
            const progressPart = part as { type: string; data?: { status?: string; message?: string; stage?: string; stepId?: string } }
            const eventData = progressPart.data

            if (eventData && eventData.status !== null && typeof eventData.status === "string" &&
                (eventData.status === "in-progress" || eventData.status === "done" || eventData.status === "error")) {
              allProgressEvents.push({
                id: `${message.id}-${part.type}-${partIndex}`,
                stage: eventData.stage ?? "workflow",
                status: eventData.status,
                message: eventData.message ?? `${part.type} ${eventData.status}`,
                stepId: eventData.stepId,
                timestamp: new Date(),
                data: eventData,
              })
            }
          }
        }

Comment on lines +15 to +22
// Import custom tool components
//import {
// WebScraperTool,
// BatchWebScraperTool,
// SiteMapExtractorTool,
// LinkExtractorTool,
//} from "@/src/components/ai-elements/tools"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This block of commented-out imports should be removed. If the feature is temporarily disabled, it's better to use a feature flag or remove the code and rely on version control to restore it later. Leaving commented-out code reduces readability and can lead to confusion.

Comment on lines +97 to +145
// Check if this is a web scraper tool and render custom component
// if (toolName === "web:scraper" && hasOutput) {
// return (
// <WebScraperTool
// key={`${id}-${toolName}-${toolState}-${groupIdx}`}
// toolCallId={id}
// input={latest.input as WebScraperUITool["input"]}
// output={latest.output as WebScraperUITool["output"]}
// errorText={errorText}
// />
// )
// }

// if (toolName === "batch-web-scraper" && hasOutput) {
// return (
// <BatchWebScraperTool
// key={`${id}-${toolName}-${toolState}-${groupIdx}`}
// toolCallId={id}
// input={latest.input as BatchWebScraperUITool["input"]}
// output={latest.output as BatchWebScraperUITool["output"]}
// errorText={errorText}
// />
// )
// }

// if (toolName === "site-map-extractor" && hasOutput) {
// return (
// <SiteMapExtractorTool
// key={`${id}-${toolName}-${toolState}-${groupIdx}`}
// toolCallId={id}
// input={latest.input as SiteMapExtractorUITool["input"]}
// output={latest.output as SiteMapExtractorUITool["output"]}
// errorText={errorText}
// />
// )
// }

//if (toolName === "link-extractor" && hasOutput) {
// return (
// <LinkExtractorTool
// key={`${id}-${toolName}-${toolState}-${groupIdx}`}
// toolCallId={id}
// input={latest.input as LinkExtractorUITool["input"]}
// output={latest.output as LinkExtractorUITool["output"]}
// errorText={errorText}
// />
// )
// }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This large block of commented-out code for rendering custom tool components should be removed. It clutters the component and makes it harder to understand the current logic. Version control is the appropriate place to keep historical code. If this is for a feature that is temporarily disabled, consider using a feature flag.

@greptile-apps
Copy link

greptile-apps bot commented Dec 12, 2025

Greptile Overview

Greptile Summary

This PR enhances the workflows and networks features with comprehensive progress tracking and adds new web scraping tool UI components.

Key Changes

  • Workflow Progress Monitoring: Added WorkflowProgressPanel component to visualize workflow execution events grouped by stage with real-time status indicators
  • Workflow Suspension & Approval: Implemented WorkflowSuspendDialog to support human-in-the-loop workflows requiring approval before continuing execution
  • Network Progress Events: Enhanced NetworkRoutingPanel to display progress events from agent execution with status icons and badges
  • Web Scraping UI Components: Created four new AI Elements tool components (WebScraperTool, BatchWebScraperTool, LinkExtractorTool, SiteMapExtractorTool) for rich visualization of scraping results
  • Enhanced Context Providers: Both workflow-context.tsx and network-context.tsx now extract and track progress events from streaming message parts, improving observability
  • Type Safety Improvements: Added explicit null checks and proper type guards for optional properties throughout the contexts

Technical Highlights

  • Progress events extracted from various message part types (data-workflow, data-tool-workflow, data-network, data-workflow-suspend)
  • Real-time streaming text display in workflow nodes shows agent processing status
  • Workflow nodes display inline progress during step execution
  • Resume workflow now accepts approval data for audit trails
  • All new UI components follow the AI Elements design patterns with proper accessibility and responsive layouts

The changes integrate well with the existing architecture and maintain consistency with the project's coding patterns.

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk - all changes are additive enhancements to existing features
  • The changes are well-structured, follow existing patterns, use proper type safety, and are primarily additive (new components and features). The commented-out code in agent-tools.tsx shows careful planning without breaking changes. No critical logic errors, security issues, or breaking changes detected.
  • No files require special attention - all implementations follow project conventions and are production-ready

Important Files Changed

File Analysis

Filename Score Overview
app/networks/providers/network-context.tsx 4/5 Enhanced context with progress events extraction from message parts, improved type safety with proper null checks
app/workflows/components/workflow-progress-panel.tsx 5/5 New component providing comprehensive progress event visualization grouped by stage with status indicators
app/workflows/components/workflow-suspend-dialog.tsx 5/5 New approval dialog component for workflow suspension with approve/reject actions and optional approver name input
app/workflows/providers/workflow-context.tsx 4/5 Added progress event extraction, workflow suspension support with approval mechanism, enhanced resumeWorkflow to accept approval data
src/components/ai-elements/tools/batch-web-scraper-tool.tsx 5/5 New UI component for batch web scraping results display with success/failure indicators and content previews
src/components/ai-elements/tools/web-scraper-tool.tsx 5/5 New comprehensive UI component for web scraping results with tabbed interface for extracted data, markdown, metadata, images, and raw HTML

Sequence Diagram

sequenceDiagram
    participant User
    participant WorkflowCanvas
    participant WorkflowContext
    participant AISDKChat
    participant MastraAPI
    participant WorkflowProgressPanel
    participant WorkflowSuspendDialog
    participant WorkflowNode

    User->>WorkflowCanvas: Initiate workflow
    WorkflowCanvas->>WorkflowContext: runWorkflow(inputData)
    WorkflowContext->>AISDKChat: sendMessage(inputText)
    AISDKChat->>MastraAPI: POST /workflow/{workflowId}
    
    loop Streaming Response
        MastraAPI-->>AISDKChat: Stream message parts
        AISDKChat-->>WorkflowContext: Update messages[]
        WorkflowContext->>WorkflowContext: Extract progress events from parts
        WorkflowContext->>WorkflowContext: Extract suspend payload if present
        WorkflowContext-->>WorkflowProgressPanel: Update progressEvents[]
        WorkflowContext-->>WorkflowNode: Update stepStreamingEvents
        WorkflowNode-->>User: Show real-time agent text
        WorkflowProgressPanel-->>User: Show progress grouped by stage
    end

    alt Workflow Suspended
        MastraAPI-->>WorkflowContext: data-workflow-suspend part
        WorkflowContext->>WorkflowContext: setSuspendPayload()
        WorkflowContext->>WorkflowContext: setWorkflowStatus("paused")
        WorkflowContext-->>WorkflowSuspendDialog: Show dialog
        WorkflowSuspendDialog-->>User: Request approval
        User->>WorkflowSuspendDialog: Approve/Reject
        WorkflowSuspendDialog->>WorkflowContext: approveWorkflow(approved, name)
        WorkflowContext->>WorkflowContext: resumeWorkflow(resumeData)
        WorkflowContext->>AISDKChat: sendMessage(resume + approval data)
        AISDKChat->>MastraAPI: Resume workflow
    end

    alt Network Execution
        User->>NetworkContext: sendMessage(text)
        NetworkContext->>AISDKChat: Send to /network route
        AISDKChat->>MastraAPI: POST /network
        loop Streaming
            MastraAPI-->>NetworkContext: Stream with data-network parts
            NetworkContext->>NetworkContext: Extract routing steps from agents[]
            NetworkContext->>NetworkContext: Extract progress events
            NetworkContext-->>NetworkRoutingPanel: Update routingSteps & progressEvents
            NetworkRoutingPanel-->>User: Visualize agent routing flow
        end
    end
Loading

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

18 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 11

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 59cc98b and 9b2981a.

📒 Files selected for processing (18)
  • app/chat/components/agent-tools.tsx (2 hunks)
  • app/chat/helpers/tool-part-transform.ts (0 hunks)
  • app/networks/components/network-routing-panel.tsx (4 hunks)
  • app/networks/providers/network-context.tsx (11 hunks)
  • app/workflows/components/workflow-canvas.tsx (2 hunks)
  • app/workflows/components/workflow-header.tsx (2 hunks)
  • app/workflows/components/workflow-input-panel.tsx (2 hunks)
  • app/workflows/components/workflow-node.tsx (3 hunks)
  • app/workflows/components/workflow-progress-panel.tsx (1 hunks)
  • app/workflows/components/workflow-suspend-dialog.tsx (1 hunks)
  • app/workflows/providers/workflow-context.tsx (9 hunks)
  • src/components/ai-elements/tools/batch-web-scraper-tool.tsx (1 hunks)
  • src/components/ai-elements/tools/index.ts (1 hunks)
  • src/components/ai-elements/tools/link-extractor-tool.tsx (1 hunks)
  • src/components/ai-elements/tools/site-map-extractor-tool.tsx (1 hunks)
  • src/components/ai-elements/tools/web-scraper-tool.tsx (1 hunks)
  • src/mastra/tools/index.ts (1 hunks)
  • src/mastra/tools/web-scraper-tool.ts (1 hunks)
💤 Files with no reviewable changes (1)
  • app/chat/helpers/tool-part-transform.ts
🧰 Additional context used
📓 Path-based instructions (26)
**/components/**/*.{js,jsx,ts,tsx}

📄 CodeRabbit inference engine (.github/instructions/next-js.instructions.md)

Use PascalCase for component file names (e.g., ComponentName.jsx or ComponentName.tsx).

Files:

  • src/components/ai-elements/tools/index.ts
  • app/workflows/components/workflow-input-panel.tsx
  • app/chat/components/agent-tools.tsx
  • app/workflows/components/workflow-node.tsx
  • app/networks/components/network-routing-panel.tsx
  • src/components/ai-elements/tools/link-extractor-tool.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • src/components/ai-elements/tools/batch-web-scraper-tool.tsx
  • src/components/ai-elements/tools/site-map-extractor-tool.tsx
  • app/workflows/components/workflow-canvas.tsx
  • src/components/ai-elements/tools/web-scraper-tool.tsx
**/*.{js,jsx,ts,tsx}

📄 CodeRabbit inference engine (.github/instructions/next-js.instructions.md)

**/*.{js,jsx,ts,tsx}: Use next/dynamic for dynamic imports to load components only when needed, improving initial load time.
Use next/image component for automatic image optimization, including lazy loading and responsive images.
Use React.memo to prevent unnecessary re-renders of components.
Use the <Link prefetch> tag to prefetch pages that are likely to be visited.
Use getServerSideProps, getStaticProps, or server components for fetching data on the server-side.
Use SWR or React Query for client-side data fetching and caching.
Use CSS Modules, Styled Components, or Tailwind CSS for component-level styling. Prefer Tailwind CSS for rapid development.
Use React Context, Zustand, Jotai, or Recoil for managing global state. Avoid Redux unless necessary.
Use react-hook-form for managing forms and validation.
Only fetch the data that is needed by the component to avoid over-fetching.
Avoid long-running synchronous operations in the main thread to prevent blocking.
Always use setState or hooks to update state instead of mutating state directly.
Include a complete dependency array in useEffect hooks to prevent unexpected behavior.
Avoid writing server-side code in client components to prevent exposing secrets or causing unexpected behavior.
Use try...catch blocks for handling errors in asynchronous operations.
Implement error boundary components using getDerivedStateFromError or componentDidCatch lifecycle methods.
Sanitize user input to prevent Cross-Site Scripting (XSS) attacks. Be especially careful when rendering HTML directly from user input.
Store authentication tokens in HTTP-only cookies or local storage securely.
Implement role-based access control to restrict access to sensitive resources.
Clean up event listeners and timers in useEffect hooks to avoid memory leaks.
Only update state when necessary to reduce the number of re-renders and improve performance.
Use immutable data structures and avoid mutating data directly to prevent unexpected...

Files:

  • src/components/ai-elements/tools/index.ts
  • app/workflows/components/workflow-input-panel.tsx
  • app/chat/components/agent-tools.tsx
  • app/workflows/components/workflow-node.tsx
  • app/networks/components/network-routing-panel.tsx
  • src/components/ai-elements/tools/link-extractor-tool.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • src/components/ai-elements/tools/batch-web-scraper-tool.tsx
  • src/components/ai-elements/tools/site-map-extractor-tool.tsx
  • app/workflows/components/workflow-canvas.tsx
  • src/mastra/tools/web-scraper-tool.ts
  • app/workflows/providers/workflow-context.tsx
  • src/mastra/tools/index.ts
  • src/components/ai-elements/tools/web-scraper-tool.tsx
  • app/networks/providers/network-context.tsx
**/*.{js,ts}

📄 CodeRabbit inference engine (.github/instructions/next-js.instructions.md)

Use parameterized queries or an ORM to prevent SQL injection attacks.

Files:

  • src/components/ai-elements/tools/index.ts
  • src/mastra/tools/web-scraper-tool.ts
  • src/mastra/tools/index.ts
**/*.{ts,tsx,js,jsx,py,java,cs,rb,go,rs,cpp,c,h,hpp,swift,kotlin,php,scala,clj,groovy,lua,sh,bash}

📄 CodeRabbit inference engine (.github/instructions/self-explanatory-code-commenting.instructions.md)

**/*.{ts,tsx,js,jsx,py,java,cs,rb,go,rs,cpp,c,h,hpp,swift,kotlin,php,scala,clj,groovy,lua,sh,bash}: Write code that speaks for itself. Comment only when necessary to explain WHY, not WHAT. Avoid obvious comments that state what the code literally does.
Avoid redundant comments that simply repeat what the code is doing
Keep comments accurate and up-to-date with code changes. Remove or update outdated comments that no longer match the implementation.
Write comments for complex business logic that explain the WHY behind specific calculations or business rules
Document non-obvious algorithms with comments explaining the algorithm choice and its reasoning
Add comments explaining what regex patterns match, especially for complex patterns
Document API constraints, rate limits, gotchas, and external dependencies with explanatory comments
Avoid commenting out dead code. Use version control instead of maintaining commented code blocks.
Do not maintain code change history or modification logs as comments. Rely on git history and commit messages instead.
Avoid decorative divider comments (e.g., lines of equals signs or asterisks) for section separation
Ensure comments are placed appropriately above or adjacent to the code they describe
Write comments using proper grammar, spelling, and professional language
Prefer self-documenting code with clear variable/function names over adding comments to explain unclear code

Files:

  • src/components/ai-elements/tools/index.ts
  • app/workflows/components/workflow-input-panel.tsx
  • app/chat/components/agent-tools.tsx
  • app/workflows/components/workflow-node.tsx
  • app/networks/components/network-routing-panel.tsx
  • src/components/ai-elements/tools/link-extractor-tool.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • src/components/ai-elements/tools/batch-web-scraper-tool.tsx
  • src/components/ai-elements/tools/site-map-extractor-tool.tsx
  • app/workflows/components/workflow-canvas.tsx
  • src/mastra/tools/web-scraper-tool.ts
  • app/workflows/providers/workflow-context.tsx
  • src/mastra/tools/index.ts
  • src/components/ai-elements/tools/web-scraper-tool.tsx
  • app/networks/providers/network-context.tsx
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (.github/instructions/self-explanatory-code-commenting.instructions.md)

**/*.{ts,tsx,js,jsx}: Document public APIs with TSDoc/JSDoc comments including parameter descriptions, return types, examples, and thrown exceptions
Add TSDoc comments to configuration constants and environment variables explaining their source, reasoning, or constraints
Use TSDoc annotation tags (TODO, FIXME, HACK, NOTE, WARNING, PERF, SECURITY, BUG, REFACTOR, DEPRECATED) to mark special comments
Include file headers with @fileoverview, @author, @copyright, and @license tags to document file purpose and ownership
Document function parameters with @param tags, return values with @returns tags, and exceptions with @throws tags in TSDoc comments
Use @see tags in TSDoc comments to reference related functions, methods, or documentation
Include @example tags in public API documentation with code examples showing typical usage

**/*.{ts,tsx,js,jsx}: Use Mastra mcp tools (#mastradocs, #mastraChanges, #mastraexamples, #mastraBlog) for Mastra framework development to stay updated with latest features and best practices
When working with Next.js projects, always utilize the next-devtools-mcp server for all Next.js related queries

Files:

  • src/components/ai-elements/tools/index.ts
  • app/workflows/components/workflow-input-panel.tsx
  • app/chat/components/agent-tools.tsx
  • app/workflows/components/workflow-node.tsx
  • app/networks/components/network-routing-panel.tsx
  • src/components/ai-elements/tools/link-extractor-tool.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • src/components/ai-elements/tools/batch-web-scraper-tool.tsx
  • src/components/ai-elements/tools/site-map-extractor-tool.tsx
  • app/workflows/components/workflow-canvas.tsx
  • src/mastra/tools/web-scraper-tool.ts
  • app/workflows/providers/workflow-context.tsx
  • src/mastra/tools/index.ts
  • src/components/ai-elements/tools/web-scraper-tool.tsx
  • app/networks/providers/network-context.tsx
**/*.{ts,tsx}

📄 CodeRabbit inference engine (.github/instructions/self-explanatory-code-commenting.instructions.md)

**/*.{ts,tsx}: Document interface and type definitions with TSDoc comments explaining their purpose and usage context
Document interface properties with /** */ comments explaining each field's purpose and constraints
Document generic type parameters with @template tags explaining what each type parameter represents
Use type guards with comments explaining the runtime validation logic being performed
Document advanced/complex TypeScript types with explanatory comments about their purpose and use cases

Files:

  • src/components/ai-elements/tools/index.ts
  • app/workflows/components/workflow-input-panel.tsx
  • app/chat/components/agent-tools.tsx
  • app/workflows/components/workflow-node.tsx
  • app/networks/components/network-routing-panel.tsx
  • src/components/ai-elements/tools/link-extractor-tool.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • src/components/ai-elements/tools/batch-web-scraper-tool.tsx
  • src/components/ai-elements/tools/site-map-extractor-tool.tsx
  • app/workflows/components/workflow-canvas.tsx
  • src/mastra/tools/web-scraper-tool.ts
  • app/workflows/providers/workflow-context.tsx
  • src/mastra/tools/index.ts
  • src/components/ai-elements/tools/web-scraper-tool.tsx
  • app/networks/providers/network-context.tsx
src/components/ai-elements/**/*.{ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Use AI Elements (30 components) from src/components/ai-elements/ for chat/reasoning/canvas UIs

Files:

  • src/components/ai-elements/tools/index.ts
  • src/components/ai-elements/tools/link-extractor-tool.tsx
  • src/components/ai-elements/tools/batch-web-scraper-tool.tsx
  • src/components/ai-elements/tools/site-map-extractor-tool.tsx
  • src/components/ai-elements/tools/web-scraper-tool.tsx
**/*.{css,tsx,ts}

📄 CodeRabbit inference engine (AGENTS.md)

Use Tailwind CSS 4 with oklch color variables for styling

Files:

  • src/components/ai-elements/tools/index.ts
  • app/workflows/components/workflow-input-panel.tsx
  • app/chat/components/agent-tools.tsx
  • app/workflows/components/workflow-node.tsx
  • app/networks/components/network-routing-panel.tsx
  • src/components/ai-elements/tools/link-extractor-tool.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • src/components/ai-elements/tools/batch-web-scraper-tool.tsx
  • src/components/ai-elements/tools/site-map-extractor-tool.tsx
  • app/workflows/components/workflow-canvas.tsx
  • src/mastra/tools/web-scraper-tool.ts
  • app/workflows/providers/workflow-context.tsx
  • src/mastra/tools/index.ts
  • src/components/ai-elements/tools/web-scraper-tool.tsx
  • app/networks/providers/network-context.tsx
**/app/**

📄 CodeRabbit inference engine (.github/instructions/next-js.instructions.md)

Use the app/ directory structure for route handlers, server components, and client components (Next.js 13+). Prefer this over the pages/ directory for new projects.

Files:

  • app/workflows/components/workflow-input-panel.tsx
  • app/chat/components/agent-tools.tsx
  • app/workflows/components/workflow-node.tsx
  • app/networks/components/network-routing-panel.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • app/workflows/components/workflow-canvas.tsx
  • app/workflows/providers/workflow-context.tsx
  • app/networks/providers/network-context.tsx
app/**/*.{tsx,ts}

📄 CodeRabbit inference engine (app/AGENTS.md)

app/**/*.{tsx,ts}: Use Tailwind CSS 4 with oklch color variables for styling in Next.js App Router pages and layouts
Use React 19 latest features in component implementations within the app directory

Files:

  • app/workflows/components/workflow-input-panel.tsx
  • app/chat/components/agent-tools.tsx
  • app/workflows/components/workflow-node.tsx
  • app/networks/components/network-routing-panel.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • app/workflows/components/workflow-canvas.tsx
  • app/workflows/providers/workflow-context.tsx
  • app/networks/providers/network-context.tsx
app/workflows/components/**/*.{ts,tsx}

📄 CodeRabbit inference engine (app/workflows/AGENTS.md)

app/workflows/components/**/*.{ts,tsx}: Use useChat from @ai-sdk/react with DefaultChatTransport to connect to Mastra workflow routes, mapping workflow-specific input fields through prepareSendMessagesRequest
Use useWorkflowContext() for state management in workflow components, providing workflow selection, execution control, status tracking, React Flow data, and streaming output
Use AI Elements components (Canvas, Node, Edge, Panel, Controls, Toolbar, Connection) for workflow visualization, with animated active edges and temporary pending edge types

Files:

  • app/workflows/components/workflow-input-panel.tsx
  • app/workflows/components/workflow-node.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • app/workflows/components/workflow-canvas.tsx
app/workflows/**/*.{ts,tsx}

📄 CodeRabbit inference engine (app/workflows/AGENTS.md)

Organize workflows feature with modular pattern: page.tsx, config/, providers/, and components/ directories with clear separation of concerns (state management, UI components, configuration)

Files:

  • app/workflows/components/workflow-input-panel.tsx
  • app/workflows/components/workflow-node.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • app/workflows/components/workflow-canvas.tsx
  • app/workflows/providers/workflow-context.tsx
app/**/*.{ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

Frontend uses Next.js 16 App Router with React 19; use shadcn/ui base components (34 components) from ui/ directory

Files:

  • app/workflows/components/workflow-input-panel.tsx
  • app/chat/components/agent-tools.tsx
  • app/workflows/components/workflow-node.tsx
  • app/networks/components/network-routing-panel.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • app/workflows/components/workflow-canvas.tsx
  • app/workflows/providers/workflow-context.tsx
  • app/networks/providers/network-context.tsx
app/**/*.tsx

📄 CodeRabbit inference engine (AGENTS.md)

Use @ai-sdk/react for streaming and AI interactions in React components

Files:

  • app/workflows/components/workflow-input-panel.tsx
  • app/chat/components/agent-tools.tsx
  • app/workflows/components/workflow-node.tsx
  • app/networks/components/network-routing-panel.tsx
  • app/workflows/components/workflow-suspend-dialog.tsx
  • app/workflows/components/workflow-progress-panel.tsx
  • app/workflows/components/workflow-header.tsx
  • app/workflows/components/workflow-canvas.tsx
  • app/workflows/providers/workflow-context.tsx
  • app/networks/providers/network-context.tsx
app/chat/components/**/*.{ts,tsx}

📄 CodeRabbit inference engine (app/chat/AGENTS.md)

app/chat/components/**/*.{ts,tsx}: Use AI SDK v5 types and patterns in chat components: import types like UIMessage, DynamicToolUIPart, TextUIPart, ReasoningUIPart from 'ai' and use type guard functions like isTextUIPart, isReasoningUIPart, isToolOrDynamicToolUIPart to filter message parts
Access message content through message.parts array using type guards rather than message.content, extracting specific parts like text using const textPart = message.parts?.find(isTextUIPart)
Handle Mastra stream chunk types correctly: use 'text-delta' for streaming text, 'reasoning-delta' for streaming reasoning (NOT 'reasoning'), 'tool-call' for tool invocation, 'tool-result' for tool completion, 'source' for research sources, and 'finish' for completion with usage data
Use AI Elements components (Conversation, Message, PromptInput, ModelSelector, Reasoning, Tool, Sources, Artifact) from the AI Elements library in their respective chat component files as specified in the architecture

Files:

  • app/chat/components/agent-tools.tsx
app/chat/components/agent-tools.tsx

📄 CodeRabbit inference engine (app/chat/AGENTS.md)

Display tool invocations with Tool AI Elements component showing input-available, output-available, and output-error states when the tools feature flag is enabled for the agent

Files:

  • app/chat/components/agent-tools.tsx
app/workflows/components/workflow-node.tsx

📄 CodeRabbit inference engine (app/workflows/AGENTS.md)

Implement custom node component with status indicators reflecting workflow step execution state

Files:

  • app/workflows/components/workflow-node.tsx
app/networks/**/*.tsx

📄 CodeRabbit inference engine (app/networks/AGENTS.md)

app/networks/**/*.tsx: Use useChat from @ai-sdk/react with DefaultChatTransport for streaming responses from the /network route in the networks feature
Support real-time streaming with stop/cancel functionality, reasoning visualization for chain-of-thought models, and tool invocation display showing agent tool calls
Provide a responsive layout with collapsible sidebar on mobile for the networks feature

Files:

  • app/networks/components/network-routing-panel.tsx
  • app/networks/providers/network-context.tsx
app/networks/components/network-routing-panel.tsx

📄 CodeRabbit inference engine (app/networks/AGENTS.md)

Implement a visual routing flow sidebar component that displays agent execution steps and routing visualization

Files:

  • app/networks/components/network-routing-panel.tsx
app/workflows/components/workflow-canvas.tsx

📄 CodeRabbit inference engine (app/workflows/AGENTS.md)

Use React Flow canvas wrapper for workflow visualization with pre-configured defaults from AI Elements Canvas component

Files:

  • app/workflows/components/workflow-canvas.tsx
src/mastra/tools/**/*.ts

📄 CodeRabbit inference engine (src/mastra/AGENTS.md)

src/mastra/tools/**/*.ts: Use the createTool pattern with Zod schemas when adding new tools under src/mastra/tools
Use explicit Zod schemas for every tool input/output

Implement tools using createTool({ id, inputSchema, outputSchema, execute }) pattern with strict Zod schemas

Files:

  • src/mastra/tools/web-scraper-tool.ts
  • src/mastra/tools/index.ts
src/mastra/{tools,workflows}/**/*.ts

📄 CodeRabbit inference engine (src/mastra/AGENTS.md)

Use RuntimeContext to enforce access control in tools and workflows

Files:

  • src/mastra/tools/web-scraper-tool.ts
  • src/mastra/tools/index.ts
src/mastra/**/*

📄 CodeRabbit inference engine (src/AGENTS.md)

mastra modules can import from utils, but must not import from app or cli (except types)

Files:

  • src/mastra/tools/web-scraper-tool.ts
  • src/mastra/tools/index.ts
src/mastra/{tools,agents,workflows}/**/*.ts

📄 CodeRabbit inference engine (AGENTS.md)

Use Zod schemas for strict input/output validation in tools, agents, and workflows

Files:

  • src/mastra/tools/web-scraper-tool.ts
  • src/mastra/tools/index.ts
app/workflows/providers/workflow-context.tsx

📄 CodeRabbit inference engine (app/workflows/AGENTS.md)

Implement workflow context provider with useChat hook from @ai-sdk/react, exporting interface with state (selectedWorkflow, workflowStatus, currentRun, activeStepIndex) and actions (selectWorkflow, runWorkflow, pauseWorkflow, resumeWorkflow, stopWorkflow, runStep, getStepStatus)

Files:

  • app/workflows/providers/workflow-context.tsx
app/networks/providers/network-context.tsx

📄 CodeRabbit inference engine (app/networks/AGENTS.md)

Implement state management using useNetworkContext() hook that provides selectedNetwork, networkConfig, networkStatus, messages, streamingOutput, routingSteps, and control functions (sendMessage, stopExecution, clearHistory)

Files:

  • app/networks/providers/network-context.tsx
🧬 Code graph analysis (8)
app/workflows/components/workflow-node.tsx (1)
app/workflows/providers/workflow-context.tsx (1)
  • useWorkflowContext (110-116)
app/networks/components/network-routing-panel.tsx (1)
app/networks/providers/network-context.tsx (2)
  • ProgressEvent (34-42)
  • useNetworkContext (71-77)
app/workflows/components/workflow-suspend-dialog.tsx (5)
app/workflows/providers/workflow-context.tsx (1)
  • useWorkflowContext (110-116)
ui/dialog.tsx (6)
  • Dialog (133-133)
  • DialogContent (135-135)
  • DialogHeader (138-138)
  • DialogTitle (141-141)
  • DialogDescription (136-136)
  • DialogFooter (137-137)
ui/label.tsx (1)
  • Label (9-20)
ui/input.tsx (1)
  • Input (21-21)
ui/button.tsx (1)
  • Button (60-60)
app/workflows/components/workflow-progress-panel.tsx (4)
app/workflows/providers/workflow-context.tsx (2)
  • WorkflowProgressEvent (35-43)
  • useWorkflowContext (110-116)
ui/badge.tsx (1)
  • Badge (46-46)
ui/card.tsx (4)
  • Card (85-85)
  • CardHeader (86-86)
  • CardTitle (88-88)
  • CardContent (91-91)
ui/scroll-area.tsx (1)
  • ScrollArea (56-56)
src/components/ai-elements/tools/site-map-extractor-tool.tsx (3)
src/mastra/tools/web-scraper-tool.ts (1)
  • SiteMapExtractorUITool (1515-1515)
ui/card.tsx (4)
  • Card (85-85)
  • CardHeader (86-86)
  • CardTitle (88-88)
  • CardContent (91-91)
ui/badge.tsx (1)
  • Badge (46-46)
app/workflows/components/workflow-canvas.tsx (7)
app/workflows/components/workflow-progress-panel.tsx (1)
  • WorkflowProgressPanel (66-139)
app/workflows/components/workflow-info-panel.tsx (1)
  • WorkflowInfoPanel (26-115)
app/workflows/components/workflow-legend.tsx (1)
  • WorkflowLegend (9-51)
app/workflows/components/workflow-actions.tsx (1)
  • WorkflowActions (10-74)
app/workflows/components/workflow-output.tsx (1)
  • WorkflowOutput (9-102)
app/workflows/components/workflow-input-panel.tsx (1)
  • WorkflowInputPanel (15-187)
app/workflows/components/workflow-suspend-dialog.tsx (1)
  • WorkflowSuspendDialog (18-98)
src/mastra/tools/web-scraper-tool.ts (1)
src/mastra/tools/index.ts (1)
  • linkExtractorTool (12-12)
app/workflows/providers/workflow-context.tsx (3)
app/workflows/config/workflows.ts (2)
  • WorkflowId (517-517)
  • WORKFLOW_CONFIGS (44-475)
src/components/ai-elements/custom/workflow-execution.tsx (1)
  • StepStatus (23-23)
src/components/ai-elements/custom/index.ts (1)
  • StepStatus (41-41)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: CodeQL analysis (javascript-typescript)
  • GitHub Check: Agent
  • GitHub Check: Codacy Security Scan
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (28)
app/networks/components/network-routing-panel.tsx (1)

206-219: LGTM!

The Progress Events section is well-implemented with appropriate conditional rendering and limiting to the last 5 events. The key uniqueness depends on proper ID generation in the context provider.

app/networks/providers/network-context.tsx (5)

34-42: LGTM!

The ProgressEvent interface is well-defined with appropriate optional fields and a discriminated status union type.


371-372: LGTM!

The explicit null/undefined checks are more robust than truthy checks, properly handling edge cases where startedAt or completedAt might be falsy but valid values (e.g., epoch timestamp 0).


416-425: LGTM!

Progress events are properly reset when selecting a new network, maintaining consistency with other state resets.


441-447: LGTM!

Progress events are properly cleared alongside other state when clearing history.


451-486: LGTM!

The progressEvents is properly included in both the context value and the useMemo dependency array, ensuring consumers receive updates when progress events change.

src/components/ai-elements/tools/site-map-extractor-tool.tsx (2)

15-50: LGTM! Error and loading states are well-structured.

The error and loading states follow consistent patterns and provide clear user feedback. The loading state appropriately displays crawling parameters.


84-132: LGTM! Site structure rendering is well-organized.

The hierarchical display with depth grouping, proper sorting, and comprehensive page metadata is well-implemented. The title fallback logic appropriately handles missing data.

src/mastra/tools/web-scraper-tool.ts (1)

1762-1762: LGTM! Type export follows established pattern.

The new LinkExtractorUITool type export is consistent with other UI tool type exports in this file.

app/chat/components/agent-tools.tsx (1)

15-144: Scaffolding prepared for future tool integration.

The commented imports and conditional rendering blocks provide clear scaffolding for integrating the new custom tool components. The existing default rendering path remains unchanged and functional.

src/components/ai-elements/tools/index.ts (1)

1-4: LGTM! Clean barrel export for tool components.

The index file appropriately exposes the new tool components through a centralized export point.

src/components/ai-elements/tools/link-extractor-tool.tsx (2)

15-50: LGTM! Error and loading states follow consistent patterns.

The error and loading states provide clear feedback and follow the established component structure.


54-141: LGTM! Link results rendering is comprehensive and secure.

The success state appropriately displays summary statistics with color-coded badges and renders individual links with proper validation indicators. The external links use correct security attributes (rel="noopener noreferrer").

src/mastra/tools/index.ts (1)

10-18: LGTM! Tool exports appropriately extend the public API.

The new exports from web-scraper-tool provide access to the expanded tool suite while maintaining consistent export patterns.

src/components/ai-elements/tools/batch-web-scraper-tool.tsx (2)

15-50: LGTM! Error and loading states provide clear feedback.

The error and loading states follow established patterns and appropriately indicate batch processing of multiple URLs.


54-126: LGTM! Batch results display is well-structured.

The success state effectively presents summary statistics and per-URL results with appropriate visual differentiation for success/failure states. The content preview with 200-character truncation provides useful context without overwhelming the UI.

src/components/ai-elements/tools/web-scraper-tool.tsx (2)

17-52: LGTM! Error and loading states are well-implemented.

The states provide appropriate feedback with loading indicators showing the selector being used for extraction.


56-221: LGTM! Tabbed content organization enhances usability.

The success state effectively organizes different content types (extracted data, markdown, metadata, images, raw HTML) into separate tabs. Conditional badge display in the header provides a quick summary of available content. External links use proper security attributes.

app/workflows/components/workflow-canvas.tsx (1)

42-42: LGTM!

The new workflow UI components are properly integrated into the canvas. The WorkflowProgressPanel is logically placed after controls for visibility, and WorkflowSuspendDialog is positioned after the input panel to maintain a coherent approval workflow.

Also applies to: 48-48

app/workflows/components/workflow-input-panel.tsx (1)

128-128: LGTM!

The conversion from arbitrary pixel values to Tailwind scale-based classes improves consistency. The numeric values are equivalent (60px → min-h-15, 120px → max-h-30/max-w-30), maintaining the original layout while following Tailwind 4 best practices.

Also applies to: 174-174

app/workflows/components/workflow-suspend-dialog.tsx (1)

18-97: LGTM!

The suspend dialog implementation is clean and well-structured:

  • Proper context consumption and guard clauses prevent rendering when not needed
  • The approval/rejection flow correctly invokes approveWorkflow with appropriate parameters
  • The optional approverName input is handled correctly (trimmed and converted to undefined if empty)
  • UI provides clear visual feedback with icons and color-coded sections
app/workflows/providers/workflow-context.tsx (2)

372-388: LGTM!

The resumeWorkflow and approveWorkflow implementations correctly handle the suspend/resume flow:

  • resumeWorkflow optionally accepts approval data and clears suspend state
  • approveWorkflow properly delegates to resumeWorkflow with structured approval data
  • The guard in approveWorkflow ensures it only acts when suspend payload exists

332-333: LGTM!

Progress events and suspend payloads are properly cleared in all workflow lifecycle transitions (select, run, stop), preventing stale state from affecting new workflow executions.

Also applies to: 352-353, 395-396

app/workflows/components/workflow-header.tsx (1)

74-74: LGTM!

The conversion from w-[280px] to w-70 maintains the same width (280px) while following Tailwind 4 scale conventions for consistency.

app/workflows/components/workflow-node.tsx (2)

76-79: LGTM!

The streaming event display is well-implemented:

  • Correctly filters progress events for the current step and "in-progress" status
  • Conditional rendering guards prevent showing the UI when not applicable
  • Visual feedback (pulsing dot, cursor animation) effectively communicates ongoing processing
  • Scrollable container with max-height handles overflow gracefully

Also applies to: 113-131


91-91: LGTM!

The width conversion from w-[280px] to w-70 maintains consistent sizing (280px) while using Tailwind 4 scale values, matching the pattern used across other workflow components.

app/workflows/components/workflow-progress-panel.tsx (2)

88-88: Verify absolute positioning doesn't cause overlap.

The panel uses absolute top-4 right-4 positioning. Ensure this doesn't overlap with other workflow UI elements (like the workflow actions panel or legend) on smaller screens or when multiple panels are visible.

Consider testing the layout with:

  • Different screen sizes (tablet, mobile)
  • All panels visible simultaneously
  • Long workflow names or many progress events

Alternatively, consider using React Flow's Panel component with position="top-right" for automatic positioning that respects other panels.


17-64: LGTM!

The progress panel implementation provides excellent visibility into workflow execution:

  • Clean separation between the internal ProgressEventItem and exported WorkflowProgressPanel
  • Proper memoization for grouping events
  • Status-specific icons and colors improve scannability
  • Conditional rendering based on workflow state prevents clutter
  • ScrollArea handles overflow gracefully

Also applies to: 66-139

Comment on lines +33 to +58
function ProgressEventItem({ event }: { event: ProgressEvent }) {
const getStatusIcon = (status: ProgressEvent["status"]) => {
switch (status) {
case "done":
return <CheckCircle2Icon className="size-3 text-green-500" />
case "in-progress":
return <Loader2Icon className="size-3 animate-spin text-blue-500" />
case "error":
return <XCircleIcon className="size-3 text-red-500" />
default:
return <CircleIcon className="size-3 text-muted-foreground" />
}
}

const getStatusColor = (status: ProgressEvent["status"]) => {
switch (status) {
case "done":
return "text-green-700 dark:text-green-300"
case "in-progress":
return "text-blue-700 dark:text-blue-300"
case "error":
return "text-red-700 dark:text-red-300"
default:
return "text-muted-foreground"
}
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Consider extracting helper functions outside the component.

getStatusIcon and getStatusColor are pure functions with no dependencies on component state or props. Defining them inside ProgressEventItem causes them to be recreated on every render.

+const getProgressStatusIcon = (status: ProgressEvent["status"]) => {
+  switch (status) {
+    case "done":
+      return <CheckCircle2Icon className="size-3 text-green-500" />
+    case "in-progress":
+      return <Loader2Icon className="size-3 animate-spin text-blue-500" />
+    case "error":
+      return <XCircleIcon className="size-3 text-red-500" />
+    default:
+      return <CircleIcon className="size-3 text-muted-foreground" />
+  }
+}
+
+const getProgressStatusColor = (status: ProgressEvent["status"]) => {
+  switch (status) {
+    case "done":
+      return "text-green-700 dark:text-green-300"
+    case "in-progress":
+      return "text-blue-700 dark:text-blue-300"
+    case "error":
+      return "text-red-700 dark:text-red-300"
+    default:
+      return "text-muted-foreground"
+  }
+}
+
 function ProgressEventItem({ event }: { event: ProgressEvent }) {
-  const getStatusIcon = (status: ProgressEvent["status"]) => {
-    switch (status) {
-      case "done":
-        return <CheckCircle2Icon className="size-3 text-green-500" />
-      case "in-progress":
-        return <Loader2Icon className="size-3 animate-spin text-blue-500" />
-      case "error":
-        return <XCircleIcon className="size-3 text-red-500" />
-      default:
-        return <CircleIcon className="size-3 text-muted-foreground" />
-    }
-  }
-
-  const getStatusColor = (status: ProgressEvent["status"]) => {
-    switch (status) {
-      case "done":
-        return "text-green-700 dark:text-green-300"
-      case "in-progress":
-        return "text-blue-700 dark:text-blue-300"
-      case "error":
-        return "text-red-700 dark:text-red-300"
-      default:
-        return "text-muted-foreground"
-    }
-  }
-
   return (
     <div className="flex items-start gap-2 py-1">
-      {getStatusIcon(event.status)}
+      {getProgressStatusIcon(event.status)}
       <div className="flex-1 min-w-0">
         <div className="flex items-center gap-2">
-          <span className={cn("text-xs font-medium", getStatusColor(event.status))}>
+          <span className={cn("text-xs font-medium", getProgressStatusColor(event.status))}>
🤖 Prompt for AI Agents
In app/networks/components/network-routing-panel.tsx around lines 33 to 58,
extract the pure helper functions getStatusIcon and getStatusColor out of the
ProgressEventItem component so they are not recreated on every render; move them
to module-level (above or below the component), keep the same signatures and
return values, export them if used elsewhere, and update any type
imports/annotations so the standalone functions still accept
ProgressEvent["status"] — no logic change, just relocate the functions to
top-level.

Comment on lines +63 to +64
selectNetwork: (_networkId: NetworkId) => void
sendMessage: (_text: string) => void
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Unconventional underscore prefix in public interface signatures.

The underscore prefix (_networkId, _text) typically denotes unused parameters. In interface definitions, parameter names serve as documentation. Consider removing the prefix for clarity in API documentation and IDE hints.

-  selectNetwork: (_networkId: NetworkId) => void
-  sendMessage: (_text: string) => void
+  selectNetwork: (networkId: NetworkId) => void
+  sendMessage: (text: string) => void
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
selectNetwork: (_networkId: NetworkId) => void
sendMessage: (_text: string) => void
selectNetwork: (networkId: NetworkId) => void
sendMessage: (text: string) => void
🤖 Prompt for AI Agents
In app/networks/providers/network-context.tsx around lines 63 to 64, the
interface parameter names use an underscore prefix (e.g., _networkId, _text)
which implies unused params; remove the leading underscores so the signatures
read selectNetwork(networkId: NetworkId) and sendMessage(text: string) to
improve documentation and IDE hints, then update any implementations/usages to
match the new parameter names to keep TypeScript compile checks passing.

Comment on lines +292 to +337
// Extract progress events from custom data parts
useEffect(() => {
const allProgressEvents: ProgressEvent[] = []
for (const message of messages) {
if (message.role === "assistant" && message.parts !== null) {
for (const part of message.parts) {
// Handle agent and workflow progress events
if (part.type === "data-tool-agent" || part.type === "data-tool-workflow") {
const agentPart = part as { data: AgentDataPart }
const eventData = agentPart.data

if (eventData?.data?.text?.trim()) {
allProgressEvents.push({
id: `${message.id}-${part.type}-${Date.now()}`,
stage: part.type.replace("data-tool-", ""),
status: "in-progress",
message: eventData.data.text,
agentId: eventData.id,
timestamp: new Date(),
data: eventData,
})
}
}

// Handle custom progress events from tools
if (typeof part.type === "string" && part.type.startsWith("data-tool-progress")) {
const progressPart = part as { type: string; data?: { status?: string; message?: string; stage?: string; agentId?: string } }
const eventData = progressPart.data

if (eventData?.status && (eventData.status === "in-progress" || eventData.status === "done" || eventData.status === "error")) {
allProgressEvents.push({
id: `${message.id}-${part.type}-${Date.now()}`,
stage: eventData.stage ?? "progress",
status: eventData.status,
message: eventData.message ?? `${part.type} ${eventData.status}`,
agentId: eventData.agentId,
timestamp: new Date(),
data: eventData,
})
}
}
}
}
}
setProgressEvents(allProgressEvents)
}, [messages])
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

ID generation may produce duplicate keys.

Using Date.now() within the same useEffect execution will produce identical timestamps for all events processed in that run. If multiple events share the same message.id and part.type, they will have colliding IDs, causing React key warnings and potential rendering issues.

Add an index to ensure uniqueness:

   useEffect(() => {
     const allProgressEvents: ProgressEvent[] = []
+    let eventIndex = 0
     for (const message of messages) {
       if (message.role === "assistant" && message.parts !== null) {
         for (const part of message.parts) {
           if (part.type === "data-tool-agent" || part.type === "data-tool-workflow") {
             const agentPart = part as { data: AgentDataPart }
             const eventData = agentPart.data

             if (eventData?.data?.text?.trim()) {
               allProgressEvents.push({
-                id: `${message.id}-${part.type}-${Date.now()}`,
+                id: `${message.id}-${part.type}-${eventIndex++}`,
                 stage: part.type.replace("data-tool-", ""),
                 status: "in-progress",
                 message: eventData.data.text,
                 agentId: eventData.id,
                 timestamp: new Date(),
                 data: eventData,
               })
             }
           }

           if (typeof part.type === "string" && part.type.startsWith("data-tool-progress")) {
             const progressPart = part as { type: string; data?: { status?: string; message?: string; stage?: string; agentId?: string } }
             const eventData = progressPart.data

             if (eventData?.status && (eventData.status === "in-progress" || eventData.status === "done" || eventData.status === "error")) {
               allProgressEvents.push({
-                id: `${message.id}-${part.type}-${Date.now()}`,
+                id: `${message.id}-${part.type}-${eventIndex++}`,
                 stage: eventData.stage ?? "progress",
                 status: eventData.status,
                 message: eventData.message ?? `${part.type} ${eventData.status}`,
                 agentId: eventData.agentId,
                 timestamp: new Date(),
                 data: eventData,
               })
             }
           }
         }
       }
     }
     setProgressEvents(allProgressEvents)
   }, [messages])
🤖 Prompt for AI Agents
In app/networks/providers/network-context.tsx around lines 292 to 337, the
generated event id uses Date.now() which is identical for all events created in
one useEffect run and can cause duplicate React keys; change id generation to
include a uniqueness suffix (e.g., a per-effect incremental counter or a
UUID/nanoid) — implement a local counter (or import a small UID lib) and append
that counter/UID to `${message.id}-${part.type}-${Date.now()}` so each pushed
event has a unique id, and ensure any consumers/keys use the new id.

import { Controls } from "@/src/components/ai-elements/controls"
import { Edge } from "@/src/components/ai-elements/edge"
import { useWorkflowContext } from "@/app/workflows/providers/workflow-context"
import { useWorkflowContext, type WorkflowProgressEvent } from "@/app/workflows/providers/workflow-context"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Remove unused type import.

The WorkflowProgressEvent type is imported but never used in this file. It's only needed in components that directly manipulate or filter progress events (like workflow-node.tsx and workflow-progress-panel.tsx).

Apply this diff:

-import { useWorkflowContext, type WorkflowProgressEvent } from "@/app/workflows/providers/workflow-context"
+import { useWorkflowContext } from "@/app/workflows/providers/workflow-context"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
import { useWorkflowContext, type WorkflowProgressEvent } from "@/app/workflows/providers/workflow-context"
import { useWorkflowContext } from "@/app/workflows/providers/workflow-context"
🤖 Prompt for AI Agents
app/workflows/components/workflow-canvas.tsx around line 7: the import currently
pulls in the WorkflowProgressEvent type which is unused in this file; remove
WorkflowProgressEvent from the import list so only useWorkflowContext is
imported, and then run a quick lint/TypeScript check to ensure no other
references to WorkflowProgressEvent remain in this file.

</Button>
) : (
<Button variant="outline" size="sm" onClick={resumeWorkflow}>
<Button variant="outline" size="sm" onClick={() => resumeWorkflow()}>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Consider removing unnecessary arrow function wrapper.

Wrapping resumeWorkflow in an arrow function creates a new function instance on each render. Since you're calling it without arguments here, you can pass it directly.

Apply this diff:

-<Button variant="outline" size="sm" onClick={() => resumeWorkflow()}>
+<Button variant="outline" size="sm" onClick={resumeWorkflow}>

If the intent is to explicitly show that resumeWorkflow accepts optional parameters, consider adding a comment instead.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
<Button variant="outline" size="sm" onClick={() => resumeWorkflow()}>
<Button variant="outline" size="sm" onClick={resumeWorkflow}>
🤖 Prompt for AI Agents
In app/workflows/components/workflow-header.tsx around line 138, the onClick
handler wraps resumeWorkflow in an unnecessary arrow function which creates a
new function each render; replace onClick={() => resumeWorkflow()} with
onClick={resumeWorkflow} (or add a brief comment if you want to document
optional params) so the handler is passed directly without recreating a closure.

Comment on lines +70 to +80
const eventsByStage = useMemo(() => {
const grouped: Record<string, WorkflowProgressEvent[]> = {}
progressEvents.forEach(event => {
const stage = event.stage || "general"
if (!(stage in grouped)) {
grouped[stage] = []
}
grouped[stage].push(event)
})
return grouped
}, [progressEvents])
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Consider using more idiomatic object key check.

Line 74 uses !(stage in grouped) which is correct but !grouped[stage] is more idiomatic for checking if a key doesn't exist.

Apply this diff:

const eventsByStage = useMemo(() => {
  const grouped: Record<string, WorkflowProgressEvent[]> = {}
  progressEvents.forEach(event => {
    const stage = event.stage || "general"
-   if (!(stage in grouped)) {
+   if (!grouped[stage]) {
      grouped[stage] = []
    }
    grouped[stage].push(event)
  })
  return grouped
}, [progressEvents])
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const eventsByStage = useMemo(() => {
const grouped: Record<string, WorkflowProgressEvent[]> = {}
progressEvents.forEach(event => {
const stage = event.stage || "general"
if (!(stage in grouped)) {
grouped[stage] = []
}
grouped[stage].push(event)
})
return grouped
}, [progressEvents])
const eventsByStage = useMemo(() => {
const grouped: Record<string, WorkflowProgressEvent[]> = {}
progressEvents.forEach(event => {
const stage = event.stage || "general"
if (!grouped[stage]) {
grouped[stage] = []
}
grouped[stage].push(event)
})
return grouped
}, [progressEvents])
🤖 Prompt for AI Agents
In app/workflows/components/workflow-progress-panel.tsx around lines 70 to 80,
the code uses the less-idiomatic check `!(stage in grouped)` when determining
whether to initialize grouped[stage]; replace that with the more idiomatic falsy
check `!grouped[stage]` (or use `grouped[stage] ||= []`) to initialize the array
if absent; update that line so grouped[stage] is created when falsy and then
push the event into it.

</Badge>
</div>
<div className="space-y-1 ml-2">
{events.slice(-3).map((event) => (
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Consider showing more events or adding pagination.

Currently showing only the last 3 events per stage (events.slice(-3)) might hide important earlier events, especially for long-running workflows. Consider:

  • Increasing the limit (e.g., last 5-10 events)
  • Adding a "Show all" toggle or pagination
  • Including a visual indicator when events are hidden
🤖 Prompt for AI Agents
In app/workflows/components/workflow-progress-panel.tsx around line 116, the
code currently renders only the last 3 events via events.slice(-3), which can
hide important earlier events; change this to either use a configurable limit
(e.g., const PAGE_SIZE = 5 or 10) or add per-stage UI state to toggle between
collapsed (last N) and expanded (all) views: implement a piece of state like
expandedStages Set/Map, render events.slice(-PAGE_SIZE) when collapsed and
events when expanded, add a "Show all"/"Show less" button to flip that state,
and when collapsed show a compact indicator ("+N more events" or similar) so
users know events are hidden; ensure the behavior is accessible and preserves
keys and styling.

Comment on lines +257 to +324
useEffect(() => {
const allProgressEvents: WorkflowProgressEvent[] = []
let detectedSuspendPayload: WorkflowSuspendPayload | null = null

for (const message of messages) {
if (message.role === "assistant" && message.parts !== null) {
for (const part of message.parts) {
// Handle workflow progress events (data-workflow, data-tool-workflow)
if (part.type === "data-workflow" || part.type === "data-tool-workflow") {
const workflowPart = part as { data?: { text?: string; status?: string; stepId?: string } }
const eventData = workflowPart.data

if (eventData && eventData.text !== null && typeof eventData.text === "string" && eventData.text.trim()) {
allProgressEvents.push({
id: `${message.id}-${part.type}-${Date.now()}`,
stage: part.type.replace("data-", "").replace("-tool-", " "),
status: "in-progress",
message: eventData.text,
stepId: eventData.stepId,
timestamp: new Date(),
data: eventData,
})
}
}

// Handle suspend payloads (data-workflow-suspend)
if (part.type === "data-workflow-suspend") {
const suspendPart = part as { data?: { message?: string; requestId?: string; stepId?: string } }
const suspendData = suspendPart.data

if (suspendData?.message !== null && suspendData?.message !== undefined &&
suspendData?.stepId !== null && suspendData?.stepId !== undefined) {
detectedSuspendPayload = {
message: suspendData.message,
requestId: suspendData.requestId,
stepId: suspendData.stepId,
}
setWorkflowStatus("paused")
}
}

// Handle custom progress events from workflow steps
if (typeof part.type === "string" && part.type.startsWith("data-workflow-progress")) {
const progressPart = part as { type: string; data?: { status?: string; message?: string; stage?: string; stepId?: string } }
const eventData = progressPart.data

if (eventData && eventData.status !== null && typeof eventData.status === "string" &&
(eventData.status === "in-progress" || eventData.status === "done" || eventData.status === "error")) {
allProgressEvents.push({
id: `${message.id}-${part.type}-${Date.now()}`,
stage: eventData.stage ?? "workflow",
status: eventData.status,
message: eventData.message ?? `${part.type} ${eventData.status}`,
stepId: eventData.stepId,
timestamp: new Date(),
data: eventData,
})
}
}
}
}
}

setProgressEvents(allProgressEvents)
if (detectedSuspendPayload) {
setSuspendPayload(detectedSuspendPayload)
}
}, [messages])
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Verify type safety and consider batching state updates.

The event extraction logic has several concerns:

  1. Type casting safety: Lines 266, 284, 300 use as to cast message parts without runtime validation. If the part structure doesn't match, accessing nested properties could fail.

  2. State update batching: Line 320 calls setProgressEvents inside the loop, and lines 321-323 conditionally update suspendPayload and workflowStatus. In React 19, automatic batching helps, but explicitly batching these related updates would be clearer.

  3. Verbose null checks: Lines 269, 287-288, 303-304 use redundant checks like !== null && typeof x === "string". TypeScript's optional chaining and type guards would be cleaner.

Consider refactoring to improve type safety:

// Extract progress events from custom data parts
useEffect(() => {
  const allProgressEvents: WorkflowProgressEvent[] = []
  let detectedSuspendPayload: WorkflowSuspendPayload | null = null
+ let shouldPause = false

  for (const message of messages) {
-   if (message.role === "assistant" && message.parts !== null) {
+   if (message.role === "assistant" && message.parts) {
      for (const part of message.parts) {
        // Handle workflow progress events (data-workflow, data-tool-workflow)
        if (part.type === "data-workflow" || part.type === "data-tool-workflow") {
-         const workflowPart = part as { data?: { text?: string; status?: string; stepId?: string } }
-         const eventData = workflowPart.data
+         const eventData = (part as any).data
          
-         if (eventData && eventData.text !== null && typeof eventData.text === "string" && eventData.text.trim()) {
+         if (eventData?.text?.trim()) {
            allProgressEvents.push({
              id: `${message.id}-${part.type}-${Date.now()}`,
              stage: part.type.replace("data-", "").replace("-tool-", " "),
              status: "in-progress",
              message: eventData.text,
              stepId: eventData.stepId,
              timestamp: new Date(),
              data: eventData,
            })
          }
        }

        // Handle suspend payloads (data-workflow-suspend)
        if (part.type === "data-workflow-suspend") {
-         const suspendPart = part as { data?: { message?: string; requestId?: string; stepId?: string } }
-         const suspendData = suspendPart.data
+         const suspendData = (part as any).data

-         if (suspendData?.message !== null && suspendData?.message !== undefined &&
-             suspendData?.stepId !== null && suspendData?.stepId !== undefined) {
+         if (suspendData?.message && suspendData?.stepId) {
            detectedSuspendPayload = {
              message: suspendData.message,
              requestId: suspendData.requestId,
              stepId: suspendData.stepId,
            }
-           setWorkflowStatus("paused")
+           shouldPause = true
          }
        }

        // Handle custom progress events from workflow steps
        if (typeof part.type === "string" && part.type.startsWith("data-workflow-progress")) {
-         const progressPart = part as { type: string; data?: { status?: string; message?: string; stage?: string; stepId?: string } }
-         const eventData = progressPart.data
+         const eventData = (part as any).data

-         if (eventData && eventData.status !== null && typeof eventData.status === "string" &&
-             (eventData.status === "in-progress" || eventData.status === "done" || eventData.status === "error")) {
+         const validStatuses = ["in-progress", "done", "error"] as const
+         if (eventData?.status && validStatuses.includes(eventData.status)) {
            allProgressEvents.push({
              id: `${message.id}-${part.type}-${Date.now()}`,
              stage: eventData.stage ?? "workflow",
              status: eventData.status,
              message: eventData.message ?? `${part.type} ${eventData.status}`,
              stepId: eventData.stepId,
              timestamp: new Date(),
              data: eventData,
            })
          }
        }
      }
    }
  }

+ // Batch state updates together
  setProgressEvents(allProgressEvents)
  if (detectedSuspendPayload) {
    setSuspendPayload(detectedSuspendPayload)
+   setWorkflowStatus("paused")
  }
}, [messages])

Committable suggestion skipped: line range outside the PR's diff.

Comment on lines +55 to +59
const pagesByDepth = pages.reduce((acc, page) => {
if (!acc[page.depth]) {acc[page.depth] = []}
acc[page.depth].push(page)
return acc
}, {} as Record<number, typeof pages>)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Minor formatting inconsistency.

Missing space after opening brace on line 56.

Apply this diff:

-    if (!acc[page.depth]) {acc[page.depth] = []}
+    if (!acc[page.depth]) { acc[page.depth] = [] }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const pagesByDepth = pages.reduce((acc, page) => {
if (!acc[page.depth]) {acc[page.depth] = []}
acc[page.depth].push(page)
return acc
}, {} as Record<number, typeof pages>)
const pagesByDepth = pages.reduce((acc, page) => {
if (!acc[page.depth]) { acc[page.depth] = [] }
acc[page.depth].push(page)
return acc
}, {} as Record<number, typeof pages>)
🤖 Prompt for AI Agents
In src/components/ai-elements/tools/site-map-extractor-tool.tsx around lines 55
to 59, there's a minor formatting inconsistency: the opening brace on line 56
lacks a space. Update the line to add a space after the '{' so the conditional
block reads with consistent spacing (e.g., change "{acc[page.depth]" to "{
acc[page.depth]") to match project formatting.

{totalPages} pages discovered
</Badge>
<Badge variant="secondary" className="text-purple-700">
Depth: {Math.max(...pages.map(p => p.depth))}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Handle empty pages array edge case.

Math.max(...pages.map(p => p.depth)) will return -Infinity if the pages array is empty, resulting in a poor user experience.

Apply this diff:

-            <Badge variant="secondary" className="text-purple-700">
-              Depth: {Math.max(...pages.map(p => p.depth))}
-            </Badge>
+            {pages.length > 0 && (
+              <Badge variant="secondary" className="text-purple-700">
+                Depth: {Math.max(...pages.map(p => p.depth))}
+              </Badge>
+            )}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Depth: {Math.max(...pages.map(p => p.depth))}
{pages.length > 0 && (
<Badge variant="secondary" className="text-purple-700">
Depth: {Math.max(...pages.map(p => p.depth))}
</Badge>
)}
🤖 Prompt for AI Agents
In src/components/ai-elements/tools/site-map-extractor-tool.tsx around line 77,
the expression Math.max(...pages.map(p => p.depth)) will return -Infinity when
pages is empty; change it to guard the empty array and provide a sensible
fallback (e.g., pages.length ? Math.max(...pages.map(p => p.depth)) : 0 or a
display string like 'N/A') so the UI shows a valid depth instead of -Infinity.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements comprehensive web scraper tool UI components and workflow/network progress event tracking capabilities. It adds new UI visualizations for various web scraping tools (web scraper, batch scraper, sitemap extractor, link extractor) and enhances workflow and network providers with progress events, suspend/resume functionality, and real-time status updates.

Key Changes:

  • Added type exports for web scraper tools to enable proper TypeScript inference in UI components
  • Created dedicated UI components for displaying web scraper tool results with rich visualizations
  • Enhanced workflow context with suspend/resume capabilities and progress event tracking
  • Added progress event tracking to network context for real-time agent status updates
  • Removed legacy tool-part-transform helper file (functionality likely moved elsewhere)

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 18 comments.

Show a summary per file
File Description
src/mastra/tools/web-scraper-tool.ts Added LinkExtractorUITool type export for UI component type inference
src/mastra/tools/index.ts Exported 9 new web scraper tool variants for external consumption
src/components/ai-elements/tools/web-scraper-tool.tsx New component for rendering web scraper results with tabs for extracted data, markdown, metadata, images, and raw HTML
src/components/ai-elements/tools/site-map-extractor-tool.tsx New component for displaying site map extraction results with hierarchical depth-based organization
src/components/ai-elements/tools/link-extractor-tool.tsx New component for visualizing extracted links with validation status and categorization
src/components/ai-elements/tools/index.ts Barrel export file for new tool UI components
src/components/ai-elements/tools/batch-web-scraper-tool.tsx New component for displaying batch scraping results with success/failure status per URL
app/workflows/providers/workflow-context.tsx Added progress events, suspend payload state, approval workflow, and event extraction from message parts
app/workflows/components/workflow-suspend-dialog.tsx New dialog component for handling workflow approval requests with user input
app/workflows/components/workflow-progress-panel.tsx New panel component displaying real-time workflow progress events grouped by stage
app/workflows/components/workflow-node.tsx Added streaming progress visualization within workflow nodes and updated styling
app/workflows/components/workflow-input-panel.tsx Updated textarea and button styling (contains invalid Tailwind classes)
app/workflows/components/workflow-header.tsx Updated select trigger styling and resume button callback
app/workflows/components/workflow-canvas.tsx Integrated progress panel and suspend dialog components
app/networks/providers/network-context.tsx Added progress event tracking, improved type safety with explicit interfaces, and updated parameter naming
app/networks/components/network-routing-panel.tsx Added progress event display section showing the last 5 events
app/chat/helpers/tool-part-transform.ts Deleted helper file (functionality likely consolidated elsewhere)
app/chat/components/agent-tools.tsx Added commented-out code for custom web scraper tool rendering

const agentPart = part as { data: AgentDataPart }
const eventData = agentPart.data

if (eventData?.data?.text?.trim()) {
Copy link

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conditional check uses optional chaining but then explicitly checks for null and undefined. When using optional chaining, the expression "eventData?.data?.text?.trim()" already returns undefined if any part of the chain is null or undefined, so the explicit truthiness check is sufficient. The verbose null/undefined checking pattern is inconsistent with the optional chaining usage.

Copilot uses AI. Check for mistakes.
disabled={isWorkflowActive}
>
<SelectTrigger className="w-[280px]">
<SelectTrigger className="w-70">
Copy link

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The className value "w-70" is not a valid Tailwind CSS utility class. Tailwind uses specific numeric scales for width utilities. Valid options include "w-64" (16rem), "w-72" (18rem), "w-80" (20rem), or "w-96" (24rem). The value "w-70" will not apply any styling and should be replaced with a valid Tailwind width class.

Suggested change
<SelectTrigger className="w-70">
<SelectTrigger className="w-72">

Copilot uses AI. Check for mistakes.

if (eventData && eventData.text !== null && typeof eventData.text === "string" && eventData.text.trim()) {
allProgressEvents.push({
id: `${message.id}-${part.type}-${Date.now()}`,
Copy link

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using Date.now() in the ID generation for progress events can lead to collisions if multiple events are processed in the same millisecond. This could cause duplicate keys in React rendering and make it difficult to track individual events. Consider using a more robust unique ID generation approach such as crypto.randomUUID() or a counter-based approach.

Copilot uses AI. Check for mistakes.
if (eventData && eventData.status !== null && typeof eventData.status === "string" &&
(eventData.status === "in-progress" || eventData.status === "done" || eventData.status === "error")) {
allProgressEvents.push({
id: `${message.id}-${part.type}-${Date.now()}`,
Copy link

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using Date.now() in the ID generation for progress events can lead to collisions if multiple events are processed in the same millisecond. This could cause duplicate keys in React rendering and make it difficult to track individual events. Consider using a more robust unique ID generation approach such as crypto.randomUUID() or a counter-based approach.

Copilot uses AI. Check for mistakes.

if (eventData?.data?.text?.trim()) {
allProgressEvents.push({
id: `${message.id}-${part.type}-${Date.now()}`,
Copy link

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using Date.now() in the ID generation for progress events can lead to collisions if multiple events are processed in the same millisecond. This could cause duplicate keys in React rendering and make it difficult to track individual events. Consider using a more robust unique ID generation approach such as crypto.randomUUID() or a counter-based approach.

Copilot uses AI. Check for mistakes.
Comment on lines +371 to +372
const startedAt = agent.startedAt !== null && agent.startedAt !== undefined ? new Date(agent.startedAt) : undefined
const completedAt = agent.completedAt !== null && agent.completedAt !== undefined ? new Date(agent.completedAt) : undefined
Copy link

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check for null before undefined is redundant when using optional chaining. The expression "agent.startedAt !== null && agent.startedAt !== undefined" can be simplified to just "agent.startedAt != null" (loose equality with null checks both null and undefined) or even just check the value directly since new Date() handles various input types. This would make the code more concise and idiomatic.

Suggested change
const startedAt = agent.startedAt !== null && agent.startedAt !== undefined ? new Date(agent.startedAt) : undefined
const completedAt = agent.completedAt !== null && agent.completedAt !== undefined ? new Date(agent.completedAt) : undefined
const startedAt = agent.startedAt != null ? new Date(agent.startedAt) : undefined
const completedAt = agent.completedAt != null ? new Date(agent.completedAt) : undefined

Copilot uses AI. Check for mistakes.

import { Badge } from "@/ui/badge"
import { Card, CardContent, CardHeader, CardTitle } from "@/ui/card"
import { CheckCircle, XCircle, Clock, AlertCircle } from "lucide-react"
Copy link

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused import AlertCircle.

Suggested change
import { CheckCircle, XCircle, Clock, AlertCircle } from "lucide-react"
import { CheckCircle, XCircle, Clock } from "lucide-react"

Copilot uses AI. Check for mistakes.

import { Badge } from "@/ui/badge"
import { Card, CardContent, CardHeader, CardTitle } from "@/ui/card"
import { TreePine, Link, ExternalLink, FileText } from "lucide-react"
Copy link

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused imports ExternalLink, Link.

Suggested change
import { TreePine, Link, ExternalLink, FileText } from "lucide-react"
import { TreePine, FileText } from "lucide-react"

Copilot uses AI. Check for mistakes.
import { Badge } from "@/ui/badge"
import { Card, CardContent, CardHeader, CardTitle } from "@/ui/card"
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/ui/tabs"
import { ExternalLink, FileText, Globe, Image as ImageIcon, Link } from "lucide-react"
Copy link

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused import Link.

Suggested change
import { ExternalLink, FileText, Globe, Image as ImageIcon, Link } from "lucide-react"
import { ExternalLink, FileText, Globe, Image as ImageIcon } from "lucide-react"

Copilot uses AI. Check for mistakes.
)
}

const { extractedData, rawContent, markdownContent, metadata, images, structuredData } = output
Copy link

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused variable structuredData.

Suggested change
const { extractedData, rawContent, markdownContent, metadata, images, structuredData } = output
const { extractedData, rawContent, markdownContent, metadata, images } = output

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant