Skip to content

fix: allow multiple pipeline chunker nodes#14845

Open
CaptainTimon wants to merge 1 commit into
infiniflow:mainfrom
CaptainTimon:fix/allow-multiple-pipeline-custom-nodes
Open

fix: allow multiple pipeline chunker nodes#14845
CaptainTimon wants to merge 1 commit into
infiniflow:mainfrom
CaptainTimon:fix/allow-multiple-pipeline-custom-nodes

Conversation

@CaptainTimon
Copy link
Copy Markdown
Contributor

What problem does this solve?

Closes #14772.

Pipeline workflows hid chunker nodes after one instance had already been added, which prevented users from adding multiple independently configured title/token chunking stages in the same pipeline.

What is changed?

  • Keep only structural pipeline nodes (Parser, Tokenizer) restricted to a single canvas instance.
  • Keep TokenChunker and TitleChunker available in the pipeline operator menu after one is already present.
  • Align the pipeline singleton operator list with that behavior.

Behavior proof

  • Before: PipelineAccordionOperators filtered TokenChunker and TitleChunker through restrictSingleOperatorOnCanvas, so the menu removed them once findNodeByName() found an existing node.
  • After: the chunker menu always exposes [Operator.TokenChunker, Operator.TitleChunker], while Parser and Tokenizer still go through the singleton filter.
  • The DSL builder already keys components by unique node ID, so multiple chunker nodes can keep separate form configs and edges.

Test plan

  • git diff --check
  • cd web && npx eslint src/pages/agent/canvas/node/dropdown/accordion-operators.tsx src/pages/agent/constant/pipeline.tsx
  • Commit hook ran prettier --write --ignore-unknown and eslint on staged files
  • cd web && npm run type-check -- --pretty false currently fails on existing unrelated TypeScript errors outside this change, e.g. src/components/document-preview/doc-preview.tsx, src/components/dynamic-form.tsx, and src/components/file-upload.tsx.

@dosubot dosubot Bot added the size:S This PR changes 10-29 lines, ignoring generated files. label May 12, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 12, 2026

Review Change Stack
No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a6af286b-e8ff-4c39-a52b-f5f92863d3dc

📥 Commits

Reviewing files that changed from the base of the PR and between ad4717f and 60afd43.

📒 Files selected for processing (2)
  • web/src/pages/agent/canvas/node/dropdown/accordion-operators.tsx
  • web/src/pages/agent/constant/pipeline.tsx

📝 Walkthrough

Walkthrough

This PR modifies operator constraints in the pipeline UI to allow multiple instances of chunker operators. The SingleOperators constant now restricts only tokenizer and parser operators, while chunker operators are handled directly in the accordion component without single-instance filtering.

Changes

Multiple Chunker Operators Support

Layer / File(s) Summary
Operator singularity constraints
web/src/pages/agent/constant/pipeline.tsx
SingleOperators constant is reduced from [Operator.Tokenizer, Operator.TokenChunker, Operator.TitleChunker, Operator.Parser] to [Operator.Tokenizer, Operator.Parser], allowing chunker operators to appear multiple times on the canvas.
Chunker operator handling in accordion
web/src/pages/agent/canvas/node/dropdown/accordion-operators.tsx
Comment clarifies that structural pipeline operators must exist at most once, and chunkerOperators is initialized as a fixed [Operator.TokenChunker, Operator.TitleChunker] array without the single-operator restriction filter, enabling multiple instances.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

A rabbit hops through operator trees,
Chunkers now dance in twos and threes—
No longer bound to singular constraint,
Multiple paths the workflow can paint. 🐰✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The PR title clearly and concisely describes the main change: allowing multiple pipeline chunker nodes, which is the primary objective of this changeset.
Description check ✅ Passed The PR description comprehensively addresses the template sections, explaining the problem solved, changes made, behavioral differences, and test plan.
Linked Issues check ✅ Passed Code changes directly fulfill issue #14772 by allowing multiple chunker nodes (TokenChunker, TitleChunker) while restricting singleton nodes (Parser, Tokenizer) to single instances, enabling multi-stage classification workflows.
Out of Scope Changes check ✅ Passed All changes are scoped to the stated objectives: modifying chunker operator filtering logic and updating the SingleOperators constant to reflect the new singleton restrictions.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

Warning

Review ran into problems

🔥 Problems

Git: Failed to clone repository. Please run the @coderabbitai full review command to re-trigger a full review. If the issue persists, set path_filters to include or exclude specific files.

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Support multiple custom nodes in Pipeline - e.g., multiple title-based classifiers

1 participant