Skip to content

feat: add composite task routing eval - #61

Merged
pertrai1 merged 2 commits into
mainfrom
feat/composite-task-routing
Jul 10, 2026
Merged

pertrai1 merged 2 commits into
mainfrom
feat/composite-task-routing

Conversation

@pertrai1-bot

@pertrai1-bot pertrai1-bot commented Jul 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Add compact Composite Task Routing guidance to directives/adaptive-routing.md.
  • Add a compositional skill routing eval scenario for multi-intent CI/debugging/test/boundary prompts.
  • Regenerate manifest.json after bumping adaptive-routing to 1.9.0.

Verification

  • npm install (dependencies were missing locally)
  • npm run manifest
  • git diff --check && git diff --cached --check
  • npm run check
  • git diff --check HEAD~1..HEAD

Parallel PR notes

This PR intentionally stays scoped to routing prose and an eval scenario. It does not add manifest routing metadata or context-audit CLI behavior, which are expected to be handled in separate parallel PRs.

Summary by CodeRabbit

  • New Features

    • Added guidance for multi-intent requests by decomposing them into atomic work items and composing the correct workflow paths, with dependency-aware ordering.
  • Documentation

    • Introduced a new “Compositional Skill Routing” scenario covering debugging, regression testing, and boundary review, including expected routing behavior and failure checks.
    • Updated the adaptive-routing directive version to 1.9.1 and refined instructions to load only selected matches while explaining excluded near-matches.

@coderabbitai

coderabbitai Bot commented Jul 10, 2026 •

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 06cbeef2-c6ca-46bb-84cc-5583a81618e9

📥 Commits

Reviewing files that changed from the base of the PR and between 30a5cb4 and 1b7b896.

📒 Files selected for processing (3)
  • directives/adaptive-routing.md
  • evals/scenarios/compositional-skill-routing.md
  • manifest.json

📝 Walkthrough

Walkthrough

Adaptive routing is updated to version 1.9.1 with composite task decomposition, dependency-ordered route merging, and selective skill loading. A new evaluation scenario verifies debugging, regression, and boundary-review routing behavior.

Changes

Adaptive routing composition

Layer / File(s) Summary
Composite routing policy and versioning
directives/adaptive-routing.md, manifest.json
Adds composite task routing guidance and updates the adaptive-routing directive version to 1.9.1.
Composite routing evaluation
evals/scenarios/compositional-skill-routing.md
Defines expected decomposition, skill selection, route ordering, anti-behaviors, and pass/fail criteria.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related issues

  • pertrai1/ai-coding-agent issue 5 — Covers adaptive, selective directive and skill routing through composite task decomposition and routing evaluations.

Possibly related PRs

Sequence Diagram(s)

sequenceDiagram
  participant Agent
  participant AdaptiveRouting
  participant DiscoveryMetadata
  participant SpecialistSkills
  Agent->>AdaptiveRouting: Decompose CI-failure request
  AdaptiveRouting->>DiscoveryMetadata: Match debugging, regression, and boundary work
  DiscoveryMetadata-->>AdaptiveRouting: Return candidate paths and skills
  AdaptiveRouting->>SpecialistSkills: Load selected specialist skills in order
  AdaptiveRouting-->>Agent: Report merged route and selected files
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the new composite task routing evaluation added by the pull request.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/composite-task-routing

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
directives/adaptive-routing.md (1)

108-110: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Clarify regression-test ordering.

“Fix before regression test” can encourage adding the regression test only after the defect is fixed, so it never demonstrates the original failure. Prefer: reproduce → add/update regression test → fix → rerun the test → boundary review. Align the scenario’s route expectations with this distinction.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@directives/adaptive-routing.md` around lines 108 - 110, Clarify the
route-ordering guidance in the “Compose the final route” step: require reproduce
→ add or update the regression test → fix → rerun the test → boundary review,
replacing the ambiguous “fix before regression test” ordering and aligning
scenario route expectations accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@directives/adaptive-routing.md`:
- Around line 108-110: Clarify the route-ordering guidance in the “Compose the
final route” step: require reproduce → add or update the regression test → fix →
rerun the test → boundary review, replacing the ambiguous “fix before regression
test” ordering and aligning scenario route expectations accordingly.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3b6e458d-9ef9-40e4-aa95-6195a38159bb

📥 Commits

Reviewing files that changed from the base of the PR and between c162eb4 and 30a5cb4.

📒 Files selected for processing (3)
  • directives/adaptive-routing.md
  • evals/scenarios/compositional-skill-routing.md
  • manifest.json

@pertrai1
pertrai1 merged commit 27406c7 into main Jul 10, 2026
1 of 2 checks passed
@pertrai1
pertrai1 deleted the feat/composite-task-routing branch July 10, 2026 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants