Skip to content

feat(cli): add an experimental diff command - #2940

Draft
vadyvas wants to merge 53 commits into
mainfrom
feat/diff-command
Draft

vadyvas wants to merge 53 commits into
mainfrom
feat/diff-command

Conversation

@vadyvas

@vadyvas vadyvas commented Jul 7, 2026 •

Copy link
Copy Markdown
Contributor

What/Why/How?

Adds the experimental redocly diff command.
It compares two versions of an API description and gives each change an impact: the semver part that a release must bump (major, minor, or patch).

redocly diff main/openapi.yaml openapi.yaml
  • Supports OpenAPI 3.x and AsyncAPI 3.
    For other description types, it lists the changes, but no rule judges them.
  • Exits with code 1 on a major change.
    --fail-on sets another threshold.
  • --check-version fails if info.version is not bumped enough.
  • Formats: stylish, json, markdown, html, github-actions, next-version.
  • The diff section of redocly.yaml sets the impact of each rule.
    diff-recommended turns on all 27 rules, and --skip-rule turns off a rule for one run.

How it works

flowchart LR
  base["base.yaml"] --> baseTree["node tree"]
  revision["revision.yaml"] --> revisionTree["node tree"]
  baseTree --> compare["compareTrees"]
  revisionTree --> compare
  compare --> changes["changes"]
  changes --> judge["judgeChanges"]
  rules[("diff rules + impacts from redocly.yaml")] -.-> judge
  judge --> report["report format"]
Loading
  1. buildNodeTree walks each document once, after the preprocessors.
    Each node keeps its type from the walker, and a $ref node points to its target.
  2. compareTrees pairs the two trees from the root down and lists what was added, removed, or modified.
    Nothing below an added or removed node is compared.
  3. judgeChanges runs the diff rules on each change, and the highest impact wins.
    A change that no rule reports is minor if something was added, and patch otherwise.

Pairing

Map entries pair by key, and list items by content.
Exact matches pair first.
Each other child pairs with the most alike child of the same type, if they are at least half alike (like git rename detection).
So a reordered list or a renamed key is one change, not a removal and an addition:

flowchart LR
  subgraph baseParameters [base parameters]
    baseLimit["0: limit, query"]
    baseSearch["1: search, query"]
  end
  subgraph revisionParameters [revision parameters]
    revisionSearch["0: search, query"]
    revisionLimits["1: limits, query"]
  end
  baseSearch -- same --> revisionSearch
  baseLimit -- "most alike: name changed" --> revisionLimits
Loading

directions

The same change can break a request and be safe in a response.
A rule calls getDirections() only when it needs the direction.
The directions come from the nearest node that knows its direction (request body, parameters, responses, readOnly/writeOnly, AsyncAPI action), above the change or above any place in referencedBy.
Callbacks and webhooks swap the directions.

flowchart BT
  change["new enum value in Order.status"] --> order["Order schema"]
  order --> requestBody["POST /orders requestBody: request"]
  order -. referencedBy .-> response["GET /orders/{id} 200: response"]
Loading

Here the directions are request and response, so enum-values-added reports the change as major.

Diff rules

A diff rule is a visitor over changes.
It is keyed by node type and nested like a lint visitor (simplified):

export const PropertyRemoved: DiffRule = () => ({
  SchemaProperties: {
    Schema(change, { report, getDirections }) {
      if (change.kind === 'removed' && getDirections().includes('response')) {
        report({ message: `Property \`${nameOf(change.node)}\` was removed.` });
      }
    },
  },
});

Changes in core

  • diffDocuments() and the diff result types.
  • node-tree/: the typed tree of a document, built by the walker.
  • diff/: compareTrees, judgeChanges, directions, and the rules in diff/rules/.
  • Config: the diff, oas3_0Diff, oas3_1Diff, oas3_2Diff, and async3Diff sections, and the diff-recommended ruleset.
    Plugins can add diff rules, but this is not documented yet.

Reference

Testing

  • Unit tests for each diff rule (one test for each reported change) and for each engine step.
  • e2e tests for user stories: output formats, OpenAPI and AsyncAPI versions, configuration, a plugin, multi-file descriptions, and CLI options.

Screenshots (optional)

image image

Check yourself

  • This PR follows the contributing guide
  • All new/updated code is covered by tests
  • Core code changed? - Tested with other Redocly products (internal contributions only)
  • New package installed? - Tested in different environments (browser/node)
  • Documentation update has been considered

Security

  • The security impact of the change has been considered
  • Code follows company security practices and guidelines

Note

Medium Risk
Large new comparison and rule engine plus default config now includes diff-recommended; misclassified impacts or bundle resolution gaps could affect CI gates, though the feature is marked experimental.

Overview
Adds an experimental redocly diff <base> <revision> command that bundles both API descriptions, compares them structurally, and reports additions, removals, and edits with a semver impact (patch, minor, major) per change.

The comparison engine lives in @redocly/openapi-core: it builds matched diff trees (identity-aware matching for parameters, paths, $ref targets, etc.), resolves request vs response direction (including callbacks/webhooks and AsyncAPI action), runs a diff rule registry for OpenAPI 3.x and AsyncAPI 3, and exposes diffDocuments. Governance is configured like lint via extends: [diff-recommended], global diff / per-spec oas3_*Diff / async3Diff blocks, and plugin diff rules; default config now also extends diff-recommended.

The CLI command adds report formats (stylish, json, markdown, html, github-actions, next-version), --fail-on (default major), optional --check-version against info.version, and --skip-rule. Documentation and a minor changeset cover both packages.

Reviewed by Cursor Bugbot for commit e01e302. Bugbot is set up for automated code reviews on this repo. Configure here.

@changeset-bot

changeset-bot Bot commented Jul 7, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 5b7ade9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages
Name Type
@redocly/cli Minor
@redocly/openapi-core Minor
@redocly/client-generator Patch
@redocly/recheck Minor
@redocly/respect-core Minor
@redocly/reunite-integration Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Comment thread packages/cli/src/commands/diff/serializers/markdown.ts Fixed
@vadyvas vadyvas self-assigned this Aug 7, 2026
@vadyvas
vadyvas force-pushed the feat/diff-command branch from fe59519 to 7d66075 Compare August 7, 2026 16:47
@github-actions

github-actions Bot commented Aug 7, 2026 •

Copy link
Copy Markdown
Contributor

Performance Benchmark (Lower is Faster)

CLI Version Bundle Lint Check Config
latest ▓░░░░░░░░░░ 1.00x ▓░░░░░░░░░░ 1.00x ▓░░░░░░░░░░ 1.00x
next ▓░░░░░░░░░░ 1.01x ± 0 ▓░░░░░░░░░░ 1.00x ± 0.01 ▓░░░░░░░░░░ 1.00x ± 0.01

@github-actions

github-actions Bot commented Aug 7, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 83.53% (🎯 80%) 21065 / 25218
🔵 Statements 83.09% (🎯 80%) 22850 / 27499
🔵 Functions 85.07% (🎯 80%) 4046 / 4756
🔵 Branches 75.74% (🎯 75%) 15428 / 20367
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
packages/cli/src/commands/diff/check-version.ts 100% 95.83% 100% 100%
packages/cli/src/commands/diff/fail-on.ts 100% 100% 100% 100%
packages/cli/src/commands/diff/index.ts 0% 0% 0% 0% 23-99
packages/cli/src/commands/diff/format/github-actions.ts 100% 100% 100% 100%
packages/cli/src/commands/diff/format/html.ts 100% 66.66% 100% 100%
packages/cli/src/commands/diff/format/index.ts 0% 100% 100% 0% 9-15
packages/cli/src/commands/diff/format/json.ts 100% 100% 100% 100%
packages/cli/src/commands/diff/format/location.ts 100% 100% 100% 100%
packages/cli/src/commands/diff/format/markdown.ts 100% 75% 100% 100%
packages/cli/src/commands/diff/format/next-version.ts 75% 50% 100% 75% 8-10
packages/cli/src/commands/diff/format/stylish.ts 98.14% 95.65% 100% 100% 55
packages/core/src/config/builtIn.ts 100% 100% 100% 100%
packages/core/src/config/config-resolvers.ts 77.29% 65.04% 94.11% 78.02% 78, 106-109, 201, 210, 266, 286-287, 311, 324, 335, 338-342, 350-359, 383-385, 401, 404, 407, 410, 423-425, 435, 438, 441-444, 447-450, 453-456, 470-472, 479, 482, 485, 488, 491, 494, 503-511, 532-534
packages/core/src/config/config.ts 62.01% 62.96% 70.83% 61.95% 200-230, 257, 261, 265-271, 274, 278-287, 319, 337-357, 417-425, 462-502
packages/core/src/config/constants.ts 100% 100% 100% 100%
packages/core/src/config/diff-recommended.ts 100% 100% 100% 100%
packages/core/src/config/rules.ts 100% 100% 100% 100%
packages/core/src/config/utils.ts 97.89% 81.81% 100% 98.93% 38, 98-100
packages/core/src/diff/changes.ts 100% 100% 100% 100%
packages/core/src/diff/diff-node.ts 100% 100% 100% 100%
packages/core/src/diff/diff-tree.ts 100% 100% 100% 100%
packages/core/src/diff/direction.ts 97.05% 97.14% 100% 96.15% 73
packages/core/src/diff/impact.ts 100% 100% 100% 100%
packages/core/src/diff/index.ts 100% 87.5% 100% 100%
packages/core/src/diff/judge.ts 100% 100% 100% 100%
packages/core/src/diff/pair-children.ts 100% 92.3% 100% 100%
packages/core/src/diff/rules/channel-address-changed.ts 91.66% 87.5% 100% 100% 6
packages/core/src/diff/rules/channel-removed.ts 100% 83.33% 100% 100%
packages/core/src/diff/rules/enum-values-added.ts 91.66% 87.5% 100% 100% 11
packages/core/src/diff/rules/enum-values-removed.ts 91.66% 87.5% 100% 100% 11
packages/core/src/diff/rules/index.ts 100% 100% 100% 100%
packages/core/src/diff/rules/media-type-removed.ts 100% 91.66% 100% 100%
packages/core/src/diff/rules/message-content-type-changed.ts 91.66% 87.5% 100% 100% 6
packages/core/src/diff/rules/message-removed.ts 100% 50% 100% 100%
packages/core/src/diff/rules/operation-action-changed.ts 75% 62.5% 100% 100% 8, 14, 15
packages/core/src/diff/rules/operation-id-changed.ts 100% 100% 100% 100%
packages/core/src/diff/rules/operation-removed.ts 90.9% 75% 100% 100% 7
packages/core/src/diff/rules/parameter-became-required.ts 88.88% 88.23% 100% 100% 11, 28
packages/core/src/diff/rules/parameter-removed.ts 95.23% 89.47% 100% 100% 7
packages/core/src/diff/rules/parameter-serialization-changed.ts 94.11% 90% 100% 100% 12
packages/core/src/diff/rules/path-removed.ts 96.15% 70% 100% 100% 53
packages/core/src/diff/rules/property-removed.ts 100% 100% 100% 100%
packages/core/src/diff/rules/request-body-became-required.ts 83.33% 87.5% 100% 100% 5
packages/core/src/diff/rules/request-body-removed.ts 100% 75% 100% 100%
packages/core/src/diff/rules/required-properties-added.ts 100% 100% 100% 100%
packages/core/src/diff/rules/required-properties-removed.ts 100% 100% 100% 100%
packages/core/src/diff/rules/response-header-removed.ts 92.85% 85.71% 100% 100% 7
packages/core/src/diff/rules/response-removed.ts 100% 100% 100% 100%
packages/core/src/diff/rules/schema-combinator-changed.ts 84.21% 77.77% 100% 100% 9, 24, 27
packages/core/src/diff/rules/schema-constraint-changed.ts 94.73% 92.47% 100% 97.29% 88, 109-112, 142, 175
packages/core/src/diff/rules/schema-type-changed.ts 100% 100% 100% 100%
packages/core/src/diff/rules/security-requirement-changed.ts 100% 97.14% 100% 100%
packages/core/src/diff/rules/security-scheme-changed.ts 100% 100% 100% 100%
packages/core/src/diff/rules/server-removed.ts 100% 100% 100% 100%
packages/core/src/diff/rules/utils.ts 93.33% 90% 100% 100% 8
packages/core/src/node-tree/access.ts 100% 95% 100% 100%
packages/core/src/node-tree/index.ts 97.43% 92.85% 100% 100% 76
packages/core/src/types/redocly-yaml.ts 90.38% 77.19% 93.33% 90.09% 518, 550, 592-599, 601, 740-750, 760-776
packages/core/src/utils/is-scalar.ts 100% 100% 100% 100%
packages/core/src/utils/items-not-in.ts 100% 100% 100% 100%
packages/core/src/utils/levenshtein.ts 100% 100% 100% 100%
Generated in workflow #12409 for commit 5b7ade9 by the Vitest Coverage Report Action

@vadyvas vadyvas changed the title Feat/diff command feat(cli): add an experimental diff command Aug 10, 2026
Comment thread packages/core/src/diff/format/markdown.ts Fixed
Comment thread packages/core/src/diff/format/markdown.ts Fixed
Comment thread packages/core/src/diff/format/markdown.ts Fixed
Comment thread packages/core/src/diff/format/markdown.ts Fixed
@vadyvas

vadyvas commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit e01e302. Configure here.

message: `Security scheme \`${change.property}\` requires new scopes: ${quoted(added)}.`,
});
},
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AND security scheme addition missed

Medium Severity

Adding another scheme to an existing security requirement (AND) produces no verdict when the new scheme has no scopes. SecurityScopesAdded only reports extra scopes, and SecurityRequirementAdded only covers going from no authentication to some.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit e01e302. Configure here.

if (change.kind === 'removed')
report({ message: `Media type \`${nameOf(change.node)}\` was removed.` });
},
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed maps skip child rules

High Severity

collectChanges reports a removed map once and does not visit its children. media-type-removed and property-removed only match those children, so deleting content or properties is rated patch even though removing the same members one by one is major.

Additional Locations (2)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit e01e302. Configure here.

const before = acceptedTypes(fieldOf(base, 'type'), fieldOf(base, 'nullable'));
const after = acceptedTypes(fieldOf(revision, 'type'), fieldOf(revision, 'nullable'));
// An absent `type` accepts anything, so there is nothing to narrow or widen.
if (!before.length || !after.length) return;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Untyped schema type changes ignored

Medium Severity

SchemaTypeChanged returns early when either side omits type. Adding a type to an untyped request schema narrows what clients may send, and dropping a type from a response widens what they must handle. Both stay unjudged.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit e01e302. Configure here.

vadyvas and others added 14 commits October 7, 2026 20:37
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Compare two API descriptions and report added, removed, and changed
parts. Structural diff works for every supported spec type via the
existing openapi-core type trees; breaking-change classification
(breaking / warning / non-breaking) applies to OpenAPI 3.x.

The diff engine lives entirely in the CLI package and consumes only the
public @redocly/openapi-core API (walkDocument, type trees, bundle) —
packages/core is untouched. Pipeline: collect each side into a flat
stable-pointer map, two-pass compare into a change list, then classify
with a polarity-aware lint-style rule registry (worst verdict wins).

Supports stylish, json, markdown, and html output and a --fail-on CI
gate. Marked [experimental]; 14 starter rules documented.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… verdicts

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ons, and path-param matching

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e case

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vadyvas added 28 commits October 7, 2026 20:38
…-recommended and configure diff once per project
…pecification-agnostic

Match the two node maps up into a tree of pairs instead of computing a string key per
node and joining the two key sets. A pair is the identity, so the report key becomes a
label, the collision suffix stops being part of matching, and a change is addressed by
its pair rather than by a key and a node.

Everything a specification knows now sits behind `DiffSpec` in `diff/specs/`, so the
comparison itself names no node type. Rules gain lint's nested visitor keys:
`SchemaProperties: { Schema }` replaces a parent check in the rule body. The usage index
becomes one fixed-point pass over the reference graph instead of a class that recursed
per change.

The report formats move to the CLI, which is their only consumer, and the semver helper
gives way to the `semver` package already in its dependencies.
…oded refs, and settle directions in linear time
… children by a segment alone, and derive diff families from the rule registry
…gs, add the next-version format, and align the report formats with the rest of the CLI
…s formats, specifications, configuration and plugins
…writeOnly, whole-map removals and untyped schemas like a client would, and merge overlapping rules
@vadyvas
vadyvas force-pushed the feat/diff-command branch from b741b3c to 5b7ade9 Compare October 7, 2026 17:41
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

📝 Recheck Summary

⚠️ Warnings found • Scanned 6 changed file(s)

  • 0 error(s)
  • 30 warning(s)
  • 17 info

📋 Findings by rule

🟡 technical-english/sentence-length: 0 errors, 11 warnings, 0 info
🔵 technical-english/passive-voice: 0 errors, 0 warnings, 17 info
🟡 no-gerund-headings: 0 errors, 1 warnings, 0 info
🟡 semantic-line-breaks: 0 errors, 17 warnings, 0 info
🟡 plain-language/excess-intensifiers: 0 errors, 1 warnings, 0 info

Errors fail the full-tree check. Warnings and info are a worklist and do not block the PR.

📖 Readability of changed files

Automated Readability Index (ARI): a U.S. grade level. Lower is easier to read.

File Base This PR Change
docs/@v2/commands/diff.md — 4.8 new file
docs/@v2/commands/index.md 10.6 10.6 ±0
docs/@v2/configuration/index.md 10.8 10.7 🟢 -0.1 easier
docs/@v2/configuration/reference/diff.md — 6.8 new file
docs/@v2/rules.md 10.8 11.2 🔴 +0.4 harder
docs/@v2/rules/diff-rules.md — 3.4 new file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants