Skip to content

test: validate saved markup against Gutenberg's own deprecation chain - #74

Merged
dknauss merged 2 commits into
mainfrom
test/deprecation-fixtures
Aug 3, 2026
Merged

test: validate saved markup against Gutenberg's own deprecation chain#74
dknauss merged 2 commits into
mainfrom
test/deprecation-fixtures

Conversation

@dknauss

@dknauss dknauss commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Closes the largest untested risk in the plugin.

The gap

Borges is a static-save block: its markup is baked into post_content, and the editor re-validates that markup against save() every time a post is opened. When save() changes, previously saved posts keep working only because an entry in src/deprecated.js still reproduces the old markup exactly. Break one and every affected post shows "Attempt Block Recovery" — the worst regression this plugin can ship, and one that surfaces in readers' content rather than in CI.

There are 6 deprecations. src/save.test.js and src/deprecated.test.js exercise those functions directly, but nothing round-tripped real saved markup through Gutenberg's actual validator — the thing that decides whether a post breaks.

What this adds

The test registers the block against the real @wordpress/blocks registry (already a dependency — no new packages) with its shipping save plus the full deprecated chain, then parses 7 committed fixtures, one per shipped save() shape, and asserts each still validates.

It asserts block.name before isValid. That ordering matters: an unregistered block parses as core/missing and reports valid vacuously, so checking validity alone would be a false green.

Why the fixtures are frozen on disk

Minting them at test time from the same deprecated array under test would be circular — editing a deprecation would rewrite its own expectation and the test would still pass, which is precisely the regression it exists to catch. Regeneration is deliberate and env-gated:

BORGES_WRITE_DEPRECATION_FIXTURES=1 npm test -- deprecation-fixtures

When adding a deprecation, the pre-existing fixtures must come back unchanged. A diff in an existing fixture means history was altered, not appended to.

A gap found in review, and closed

The first version of this test contained zero <a href> anchors across all 7 fixtures. linkVisibleUrls splits the citation's formattedText on literal URL substrings — it never reads the CSL URL field, which is where the URL had been placed. The three linkVisibleUrls: true deprecations and the three false ones therefore serialized identically on that dimension, while the test's own doc comment claimed the path was covered.

Fixed by putting a literal URL in formattedText. The split is now real — anchors in v00, v04, v05, v06 (the true entries plus current), none in v01, v02, v03 — and a dedicated test asserts both sides still differ, so dropping the URL later fails loudly rather than quietly shrinking coverage.

Verification

Proven non-vacuous by sabotage in two independent dimensions, each failing exactly the affected fixture and no others:

Sabotage Result
deprecation 1: headingTag h2h4 only deprecation 1 (index 5) fails
deprecation 4: linkVisibleUrls truefalse only deprecation 4 (index 2) fails

Both restored, git diff clean, suite green. The regenerated fixtures were byte-diffed against reconstructed pre-fix versions: every file changed by exactly two additive insertions (the JSON attribute and the rendered text/anchor), with no incidental reordering or drift.

Full suite: 42 suites / 669 tests (was 41/660). Lint and verify-metrics clean.

One config change worth noting

Jest needed transformIgnorePatterns for uuid. The repo pins uuid: ^14 in overrides, which is ESM-only with no CJS build, and @wordpress/blocks requires it transitively. The pattern only adds a transformable path and preserves Jest's default .pnp. entry; suite runtime is unchanged (~6s). The alternative — unpinning uuid — would undo a deliberate pin.

Risk

Low. Test-only plus one Jest config line. No plugin runtime code is touched, and nothing ships in the distributed package.

🤖 Generated with Claude Code

Borges bakes its markup into post content, and the editor re-validates it
against save() every time a post is opened. Old posts keep working only
because an entry in deprecated reproduces their markup exactly; break one
and every affected post shows "Attempt Block Recovery". save.test.js and
deprecated.test.js exercise those functions directly, so nothing caught a
break in the round trip through Gutenberg's real validator.

Register the block with the shipping save plus the full deprecated chain
against the real @wordpress/blocks registry, then parse a committed
fixture per shipped save() shape and assert each still validates. Assert
the block name before validity: an unregistered block parses as
core/missing and reports valid vacuously.

Fixtures are frozen on disk rather than generated per run. Minting them
from the same deprecated array under test would be circular -- editing a
deprecation would rewrite its own expectation and still pass, which is
the regression this exists to catch. Regeneration is deliberate, behind
BORGES_WRITE_DEPRECATION_FIXTURES.

The fixture citation carries a literal URL in formattedText because
linkVisibleUrls splits the rendered text on URL substrings and never
reads the CSL URL field; without it the linkVisibleUrls true and false
deprecations serialize identically and that path goes uncovered. A
dedicated test asserts both sides of the split still differ.

Jest needs transformIgnorePatterns for uuid: the repo pins uuid ^14,
which is ESM-only with no CJS build, and @wordpress/blocks requires it.

Verified by sabotage in both dimensions -- altering a deprecation's
heading tag, and flipping its linkVisibleUrls -- each failing exactly the
affected fixture and no others.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dknauss

dknauss commented Aug 3, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.04%. Comparing base (1140ad7) to head (d70911c).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #74      +/-   ##
==========================================
+ Coverage   81.01%   81.04%   +0.03%     
==========================================
  Files          43       43              
  Lines        3113     3113              
  Branches      522      522              
==========================================
+ Hits         2522     2523       +1     
+ Misses        185      184       -1     
  Partials      406      406              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4058d16da4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/deprecation-fixtures.test.js Outdated
With outputCiteExport off, the current save() and deprecated[0] emit the
same markup -- v00-current.html and v06-deprecated.html were byte
identical. deprecated[0] exists only to freeze the shape from before
per-entry cite/export panels, so breaking or deleting it left its fixture
validating against the current save and the newest compatibility boundary
went uncovered. Turn the panels on so the two shapes diverge.

Add a structural guard rather than only fixing the instance: any two
fixtures with identical markup now fail the suite. Two identical fixtures
mean their save() shapes do not differ under the fixture attributes, so
one of them proves nothing. This catches the class, including whichever
attribute defaults happen to collapse a future pair.

Verified: sabotaging deprecated[0]'s entryTag now fails exactly its own
fixture, which was impossible before. Reported by Codex as P2 on #74.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@dknauss
dknauss merged commit 06df0fb into main Aug 3, 2026
16 checks passed
@dknauss
dknauss deleted the test/deprecation-fixtures branch August 3, 2026 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant