Removing a block type silently drops stored content from the next save #101

This issue concerns a schema change: a developer removes a block type from a collection's definition, so it is no longer listed in a blocks field's blocks: [...] array. It does not concern everyday editorial block building — adding, reordering, or deleting block instances in the admin editor works correctly and is unaffected.

Removing a block type from a collection's schema does not remove the rows already stored for that type, but reconstruction drops those rows from the document without a warning. Depending on where the removed block sat, the document either becomes unsaveable or loses that content from the next version with no error at any layer.

This is independent of collection-versioning Phases 2–5: it affects current documents read against the current schema, not historical rendering.

Behaviour

restoreBlocksFieldData skips rows whose block type is absent from the live definition, and surviving blocks keep their original array indices. With retired a removed type and live a declared one:

The trailing case is the more serious one: nothing fails, so an ordinary editorial save silently drops content from the current version. Rows from earlier versions remain in storage but are not readable through the current schema.

Read paths

Reads never fail and never warn, and the document is not unloadable. Every consumer that walks the blocks array skips the missing item, so the document opens everywhere — it simply contains less than storage holds:

Which failure you get depends on the serializer in the path, and the three in use here disagree:

• seroval (TanStack Start's transport) preserves the hole — it encodes [,{…}], and 0 in array is still false after a round trip. SSR and hydration both skip the block; nothing throws.
• Keyv with KeyvCacheableMemory (the public cache in apps/webapp/src/lib/cache/cache-manager.ts) compacts the array. Storing length 2 with [0] empty and [1] live returns length 1 with the live block at [0]: surviving blocks renumber, so a cache hit and a cache miss hand the renderer different arrays.
• JSON (JSON.parse(JSON.stringify(...))) turns the hole into null. map does not skip null, so RenderBlocks throws Cannot read properties of null (reading '_type'), and validateDocumentFields reports content[0]: Item must be an object.

So the leading-block case is only loud where JSON is involved. Byline's own transport is not JSON, which is why the record we found sat unnoticed: it rendered, it indexed, it exported to markdown, and it failed only when someone tried to save it.

Read-side degradation is intended

Serving the rest of the document when a block type is no longer declared is wanted behaviour, not the defect. A retired block type must not fail a public page or lock an editor out of the content around it, so a fix must keep reads succeeding. What is missing on the read side is the signal, not the failure: the omission should be diagnosable — logged, and surfaced to an editor at the block's position — rather than fatal.

Scoped that way, the defect is:

• an ordinary save drops the unknown block's content with no signal (trailing or only position);
• flattenFieldSetData throws an unhelpful destructure error on a sparse hole (leading or middle position);
• seroval, Keyv and JSON disagree about what the document contains;
• nothing reports the omission at any layer.

A change that makes reads throw on an unknown block type would turn one retired block into a broken page, and is explicitly not wanted.

Reproduction

Database-free, from apps/webapp:

node --import tsx --input-type=module <<'JS'
import { flattenFieldSetData } from '../../packages/core/src/storage/storage-flatten.ts'
import { restoreFieldSetData } from '../../packages/core/src/storage/storage-restore.ts'
import { validateDocumentFields } from '../../packages/core/src/validation/document-fields.ts'

const retired = { blockType: 'retired', fields: [{ name: 'text', type: 'text' }] }
const live = { blockType: 'live', fields: [{ name: 'text', type: 'text' }] }
const before = [{ name: 'content', type: 'blocks', blocks: [retired, live] }]
const after = [{ name: 'content', type: 'blocks', blocks: [live] }]

for (const types of [['retired', 'live'], ['live', 'retired'], ['retired']]) {
  const rows = flattenFieldSetData(before, { content: types.map(_type => ({ _type, text: _type })) }, 'en')
  const { data, warnings } = restoreFieldSetData(after, rows, 'en')
  const serialized = JSON.parse(JSON.stringify(data))
  let nextRows
  try {
    nextRows = flattenFieldSetData(after, data, 'en').map(row => row.field_path.join('.'))
  } catch (error) { nextRows = `THREW: ${error.message}` }
  console.log({ types, warnings, serialized,
    directIssues: validateDocumentFields(after, data),
    serializedIssues: validateDocumentFields(after, serialized), nextRows })
}
JS

Found on a real record: a pages document holding an uploadTestBlock, orphaned when 38b1cd5 removed that block from the collection. It had been unsaveable ever since, and field validation reported only content[0]: Item must be an object — the position, not the cause.

Where it is

• packages/core/src/storage/storage-restore.ts — the skip, at the field.blocks.find(...) == null branch. The silence is deliberate and commented (// Block type was removed from the schema; silently skip orphaned rows.) while every adjacent branch in that function pushes a warning. A fix revisits a recorded decision rather than repairing an oversight.
• packages/core/src/validation/document-fields.ts — block validation uses value.forEach, which skips sparse slots.
• packages/core/src/storage/storage-flatten.ts — block flattening destructures each item, so a hole throws.
• packages/admin/src/fields/blocks/blocks-field.tsx — renderItem returns null for an undeclared type, so the editor shows nothing at that position.
• packages/admin/src/forms/form-page-chrome.tsx — a document-level restoreWarnings banner exists, but removed block types produce no warnings to show in it.

What a fix needs

1. Surface unknown stored block types with document, field-path, type, and item-identity context. Strict and lenient reads each need defined behaviour; neither should report a complete document after omitting content.
2. Reject sparse or null block items with actionable field issues before flattening.
3. Preserve unknown block identity, order, and values, or refuse the write until an explicit migration or removal resolves them. Compacting the array is not a content-preserving repair.
4. Define behaviour for historical versions as well as current documents.
5. Cover first, middle, last, and only removed blocks; required and optional blocks fields; direct and serialized payloads; revision-guarded updates and restores; both adapters.
6. Keep public reads under a separate policy — no internal schema diagnostics leaking to public visitors.
7. Make the chosen representation behave identically across seroval, Keyv, and JSON. Today those three produce a preserved hole, a compacted array, and a null respectively from the same document, so "what this document contains" depends on which path served it. A fix that only addresses the sparse array in core leaves that divergence in place.

The open design decision is how an unknown block is represented: retained compatibility definitions, an explicit developer migration, or an opaque passthrough. That choice determines whether a save can ever proceed with unknown content present, and is not settled here.

A proposed editor recovery strategy — a placeholder card at the block's original position plus a server-side save safeguard — is written up in specs/2026-09-16-removed-block-reconstruction.md.

Documentation

The limitation is documented in docs/04-collections/08-collection-versioning.md under "Removed block types", and docs/03-architecture/01-document-storage.md now narrows its first property to "collection changes need no database DDL migration", since content migration can still be required.

Related

• Historical config snapshots: collection_versions history table #11 — historical config snapshots (Phase 2); a prerequisite for rendering a removed block from its original definition, but not for the diagnostics or the save safeguard here.
• Collection-versioning Phases 3-5 #31 — collection-versioning Phases 3–5 (deferred).