[lexical-code-prism][lexical-code-shiki][lexical-playground] Feature: Allow null Tokenizer.defaultLanguage to preserve markdown ``` round-trip

[mayrang]

Description

Markdown (no info string) imported into a wysiwyg editor with the Prism or Shiki highlighter active permanently rewrote the `CodeNode`'s language to the tokenizer's default ('javascript'), and the next markdown export emittedjavascript instead of preserving the round-trip.

Per the maintainer suggestion on this PR, this widens Tokenizer.defaultLanguage from string to string | null so consumers can opt out of the implicit fallback. When set to null:

• The highlight transform leaves __language as undefined (no setLanguage mutation)
• getIsSyntaxHighlightSupported reports false, so createDOM emits no data-language / data-highlight-language attributes
• $tokenize returns a plain split of the text (TextNode + LineBreakNode + TabNode) instead of highlight tokens, so the code block renders as plain monospace with real line breaks — matching what GitHub / Slack / VS Code do for ``` with no language

PrismTokenizer / ShikiTokenizer keep their existing defaultLanguage: DEFAULT_CODE_LANGUAGE so default consumers see no behavior change. Playground opts in by spreading {...PrismTokenizer, defaultLanguage: null} (and the Shiki equivalent) into the CodeHighlightExtension dependency.

Playground toolbar and CodeActionMenu gained UX support for the unset state — the language dropdown shows (No language) as the label and as a selectable item that calls setLanguage(null) (which still zaps to undefined per existing semantics, but lets users return a code block to the unset state without going through markdown). CodeActionMenu falls back to the syntax-highlight default for canBePrettier / the friendly name so Prettier still works on unset blocks.

Cleanup

getDefaultCodeLanguage gained @deprecated — it is unused internally and DEFAULT_CODE_LANGUAGE can be read directly. DEFAULT_CODE_LANGUAGE is still used internally so left alone.

Backwards compatibility

Tokenizer.defaultLanguage widens from string to string | null. Existing consumers passing a string are unaffected; consumers reading tokenizer.defaultLanguage need a null guard if they handle the opt-out case. The default PrismTokenizer / ShikiTokenizer instances are unchanged, so any consumer that registers CodePrismExtension / CodeShikiExtension without overriding the tokenizer keeps the old behavior.

Closes #7235

Test plan

• pnpm tsc --noEmit -p tsconfig.json clean.
• pnpm flow clean.
• pnpm vitest run --project unit — 2599 passed, 1 skipped.
• npx prettier --check clean / pnpm eslint clean.
• New unit tests (CodePrismNullDefaultLanguage.test.ts, CodeShikiNullDefaultLanguage.test.ts) register the highlight with defaultLanguage: null and confirm getLanguage() stays undefined after the transform runs.
• Manual playground (Prism mode):
  • Markdown (no lang) → wysiwyg → plain monospace (no highlight tokens), toolbar shows `(No language)`; M-toggle back → markdown still.
  • Markdown python → wysiwyg → tokens + toolbar `Python`; M-toggle back → python.
  • /code slash menu → empty code block → toolbar (No language), no highlight (default opted out).
  • Toolbar (No language) item on a highlighted Python block → highlight clears, block becomes plain; markdown export ```.
  • CodeActionMenu Prettier on a (No language) block → format applied, \n rendered as real line breaks (not literal characters).
• Same scenarios in Shiki mode.
• Updated existing e2e snapshots (CodeBlock, CodeActionMenu, Indentation, Markdown, Tab, 1384-insert-nodes) — data-language="javascript" / data-highlight-language="javascript" no longer present when the block was created via markdown ``` shortcut.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
lexical	Ready	Preview, Comment	May 25, 2026 6:46pm
lexical-playground	Ready	Preview, Comment	May 25, 2026 6:46pm

[etrepum]

I wonder if this could be done in a backwards compatible way, perhaps by allowing defaultLanguage to be explicitly null or undefined in the extension configurations?

We should probably go ahead and deprecate getDefaultCodeLanguage while we're in here since it's unused internally and doesn't really have any bearing on how things work. DEFAULT_CODE_LANGUAGE could probably also be deprecated although that is currently used internally.

[mayrang]

Switched to the Tokenizer.defaultLanguage: string | null route — default tokenizers keep their existing string, playground opts in by spreading {...PrismTokenizer, defaultLanguage: null} (and the Shiki equivalent) into CodeHighlightExtension. The earlier 3-state CodeNode.__language change is fully reverted, so no setLanguage / updateFromJSON BC impact.

getDefaultCodeLanguage is now @deprecated. Left DEFAULT_CODE_LANGUAGE as-is since it's still used internally.

[etrepum]

Do we still have e2e tests that explicitly set a language in the markdown blocks to make sure these highlighters are still working correctly with the playground config in those cases?

[etrepum]

I think this can be marked as @internal - it's not really an API and doesn't make sense when it's overridden in the configs

[mayrang]

Switched to @internal.

[etrepum]

I don't think we need to point people anywhere else, maybe just say it's configurable in the extensions

[mayrang]

Trimmed the JSDoc — just says it's configurable through the extensions now.

[etrepum]

We should unify these into @lexical/code-core since we have the code repeated verbatim in each highlighter package

[mayrang]

Moved $plainifyCodeContent to @lexical/code-core and imported from both highlighter packages.

[mayrang]

Yes — CodeBlock.spec.mjs l1350/l1410 type ```diff / ```diff-javascript as the markdown opener and assert the rendered data-language + highlight tokens against the playground config. The new defaultLanguage: null branch only fires when getLanguage() === undefined, so these two cases stay on the existing path.