🐛 needflow: fix the graphviz label escaper, and stop bad configuration ending the build - #1780
Merged
Merged
Conversation
added 4 commits
August 20, 2026 00:07
A need title is escaped for the graphviz HTML-like label and then wrapped, so the wrapper counted the characters of an entity and could break inside one: a title holding a quote wrapped to `&quo<br/>t;`. That is not merely invalid markup -- graphviz refuses the label outright, so the build reports `not well-formed (invalid token)` and produces no diagram at all. The wrap width was also counting escaped characters rather than what the reader sees. Wrapping now happens first and each piece is escaped after, which fixes both.
`:debug:` emitted raw HTML on plantuml and a literal block on graphviz, so the same option gave the diagram source line numbers and the theme's code styling on one engine only. Both now emit a literal block. Pygments has no PlantUML lexer, so that source is shown unhighlighted rather than wrongly highlighted. This changes the rendered page rather than fixing a fault, so it is recorded as an improvement, and the one existing assertion that depended on the old markup -- a plantuml legend matched at the start of a line -- is rewritten to match the way the graphviz assertions already do, i.e. without depending on the line numbering. `:class:` is pinned alongside it. The plantuml engine never sets it on the figure: it arrives there only because docutils copies the classes of a replaced node onto the first node replacing it, which is the figure. Graphviz writes it on the image instead. Both honour the option, in different places, and the test now says so.
Three ways in which the needflow directive turned bad configuration into a failed build, or into a diagram styled by the one before it: - A `needs_graphviz_styles` element type holding something other than a mapping of attributes travelled unchecked into the emitter, where `'str' object has no attribute 'items'` ended the build with a traceback instead of a message. (An *entry* that is not a mapping was already reported, by the broad `except` around the merge; only the element type reached the emitter.) It is now reported where it is read, and the diagram is drawn without it. - `needs_flow_engine` was checked with a bare `assert`, which ends the build with a traceback rather than a message -- and which `python -O` strips altogether, leaving the unknown name to fail somewhere further downstream. An unknown value is now reported once for the project, and the default engine draws the diagram. The `:engine:` option cannot reach this branch, because docutils validates it as it is parsed. - Merging several named styles took the first style's attributes by reference and then updated that same dictionary with the second style's, rewriting the configured -- and built-in -- styles in place. Every later diagram naming the first style inherited the second one's attributes, for the rest of the build; the built-in styles stayed rewritten for the life of the process. The attributes are now copied before they are merged.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1780 +/- ##
==========================================
+ Coverage 86.87% 90.64% +3.76%
==========================================
Files 56 76 +20
Lines 6532 11340 +4808
==========================================
+ Hits 5675 10279 +4604
- Misses 857 1061 +204
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ubmarco
approved these changes
Aug 20, 2026
This was referenced Aug 20, 2026
chrisjsewell
added a commit
that referenced
this pull request
Aug 20, 2026
Third slice of the #1770 split (after #1780 and #1781): the presentation options. It gives `needflow` an engine-neutral way to say which way the graph flows, what the edges are labelled with, and what the legend contains — additive only, **zero deprecations**: two existing options are widened rather than replaced, so the spellings in existing documents keep working and keep producing the same bytes. ## Options - **`:direction:`** (new) — `down` (default) / `up` / `right` / `left`, plus the `TB`/`TD`/`BT`/`LR`/`RL` two-letter forms Graphviz and Mermaid users already know, with a `needs_flow_direction` project default. An explicit option beats an engine-config(`:config:`)-derived direction on both engines, and disagreement warns. PlantUML has no bottom-to-top or right-to-left primitive (probed: both are syntax errors), so those degrade to their axis mate with a single warning per project. - **`:show_link_names:` widened** — the flag now takes an optional value `none` / `outgoing` / `incoming` / `type`, and written bare still means exactly what it always meant (`outgoing`), byte-for-byte. `needs_flow_show_links` likewise accepts a string as well as a boolean (`True` ≡ `outgoing`, `False` ≡ `none`). The old OR of flag and config becomes a precedence — only an unset option consults the config — which fixes the interaction that made "project default on, this one diagram off" impossible to express. - **`:show_legend:` widened** — it now takes the *name* of a legend defined in `needs_flow_legends`, or nothing. Deliberately key-only: an inline value set would collide with user-chosen names and need a precedence rule. Written bare it renders today's in-diagram legend, byte-for-byte. ## Config `needs_flow_legends = {name: {...}}` defines legends — `parts`, an **ordered list** of the sections to draw (`types`, `links`; a list only, and order is rendered as listed), and `placement` (`internal` / `external`), a preference: unset takes the engine's own default placement, which is internal for both engines here. A legend that includes `links` renders as a table beside the diagram, since neither engine can draw link rows inside it. `needs_flow_show_legend` names the legend a diagram gets when it asks for one without naming its own; it selects *which*, never *whether* — presence stays per-directive, so there is no off-switch value for a legend name to collide with. An undefined key resolves as a **chain** — the directive's key, else the project's, else the engine default — warning at each undefined step and handing on, because an unusable value is treated as unset everywhere in this vocabulary. The two steps warn at different tiers: a bad directive key per directive (`needs.needflow`), a bad `needs_flow_show_legend` once per project (`needs.config`, no directive location — it is a `conf.py` mistake). All four new configs are validated as they are read, so a project that misconfigures one and happens to have no needflow is still told; the checks warn and fall back, never crash (including non-string values, `parts: 5`, and `parts: "links"`). Enumerated config values are matched the way the option parsers already match theirs — case-insensitively, ignoring surrounding whitespace — and `needs_flow_engine`'s membership check from #1780 gets the same treatment. ## Nothing moves for existing projects — with one named edge Verified by cross-commit diff of the generated diagram source on both engines (a five-needflow probe using only pre-existing spellings, and a ten-shape matrix over every reachable `show_link_names` shape), and independently reproduced in review: projects using only existing spellings produce byte-identical output. No test fixture changed: `git diff master -- tests/doc_test/` is empty, `.rst` and `conf.py` alike. The five conformance cases merged in #1781 — including the bare-`:show_legend:` and bare-`:show_link_names:` parity targets — pass **unregenerated**, with unchanged checksums. The one edge, and why the changelog has a Breaking section: `needs_flow_show_links` set to a **string** was never a supported spelling (the config was declared `bool`, and Sphinx already warned about the type) but drew labels via truthiness. It is now read as a value: an unrecognised string warns and draws no labels, and `'none'` — the sharpest case — now silently means what its author meant instead of the opposite. Non-string truthiness (`1`, `0`) is deliberately preserved. ## Environment version `ENV_DATA_VERSION` goes 6 → 7: the directive persists the resolved options in doctrees, and an older reader over a newer doctree fails on the missing keys (reproduced by rebuilding across the change and watching it raise, then fixed by the bump). No other open PR claims 7. ## Conformance corpus `corpus_version` 1 → 2, 5 → 23 cases: direction (one per value, including the two PlantUML degradations and the option-vs-engine-config conflict), the four link-label values plus config-driven and option-beats-config cases, and six legend cases (explicit keys, order pinning, the chain). Fifteen case files are byte-identical to the reviewed umbrella branch and pass here unregenerated — a cross-check of this carve rather than a snapshot of it. The degradation mapping table gains its first three live rows (tier + subtype + pattern), regeneration now records degradation entries, and the per-engine `expect.<engine>.legend` key is exercised for the first time by the external-placement cases. Regeneration is idempotent (verified over two full runs). ## Tests and review Every behaviour was recorded failing before its implementation landed, and seven targeted mutations each turn a test red — one of them (an option-beating-`:config:` assertion satisfiable by a `rankdir` emitted in the wrong place) was caught by the byte-exact corpus case and now has a dedicated ordering fence. The adversarial review reproduced the byte-preservation, the matrix, the tier split, and the ENV crash independently, and found one real defect — a non-string `needs_flow_show_legend` crashed the build — which is fixed at both affected sites with red-first tests for both paths. Full suite: 1637 passed; the 47 remaining failures/errors are pre-existing environmental ones, byte-identical to master's set. Docs build warning-count and warning-set identical to master. ## Follow-ups (later slices of the #1770 split) - Slice 4: link and type styling (`needs_links[].line/part_line/color/part_color/arrow`, `needs_types[].shape`) with the first deprecations. - Slice 5: `:styles:` + `needs_flow_styles`, `:engine_config:`, the remaining deprecations, and moving the engine membership check into `validate_flow_config` (review measured today's #1780 behaviour reporting a `conf.py` engine mistake against a directive location, and staying silent when no needflow exists — inherited there deliberately).
This was referenced Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Slice 1 of the #1770 split (see the plan there): the pure fixes, carved out so they can be
reviewed and merged ahead of any vocabulary discussion.
Five independent needflow defects, each with a regression test that fails on
master, plusone deliberate output change that is called out separately below. No new directive option,
no new configuration key, no deprecation, and no stored-data version change.
Fixes
A wrapped graphviz label could break an HTML entity in two. A need title is wrapped
to the label width and escaped for graphviz's HTML-like labels, but the escaping ran
first, so the wrapper counted the characters of an entity and could break inside one. A
title containing a quote wrapped to
&quo<br/>t;, and graphviz then refuses the labeloutright: the build reports
not well-formed (invalid token)and draws no diagram.Wrapping now happens first and each piece is escaped after, which also makes the wrap
width count the characters the reader sees instead of the ones the escaper wrote.
A
needs_graphviz_styleselement type holding something other than a mapping ofattributes ended the build. The value travelled unchecked into the emitter, where
'str' object has no attribute 'items'surfaced as anExtensionErrortraceback fromthe
doctree-resolvedhandler, naming no configuration key and no file. It is nowreported where it is read, and the diagram is drawn without the offending style. (An
entry that is not a mapping was already reported; only an element type got through.)
An unknown
needs_flow_enginevalue failed a bareassert. That ends the build witha traceback rather than a message, and
python -Ostrips it entirely, leaving theunknown name to fail somewhere further downstream instead. It is now a
needs.configwarning, said once for the project, and the default engine draws the diagram. The
:engine:option cannot reach this path, because docutils validates it as it is parsed.Naming several graphviz
:config:styles leaked the merge into later diagrams. Themerge took the first style's attribute mapping by reference and then
update()d thatsame dictionary with the second style's, so the configured — and the built-in — styles
were rewritten in place. Every later diagram naming the first style inherited the second
one's attributes for the rest of the build, and in the built-in case for the life of the
process, so a page rendered differently depending on which diagrams came before it. The
attributes are now copied before they are merged.
Deliberate output change (not a fix)
:debug:now renders the diagram source as a code block under the plantuml enginetoo. It emitted raw HTML — an unnumbered, unstyled
<pre>— where graphviz emitted aliteral block, so the same option gave the source line numbers and the theme's code
styling on one engine only. Both now emit a literal block; the plantuml source is shown
unhighlighted, because Pygments has no PlantUML lexer. A project that styles or scrapes
the debug block will see the new markup, which is why this is listed as an improvement
rather than a fix. One existing test assertion that matched the plantuml legend at the
start of a line is rewritten not to depend on the line numbering, exactly as the
graphviz half of the same test already does.
Also pinned, not changed:
:class:reaches the plantuml output only because docutilscopies a replaced node's classes onto the first node replacing it, and graphviz writes the
class on the image rather than the figure. Both honour the option, in different places; the
new test records that so the placement cannot drift unnoticed.
Review notes
:debug:block, the threeconfiguration fixes (one code region), and the changelog.
sphinx_needs/directives/needflow/— 70 changed lines of source acrossthree files, the rest is tests, changelog and docs.
assert warnings == ""fences intests/test_needflow.pyareuntouched and still green: nothing here warns on a default path, and the two new warnings
only fire on input that used to end the build.