Skip to content

🐛 needflow: fix the graphviz label escaper, and stop bad configuration ending the build - #1780

Merged
chrisjsewell merged 4 commits into
masterfrom
claude/needflow-emitter-fixes
Aug 20, 2026
Merged

🐛 needflow: fix the graphviz label escaper, and stop bad configuration ending the build#1780
chrisjsewell merged 4 commits into
masterfrom
claude/needflow-emitter-fixes

Conversation

@chrisjsewell

@chrisjsewell chrisjsewell commented Aug 20, 2026

Copy link
Copy Markdown
Member

Slice 1 of the #1770 split (see the plan there): the pure fixes, carved out so they can be
reviewed and merged ahead of any vocabulary discussion.

Five independent needflow defects, each with a regression test that fails on master, plus
one deliberate output change that is called out separately below. No new directive option,
no new configuration key, no deprecation, and no stored-data version change.

Fixes

  • A wrapped graphviz label could break an HTML entity in two. A need title is wrapped
    to the label width and escaped for graphviz's HTML-like labels, but the escaping ran
    first, so the wrapper counted the characters of an entity and could break inside one. A
    title containing a quote wrapped to &quo<br/>t;, and graphviz then refuses the label
    outright: the build reports not well-formed (invalid token) and draws no diagram.
    Wrapping now happens first and each piece is escaped after, which also makes the wrap
    width count the characters the reader sees instead of the ones the escaper wrote.

  • A needs_graphviz_styles element type holding something other than a mapping of
    attributes ended the build.
    The value travelled unchecked into the emitter, where
    'str' object has no attribute 'items' surfaced as an ExtensionError traceback from
    the doctree-resolved handler, naming no configuration key and no file. It is now
    reported where it is read, and the diagram is drawn without the offending style. (An
    entry that is not a mapping was already reported; only an element type got through.)

  • An unknown needs_flow_engine value failed a bare assert. That ends the build with
    a traceback rather than a message, and python -O strips it entirely, leaving the
    unknown name to fail somewhere further downstream instead. It is now a needs.config
    warning, said once for the project, and the default engine draws the diagram. The
    :engine: option cannot reach this path, because docutils validates it as it is parsed.

  • Naming several graphviz :config: styles leaked the merge into later diagrams. The
    merge took the first style's attribute mapping by reference and then update()d that
    same dictionary with the second style's, so the configured — and the built-in — styles
    were rewritten in place. Every later diagram naming the first style inherited the second
    one's attributes for the rest of the build, and in the built-in case for the life of the
    process, so a page rendered differently depending on which diagrams came before it. The
    attributes are now copied before they are merged.

Deliberate output change (not a fix)

  • :debug: now renders the diagram source as a code block under the plantuml engine
    too.
    It emitted raw HTML — an unnumbered, unstyled <pre> — where graphviz emitted a
    literal block, so the same option gave the source line numbers and the theme's code
    styling on one engine only. Both now emit a literal block; the plantuml source is shown
    unhighlighted, because Pygments has no PlantUML lexer. A project that styles or scrapes
    the debug block will see the new markup, which is why this is listed as an improvement
    rather than a fix. One existing test assertion that matched the plantuml legend at the
    start of a line is rewritten not to depend on the line numbering, exactly as the
    graphviz half of the same test already does.

Also pinned, not changed: :class: reaches the plantuml output only because docutils
copies a replaced node's classes onto the first node replacing it, and graphviz writes the
class on the image rather than the figure. Both honour the option, in different places; the
new test records that so the placement cannot drift unnoticed.

Review notes

  • Four commits, each self-contained: the label escaper, the :debug: block, the three
    configuration fixes (one code region), and the changelog.
  • Everything is in sphinx_needs/directives/needflow/ — 70 changed lines of source across
    three files, the rest is tests, changelog and docs.
  • The two whole-project assert warnings == "" fences in tests/test_needflow.py are
    untouched and still green: nothing here warns on a default path, and the two new warnings
    only fire on input that used to end the build.

Chris Sewell added 4 commits August 20, 2026 00:07
A need title is escaped for the graphviz HTML-like label and then wrapped, so the
wrapper counted the characters of an entity and could break inside one: a title
holding a quote wrapped to `&quo<br/>t;`. That is not merely invalid markup --
graphviz refuses the label outright, so the build reports `not well-formed
(invalid token)` and produces no diagram at all. The wrap width was also counting
escaped characters rather than what the reader sees.

Wrapping now happens first and each piece is escaped after, which fixes both.
`:debug:` emitted raw HTML on plantuml and a literal block on graphviz, so the same
option gave the diagram source line numbers and the theme's code styling on one
engine only. Both now emit a literal block. Pygments has no PlantUML lexer, so that
source is shown unhighlighted rather than wrongly highlighted.

This changes the rendered page rather than fixing a fault, so it is recorded as an
improvement, and the one existing assertion that depended on the old markup -- a
plantuml legend matched at the start of a line -- is rewritten to match the way the
graphviz assertions already do, i.e. without depending on the line numbering.

`:class:` is pinned alongside it. The plantuml engine never sets it on the figure:
it arrives there only because docutils copies the classes of a replaced node onto
the first node replacing it, which is the figure. Graphviz writes it on the image
instead. Both honour the option, in different places, and the test now says so.
Three ways in which the needflow directive turned bad configuration into a failed
build, or into a diagram styled by the one before it:

- A `needs_graphviz_styles` element type holding something other than a mapping of
  attributes travelled unchecked into the emitter, where `'str' object has no
  attribute 'items'` ended the build with a traceback instead of a message. (An
  *entry* that is not a mapping was already reported, by the broad `except` around
  the merge; only the element type reached the emitter.) It is now reported where it
  is read, and the diagram is drawn without it.

- `needs_flow_engine` was checked with a bare `assert`, which ends the build with a
  traceback rather than a message -- and which `python -O` strips altogether,
  leaving the unknown name to fail somewhere further downstream. An unknown value is
  now reported once for the project, and the default engine draws the diagram. The
  `:engine:` option cannot reach this branch, because docutils validates it as it is
  parsed.

- Merging several named styles took the first style's attributes by reference and
  then updated that same dictionary with the second style's, rewriting the
  configured -- and built-in -- styles in place. Every later diagram naming the first
  style inherited the second one's attributes, for the rest of the build; the
  built-in styles stayed rewritten for the life of the process. The attributes are
  now copied before they are merged.
@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.64%. Comparing base (4e10030) to head (b6af7e2).
⚠️ Report is 328 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1780      +/-   ##
==========================================
+ Coverage   86.87%   90.64%   +3.76%     
==========================================
  Files          56       76      +20     
  Lines        6532    11340    +4808     
==========================================
+ Hits         5675    10279    +4604     
- Misses        857     1061     +204     
Flag Coverage Δ
pytests 90.64% <100.00%> (+3.76%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@chrisjsewell
chrisjsewell merged commit 53c7432 into master Aug 20, 2026
25 checks passed
@chrisjsewell
chrisjsewell deleted the claude/needflow-emitter-fixes branch August 20, 2026 06:50
chrisjsewell added a commit that referenced this pull request Aug 20, 2026
Third slice of the #1770 split (after #1780 and #1781): the presentation
options.
It gives `needflow` an engine-neutral way to say which way the graph
flows, what the edges are
labelled with, and what the legend contains — additive only, **zero
deprecations**: two existing
options are widened rather than replaced, so the spellings in existing
documents keep working
and keep producing the same bytes.

## Options

- **`:direction:`** (new) — `down` (default) / `up` / `right` / `left`,
plus the `TB`/`TD`/`BT`/`LR`/`RL`
two-letter forms Graphviz and Mermaid users already know, with a
`needs_flow_direction` project
default. An explicit option beats an engine-config(`:config:`)-derived
direction on both engines,
and disagreement warns. PlantUML has no bottom-to-top or right-to-left
primitive (probed: both are
syntax errors), so those degrade to their axis mate with a single
warning per project.
- **`:show_link_names:` widened** — the flag now takes an optional value
`none` / `outgoing` /
`incoming` / `type`, and written bare still means exactly what it always
meant (`outgoing`),
byte-for-byte. `needs_flow_show_links` likewise accepts a string as well
as a boolean
(`True` ≡ `outgoing`, `False` ≡ `none`). The old OR of flag and config
becomes a precedence —
only an unset option consults the config — which fixes the interaction
that made
  "project default on, this one diagram off" impossible to express.
- **`:show_legend:` widened** — it now takes the *name* of a legend
defined in
`needs_flow_legends`, or nothing. Deliberately key-only: an inline value
set would collide with
user-chosen names and need a precedence rule. Written bare it renders
today's in-diagram legend,
  byte-for-byte.

## Config

`needs_flow_legends = {name: {...}}` defines legends — `parts`, an
**ordered list** of the
sections to draw (`types`, `links`; a list only, and order is rendered
as listed), and `placement`
(`internal` / `external`), a preference: unset takes the engine's own
default placement, which is
internal for both engines here. A legend that includes `links` renders
as a table beside the
diagram, since neither engine can draw link rows inside it.

`needs_flow_show_legend` names the legend a diagram gets when it asks
for one without naming its
own; it selects *which*, never *whether* — presence stays per-directive,
so there is no off-switch
value for a legend name to collide with.

An undefined key resolves as a **chain** — the directive's key, else the
project's, else the
engine default — warning at each undefined step and handing on, because
an unusable value is
treated as unset everywhere in this vocabulary. The two steps warn at
different tiers: a bad
directive key per directive (`needs.needflow`), a bad
`needs_flow_show_legend` once per project
(`needs.config`, no directive location — it is a `conf.py` mistake). All
four new configs are
validated as they are read, so a project that misconfigures one and
happens to have no needflow
is still told; the checks warn and fall back, never crash (including
non-string values, `parts: 5`,
and `parts: "links"`). Enumerated config values are matched the way the
option parsers already
match theirs — case-insensitively, ignoring surrounding whitespace — and
`needs_flow_engine`'s
membership check from #1780 gets the same treatment.

## Nothing moves for existing projects — with one named edge

Verified by cross-commit diff of the generated diagram source on both
engines (a five-needflow
probe using only pre-existing spellings, and a ten-shape matrix over
every reachable
`show_link_names` shape), and independently reproduced in review:
projects using only existing
spellings produce byte-identical output. No test fixture changed:
`git diff master -- tests/doc_test/` is empty, `.rst` and `conf.py`
alike. The five conformance
cases merged in #1781 — including the bare-`:show_legend:` and
bare-`:show_link_names:` parity
targets — pass **unregenerated**, with unchanged checksums.

The one edge, and why the changelog has a Breaking section:
`needs_flow_show_links` set to a
**string** was never a supported spelling (the config was declared
`bool`, and Sphinx already
warned about the type) but drew labels via truthiness. It is now read as
a value: an unrecognised
string warns and draws no labels, and `'none'` — the sharpest case — now
silently means what its
author meant instead of the opposite. Non-string truthiness (`1`, `0`)
is deliberately preserved.

## Environment version

`ENV_DATA_VERSION` goes 6 → 7: the directive persists the resolved
options in doctrees, and an
older reader over a newer doctree fails on the missing keys (reproduced
by rebuilding across the
change and watching it raise, then fixed by the bump). No other open PR
claims 7.

## Conformance corpus

`corpus_version` 1 → 2, 5 → 23 cases: direction (one per value,
including the two PlantUML
degradations and the option-vs-engine-config conflict), the four
link-label values plus
config-driven and option-beats-config cases, and six legend cases
(explicit keys, order pinning,
the chain). Fifteen case files are byte-identical to the reviewed
umbrella branch and pass here
unregenerated — a cross-check of this carve rather than a snapshot of
it. The degradation mapping
table gains its first three live rows (tier + subtype + pattern),
regeneration now records
degradation entries, and the per-engine `expect.<engine>.legend` key is
exercised for the first
time by the external-placement cases. Regeneration is idempotent
(verified over two full runs).

## Tests and review

Every behaviour was recorded failing before its implementation landed,
and seven targeted
mutations each turn a test red — one of them (an
option-beating-`:config:` assertion satisfiable
by a `rankdir` emitted in the wrong place) was caught by the byte-exact
corpus case and now has a
dedicated ordering fence. The adversarial review reproduced the
byte-preservation, the matrix, the
tier split, and the ENV crash independently, and found one real defect —
a non-string
`needs_flow_show_legend` crashed the build — which is fixed at both
affected sites with red-first
tests for both paths. Full suite: 1637 passed; the 47 remaining
failures/errors are pre-existing
environmental ones, byte-identical to master's set. Docs build
warning-count and warning-set
identical to master.

## Follow-ups (later slices of the #1770 split)

- Slice 4: link and type styling
(`needs_links[].line/part_line/color/part_color/arrow`,
  `needs_types[].shape`) with the first deprecations.
- Slice 5: `:styles:` + `needs_flow_styles`, `:engine_config:`, the
remaining deprecations, and
moving the engine membership check into `validate_flow_config` (review
measured today's #1780
behaviour reporting a `conf.py` engine mistake against a directive
location, and staying silent
  when no needflow exists — inherited there deliberately).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants