Skip to content

engine: test_render_arc_svg_byte_identical fails on main from ~1 ULP float-formatting drift; byte-identical SVG golden test is fragile #639

Description

@bpowers

Summary

The unit test simlin_engine::diagram::connector::tests::test_render_arc_svg_byte_identical (src/simlin-engine/src/diagram/connector.rs:712) is failing on main (HEAD 0ce193d1). This makes cargo test -p simlin-engine red and blocks the pre-commit hook for everyone.

Reproduce:

cargo test -p simlin-engine --lib diagram::connector::tests::test_render_arc_svg_byte_identical

Nature of the failure

The test asserts byte-for-byte equality between the live render_connector SVG output and a hardcoded expected string. The live output now differs from the golden string by ~1 ULP of f64 formatting in several coordinates. Actual (left) vs expected (right):

  • arc radius: 273.2050807568877 vs 273.20508075688764
  • arrowhead coords: ...192.27852897536675 vs ...192.27852897536678
  • rotate angle: rotate(58.111862977287586,...) vs rotate(58.1118629772876,...)

The differences are purely last-digit floating-point formatting; the geometry is identical to within rounding.

Root cause

The 0ce193d1 Revert "engine: small layout tweaks" commit was the prime suspect, but it is not the cause of this test failure. Investigation:

  • Neither the tweak (60ee4ceb) nor the revert (0ce193d1) touched src/simlin-engine/src/diagram/connector.rs. The revert only touched layout/ (config.rs, metrics.rs, mod.rs, declutter.rs, layout_selection_tests.rs), examples/layout_eval*, and a CLAUDE.md.
  • The connector arc-rendering source AND the golden string are byte-identical between 930bb5f8 (where they were last committed together) and HEAD: git diff 930bb5f8 HEAD -- src/simlin-engine/src/diagram/connector.rs is empty, as is the diff across src/simlin-engine/src/diagram/, float.rs, and common.rs.

So this is not a code/golden desync left behind by an incomplete revert. The same source computes a ~1-ULP-different f64 than the environment in which the golden string was originally captured. In other words, the golden string encodes a platform/compiler-specific last-ULP result, and the build environment producing the failure rounds the other way. (Toolchain pinned at rustc 1.95.0.)

Why it matters

  • Correctness gate is red on main. cargo test -p simlin-engine fails, which blocks the pre-commit hook and CI for all contributors until resolved.
  • Recurring-flake risk. Even once this specific mismatch is reconciled, byte-identical SVG golden-string tests are inherently fragile to last-ULP float-formatting differences across platforms, compilers, optimization levels, and f64-contraction settings. This class of test is likely to break again the next time anyone builds on a slightly different host. There are 2 such golden-string assert_eq!(svg, expected) blocks in connector.rs.

Component(s) affected

  • src/simlin-engine/src/diagram/connector.rs (the failing test and a second golden-string assertion)
  • The intended (per the test's doc comment) cross-language parity contract that Rust SVG output equals the TypeScript renderer's SVG output, byte for byte.

Possible approaches

Primary (unblock main):

  1. Re-capture the golden string from the current build so the test passes again. This is the minimal fix but leaves the fragility in place (and a re-captured string is only correct for whatever host captured it).

Secondary (durable fix; the real ask):
2. Replace the byte-identical string comparison with a tolerance-based geometry comparison: parse the path/transform numbers out of both SVGs and assert each coordinate matches within an epsilon (e.g. 1e-9 absolute, or ULP-based via the existing crate::float::approx_eq), while still asserting the structural/string scaffolding (element nesting, class names, command letters) exactly.
3. Alternatively, round coordinates to a fixed number of decimal places at SVG-emit time (e.g. via a shared formatting helper), which both stabilizes the golden test and produces stabler diffs/output for consumers. This also keeps the Rust-vs-TS parity contract achievable across platforms, since both sides would round identically.

Approach 2 or 3 addresses the root fragility; approach 1 alone will likely just defer the next break.

Discovery context

Found incidentally while committing an unrelated docs-only design plan on main (the pre-commit hook tripped on this test). The failure is pre-existing on main and is not caused by that docs change.

Activity

  1. bpowers commented on Jun 8, 2026

    @bpowers
    OwnerAuthor

    Status check while triaging open issues (2026-06-07): test_render_arc_svg_byte_identical currently PASSES at HEAD on this macOS/rustc dev machine, and connector.rs hasn't changed -- consistent with the analysis that the failure is environmental (last-ULP float formatting differing across hosts/toolchains). Leaving this open for the durable fix (tolerance-based geometry comparison or fixed-precision coordinate formatting), since the byte-identical golden strings remain a recurring-flake hazard on any host that rounds the other way.

  2. added
    engineIssues with the rust-based simulation engine
    ciCI, build pipeline, test hygiene
    on Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ciCI, build pipeline, test hygieneengineIssues with the rust-based simulation engine

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions