Markdown region-kind golden fixture coverage (2.1.2)

[leynos]

Summary

Implements roadmap item 2.1.2: every shipped v1 Markdown region kind is now
covered by golden fixtures and structural assertions.

The implementation adds the missing Markdown region surfaces (list_item,
blockquote, frontmatter, image_alt, and link_title), keeps
frontmatter_field reserved through ADR 005/RFC 0001, and validates IR segment
source backing by re-slicing the original source bytes.

Review Follow-up

• Reject duplicate IR node and region IDs before building validator lookup sets.
• Make synthetic segment construction use SyntheticReason rather than a
  free-form string.
• Keep normalized, unmappable inline code spans as decoded_text synthetic
  segments.
• Skip empty link_title regions and cover the negative case in the fixture.
• Sort fixture directory reads for deterministic test ordering.
• Reject unexpected Python bridge keyword arguments.
• Compare the full supported_region_kinds() tuple against RegionKind::ALL.
• Pin the smoke workflow cargo-nextest install.
• Restructure ADR 005 and trim user-facing docs so internal bridge details are
  not advertised as public API.

Validation

• make check-fmt
• make lint
• make typecheck
• make test
• make markdownlint
• make nixie

References

• Lody session: https://lody.ai/leynos/sessions/276ddc47-4455-4a41-8a0f-6ca1de88aa10

[sourcery-ai]

Sorry @leynos, you have reached your weekly rate limit of 2500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

Walkthrough

Update the Markdown IR vocabulary, emission, validation, tests, and Python bridge so region kinds stay stable, emit consistently, and surface through Rust and Python APIs. Align CI, fixture access, and docs with the same contracts.

Changes

Markdown IR rollout

Layer / File(s)	Summary
CI and test workflow .github/workflows/smoke.yml, Makefile, clippy.toml, tests/test_build_spine_units.py, tests/test_makefile_recipes.py, tests/test_maturin_build.py, docs/developers-guide.md, docs/contents.md	Tighten the smoke workflow, Makefile recipes, Clippy thresholds, and the assertions/docs that track lint, typecheck, doc-test, and build-step commands.
Repository fixture access Cargo.toml, .gitignore, crates/stilyagi-test-support/*, crates/stilyagi-extract/*, crates/stilyagi-tree-sitter/src/lib.rs	Switch fixture reads to repository-root cap-std directory handles, export byte and listing helpers, and simplify related test harness error handling and module docs.
Canonical IR vocabularies crates/stilyagi-ir/*, docs/adr-005-*, docs/rfcs/0001-*, docs/stilyagi-design.md, docs/users-guide.md, docs/contents.md, docs/developers-guide.md, tests/fixtures/corpus/markdown/*	Introduce stable RegionKind and SyntheticReason spellings, typed synthetic origins, and matching contracts, docs, and fixtures for the Markdown region vocabulary.
Markdown builder, emission, and validation crates/stilyagi-markdown/src/builder.rs, crates/stilyagi-markdown/src/flatten.rs, crates/stilyagi-markdown/src/lib.rs, crates/stilyagi-markdown/src/region_emission.rs, crates/stilyagi-markdown/src/validation.rs	Move Markdown AST traversal into MarkdownIrBuilder, split region emission into dedicated helpers, centralise source-text cursor handling, and validate hashes, IDs, parents, origins, and spans.
Markdown invariant tests crates/stilyagi-markdown/Cargo.toml, crates/stilyagi-markdown/src/tests.rs, crates/stilyagi-markdown/src/tests/*.rs	Revise shared Markdown test helpers around ExpectedText, must_ok!, and SyntheticReason::ALL, and expand regression/property tests for duplicate IDs, parent links, reconstruction, parser panic messages, and segment origin rules.
Corpus coverage and rollout crates/stilyagi-markdown/src/tests/coverage.rs, tests/fixtures/corpus/markdown/*, docs/execplans/2-1-2-golden-fixture-coverage.md, docs/roadmap.md	Cover the promised Markdown region vocabulary across valid, malformed, CRLF, and recovery fixtures, and record the completed rollout in the execplan and roadmap.
PyO3 bridge and Python adapter crates/stilyagi-pyext/*, python/stilyagi/*, tests/test_package_*.py, crates/stilyagi-pyext/tests/ui/pass/pyo3_pymodule.rs, docs/users-guide.md, docs/developers-guide.md	Export supported_region_kinds, validate positional extraction calls, warn on unknown IR region kinds, and align the Python package, smoke checks, package tests, and docs with the same region vocabulary.

Possibly related issues

• Review architectural abstractions introduced in PR #31 for necessity and reuse #32 — Reuse the same Markdown IR abstractions (MarkdownIrBuilder, region_emission, validation helpers) that this PR introduces and exercises.
• tests/test_maturin_build.py: add docstrings to all remaining private helper functions #28 — Update the same tests/test_maturin_build.py helper docstrings touched by this PR.

Possibly related PRs

• leynos/wildside-engine#65 — Tightens clippy.toml and Makefile-driven lint/test invocation in the same style as this PR.
• leynos/stilyagi#15 — Evolves the Markdown IR envelope and bridge path that this PR extends with stable vocabularies and region emission.
• leynos/stilyagi#22 — Updates the PyO3 UI module and exported bridge surface in the same area as supported_region_kinds.

Suggested reviewers: codescene-delta-analysis

Poem

Let headings wear their proper crown,
And thin containers hold it down.
Cursor, bridge, and fixtures sing,
Stable names for every thing;
Warnings hush, the regions sing.

---

Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

• Ignore

❌ Failed checks (1 error, 1 warning)

Check name	Status	Explanation	Resolution
Rust Compiler Lint Integrity	❌ Error	crates/stilyagi-pyext/src/bridge_bdd.rs adds multiple let _ = bridge_state; no-op references to silence unused_variables in BDD steps.	Rename unused parameters to _bridge_state (or remove them if the macro permits) and delete the no-op reads; use only a narrow #[expect] for a tracked real use.
Docstring Coverage	⚠️ Warning	Docstring coverage is 77.40% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (18 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly matches the PR’s Markdown region-kind coverage work and includes the roadmap reference (2.1.2).
Description check	✅ Passed	The description is directly related to the changeset and accurately summarises the roadmap item and follow-up fixes.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Testing (Overall)	✅ Passed	PASS: The PR adds focused regression, property, snapshot, concurrency, and boundary tests that exercise the new Markdown IR, Python bridge, fixture, and workflow behaviour.
User-Facing Documentation	✅ Passed	PASS: docs/users-guide.md documents the new extraction contract, supported_region_kinds(), the new region kinds, unknown-kind warnings, and blank-input behaviour.
Developer Documentation	✅ Passed	Docs cover the new APIs, ADR 005/RFC 0001, tooling, and workflow; roadmap 2.1.2 is checked off and the execplan is marked COMPLETE.
Module-Level Documentation	✅ Passed	PASS: Repo-wide scan found file-level doc comments/docstrings on all Python and Rust source modules; no module was missing docs.
Testing (Unit And Behavioural)	✅ Passed	Pass: add unit tests for edge cases/invariants and behavioural tests at public boundaries (subprocess package smoke, BDD, Makefile/workflow contracts).
Testing (Property / Proof)	✅ Passed	PASS: the PR adds proptest coverage for the new segment/parent/layout invariants in crates/stilyagi-markdown/src/tests/properties.rs and crates/stilyagi-ir/src/tests/segment_properties.rs; no lemma...
Testing (Compile-Time / Ui)	✅ Passed	PASS: PR adds a trybuild harness with a new PyO3 UI pass case, and its snapshot tests use canonical JSON plus focused semantic assertions.
Unit Architecture	✅ Passed	PASS: query adapters stay read-only, fallible fixture I/O is explicit via Result, and command-side warnings/state are isolated and tested at the public boundary.
Domain Architecture	✅ Passed	PASS: core IR now uses stable enums/errors (RegionKind, SyntheticReason, InvalidRegionKind) and adapter-only code handles bridge/filesystem concerns; no domain leak found.
Observability	✅ Passed	PASS: unknown IR kinds are logged at the public boundary with operation, index, and kind; Rust validation emits structured rule IDs and failure details at boundaries.
Security And Privacy	✅ Passed	PASS: changes only add a pinned docstring-lint CI step and matching Makefile/docs/tests; no secrets, auth, permission, injection, or sensitive-data exposure paths were introduced.
Performance And Resource Use	✅ Passed	PASS: Added loops are bounded, caches are memoised, and no unbounded retries or hot-path I/O regressions are introduced.
Concurrency And State	✅ Passed	Approve: the new process-wide cache and syntax flag have a clear owner, module lock, reset hook, and a concurrent test proving single validation.
Architectural Complexity And Maintainability	✅ Passed	PASS: the new modules isolate traversal, emission, validation, and bridge adaptation with explicit boundaries, documented caches, and no circular crate edges.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 2-1-2-golden-fixture-coverage

---

_{Comment @coderabbitai help to get the list of available commands.}

[leynos]

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response.

crates/stilyagi-markdown/src/tests/coverage.rs

Comment on file

//! Coverage tests for the promised Markdown IR region vocabulary.

❌ New issue: String Heavy Function Arguments
In this module, 41.7% of all arguments to its 17 functions are strings. The threshold for string arguments is 39.0%

[leynos]

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response.

crates/stilyagi-markdown/src/tests.rs

Comment on file

//! snapshots remain anchored to `src/snapshots/`. Other test categories live
//! in focused sibling modules to keep each file within the size budget.

macro_rules! must_ok {

❌ New issue: String Heavy Function Arguments
In this module, 41.7% of all arguments to its 11 functions are strings. The threshold for string arguments is 39.0%

[leynos]

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response.

crates/stilyagi-markdown/src/tests/ir_consistency.rs

Comment on file

                .regions
                .first_mut()
                .unwrap_or_else(|| panic!("expected at least one Markdown IR region"));
                .expect("expected at least one Markdown IR region");

❌ New issue: Code Duplication
The module contains 3 functions with similar structure: validate_ir_consistency_reports_invalid_origin_nodes,validate_ir_consistency_reports_region_text_mismatches,validate_ir_consistency_reports_unresolved_parent_regions

[leynos]

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response.

Large Assertion Blocks

crates/stilyagi-markdown/src/tests/coverage.rs:

What lead to degradation?

The test suite contains 4 assertion blocks with at least 4 assertions, threshold = 4

Why does this problem occur?

Measures the number of lines with consecutive assert statements in a unit test. The more consecutive assert statements, the lower the code health. The threshold for the Rust language is 4 consecutive lines of assert statements.

How to fix it?

Consider to encapsulate the assertions (i.e. test criteria) in a custom assert statement. This simplifies the test by letting you communicate the test criteria in the language of your domain. We also recommend to consider the granularity of the tests; sometimes a single test tests too many things; extracting smaller tests often help to simplify the test criteria. Finally, large assertion blocks can also signal missing functionality in the code under test: instead of fetching and comparing individual properties of an object, maybe that class should encapsulate what equality means in an Equals() method?

Helpful refactoring examples

To get a general understanding of what this code health issue looks like - and how it might be addressed - we have prepared some diffs for illustrative purposes.

SAMPLE

# large_assertion_block_example.js
 test('T-1000 automatically repairs when damaged', () => {
     robT1000.heal();
 
-    assert.equal(100, robT1000.cpuCapacity);
-    assert.isOk(robT1000.ramCheckPasses;
-    assert.isOK(robT1000.diskAccessible);
-    assert.equal(100, robT1000.vision);
-    assert.equal(CONSTANTS.FUNCTIONAL, robT1000.equipment);
+    // Replace the low-level assertions with a custom assert that lets
+    // us communicate in the language of our domain. Also encapsulates
+    // the criteria so that we only have one place to change if/when
+    // more properties are added.
+    // Most test frameworks and assertion libraries have support for
+    // custom asserts.
+    assert.fullyOperational(robT1000);
 }

[leynos]

@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response.

Overall Code Complexity

python/stilyagi/engine/extraction.py:

What lead to degradation?

This module has a mean cyclomatic complexity of 4.11 across 9 functions. The mean complexity threshold is 4

Why does this problem occur?

Overall Code Complexity is measured by the mean cyclomatic complexity across all functions in the file. The lower the number, the better.
Cyclomatic complexity is a function level metric that measures the number of logical branches (if-else, loops, etc.). Cyclomatic complexity is a rough complexity measure, but useful as a way of estimating the minimum number of unit tests you would need. As such, prefer functions with low cyclomatic complexity (2-3 branches).

How to fix it?

You address the overall cyclomatic complexity by a) modularizing the code, and b) abstract away the complexity. Let's look at some examples:
Modularizing the Code: Do an X-Ray and inspect the local hotspots. Are there any complex conditional expressions? If yes, then do a DECOMPOSE CONDITIONAL refactoring. Extract the conditional logic into a separate function and put a good name on that function. This clarifies the intent and makes the original function easier to read. Repeat until all complex conditional expressions have been simplified.

[sourcery-ai]

Sorry @leynos, your pull request is larger than the review limit of 150000 diff characters

[leynos]

@coderabbitai Have the following now been resolved?

docs/developers-guide.md (1)

638-647: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Add the missing Interrogate bullet to the make lint summary.
This list still names only Ruff, Pylint, cargo doc, cargo clippy, and Whitaker, but the Makefile now runs Interrogate between Ruff and Pylint (Line 114), and section 6a correctly documents Interrogate as tier two of three. Bring this earlier summary in line with the actual target order so readers don't get two conflicting pictures of make lint within the same guide.

📝 Suggested fix

- `make lint`
   - run Ruff checks through `uv`
+  - run Interrogate docstring-coverage checks requiring 100% coverage
   - run focused Pylint checks through the pinned `pylint-pypy-shim` wrapper
     under PyPy

🤖 Detailed instructions

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @docs/developers-guide.md around lines 638 - 647, Add the missing Interrogate
item to the make lint summary so it matches the actual make lint target
order and the later Interrogate documentation. Update the bullet list in the
make lint section to include Interrogate between Ruff and the focused Pylint
checks, keeping the wording consistent with the existing entries and with the
make lint behavior described by the Makefile and the Interrogate section.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
rg -n -C6 'make lint' docs/developers-guide.md | sed -n '1,80p'
echo "---"
sed -n '600,660p' docs/developers-guide.md

Length of output: 7090

---

@leynos Resolved. Line 645 now reads run Interrogate docstring-coverage checks requiring 100% coverage, sitting between the Ruff bullet (Line 644) and the Pylint bullet (Line 646). This matches the actual make lint target order and the section 6a tier breakdown at Line 685-689, so the two summaries no longer conflict.

No further work is required for this finding.

[leynos]

@coderabbitai Have the following now been resolved?

❌ Failed checks (1 error, 1 warning)

Check name	Status	Explanation	Resolution
Rust Compiler Lint Integrity	❌ Error	crates/stilyagi-pyext/src/bridge_bdd.rs adds multiple let _ = bridge_state; no-op references to silence unused_variables in BDD steps.	Rename unused parameters to _bridge_state (or remove them if the macro permits) and delete the no-op reads; use only a narrow #[expect] for a tracked real use.
Docstring Coverage	⚠️ Warning	Docstring coverage is 77.40% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.