feat(storage,api): persisted property-equality index for node id anchors (#1983)

[DecisionNerd]

Closes #1983

What

A persisted property-equality index so a node anchor (:L {id: $x}), or WHERE n.id = $x on one label, resolves its candidate node UUIDs from the index instead of reading the route's property column.

• crates/graphforge-storage/src/property_equality_index.rs: manifest (indexes/property/<stem>.eq: XXH64 line then JSON) plus sorted Arrow IPC shards of 4096 (value, node_uuid) rows (<stem>.shards/*.eqs). A lookup binary-searches the manifest and reads only the shards whose range holds the value, each checked against the manifest XXH64. Bytes are a pure function of the route content (no clock or identity).
• The scan (property_readers.rs) takes candidates from the index when a current one exists, else from the existing footer-statistics path. The plan's filter still decides each row, so semantics are unchanged.
• EXPLAIN (physical plan) names it: PropertyOverlayExec: route=_untyped, equality=id=Int(5), index=property_equality(id).
• Built at publish: the initial bulk construction (seal_and_publish, import session onto an empty project; in-memory and durable) writes the index inside the same generation, registered in the generation inventory. GraphForge::rebuild_property_indexes rebuilds from the current generation and publishes a new one.
• Authentication: manifest and shards are in the graph-files inventory (graph_admission::classify); the lookup checks the manifest checksum and every shard it reads. A failure returns GF_VALIDATION; the scan is not substituted and nothing is rebuilt by a read.

Decisions

• Indexed pairs: every property named id (Int64 or Utf8 stored type). The import session / load mapping has no key declaration (id_column is a benchmark-converter notion), so id only.
• Pre-index projects, and routes changed by a mutation after the index was built: the index is stale or absent (source digest mismatch), so the scan answers. A read never builds it. rebuild_property_indexes builds it. Covered by a_project_without_the_index_scans_until_rebuilt_and_after_a_write (durable, reopen, write, rebuild).
• Staged append construction (non-empty project) does not write the index (it does not write adjacency either); it is stale until rebuilt.
• Float, boolean and null equality are not pushed to the scan today (is_stored_equality; DataFusion rewrites id = true to id) and still use the scan.
• Files touched outside the index: property_scan.rs (Display and an is_current check at construction), property_overlay/inventory.rs (+ new route_digest.rs, because inventory.rs was at the source-size bound), graph_admission.rs (classification), graph_construction_encoding/bulk.rs (one call before the adjacency registration). Not touched: session.rs, expand_exec.rs, property_filter_approval.rs, left_enrichment.rs.

Verification (host load 15 to 40, shared)

• cargo nextest run -p graphforge-api --test property_equality_index --test property_read_pushdown: 23 passed, 0 failed (8 new). Covers: index in the generation inventory and named by EXPLAIN (literal and WHERE); read bytes bounded at 1/2/4 partitions with 12,288 and 24,576 nodes (bound 2*largest_fragment + 1 MiB, growth under 256 KiB, a miss reads under 1 MiB); unindexed (Float) property scans and answers; in-memory project; string ids exact, boolean ids exact; pre-index durable project scan / rebuild / reopen / write / rebuild; byte-identical index across two builds; corrupted shard and corrupted manifest each refused with GF_VALIDATION while a non-index statement still runs.
• Revert proof: with lookup returning Ok(None), an_anchored_lookup_reads_the_match_not_the_label fails: partitions=1, 6 fragments, largest 2094115: read 6521862 bytes, bound 5236806. With the index it passes.
• cargo test -p graphforge-api --test bdd: API BDD 118 passed, 0 failed; openCypher TCK 3898 of 3898 passing (0 regressed).
• cargo nextest run -p graphforge-storage: 2024 tests, all passed (the one long RSS test was re-run alone: passed).
• make check: exit 0.

Measurement

See the comment on #1983: the index is used and the bare anchor drops from about 2.0 s to about 0.7 s on SF1, but IC2, IC4 and TCR1 do not move because their floor is elsewhere (node-topology label filter and enrichment join, owned by #1984).

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[greptile-apps]

Your trial has ended. Reactivate Greptile to resume code reviews.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Repository: CurateLabs/graphforge/.coderabbit.yaml
• Review profile: CHILL
• Plan: Advanced
• Run ID: b053ab7a-a00e-4131-a688-e7995aeda21d

---

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

---

_{Comment @coderabbitai help to get the list of available commands.}