Parallel integ check in producer cycle

[spjegan]

Integrity checks in the producer side are currently parallelized across types. This means, if a type has many shards, the integrity check happens sequentially for each shard. This PR changes that by parallelizing it "per type and per shard". This feature is an opt-in via the builder. Opting out retains the old functionality. The benefit we get by parallelizing by shard is applicable only when type-sharding is enabled.

[Copilot]

Pull request overview

This PR introduces opt-in per-shard parallelization for producer integrity checks (checksums) and snapshot serialization, aiming to improve performance for large multi-shard types while preserving the default (legacy) behavior when flags are unset.

Changes:

• Add an opt-in per-(type, shard) parallel checksum computation and plumb it through producer checkIntegrity with a single per-cycle flag read.
• Add opt-in per-shard snapshot computation hooks to write states and use them in HollowBlobWriter to parallelize snapshot calculation while keeping byte-identical output.
• Add targeted sharding tests covering correctness, sensitivity, resharding cycles, and byte-identical snapshot output.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
hollow/src/main/java/com/netflix/hollow/tools/checksum/HollowChecksum.java	Adds opt-in per-shard checksum mode and an explicit-flag overload to keep integrity checks consistent within a cycle.
hollow/src/main/java/com/netflix/hollow/api/producer/AbstractHollowProducer.java	Reads checksum mode once per integrity check and applies it consistently to all four checksum computations.
hollow/src/main/java/com/netflix/hollow/core/read/engine/HollowTypeReadState.java	Adds per-shard checksum API (getShardChecksum) and a new applyShardToChecksum hook.
hollow/src/main/java/com/netflix/hollow/core/read/engine/object/HollowObjectTypeReadState.java	Implements per-shard checksum application for OBJECT read state.
hollow/src/main/java/com/netflix/hollow/core/read/engine/list/HollowListTypeReadState.java	Implements per-shard checksum application for LIST read state.
hollow/src/main/java/com/netflix/hollow/core/read/engine/set/HollowSetTypeReadState.java	Implements per-shard checksum application for SET read state.
hollow/src/main/java/com/netflix/hollow/core/read/engine/map/HollowMapTypeReadState.java	Implements per-shard checksum application for MAP read state.
hollow/src/main/java/com/netflix/hollow/core/write/HollowTypeWriteState.java	Adds extensibility points for per-shard snapshot computation (numSnapshotShards, prepareCalculateSnapshot, calculateSnapshotShard).
hollow/src/main/java/com/netflix/hollow/core/write/HollowObjectTypeWriteState.java	Implements per-shard snapshot calculation for OBJECT write state.
hollow/src/main/java/com/netflix/hollow/core/write/HollowBlobWriter.java	Adds opt-in per-shard fan-out during snapshot calculation while writing shards serially in-order for identical bytes.
hollow/src/test/java/com/netflix/hollow/tools/checksum/HollowChecksumShardedTest.java	New tests validating per-shard checksum determinism, sensitivity, and resharding integrity behavior.
hollow/src/test/java/com/netflix/hollow/core/write/HollowObjectTypeWriteStateShardedTest.java	New tests asserting per-shard snapshot computation is byte-identical to serial, including ordinal-hole scenarios.
hollow/src/test/java/com/netflix/hollow/core/write/HollowBlobWriterShardedTest.java	New end-to-end tests asserting parallel snapshot blobs match serial bytes and round-trip correctly.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.