Skip to content

Skip read validation for immutable shards - #853

Merged
DanielThomas merged 4 commits into
masterfrom
dannyt/non-recycling-read-fast-path
Oct 7, 2026
Merged

DanielThomas merged 4 commits into
masterfrom
dannyt/non-recycling-read-fast-path

Conversation

@DanielThomas

@DanielThomas DanielThomas commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

This avoids the trailing read validation when the underlying arrays are known to never be reused. This is a first in a series of pull requests in a stack that use the fact that arrays are not recycled as an invariant for further optimisation.

This builds on the work in #647. It added GarbageCollectorAwareRecycler and selects WastefulRecycler for concurrent garbage collectors, but does not remove the synchronization cost required by RecyclingRecycler.

Optimization Opportunities

In a 30-second Images CPU profile on JDK 25 with ZGC, Hollow appeared in 26.8% of samples. More than half of that time was concentrated in two low-level read operations:

  • Samples attributed to trailing read validation: 7.4% of total CPU
  • Fixed-length value extraction: 7.0% of total CPU

The attribution to validation may include stalls from the preceding data load, rather than time spent executing the fence itself. The remaining cost was spread across object fields, strings, collections, and primary-key lookups. This PR starts a stack that removes unnecessary validation for immutable shards, then optimizes the underlying fixed-length, string, collection, and byte read paths.

Local x86-64 microbenchmarks on JDK 25 with ZGC have not shown a repeatable improvement for this PR alone: one comparison of random long reads was slower, and a repeat was at parity. This is a different workload from Images, and the later PRs address more of the read path.

@DanielThomas
DanielThomas force-pushed the dannyt/non-recycling-read-fast-path branch from eef7257 to e0f2d65 Compare September 21, 2026 15:45
@DanielThomas DanielThomas changed the title Skip shard read validation when arrays are not recycled Skip read validation for immutable shards Sep 21, 2026
@DanielThomas
DanielThomas marked this pull request as ready for review September 22, 2026 07:27
@DanielThomas
DanielThomas added this pull request to stack #857 September 22, 2026 07:27
@DanielThomas
DanielThomas force-pushed the dannyt/non-recycling-read-fast-path branch from e0f2d65 to 031f1e9 Compare September 23, 2026 03:04
@Sunjeet

Sunjeet commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator

CPU the win on ARM should be larger than x86.

A minor plumbing thing that'd be good to fix-

HollowTypeReadState: derives the flag from stateEngine.getMemoryRecycler(). But the recycler that actually allocates and later recycles those segments is the one passed to the public readSnapshot(in,      
  recycler, numShards) / applyDelta(in, schema, recycler, numShards) and stored as HollowTypeDataElements.memoryRecycler (HollowTypeDataElements.java:14). Two independent sources.                            
                                                                                                                                                                                                               
  In-tree they always agree — HollowBlobReader.java:365,378 passes stateEngine.getMemoryRecycler(), and every other caller routes through HollowBlobReader. But nothing enforces it, and both entry points are 
  public API taking an arbitrary recycler. Hand a RecyclingRecycler to an engine constructed with a non-recycling one and you get validation skipped while arrays are reused — precisely the failure mode the  
  package doc names: "both validations pass while the value returned is recycled memory, with no exception and no detection."                                                                                  
                                                                                                                                                                                                               
  Secondary issue in the same spot: the flag is a constructor-time snapshot of a property owned by an object that doesn't exist yet and is rebuilt on every delta. Even with no mismatch, the derivation is    
  chronologically backwards.                                                                                                                                                                                   
                                                                                                                                                                                                               
  Fix, cheapest first:                                                                                                                                                                                         
  // in readSnapshot/applyDelta, after data elements are built                                                                                                                                                 
  if (shardsAreImmutable && dataElements.memoryRecycler.recyclesArrays())                                                                                                                                      
      throw new IllegalStateException("...");                                                                                                                                                                  
  Better: derive the flag from dataElements.memoryRecycler at publish time rather than from the engine at construction time.                                                                                   

@DanielThomas

Copy link
Copy Markdown
Member Author

I dug into this a little more: while async-profiler attributes Unsafe.loadFence at the top of a hot Hollow read stack, the fence itself accounts for those samples. On x86, hsdis places the samples on the value mask after the packed-data load. We repeated the benchmark with only the fence removed: the mask remained hot, now attributed to value extraction, with no throughput gain. On AArch64, the fence emits a hardware barrier instruction.

Will follow up with some improvements based on your notes.

@jasonk000 jasonk000 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Comment thread hollow/src/main/java/com/netflix/hollow/core/read/engine/HollowTypeReadState.java Outdated
@jasonk000
jasonk000 requested a review from kilink October 2, 2026 18:17
@DanielThomas

Copy link
Copy Markdown
Member Author

Added checks at the public snapshot and delta entry points, as you suggested. When a type uses the immutable-shard fast path, passing a recycling recycler now fails before any state is changed. Added regression tests for both entry points across objects, lists, sets, and maps.

@DanielThomas
DanielThomas force-pushed the dannyt/non-recycling-read-fast-path branch from d22d484 to 3b27656 Compare October 5, 2026 04:49
@joegoogle123
joegoogle123 force-pushed the dannyt/non-recycling-read-fast-path branch from 3b27656 to 2bf476a Compare October 5, 2026 14:02
@DanielThomas

Copy link
Copy Markdown
Member Author

Added withExperimentalFeatures(...) and the shared read configuration at the base of the stack. Validation elision is now gated by SHARD_READ_FAST_PATHS, default off.

Also added withMemoryRecyclingMode(AUTO | ENABLED | DISABLED) so recycler selection can be overridden independently of the GC.

@DanielThomas
DanielThomas merged commit 6241032 into master Oct 7, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants