SGLang L3 read fixes + diagnostics + multi-master discovery with pinned head (0.6.5–0.7.1) - #24
Merged
Merged
Conversation
SGLang's generic HiCache path writes via batch_set(keys, data), which stores one blob per RAW key, but batch_exists() looked keys up as SUFFIXED K/V component keys (_component_keys). The prefetch probe therefore missed every page -- exists_pages_found stayed 0 while write_requests climbed into the thousands -- so SGLang never issued a get and read_requests/read_remote_hits were always 0. batch_exists()/exists() now resolve keys through the active keyspace: raw for the generic value/pointer path (set by batch_set/batch_get), suffixed for the zero-copy v1/v2 path. A read-only node that has not observed the producer's keyspace self-heals by probing the other namespace once on a full miss and locking onto whichever hits. Adds a regression test: generic batch_set on the producer + batch_exists on a fresh consumer must report the pages present (and then batch_get fills them). Co-authored-by: 兰州小红鸡 <flyphp@outlook.com>
flymysql
marked this pull request as ready for review
June 2, 2026 06:56
cursor
Bot
changed the title
cursor
Bot
changed the title
cursor
Bot
changed the title
cursor
Bot
changed the title
cursor
Bot
changed the title
cursor
Bot
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Functional fixes (the "writes work, reads always 0" saga)
batch_existskeyspace match. Genericbatch_setstores raw keys butbatch_existslooked them up as suffixed component keys → prefetch probed the wrong namespace, missed every page, SGLang never read. Now resolves keys through the active keyspace + self-heals on read-only nodes.rdma_local_reg_misses == read_failures): SGLang's genericbatch_gethands a destination outside the registered host KV pool, solkey_for(addr)=0and the WR was never posted.RdmaContext::lkey_for_ensure()now lazily registers + caches an MR. Result on a 2-node test:read_hits=906/906,read_remote_hits=163,read_disk_hits=743,read_failures=0.Diagnostics
rdma_read_wc_errors/rdma_last_wc_status(+ibv_wc_status_strlogged).rdma_local_reg_misses/rdma_post_failures/rdma_lease_failures.exists_requests/exists_pages_found,directory_lookups_saved.Multi-master discovery (no single meta SPOF)
max_masters(default 3). If the head is down, live hosts still fill all slots; a small cluster has all hosts as masters.New:
config.max_masters,discovery_seeds(),meta_port(),DiscoveryClient.master_hosts(). Backward compatible with a singlediscovery_addr. Tests cover head pinning (even when the head doesn't sort first), 3-master selection, automatic failover to surviving hosts, and small-cluster all-master.Also
C++ built locally (syntax + full build OK; RDMA runtime untestable on a non-RDMA VM). Full Python suite: 58 passed. Tags
v0.6.5–v0.7.1on PyPI.