[DataFlow runtime · M5 3/4] Durable recovery (SQLite + restart reconcile) + fault/soak gates#608

Merged

jiapingW merged 26 commits into

mainfrom

dataflow-up-11-m5-recovery

Jun 30, 2026

Commits on Jun 27, 2026

runtime-m5(3/4): durable recovery — SQLiteMetadataStore + restart reconcile (B4)

maocheng23
and
claude
committed
runtime-m5(4/4): fault-injection suite + sustained-lag soak (exit gates)

maocheng23
and
claude
committed
style: apply pre-commit (black) to M5 durable-recovery files

maocheng23
and
claude
committed
runtime-m6(1/2): disaggregation seam + resharding contract + auth (B5/B9)

maocheng23
and
claude
committed
runtime-m6(2/2): >=4-rank tp>1 & sp>1 equivalence test (GPU gate)

maocheng23
and
claude
committed
style: apply pre-commit (black/isort) to M6 disagg files

maocheng23
and
claude
committed

Commits on Jun 29, 2026

runtime: readability — split SampleRefQueue lease into _lease_any/_lease_partition helpers

maocheng23
and
claude
committed
runtime(m6): per-generation-file SharedDirFeatureStore + gen-aware release (B5)

maocheng23
and
claude
committed
feat(runtime): disaggregated offline EAGLE3 assemble example

maocheng23
and
claude
committed
fix(disagg example): call sanity_check so builders see target_batch_size

maocheng23
and
claude
committed
feat(disagg example): expose log_interval on the launchers

maocheng23
and
claude
committed
fix(disagg): retain_on_release mode for re-iterable offline feature sets

maocheng23
and
claude
committed
feat(disagg example): add colocated role for head-to-head comparison

maocheng23
and
claude
committed
docs(disagg example): record head-to-head accept-length vs colocated

maocheng23
and
claude
committed
style: apply pre-commit (black/isort/autoflake) to disagg example files

maocheng23
and
claude
committed
feat(runtime/m6): MooncakeFeatureStore — RDMA fast-path backend for disaggregation

maocheng23
and
claude
committed
feat(disagg example): env-selectable Mooncake backend (DISAGG_BACKEND=mooncake)

maocheng23
and
claude
committed
feat(disagg example): size Mooncake segment via MOONCAKE_GLOBAL_SEGMENT_SIZE

maocheng23
and
claude
committed
fix(mooncake): immediate logical free (tombstone) for B5 under lease-deferred remove

maocheng23
and
claude
committed
runtime: M5/M6 hardening — LocalFeatureStore gc/release, SQLite durability, reconcile gate, USP 4-rank test

maocheng23
and
claude
committed
style: apply pre-commit (black/isort) to MooncakeFeatureStore files

maocheng23
and
claude
committed
Merge pull request #609 from sgl-project/dataflow-up-12-m6-disagg
jiapingW
authored
Merge pull request #610 from sgl-project/dataflow-up-13-disagg-example
jiapingW
authored
Merge pull request #611 from sgl-project/dataflow-up-14-hardening
jiapingW
authored
Merge pull request #612 from sgl-project/dataflow-up-15-mooncake
jiapingW
authored
docs(plan): fix draft-model naming, dedupe drafts layout, rm scratch files

maocheng23
and
claude
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[DataFlow runtime · M5 3/4] Durable recovery (SQLite + restart reconcile) + fault/soak gates#608

[DataFlow runtime · M5 3/4] Durable recovery (SQLite + restart reconcile) + fault/soak gates#608
jiapingW merged 26 commits into
mainfrom
dataflow-up-11-m5-recovery

Commits on Jun 27, 2026

runtime-m5(3/4): durable recovery — SQLiteMetadataStore + restart reconcile (B4)

runtime-m5(4/4): fault-injection suite + sustained-lag soak (exit gates)

style: apply pre-commit (black) to M5 durable-recovery files

runtime-m6(1/2): disaggregation seam + resharding contract + auth (B5/B9)

runtime-m6(2/2): >=4-rank tp>1 & sp>1 equivalence test (GPU gate)

style: apply pre-commit (black/isort) to M6 disagg files

Commits on Jun 29, 2026

runtime: readability — split SampleRefQueue lease into _lease_any/_lease_partition helpers

runtime(m6): per-generation-file SharedDirFeatureStore + gen-aware release (B5)

feat(runtime): disaggregated offline EAGLE3 assemble example

fix(disagg example): call sanity_check so builders see target_batch_size

feat(disagg example): expose log_interval on the launchers

fix(disagg): retain_on_release mode for re-iterable offline feature sets

feat(disagg example): add colocated role for head-to-head comparison

docs(disagg example): record head-to-head accept-length vs colocated

style: apply pre-commit (black/isort/autoflake) to disagg example files

feat(runtime/m6): MooncakeFeatureStore — RDMA fast-path backend for disaggregation

feat(disagg example): env-selectable Mooncake backend (DISAGG_BACKEND=mooncake)

feat(disagg example): size Mooncake segment via MOONCAKE_GLOBAL_SEGMENT_SIZE

fix(mooncake): immediate logical free (tombstone) for B5 under lease-deferred remove

runtime: M5/M6 hardening — LocalFeatureStore gc/release, SQLite durability, reconcile gate, USP 4-rank test

style: apply pre-commit (black/isort) to MooncakeFeatureStore files

Merge pull request #609 from sgl-project/dataflow-up-12-m6-disagg

Merge pull request #610 from sgl-project/dataflow-up-13-disagg-example

Merge pull request #611 from sgl-project/dataflow-up-14-hardening

Merge pull request #612 from sgl-project/dataflow-up-15-mooncake

docs(plan): fix draft-model naming, dedupe drafts layout, rm scratch files