🎯 Problem to be solved
The duties cache can serve stale committee assignments when the beacon node imports a duty-anchor block after charon has cached the epoch's duties. A late-filled anchor slot changes the shuffle without emitting a chain_reorg SSE event, so even with the sse_reorg_duties feature enabled the cache is never invalidated — every attestation for the epoch is then rejected by the BN with AttesterNotInCommittee, and the VCs signing off charon-served duties attest to nonexistent assignments.
Observed live on the Plataberget (glamsterdam devnet) CDVC cluster, 2026-09-22:
- The cluster's lighthouse BN lagged block imports (gloas custody/
InvalidKzg backfill errors, up to ~28 slots behind).
- Charon-served duties for epoch 8978 were anchored at
dependent_root 0x9df6… (slot 287310, committee 6) while the same BN's live answer was 0x4cb7… (slot 287304, committee 12).
- Every submission bounced:
AttesterNotInCommittee { attester_index: 85338, … }, four consecutive epochs.
- Enabling
sse_reorg_duties + restart fixed it for two epochs (distance-0 inclusions), then the pattern recurred (epochs 8982–8984) with the flag active — confirming reorg events alone don't cover the late-import case.
🛠️ Proposed solution
Track the dependent_root returned with each duties fetch (proposer/attester/sync) and invalidate the cached epoch when a fresh fetch (or the next scheduled refresh) returns a different root — the same protection Lighthouse's own VC implements by comparing dependent_root and re-fetching. Applies to the DutiesCache in app/eth2wrap.
Post-gloas this gets more load-bearing: proposer preferences sign over the E-2 shuffling dependent_root served by the v2 proposer duties endpoint, and the quarter-slot attestation deadline makes DVs extra sensitive to BN import lag.
category: bug
ticket: #4324
🎯 Problem to be solved
The duties cache can serve stale committee assignments when the beacon node imports a duty-anchor block after charon has cached the epoch's duties. A late-filled anchor slot changes the shuffle without emitting a
chain_reorgSSE event, so even with thesse_reorg_dutiesfeature enabled the cache is never invalidated — every attestation for the epoch is then rejected by the BN withAttesterNotInCommittee, and the VCs signing off charon-served duties attest to nonexistent assignments.Observed live on the Plataberget (glamsterdam devnet) CDVC cluster, 2026-09-22:
InvalidKzgbackfill errors, up to ~28 slots behind).dependent_root 0x9df6…(slot 287310, committee 6) while the same BN's live answer was0x4cb7…(slot 287304, committee 12).AttesterNotInCommittee { attester_index: 85338, … }, four consecutive epochs.sse_reorg_duties+ restart fixed it for two epochs (distance-0 inclusions), then the pattern recurred (epochs 8982–8984) with the flag active — confirming reorg events alone don't cover the late-import case.🛠️ Proposed solution
Track the
dependent_rootreturned with each duties fetch (proposer/attester/sync) and invalidate the cached epoch when a fresh fetch (or the next scheduled refresh) returns a different root — the same protection Lighthouse's own VC implements by comparingdependent_rootand re-fetching. Applies to theDutiesCacheinapp/eth2wrap.Post-gloas this gets more load-bearing: proposer preferences sign over the
E-2shufflingdependent_rootserved by the v2 proposer duties endpoint, and the quarter-slot attestation deadline makes DVs extra sensitive to BN import lag.category: bug
ticket: #4324