fix(manifest): bound live propagation and durable recovery - #795
fix(manifest): bound live propagation and durable recovery#795sublimator wants to merge 35 commits into
Conversation
A naked validation that authenticates against the current cached manifest for its signing key is an implicit request for that manifest. The recipient returns one singleton TMManifests on the same connection, bounded by a strand-owned per-connection repair ledger (256 rows, cleared on overflow, at most one singleton per master/sequence). Forged, malformed, unknown, and paired traffic draws no response. A validation with no available manifest is still sent naked.
Three scenarios on the new scripts primitives: mid-run rotation to sequence 2 through an old relay (supersede, admission, repair re-arm); a config-injected revocation seeded by the old node with the remaining four of five validators holding exact quorum; and wallet-wipe recovery through an old upstream with resumed paired forwarding. Waits are event-driven on terminal log facts with heartbeat logging.
The original design sketch had honest nodes informing senders of fresher manifests upon receipt of stale; it was parked as an amplification surface pending an explicit bounded one-shot model. The repair ledger is that model, so the lane lands as one more caller of sendManifestRepair at the singleton global-sequence guard, reusing the snapshot the guard already fetches. Strictly-stale only: equal-sequence arrivals are ordinary always-send traffic and draw nothing. The trigger needs no signature verification; safety comes from bounds, not authentication - answers are limited to masters this node retains, once per sequence per connection via the shared ledger, and monotone sequences make correction exchanges converge in one round trip. A retained revocation is itself the answer: a stale normal manifest for a revoked master draws the tombstone, giving revocations a demand-driven re-supply path to any peer that demonstrably still uses the dead identity. Retention, not listing, qualifies a master for an answer. Deferred: the legacy batch lane does not correct (singletons only); no live-scenario control yet (the mixed old-binary connect dump is the natural trigger); design-doc sync for the reopened corrections language left to the integration pass.
# Conflicts: # src/xrpld/app/misc/Manifest.h # src/xrpld/app/misc/detail/Manifest.cpp # src/xrpld/overlay/detail/OverlayImpl.cpp
Cold lookup treated the open ledger as ledger-authoritative, so an unvalidated SetManifest could stick in ManifestCache. Unknown naked validations also occupied the validation hash before the probe, poisoning a later verified pair. Probe only the validated ledger and refuse open views. Restore drop-before-claim when no probe will run. Give cold probes their own HashRouter domain until the signer resolves. Comments match the settled split: paired TMManifests for live rotation, standalone revocations, paid SetManifest for durability only.
A cache-miss naked validation queued a probe job per hash, so a burst of unknown signers bypassed the connection's one-in-flight token. Share that token with the cold lookup, and do not probe while a pending candidate occupies the slot.
The early naked-unknown drop concluded what checkValidation used to conclude after verifying a signature, but returned without any resource charge. Since a cold probe now holds the connection's one in-flight token, that free path is the one every further unknown naked validation on the connection takes, so an unresolvable signing key could be replayed at no cost. Charge feeUselessData at the drop, matching the job's charge for the same conclusion. Also correct the duplicate-repair comment: the repair is a cache lookup that no-ops unless some connection's probe already resolved the key, not a check that the first probe resolved.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## onchain-manifests #795 +/- ##
===================================================
+ Coverage 77.3% 77.8% +0.5%
===================================================
Files 839 839
Lines 78958 79627 +669
Branches 11592 11429 -163
===================================================
+ Hits 61064 61971 +907
+ Misses 17884 17646 -238
Partials 10 10
... and 15 files with indirect coverage changes 🚀 New features to boost your workflow:
|
|
|
||
| LEDGER_ENTRY(ltMANIFEST, 0x004D, Manifest, manifest_entry, ({ | ||
| {sfAccount, soeREQUIRED}, | ||
| {sfOwnerNode, soeREQUIRED}, |
There was a problem hiding this comment.
manifests are not directory objects, cannot be deleted and do not need to be referenced in this way
There was a problem hiding this comment.
Arguably they should even pay 2 reserves here (this exploration does not do that), for all the reasons reserves exist in the first place. They're taking 2 slots in the memory-resident state tree.
I think if you're charging a present AccountRoot the Fee, it's clearly an owned object
Pick a lane, right?
It should pay a reserve for it like anything else as it's on the ledger
It can be listed under account_objects, be seen in explorers, and it can be an obligation blocking account delete
I would also say the enrollment to online manifests should be a user txn, not really much more hassle than setting up the .well-known/xahaud.toml
Then you have something that fits in with the existing reserve system and validators can keep their authority.
So basically, a normal Account object, with the convenient auto-submitted update txns with canonical Fee
re: deletion, you probably don't need them to be super-duper-for-reals-permanent and they could be deleted.
No point having tombstones in 2035 for a defunct business that was a validator, that no one even has configured anymore.
But that's a "later" provided AccountDelete sees manifests as obligations with no current method to delete.
But this doesn't really matter, the main thing is bounding the cache, and tying precisely the manifests to validations where possible, rather than dumping the whole cache on connection, or some smaller yet still arbitrary LRU subset.
2 participants
This is a stacked draft against #779, not an alternative to it. It combines
that PR's on-ledger durability with the bounded live-propagation work from
#794, then makes the boundary between those two jobs explicit.
The existing forever-growing manifest cache quietly does two jobs. It moves
new manifests through the network, but it also acts as accidental durable
memory: once enough long-lived nodes have seen a manifest, reconnecting peers
keep teaching it to each other indefinitely. That gives useful cockroach-like
survival, but only by retaining and exchanging an unbounded cache.
This stack replaces those two properties separately. Validations drive fast
live propagation; the ledger provides deliberate durable recovery.
Before sending a validation, a node sends its best manifest for that signing
key using the existing envelopes. The receiver holds at most one unverified
normal manifest per connection and does not verify or retain it until a
matching validation arrives. Once both signatures pass, locally listed
validators may enter the durable local cache; other permitted validations can
carry the association ephemerally without growing it. Peer resource charges
cover malformed, invalid, and useless work.
The process cache now has an explicit retention boundary. Current policy and
locally configured manifests remain protected. Everything else that is worth
retaining -- including old wallet rows, cold on-ledger discoveries, and the
narrow unlisted-revocation correction lane -- shares one 1,000-master LRU.
Eviction removes the manifest, reverse signing-key mapping, and recency row
together. Losing that residue is safe: a later paired validation or the
validated-ledger cold lookup can recover it. This is a narrow cache bound, not
the tier-2 list, candidate population, or permit machinery from #777.
This also repairs missed associations without a new protocol negotiation. A
naked validation from a known signer is an implicit request: after it
authenticates, the receiver returns its current manifest on that connection.
An unknown signer may use the on-ledger signing-key index only after paying the
cost of a valid validation signature; successful recovery carries the manifest
onward and repairs the sender, while unresolved work is charged and dropped.
Revocations retain a narrow bounded correction lane because a revoked
validator deliberately has no later validation to carry them.
SetManifestis not a second fast-gossip plane. It follows ordinarytransaction relay and becomes authoritative durable state only through normal
ledger admission. A fee-blocked or otherwise unapplied wrapper is not
harvested into the cache and does not trigger a separate full-mesh
TMManifestsbroadcast. The node that submits an automatic wrapper alreadylearned the manifest through live propagation; the wrapper's remaining job is
to make that binding recoverable after the live network forgets it.
Registration and updates now have distinct authority, which is the main
behavioural change from #779 as drafted: there, any relayer could place any
validator's manifest on the ledger; here, the first registration is an
ordinary account-signed
SetManifest, so joining the durable registry is thekey holder's explicit act and no previously signed manifest can be conscripted
by a third party. Registration creates one canonical owned
ltMANIFEST, addsit to the owner directory, increments
OwnerCount, and charges one ownerreserve. Once that slot exists, rotations and revocations may use either
another ordinary account-signed transaction or a canonical unsigned wrapper
whose authority comes from the embedded manifest signature — so the network
can carry a registered validator's updates for it, but only its owner can
enrol it.
The unsigned form is deliberately one exact transaction, so anyone can relay
it but nobody can vary it. Canonicality covers the complete envelope, including
the Fee and the absence of memos or other extension fields. Its ruleset-fixed
Fee is 1,000 drops plus 100 drops per manifest byte. It receives no free or
priority queue lane; account-signed updates retain ordinary fee behaviour.
Sequence handling stays intentionally small. A valid higher sequence becomes
authoritative immediately; lower and identical manifests are ignored. If two
different manifests use the same sequence, the version in the validated ledger
replaces the cache. Equivocation and validator-list dismissal policy remain
outside this PR.
For an active manifest, the complete object is stored under both lookup
directions, matching #779's direct lookup geometry. The stable master-key copy
is the one owned, reserve-backed directory object; the active signing-key copy
is a full duplicate, and each copy points to the other. Revocation removes the
signing-key copy and leaves the owned master-key record. The master-key lookup
is the regular reconciliation path for locally trusted validators. Resolving
an unknown validation signing key takes one bounded read, but remains a rare
cold-cache fallback — validation prerequisites normally populate the mapping,
so startup, missed propagation, and local cache loss are its expected
occasions. Distinct negative probes are capped per ledger so valid-signature
traffic cannot manufacture unbounded ledger reads.
For now, the owned manifest is an explicit account obligation, so
AccountDeletereturnstecHAS_OBLIGATIONS. Expiration, tombstone collection,and any later rule for releasing that obligation are deferred rather than
being bundled into this refinement.
The PR remains a draft so these refinements can be discussed alongside #779
before either protocol shape is treated as final.