Extended ledger#4981
Open
Exxenoz wants to merge 14 commits into
Open
Conversation
Test Results for Commit e864619Pull Request 4981: Results Test Case Results
Last updated: 2026-07-09 08:40:11 UTC |
Exxenoz
force-pushed
the
extended_ledger
branch
5 times, most recently
from
January 8, 2026 13:30
2dcdb2c to
6d4a2ed
Compare
Exxenoz
force-pushed
the
extended_ledger
branch
2 times, most recently
from
January 24, 2026 16:42
40e2e6f to
28b8749
Compare
Exxenoz
force-pushed
the
extended_ledger
branch
4 times, most recently
from
January 29, 2026 02:08
22a66c3 to
6bcaea9
Compare
Exxenoz
force-pushed
the
extended_ledger
branch
from
February 22, 2026 00:28
6bcaea9 to
77ba2d7
Compare
Exxenoz
force-pushed
the
extended_ledger
branch
3 times, most recently
from
June 4, 2026 23:36
ba56a86 to
f15ba60
Compare
Exxenoz
force-pushed
the
extended_ledger
branch
3 times, most recently
from
July 5, 2026 02:34
b6fe35d to
da01eb0
Compare
Exxenoz
marked this pull request as ready for review
July 5, 2026 02:35
Exxenoz
force-pushed
the
extended_ledger
branch
2 times, most recently
from
July 5, 2026 03:18
0406cd3 to
f1c2921
Compare
When the extended `account_delegators_by_weight` index is enabled, the `delegators` RPC now returns delegators in descending weight order instead of delegator account order. This differs from the fallback path because the fallback scans the account table, which is ordered by account, while the extended index is keyed by representative, weight, and delegator. Re-sorting the indexed results by delegator account would defeat part of the purpose of using the weight index, and `count` cannot exactly match both the account-table ordering and the weight-index ordering at the same time (without scanning/sorting ALL delegator entries). For the indexed path, the RPC therefore follows the natural index semantics and returns the highest-weight delegators first.
Populate the account-height block lookup index by scanning the block table directly instead of walking each account chain through successor lookups. This uses the sideband height already available from the block iterator and avoids extra random block/successor reads during ledger upgrade.
Use crawlers when populating extended ledger indices so long-running write transactions can be refreshed in batches, matching the topology index population flow. Progress logging is now time-based while refreshes remain tied to the population batch size. Add crawler::next_entry(): The existing crawler increment advances by logical/group key for specialized tables such as pending. This would skip additional receivables for the same account when populating the account receivables index with crawler++.
Contributor
Author
Extended ledger indices populate performance observationsI tested the initial populate of all four extended ledger index tables with both LMDB and RocksDB. Test hardware
Populate results
Open questionThe main open question is how we should handle the LMDB behavior. The additional LMDB file growth when using Should this be investigated further and potentially optimized for LMDB? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch adds optional extended ledger indices to speed up RPC paths that previously required expensive ledger scans. The indices are persisted in the store, maintained by ledger runtime hooks, and can be populated or dropped through dedicated CLI commands.
The extended ledger indices added are:
account_block_by_height{ account, height } -> block_hash.account_history, especially offset-based lookups.account_delegator_by_weight{ representative, weight, delegator }.delegatorsanddelegators_count.account_receivable_by_amount{ account, amount, send_block_hash }.receivableandaccounts_receivablewhen sorted results are requested.receive_block_by_send_blockblocks_infowhenreceive_hash=true.The branch also adds shared extended ledger lifecycle handling, including persisted initialized flags, startup population, drop behavior, runtime add/remove/update hooks, and logging for index population and maintenance.
Note: when the extended
account_delegator_by_weightindex is enabled, thedelegatorsRPC now returns delegators in descending weight order instead of delegator account order.Note: when the extended
account_receivable_by_amountindex is enabled, theaccounts_receivableandreceivableRPCs use that index only for sorted requests (sorting: true).CLI
New maintenance commands:
--populate_extended_ledger_indices--drop_extended_ledger_indicesThe extended ledger index feature is controlled through the node configuration option
extended_ledger_index.Tests
Adds focused core tests for extended ledger index population, runtime maintenance, rollback/delete behavior, drop behavior, disabled reopen behavior, and direct index-table verification.
The extended ledger index test cases were created with GPT-5.5 assistance (using Codex), and all changes were manually reviewed by me.