Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvement/check block sequencing #299

Merged
merged 47 commits into from
Feb 7, 2024

Conversation

grooviegermanikus
Copy link
Collaborator

@grooviegermanikus grooviegermanikus commented Jan 22, 2024

note: requires a not-yet-published version of geyser-grpc-connector (open PR); unfortunately lite-rpc and geyser-grpc-connector depend in turn on an unpublished version of geyser-grpc-connection crate! keep the git pointer for now

Configuration:
Adds parameter enable_grpc_stream_inspection to config

Extensive Test on solana-lite-rpc-staging against testnet:

2024-01-24T09:56:23Z app[148e271c30e268] ams [info]2024-01-24T09:56:23.627712Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_multiplex: got confirmed blockmeta 243790968 with blockhash HcG7tEyP1WUt31WHWkx28ofzU93v3s7YjREK
t1hjvXCi
2024-01-24T09:56:23Z app[148e271c30e268] ams [info]2024-01-24T09:56:23.631753Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: Saw block: 243790968#HcG7tEyP1WUt31WHWkx28ofzU93v3s7YjREKt1hjvXCi @ confirmed with 2345
txs
2024-01-24T09:56:23Z app[148e271c30e268] ams [info]2024-01-24T09:56:23.845075Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: Saw block: 243790969#FcNzhtPA7rQVfGRviPpMyB2FkCur7tHHiRgmXyRdbHQB @ processed with 1711
txs
2024-01-24T09:56:23Z app[148e271c30e268] ams [info]2024-01-24T09:56:23.873648Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_multiplex: got finalized blockmeta 243790939 with blockhash 22FWRcqQMXG8oDVdniofaW6NXKHDUQg5uyQz
mvcoAMBu
2024-01-24T09:56:23Z app[148e271c30e268] ams [info]2024-01-24T09:56:23.875128Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: Saw block: 243790939#22FWRcqQMXG8oDVdniofaW6NXKHDUQg5uyQzmvcoAMBu @ finalized with 583
txs
2024-01-24T09:56:23Z app[148e271c30e268] ams [info]2024-01-24T09:56:23.875215Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: block sequence seen on channel for block 22FWRcqQMXG8oDVdniofaW6NXKHDUQg5uyQzmvcoAMBu (
slot 243790939): "2024-01-24 09:56:11.366533624" -> "2024-01-24 09:56:11.793724844" -> "2024-01-24 09:56:23.875196405"
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.194747Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_multiplex: got confirmed blockmeta 243790969 with blockhash FcNzhtPA7rQVfGRviPpMyB2FkCur7tHHiRgm
XyRdbHQB
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.199163Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: Saw block: 243790969#FcNzhtPA7rQVfGRviPpMyB2FkCur7tHHiRgmXyRdbHQB @ confirmed with 1711
txs
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.297512Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: Saw block: 243790970#BW2ekGVf9Ys492BXzywBTVaN7vLhcBj1EUAtop8b7bRU @ processed with 1636
txs
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.301241Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_multiplex: got finalized blockmeta 243790940 with blockhash ANGMHAyQvjWfSSphsdMvXmDea4TDKRsY8CcX
dSwp9XaD
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.303421Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: Saw block: 243790940#ANGMHAyQvjWfSSphsdMvXmDea4TDKRsY8CcXdSwp9XaD @ finalized with 2038
txs
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.303527Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: block sequence seen on channel for block ANGMHAyQvjWfSSphsdMvXmDea4TDKRsY8CcXdSwp9XaD (
slot 243790940): "2024-01-24 09:56:11.791707321" -> "2024-01-24 09:56:12.137343653" -> "2024-01-24 09:56:24.303507716"
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.561511Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: Saw block: 243790971#FhoZuDyRi8FfRJU2GGJekrCSfNCugLWtSVYUs3GQzUe7 @ processed with 718
txs
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.653176Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_multiplex: got confirmed blockmeta 243790970 with blockhash BW2ekGVf9Ys492BXzywBTVaN7vLhcBj1EUAt
op8b7bRU
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.658157Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: Saw block: 243790970#BW2ekGVf9Ys492BXzywBTVaN7vLhcBj1EUAtop8b7bRU @ confirmed with 1636
txs
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.712722Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_multiplex: got confirmed blockmeta 243790971 with blockhash FhoZuDyRi8FfRJU2GGJekrCSfNCugLWtSVYU
s3GQzUe7
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.715025Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: Saw block: 243790971#FhoZuDyRi8FfRJU2GGJekrCSfNCugLWtSVYUs3GQzUe7 @ confirmed with 718
txs
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.811835Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_multiplex: got finalized blockmeta 243790941 with blockhash 8iNTe7mBatH4yxJu3BGtjYBhfr2vaS5ko26v
kTSxEeSF
2024-01-24T09:56:24Z app[148e271c30e268] ams [info]2024-01-24T09:56:24.815630Z DEBUG solana_lite_rpc_cluster_endpoints::grpc_inspect: Saw block: 243790941#8iNTe7mBatH4yxJu3BGtjYBhfr2vaS5ko26vkTSxEeSF @ finalized with 1523
txs

the message block sequence seen is produces when the artificial consumer was able to receive three full-blocks in correct order and without duplicates

@godmodegalactus
Copy link
Collaborator

Fix conflicts and clippy

if block.commitment_config.is_processed() {
let prev_value =
saw_processed_at.insert(blockhash.clone(), (slot, SystemTime::now()));
match prev_value {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace it with if let Some(prev) = ...

if block.commitment_config.is_confirmed() {
let prev_value =
saw_confirmed_at.insert(blockhash.clone(), (slot, SystemTime::now()));
match prev_value {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

let prev_value =
saw_finalized_at.insert(blockhash.clone(), (slot, SystemTime::now()));
match prev_value {
None => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here


// rule: if confirmed, we should have seen processed but not finalized
if block.commitment_config.is_confirmed() {
if saw_processed_at.contains_key(&blockhash) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here should be like .is_none()

} else {
warn!("should not see confirmed slot without seeing processed slot first ({})", blockhash);
}
if saw_finalized_at.contains_key(&blockhash) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

"should not see confirmed slot after seeing finalized slot ({})",
blockhash
);
} else {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove else, no need to add more code for comments.

@@ -164,6 +162,17 @@ pub async fn start_lite_rpc(args: Config, rpc_client: Arc<RpcClient>) -> anyhow:
vote_account_notifier,
} = subscriptions;

// note: check failes for commitment_config processed because sources might disagree on the blocks
debugtask_blockstream_slot_progression(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be optional.

godmodegalactus
godmodegalactus previously approved these changes Feb 7, 2024
@grooviegermanikus
Copy link
Collaborator Author

hide grpc_inspect behind flag

@grooviegermanikus grooviegermanikus merged commit 38da1ed into main Feb 7, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants