Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remote Externalities: Start downloading KVs early #2494

Open
eagr opened this issue Nov 25, 2023 · 2 comments · May be fixed by #3584
Open

Remote Externalities: Start downloading KVs early #2494

eagr opened this issue Nov 25, 2023 · 2 comments · May be fixed by #3584
Labels
C1-mentor A task where a mentor is available. Please indicate in the issue who the mentor could be. D1-medium Can be fixed by a coder with good Rust knowledge but little knowledge of the codebase. I5-enhancement An additional feature request. T0-node This PR/Issue is related to the topic “node”.

Comments

@eagr
Copy link
Contributor

eagr commented Nov 25, 2023

Follow up of #1985

As described, we don't necessarily have to wait till we have collected all the keys. By starting as soon as we have collected some keys, we could potentially save a couple more minutes. I presume the benefit will increase with the growth of the total amount of KVs as it would take more time to collect all the keys.

The downside is we might not be able to show the downloading progress as we do now. But if we does manage to speed things up, I guess this is tradeoff acceptable?

CC @ggwpez @liamaharon
@ggwpez
Copy link
Member

ggwpez commented Nov 25, 2023 •
edited
Loading

The downside is we might not be able to show the downloading progress as we do now.

It could have two bars i assume; one for keys and one for values.
NVM just read the other issue since we dont know the total number of KV pairs.

@liamaharon liamaharon added T0-node This PR/Issue is related to the topic “node”. C1-mentor A task where a mentor is available. Please indicate in the issue who the mentor could be. labels Nov 27, 2023
@liamaharon
Copy link
Contributor

The downside is we might not be able to show the downloading progress as we do now. But if we does manage to speed things up, I guess this is tradeoff acceptable?

This is ok

@ggwpez ggwpez added D1-medium Can be fixed by a coder with good Rust knowledge but little knowledge of the codebase. I5-enhancement An additional feature request. labels Feb 5, 2024
@kianenigma kianenigma changed the title Start downloading KVs early Remote Externalities: Start downloading KVs early Feb 5, 2024
@dharjeezy dharjeezy linked a pull request Mar 5, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C1-mentor A task where a mentor is available. Please indicate in the issue who the mentor could be. D1-medium Can be fixed by a coder with good Rust knowledge but little knowledge of the codebase. I5-enhancement An additional feature request. T0-node This PR/Issue is related to the topic “node”.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Download KeyValues in remote externalities early dharjeezy/polkadot-sdk
3 participants
@ggwpez @liamaharon @eagr