Optimize artifact downloads when restoring an Incremental Cardano DB

[jpraynaud]

Why

We want to optimize the download of the artifacts when restoring a Cardano database: download the ancillary first and avoid chunking the download of the immutable files. This will allow to use the resources of the user in a more consistent way.

What

Optimize the implementation of the download of the ancillary and artifact files.

How

• Pre-compute the list of futures to be run (first will be the ancillary files if needed)
• Fill the JoinSet used to run the futures with the maximum parameter max_parallel_downloads
• When a task is completed, refill with one remaining task in the list if exists