Bundle Download Large Files

[ericspod]

Is your feature request related to a problem? Please describe.
Many bundles have the large_files.yml file specifying large files (specifically pre-trained weights). The bundles in the GenerativeModels repo are kept separately currently from those in the model zoo so the bundle download program doesn't work quite as expected so users would want to use bundles from a checkout of the repo. These bundles will be moved over but other users may want their own separate hosting for bundles so I feel there is still a need to make it easier to start using a bundle from a checkout.

One step for this is to make it easier to download these large files on the command line or programmatically. This would help people working with checkout code and make it easier to get bundles and their data into Colab and other environments.

I would also suggest changes to the download command we have for bundles to work with a URL pointing to a directory in an arbitrary repo. This is much more complicated now but would allow a user to provide this as the URL and get that whole bundle large files and all. Again this would be helpful in Colab and elsewhere.

Describe the solution you'd like
I would like the code for downloading the large files here to be put into a command in the monai/bundle/scripts.py file so that users can run this and get the files they need to use the bundle.

Describe alternatives you've considered
One has to manually download weights and other files currently which isn't impossible but adds an extra layer of complexity to tell people to look into the yaml file.

Additional context
This was from experience trying to get going quickly with a bundle. We do also need a way of programmatically getting a bundle's network with pre-trained weights like torch.hub.load though this will require some added specification in the bundle.

[ericspod]

@virginiafdez This was from our experience with the bundle you needed yesterday.

[Nic-Ma]

Hi @ericspod ,

is it possible to contribute the bundles to MONAI model-zoo directly?
CC @yiheng-wang-nv .

Thanks.

[ericspod]

Hi @Nic-Ma, yes there's work going on to do that but I feel there will be other use cases that this would be useful for.

[ericspod]

I still think this feature is relevant so that I can get files for bundles on the command line.

[KumoLiu]

I can take a look at updating the download API we have for bundles to work with a URL pointing to a directory in an arbitrary repo. In this case, the large file will also be automatically downloaded.

I didn't get the point of why users want to download the large file manually in a command line.

[wyli]

I didn't get the point of why users want to download the large file manually in a command line.

To quickly get started, we have the CLI usage of preparing a bundle locally, please see the model zoo readme https://github.com/Project-MONAI/model-zoo/blob/b51483d8403d35ca007059c7035762cd51a1514b/README.md?plain=1#L10 I think this feature request would make it convenient for the ones with large_files.yml

[ericspod]

I didn't get the point of why users want to download the large file manually in a command line.

The use case I run into is that I have a checkout of the model zoo and want to get the large files to do something else with the bundle that's not in a context where automatic download would have occurred. I can see people downloading the code for a bundle outside of git (either from our zoo or someone's repo or from HF) and need to get the large files then to use the bundle as intended.