Define initialize, run, finalize sections in bundle config

[Nic-Ma]

Is your feature request related to a problem? Please describe.
Currently, to support code execution in order, we defined expressions in a list of bundle config, like:
https://github.com/Project-MONAI/model-zoo/blob/dev/models/spleen_ct_segmentation/configs/multi_gpu_train.json#L27
But when loading the bundle in other applications, like MONAI Label, MONAI FL, we want to get the trainer or other components and hard to execute all the expressions in the list immediately, so we re-define much logic in the python parser, like:
https://github.com/Project-MONAI/MONAI/blob/dev/monai/fl/client/monai_algo.py#L460

To easily connect with other apps, I think we should clearly define the expressions in initialize, run, finalize sections, then others can call the operations separately. For example:

{
    "initialize": [
        "$import torch.distributed as dist",
        "$dist.is_initialized() or dist.init_process_group(backend='nccl')",
        "$torch.cuda.set_device(@device)",
        "$monai.utils.set_determinism(seed=123)",
        "$setattr(torch.backends.cudnn, 'benchmark', True)",
        "$import logging",
        "$@train#trainer.logger.setLevel(logging.WARNING if dist.get_rank() > 0 else logging.INFO)",
        "$@validate#evaluator.logger.setLevel(logging.WARNING if dist.get_rank() > 0 else logging.INFO)"
    ],
    "run": [
        "$@train#trainer.run()"
    ],
    "finalize": [
        "$dist.destroy_process_group()"
    ]
}

Will also create another ticket to define a high level API Bundle to simplify the bundle loading in other apps.

[Nic-Ma]

CC @wyli @SachidanandAlle @holgerroth for more discussion.

Thanks.

[Nic-Ma]

Hi @yiheng-wang-nv ,

As PR Project-MONAI/MONAI#5822 had been merged into MONAI 1.2rc1, could you please help update existing bundles to this initialize, run, finalize sections and update the MONAI dependency version to 1.2?
Let's merge it to dev-1.2 branch according to the plan.

Thanks in advance.

[yiheng-wang-nv]

Update: I'm now working on this ticket. According to the latest discussion with @Nic-Ma , we will use dev branch only.
As for the bundles, I will update spleen_ct_segmentation first, and submit a separate PR for others.

[yiheng-wang-nv]

This ticket is resolved