bundle script - pprint doesn't take care of long list correctly #5762
Description
Describe the bug
All the key/values are getting print here. But when the value is a huge list, the print/log is too big
https://github.com/Project-MONAI/MONAI/blob/dev/monai/bundle/scripts.py#L84
Looks like the pprint.pformat doesn't handle large lists correct.
To Reproduce
Steps to reproduce the behavior:
- You can use any monai bundle and pass the list of image/value pairs to the train/validation dataset
- Run single gpu training
Expected behavior
Smaller (first few elements and ...) log for the list; And if possible len of the list. Same should be in case of dict.
Screenshots
Currently it prints something huge.. like
2022-12-16 00:30:00,174 - INFO - > validate#dataset#data: [{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_039_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_039_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_039_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_040_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_040_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_040_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_041_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_041_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_041_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_042_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_042_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_042_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_043_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_043_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_043_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_044_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_044_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_044_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_045_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_045_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_045_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_046_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_046_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_046_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_047_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_047_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_047_type_map.npy'},
{'image': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_048_image.npy',
'label_inst': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_048_inst_map.npy',
'label_type': '/localhome/sachi/Projects/MONAILabel/sample-apps/pathology/model/pathology_nuclei_segmentation_classification/cache/train_ds/nuclei_hovernet/test_10_0_0_1000_1000_0x0_048_type_map.npy'}]
Environment
Ensuring you use the relevant python executable, please paste the output of:
================================
Printing MONAI config...
================================
MONAI version: 1.1.0rc2
Numpy version: 1.23.4
Pytorch version: 1.13.0+cu116
MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
MONAI rev id: d8652642d7ae849ccf7f53cbb483344682199043
MONAI __file__: /localhome/sachi/.local/lib/python3.10/site-packages/monai/__init__.py
Optional dependencies:
Pytorch Ignite version: 0.4.10
Nibabel version: 4.0.2
scikit-image version: 0.19.3
Pillow version: 9.0.1
Tensorboard version: 2.10.1
gdown version: 4.5.3
TorchVision version: 0.14.0+cu116
tqdm version: 4.64.1
lmdb version: 1.3.0
psutil version: 5.9.3
pandas version: 1.5.1
einops version: 0.5.0
transformers version: 4.21.3
mlflow version: 2.0.1
pynrrd version: 0.4.3
For details about installing the optional dependencies, please visit:
https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
================================
Printing system config...
================================
System: Linux
Linux version: Ubuntu 22.04 LTS
Platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.35
Processor: x86_64
Machine: x86_64
Python version: 3.10.6
Process name: python
Command: ['python', '-c', 'import monai; monai.config.print_debug_info()']
Open files: []
Num physical CPUs: 16
Num logical CPUs: 32
Num usable CPUs: 32
CPU usage (%): [5.4, 5.4, 5.1, 6.5, 7.1, 5.4, 6.4, 5.7, 5.1, 6.8, 5.4, 5.1, 6.1, 6.8, 5.4, 5.1, 6.7, 4.7, 6.1, 5.4, 5.1, 6.1, 5.4, 5.1, 5.1, 4.7, 5.4, 5.1, 5.7, 5.4, 4.7, 99.3]
CPU freq. (MHz): 1062
Load avg. in last 1, 5, 15 mins (%): [8.5, 3.7, 3.9]
Disk usage (%): 4.7
Avg. sensor temp. (Celsius): UNKNOWN for given OS
Total physical memory (GB): 251.6
Available memory (GB): 236.0
Used memory (GB): 13.5
================================
Printing GPU config...
================================
Num GPUs: 2
Has CUDA: True
CUDA version: 11.6
cuDNN enabled: True
cuDNN version: 8302
Current device: 0
Library compiled for CUDA architectures: ['sm_37', 'sm_50', 'sm_60', 'sm_70', 'sm_75', 'sm_80', 'sm_86']
GPU 0 Name: NVIDIA A40
GPU 0 Is integrated: False
GPU 0 Is multi GPU board: False
GPU 0 Multi processor count: 84
GPU 0 Total memory (GB): 44.4
GPU 0 CUDA capability (maj.min): 8.6
GPU 1 Name: NVIDIA A40
GPU 1 Is integrated: False
GPU 1 Is multi GPU board: False
GPU 1 Multi processor count: 84
GPU 1 Total memory (GB): 44.4
GPU 1 CUDA capability (maj.min): 8.6
Additional context
Add any other context about the problem here.