Skip to content

CheckpointSaver logs wrong filenames when final_filename is set; doesnt log saving of key metric checkpoint #6024

@jak0bw

Description

@jak0bw

Describe the bug
When final_filename is set during CheckpointSaver creation it still uses the default name for logging. The correct (the set name) is used for saving. If save key metric is set, the saving of best metric Checkpoint is not logged at all

To Reproduce
Steps to reproduce the behavior:

  1. Create CheckpointSaver and set final_filename, save_key_metric etc. (see screenshot)
  2. Run CheckpointSaver until Checkpoint is saved and look at logged output

Expected behavior
Use correct filename for saved final checkpoint and log saving/deletion of best metric checkpoint

Screenshots
image
image

Environment

================================
Printing MONAI config...

MONAI version: 1.1.0
Numpy version: 1.24.2
Pytorch version: 1.13.1+cu117
MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
MONAI rev id: a2ec375
MONAI file: C:\Users\jakob\miniconda3\envs\sr3915\lib\site-packages\monai_init_.py

Optional dependencies:
Pytorch Ignite version: 0.4.10
Nibabel version: 5.0.0
scikit-image version: 0.19.3
Pillow version: 9.4.0
Tensorboard version: NOT INSTALLED or UNKNOWN VERSION.
gdown version: NOT INSTALLED or UNKNOWN VERSION.
TorchVision version: 0.14.1+cu117
tqdm version: 4.64.1
lmdb version: NOT INSTALLED or UNKNOWN VERSION.
psutil version: 5.9.4
pandas version: 1.5.3
einops version: NOT INSTALLED or UNKNOWN VERSION.
transformers version: 4.26.0
mlflow version: NOT INSTALLED or UNKNOWN VERSION.
pynrrd version: NOT INSTALLED or UNKNOWN VERSION.

For details about installing the optional dependencies, please visit:
https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies

================================
Printing system config...

System: Windows
Win32 version: ('10', '10.0.19045', 'SP0', 'Multiprocessor Free')
Win32 edition: Core
Platform: Windows-10-10.0.19045-SP0
Processor: Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
Machine: AMD64
Python version: 3.9.15
Process name: python.exe
Command: ['C:\Users\jakob\miniconda3\envs\sr3915\python.exe', '-c', 'import monai; monai.config.print_debug_info()']
Open files: [popenfile(path='C:\Program Files\WindowsApps\Microsoft.LanguageExperiencePackde-DE_19041.56.184.0_neutral__8wekyb3d8bbwe\Windows\System32\de-DE\tzres.dll.mui', fd=-1), popenfile(path='C:\Program Files\WindowsApps\Microsoft.LanguageExperiencePackde-DE_19041.56.184.0_neutral__8wekyb3d8bbwe\Windows\System32\de-DE\kernel32.dll.mui', fd=-1), popenfile(path='C:\Program Files\WindowsApps\Microsoft.LanguageExperiencePackde-DE_19041.56.184.0_neutral__8wekyb3d8bbwe\Windows\System32\de-DE\KernelBase.dll.mui', fd=-1)]
Num physical CPUs: 4
Num logical CPUs: 8
Num usable CPUs: 8
CPU usage (%): [8.1, 1.3, 8.5, 2.1, 4.3, 1.7, 4.3, 78.2]
CPU freq. (MHz): 4001
Load avg. in last 1, 5, 15 mins (%): [0.0, 0.0, 0.0]
Disk usage (%): 81.0
Avg. sensor temp. (Celsius): UNKNOWN for given OS
Total physical memory (GB): 16.0
Available memory (GB): 6.9
Used memory (GB): 9.0

================================
Printing GPU config...

Num GPUs: 1
Has CUDA: True
CUDA version: 11.7
cuDNN enabled: True
cuDNN version: 8500
Current device: 0
Library compiled for CUDA architectures: ['sm_37', 'sm_50', 'sm_60', 'sm_61', 'sm_70', 'sm_75', 'sm_80', 'sm_86', 'compute_37']
GPU 0 Name: NVIDIA GeForce GTX 1060 6GB
GPU 0 Is integrated: False
GPU 0 Is multi GPU board: False
GPU 0 Multi processor count: 10
GPU 0 Total memory (GB): 6.0
GPU 0 CUDA capability (maj.min): 6.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions