Is "Prepare a config file for the CLI" out of date?

[zengchang233]

Bug description

I would like to generate a config file for my project. However, when I followed the example described in Prepare a config file for the CLI, the error was raised. Is the content in the document out of date?

• error

python prepare_config.py fit --model DemoModel --print_config   
usage: prepare_config.py [options] fit [-h] [-c CONFIG] [--print_config[=flags]] [--seed_everything SEED_EVERYTHING] [--trainer CONFIG] [--trainer.accelerator.help CLASS_PATH_OR_NAME] [--trainer.accelerator ACCELERATOR] [--trainer.strategy.help CLASS_PATH_OR_NAME] [--trainer.strategy STRATEGY] [--trainer.devices DEVICES] [--trainer.num_nodes NUM_NODES]
                                       [--trainer.precision PRECISION] [--trainer.logger.help CLASS_PATH_OR_NAME] [--trainer.logger LOGGER] [--trainer.callbacks.help CLASS_PATH_OR_NAME] [--trainer.callbacks CALLBACKS] [--trainer.fast_dev_run FAST_DEV_RUN] [--trainer.max_epochs MAX_EPOCHS] [--trainer.min_epochs MIN_EPOCHS] [--trainer.max_steps MAX_STEPS]
                                       [--trainer.min_steps MIN_STEPS] [--trainer.max_time MAX_TIME] [--trainer.limit_train_batches LIMIT_TRAIN_BATCHES] [--trainer.limit_val_batches LIMIT_VAL_BATCHES] [--trainer.limit_test_batches LIMIT_TEST_BATCHES] [--trainer.limit_predict_batches LIMIT_PREDICT_BATCHES] [--trainer.overfit_batches OVERFIT_BATCHES]
                                       [--trainer.val_check_interval VAL_CHECK_INTERVAL] [--trainer.check_val_every_n_epoch CHECK_VAL_EVERY_N_EPOCH] [--trainer.num_sanity_val_steps NUM_SANITY_VAL_STEPS] [--trainer.log_every_n_steps LOG_EVERY_N_STEPS] [--trainer.enable_checkpointing {true,false,null}] [--trainer.enable_progress_bar {true,false,null}]
                                       [--trainer.enable_model_summary {true,false,null}] [--trainer.accumulate_grad_batches ACCUMULATE_GRAD_BATCHES] [--trainer.gradient_clip_val GRADIENT_CLIP_VAL] [--trainer.gradient_clip_algorithm GRADIENT_CLIP_ALGORITHM] [--trainer.deterministic DETERMINISTIC] [--trainer.benchmark {true,false,null}] [--trainer.inference_mode {true,false}]
                                       [--trainer.use_distributed_sampler {true,false}] [--trainer.profiler.help CLASS_PATH_OR_NAME] [--trainer.profiler PROFILER] [--trainer.detect_anomaly {true,false}] [--trainer.barebones {true,false}] [--trainer.plugins.help CLASS_PATH_OR_NAME] [--trainer.plugins PLUGINS] [--trainer.sync_batchnorm {true,false}]
                                       [--trainer.reload_dataloaders_every_n_epochs RELOAD_DATALOADERS_EVERY_N_EPOCHS] [--trainer.default_root_dir DEFAULT_ROOT_DIR] [--model CONFIG] [--model.out_dim OUT_DIM] [--model.learning_rate LEARNING_RATE] [--optimizer.help CLASS_PATH_OR_NAME] [--optimizer CONFIG | CLASS_PATH_OR_NAME | .INIT_ARG_NAME VALUE]
                                       [--lr_scheduler.help CLASS_PATH_OR_NAME] [--lr_scheduler CONFIG | CLASS_PATH_OR_NAME | .INIT_ARG_NAME VALUE] [--ckpt_path CKPT_PATH]
error: Parser key "model":
Unable to load config 'DemoModel'
  - Parser key "model": Unable to load config "DemoModel"

What version are you seeing the problem on?

v2.2

How to reproduce the bug

from lightning.pytorch.cli import LightningCLI
from lightning.pytorch.demos.boring_classes import DemoModel, BoringDataModule

def main():
    cli = LightningCLI(DemoModel, BoringDataModule)

if __name__ == "__main__":
    main()

python prepare_config.py fit --model DemoModel --print_config 

Error messages and logs

python prepare_config.py fit --model DemoModel --print_config   
usage: prepare_config.py [options] fit [-h] [-c CONFIG] [--print_config[=flags]] [--seed_everything SEED_EVERYTHING] [--trainer CONFIG] [--trainer.accelerator.help CLASS_PATH_OR_NAME] [--trainer.accelerator ACCELERATOR] [--trainer.strategy.help CLASS_PATH_OR_NAME] [--trainer.strategy STRATEGY] [--trainer.devices DEVICES] [--trainer.num_nodes NUM_NODES]
                                       [--trainer.precision PRECISION] [--trainer.logger.help CLASS_PATH_OR_NAME] [--trainer.logger LOGGER] [--trainer.callbacks.help CLASS_PATH_OR_NAME] [--trainer.callbacks CALLBACKS] [--trainer.fast_dev_run FAST_DEV_RUN] [--trainer.max_epochs MAX_EPOCHS] [--trainer.min_epochs MIN_EPOCHS] [--trainer.max_steps MAX_STEPS]
                                       [--trainer.min_steps MIN_STEPS] [--trainer.max_time MAX_TIME] [--trainer.limit_train_batches LIMIT_TRAIN_BATCHES] [--trainer.limit_val_batches LIMIT_VAL_BATCHES] [--trainer.limit_test_batches LIMIT_TEST_BATCHES] [--trainer.limit_predict_batches LIMIT_PREDICT_BATCHES] [--trainer.overfit_batches OVERFIT_BATCHES]
                                       [--trainer.val_check_interval VAL_CHECK_INTERVAL] [--trainer.check_val_every_n_epoch CHECK_VAL_EVERY_N_EPOCH] [--trainer.num_sanity_val_steps NUM_SANITY_VAL_STEPS] [--trainer.log_every_n_steps LOG_EVERY_N_STEPS] [--trainer.enable_checkpointing {true,false,null}] [--trainer.enable_progress_bar {true,false,null}]
                                       [--trainer.enable_model_summary {true,false,null}] [--trainer.accumulate_grad_batches ACCUMULATE_GRAD_BATCHES] [--trainer.gradient_clip_val GRADIENT_CLIP_VAL] [--trainer.gradient_clip_algorithm GRADIENT_CLIP_ALGORITHM] [--trainer.deterministic DETERMINISTIC] [--trainer.benchmark {true,false,null}] [--trainer.inference_mode {true,false}]
                                       [--trainer.use_distributed_sampler {true,false}] [--trainer.profiler.help CLASS_PATH_OR_NAME] [--trainer.profiler PROFILER] [--trainer.detect_anomaly {true,false}] [--trainer.barebones {true,false}] [--trainer.plugins.help CLASS_PATH_OR_NAME] [--trainer.plugins PLUGINS] [--trainer.sync_batchnorm {true,false}]
                                       [--trainer.reload_dataloaders_every_n_epochs RELOAD_DATALOADERS_EVERY_N_EPOCHS] [--trainer.default_root_dir DEFAULT_ROOT_DIR] [--model CONFIG] [--model.out_dim OUT_DIM] [--model.learning_rate LEARNING_RATE] [--optimizer.help CLASS_PATH_OR_NAME] [--optimizer CONFIG | CLASS_PATH_OR_NAME | .INIT_ARG_NAME VALUE]
                                       [--lr_scheduler.help CLASS_PATH_OR_NAME] [--lr_scheduler CONFIG | CLASS_PATH_OR_NAME | .INIT_ARG_NAME VALUE] [--ckpt_path CKPT_PATH]
error: Parser key "model":
Unable to load config 'DemoModel'
  - Parser key "model": Unable to load config "DemoModel"

Environment

Current environment

• CUDA:
  - GPU:
  - NVIDIA GeForce RTX 4090
  - NVIDIA GeForce RTX 4090
  - NVIDIA GeForce RTX 4090
  - NVIDIA GeForce RTX 4090
  - NVIDIA GeForce RTX 4090
  - NVIDIA GeForce RTX 4090
  - NVIDIA GeForce RTX 4090
  - NVIDIA GeForce RTX 4090
  - available: True
  - version: 12.1
• Lightning:
  - lightning: 2.2.1
  - lightning-utilities: 0.11.2
  - pytorch-lightning: 2.2.4
  - torch: 2.2.0+cu121
  - torch-complex: 0.4.3
  - torchaudio: 2.2.0+cu121
  - torchdiffeq: 0.2.3
  - torchmetrics: 1.4.0
  - torchvision: 0.17.0+cu121
• Packages:
  - absl-py: 2.1.0
  - aiohttp: 3.9.5
  - aiosignal: 1.3.1
  - annotated-types: 0.6.0
  - antlr4-python3-runtime: 4.9.3
  - anyio: 4.3.0
  - appdirs: 1.4.4
  - argon2-cffi: 23.1.0
  - argon2-cffi-bindings: 21.2.0
  - arrow: 1.3.0
  - asteroid-filterbanks: 0.4.0
  - asttokens: 2.4.1
  - async-lru: 2.0.4
  - async-timeout: 4.0.3
  - attrs: 23.2.0
  - audioread: 3.0.1
  - babel: 2.15.0
  - beautifulsoup4: 4.12.3
  - bleach: 6.1.0
  - cachetools: 5.3.3
  - certifi: 2024.2.2
  - cffi: 1.16.0
  - charset-normalizer: 3.3.2
  - ci-sdr: 0.0.2
  - click: 8.1.7
  - colorama: 0.4.6
  - comm: 0.2.2
  - configargparse: 1.7
  - contourpy: 1.2.1
  - ctc-segmentation: 1.7.4
  - cycler: 0.12.1
  - cython: 3.0.10
  - debugpy: 1.8.1
  - decorator: 5.1.1
  - defusedxml: 0.7.1
  - distance: 0.1.3
  - docker-pycreds: 0.4.0
  - docstring-parser: 0.16
  - editdistance: 0.8.1
  - einops: 0.8.0
  - encodec: 0.1.1
  - espnet: 202402
  - espnet-tts-frontend: 0.0.3
  - exceptiongroup: 1.2.1
  - executing: 2.0.1
  - fast-bss-eval: 0.1.3
  - fastjsonschema: 2.19.1
  - filelock: 3.14.0
  - fonttools: 4.51.0
  - fqdn: 1.5.1
  - frozenlist: 1.4.1
  - fsspec: 2024.3.1
  - g2p-en: 2.1.0
  - gitdb: 4.0.11
  - gitpython: 3.1.43
  - google-auth: 2.29.0
  - google-auth-oauthlib: 0.4.6
  - grpcio: 1.63.0
  - h11: 0.14.0
  - h5py: 3.11.0
  - httpcore: 1.0.5
  - httpx: 0.27.0
  - huggingface-hub: 0.20.1
  - humanfriendly: 10.0
  - hydra-core: 1.3.2
  - idna: 3.7
  - importlib-metadata: 4.13.0
  - importlib-resources: 6.4.0
  - inflect: 7.0.0
  - ipdb: 0.13.13
  - ipykernel: 6.29.4
  - ipython: 8.24.0
  - ipywidgets: 8.1.2
  - isoduration: 20.11.0
  - jaconv: 0.3.4
  - jamo: 0.4.1
  - jedi: 0.19.1
  - jinja2: 3.1.4
  - joblib: 1.4.2
  - json5: 0.9.25
  - jsonargparse: 4.28.0
  - jsonpointer: 2.4
  - jsonschema: 4.22.0
  - jsonschema-specifications: 2023.12.1
  - julius: 0.2.7
  - jupyter: 1.0.0
  - jupyter-client: 8.6.1
  - jupyter-console: 6.6.3
  - jupyter-core: 5.7.2
  - jupyter-events: 0.10.0
  - jupyter-lsp: 2.2.5
  - jupyter-server: 2.14.0
  - jupyter-server-terminals: 0.5.3
  - jupyterlab: 4.2.0
  - jupyterlab-pygments: 0.3.0
  - jupyterlab-server: 2.27.1
  - jupyterlab-widgets: 3.0.10
  - kaldiio: 2.18.0
  - kiwisolver: 1.4.5
  - librosa: 0.9.2
  - lightning: 2.2.1
  - lightning-utilities: 0.11.2
  - llvmlite: 0.42.0
  - markdown: 3.6
  - markupsafe: 2.1.5
  - matplotlib: 3.8.4
  - matplotlib-inline: 0.1.7
  - mistune: 3.0.2
  - mpmath: 1.3.0
  - multidict: 6.0.5
  - nbclient: 0.10.0
  - nbconvert: 7.16.4
  - nbformat: 5.10.4
  - nest-asyncio: 1.6.0
  - networkx: 3.3
  - nltk: 3.8.1
  - notebook: 7.2.0
  - notebook-shim: 0.2.4
  - numba: 0.59.1
  - numpy: 1.23.5
  - nvidia-cublas-cu12: 12.1.3.1
  - nvidia-cuda-cupti-cu12: 12.1.105
  - nvidia-cuda-nvrtc-cu12: 12.1.105
  - nvidia-cuda-runtime-cu12: 12.1.105
  - nvidia-cudnn-cu12: 8.9.2.26
  - nvidia-cufft-cu12: 11.0.2.54
  - nvidia-curand-cu12: 10.3.2.106
  - nvidia-cusolver-cu12: 11.4.5.107
  - nvidia-cusparse-cu12: 12.1.0.106
  - nvidia-ml-py: 12.535.161
  - nvidia-nccl-cu12: 2.19.3
  - nvidia-nvjitlink-cu12: 12.4.127
  - nvidia-nvtx-cu12: 12.1.105
  - nvitop: 1.3.2
  - oauthlib: 3.2.2
  - omegaconf: 2.3.0
  - opt-einsum: 3.3.0
  - overrides: 7.7.0
  - packaging: 24.0
  - pandas: 2.2.2
  - pandocfilters: 1.5.1
  - parso: 0.8.4
  - pathtools: 0.1.2
  - pexpect: 4.9.0
  - pillow: 10.3.0
  - pip: 24.0
  - platformdirs: 4.2.1
  - pooch: 1.8.1
  - pretty-errors: 1.2.25
  - prometheus-client: 0.20.0
  - prompt-toolkit: 3.0.43
  - protobuf: 3.19.6
  - psutil: 5.9.8
  - ptyprocess: 0.7.0
  - pure-eval: 0.2.2
  - pyasn1: 0.6.0
  - pyasn1-modules: 0.4.0
  - pycparser: 2.22
  - pydantic: 2.7.1
  - pydantic-core: 2.18.2
  - pygments: 2.18.0
  - pyparsing: 3.1.2
  - pypinyin: 0.44.0
  - python-dateutil: 2.9.0.post0
  - python-json-logger: 2.0.7
  - pytorch-lightning: 2.2.4
  - pytz: 2024.1
  - pyworld: 0.3.4
  - pyyaml: 6.0.1
  - pyzmq: 26.0.3
  - qtconsole: 5.5.2
  - qtpy: 2.4.1
  - referencing: 0.35.1
  - regex: 2024.5.10
  - requests: 2.31.0
  - requests-oauthlib: 2.0.0
  - resampy: 0.4.3
  - rfc3339-validator: 0.1.4
  - rfc3986-validator: 0.1.1
  - rpds-py: 0.18.1
  - rsa: 4.9
  - safetensors: 0.4.3
  - scikit-learn: 1.4.2
  - scipy: 1.13.0
  - send2trash: 1.8.3
  - sentencepiece: 0.1.97
  - sentry-sdk: 2.1.1
  - setproctitle: 1.3.3
  - setuptools: 69.5.1
  - six: 1.16.0
  - smmap: 5.0.1
  - sniffio: 1.3.1
  - soundfile: 0.12.1
  - soupsieve: 2.5
  - stack-data: 0.6.3
  - sympy: 1.12
  - tensorboard: 2.9.1
  - tensorboard-data-server: 0.6.1
  - tensorboard-plugin-wit: 1.8.1
  - termcolor: 2.4.0
  - terminado: 0.18.1
  - threadpoolctl: 3.5.0
  - tinycss2: 1.3.0
  - tokenizers: 0.13.3
  - tomli: 2.0.1
  - torch: 2.2.0+cu121
  - torch-complex: 0.4.3
  - torchaudio: 2.2.0+cu121
  - torchdiffeq: 0.2.3
  - torchmetrics: 1.4.0
  - torchvision: 0.17.0+cu121
  - tornado: 6.4
  - tqdm: 4.66.4
  - traitlets: 5.14.3
  - transformers: 4.31.0
  - triton: 2.2.0
  - typeguard: 2.13.3
  - types-python-dateutil: 2.9.0.20240316
  - typeshed-client: 2.5.1
  - typing-extensions: 4.11.0
  - tzdata: 2024.1
  - unidecode: 1.3.8
  - uri-template: 1.3.0
  - urllib3: 2.2.1
  - wandb: 0.15.8
  - wcwidth: 0.2.13
  - webcolors: 1.13
  - webencodings: 0.5.1
  - websocket-client: 1.8.0
  - werkzeug: 3.0.3
  - wheel: 0.43.0
  - widgetsnbextension: 4.0.10
  - xformers: 0.0.24
  - yarl: 1.9.4
  - zipp: 3.18.1
• System:
  - OS: Linux
  - architecture:
  - 64bit
  - ELF
  - processor: x86_64
  - python: 3.10.14
  - release: 5.10.101-1.el8.ssai.x86_64
  - version: Proposal for help #1 SMP Mon Apr 25 12:35:34 UTC 2022

More info

No response

[synapticarbors]

If you define your cli as cli = LightningCLI(pl.LightningModule, BoringDataModule, subclass_mode_model=True) or cli = LightningCLI(datamodule_class=BoringDataModule) and then run:

python prepare_config.py fit --model DemoModel --print_config

you get the expected result. If you define the cli as you show, then you need to drop the --model DemoModel for it not to error and give you the expected result.

I think what the documentation is suggesting when it shows the example you're using is that if no model is passed to the LightningCLI -- "By default, no model is selected, meaning the printed config will not include model settings. To get a config with the default values of a particular model would be: python main.py fit --model DemoModel --print_config". I think what isn't clear is that the cli setup is different from what is shown directly above with the hardcoded DemoModel passed in.

[zengchang233]

If you define your cli as cli = LightningCLI(pl.LightningModule, BoringDataModule, subclass_mode_model=True) or cli = LightningCLI(datamodule_class=BoringDataModule) and then run:

python prepare_config.py fit --model DemoModel --print_config

you get the expected result. If you define the cli as you show, then you need to drop the --model DemoModel for it not to error and give you the expected result.

I think what the documentation is suggesting when it shows the example you're using is that if no model is passed to the LightningCLI -- "By default, no model is selected, meaning the printed config will not include model settings. To get a config with the default values of a particular model would be: python main.py fit --model DemoModel --print_config". I think what isn't clear is that the cli setup is different from what is shown directly above with the hardcoded DemoModel passed in.

@synapticarbors Thanks! Now I totally get it.