Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

is there way to make both --opt-with-dash and environment variable work at the same time? #469

Open
braindevices opened this issue Nov 5, 2024 · 0 comments
Assignees

Comments

@braindevices
Copy link

braindevices commented Nov 5, 2024

test using #468

import os
import traceback
from pydantic import AliasChoices, AliasGenerator, BaseModel
from pydantic_settings import BaseSettings, SettingsConfigDict, version
import sys
import pydantic.version

class SubModel(BaseModel):
    v1: str = "default"
    v2: bytes = b"hello"
    v3: int


class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_prefix="MYTEST_",
        env_nested_delimiter="__",
        cli_parse_args=True,
        nested_model_default_partial_update=True,
        alias_generator=AliasGenerator(lambda s: s.replace('_', '-'))
    )

    v0: str = "ok"
    sub_model: SubModel = SubModel(v1="top default", v3=33)


class Settings1(BaseSettings):
    model_config = SettingsConfigDict(
        env_prefix="MYTEST_",
        env_nested_delimiter="__",
        cli_parse_args=True,
        nested_model_default_partial_update=True
    )

    v0: str = "ok"
    sub_model: SubModel = SubModel(v1="top default", v3=33)


class Settings2(BaseSettings):
    model_config = SettingsConfigDict(
        env_prefix="MYTEST_",
        env_nested_delimiter="__",
        cli_parse_args=True,
        extra="ignore",
        nested_model_default_partial_update=True,
        alias_generator=AliasGenerator(validation_alias=lambda field_name: AliasChoices(field_name, field_name.replace("_", "-")))
    )

    v0: str = "ok"
    sub_model: SubModel = SubModel(v1="top default", v3=33)


def help(cls: BaseSettings):
    sys.argv = [
        cls.__name__,
        "-h"
    ]
    try:
        cls()
    except SystemExit:
        print("sys.exit() was called and handled.")
    print("After exit")

def test_cli(cls: BaseSettings, dash: bool, disable_cli: bool = False):
    sys.argv = [
        'setting',
        *(["--sub-model.v1=cli" if dash else "--sub_model.v1=cli"] if not disable_cli else [])
    ]
    print("argv:", sys.argv)
    # for k, v in os.environ.items():
    #     if k.startswith("MYTEST_"):
    #         print(k, v)
    try:
        a = cls()
        print(f"in test cli: {cls.__name__}()=", a)
    except Exception:
        traceback.print_exc()

def main():
    print("pydnatic version:", pydantic.version.VERSION)
    print("pydantic_settings version:", version.VERSION)

    help(Settings)
    help(Settings1)
    help(Settings2)

    # test_cli(Settings, dash=True)
    os.environ["V0"] = "env no prefix"
    os.environ["SUB_MODEL_V1"] = "env no prefix"
    os.environ["SUB_MODEL_V2"] = "env no prefix"
    os.environ["SUB_MODEL__V1"] = "env__no__prefix"
    os.environ["SUB_MODEL__V2"] = "env__no__prefix"
    os.environ["MYTEST_V0"] = "env with prefix"
    os.environ["MYTEST_SUB_MODEL__V1"] = "env with prefix"
    os.environ["MYTEST_SUB_MODEL__V2"] = "env with prefix"
    test_cli(Settings1, dash=False)
    test_cli(Settings, dash=True)
    test_cli(Settings2, dash=False)
    test_cli(Settings2, dash=True)
    test_cli(Settings2, dash=False, disable_cli=True)


if __name__ == "__main__":
    main()

output:

pydnatic version: 2.9.2
pydantic_settings version: 2.6.1
usage: Settings [-h] [--v0 str] [--sub-model JSON] [--sub-model.v1 str] [--sub-model.v2 bytes] [--sub-model.v3 int]

options:
  -h, --help            show this help message and exit
  --v0 str              (default: ok)

sub-model options:
  --sub-model JSON      set sub-model from JSON string
  --sub-model.v1 str    (default: top default)
  --sub-model.v2 bytes  (default: b'hello')
  --sub-model.v3 int    (default: 33)
sys.exit() was called and handled.
After exit
usage: Settings1 [-h] [--v0 str] [--sub_model JSON] [--sub_model.v1 str] [--sub_model.v2 bytes] [--sub_model.v3 int]

options:
  -h, --help            show this help message and exit
  --v0 str              (default: ok)

sub_model options:
  --sub_model JSON      set sub_model from JSON string
  --sub_model.v1 str    (default: top default)
  --sub_model.v2 bytes  (default: b'hello')
  --sub_model.v3 int    (default: 33)
sys.exit() was called and handled.
After exit
usage: Settings2 [-h] [--v0 str] [--sub_model JSON] [--sub_model.v1 str] [--sub_model.v2 bytes] [--sub_model.v3 int]

options:
  -h, --help            show this help message and exit
  --v0 str              (default: ok)

sub_model options:
  --sub_model JSON, --sub-model JSON
                        set sub_model from JSON string
  --sub_model.v1 str, --sub-model.v1 str
                        (default: top default)
  --sub_model.v2 bytes, --sub-model.v2 bytes
                        (default: b'hello')
  --sub_model.v3 int, --sub-model.v3 int
                        (default: 33)

argv: ['setting', '--sub_model.v1=cli']
in test cli: Settings1()= v0='env with prefix' sub_model=SubModel(v1='cli', v2=b'env with prefix', v3=33)
argv: ['setting', '--sub-model.v1=cli']
in test cli: Settings()= v0='env no prefix' sub_model=SubModel(v1='cli', v2=b'hello', v3=33)
argv: ['setting', '--sub_model.v1=cli']
in test cli: Settings2()= v0='env no prefix' sub_model=SubModel(v1='top default', v2=b'hello', v3=33)
argv: ['setting', '--sub-model.v1=cli']
in test cli: Settings2()= v0='env no prefix' sub_model=SubModel(v1='top default', v2=b'hello', v3=33)
argv: ['setting']
in test cli: Settings2()= v0='env no prefix' sub_model=SubModel(v1='top default', v2=b'hello', v3=33)

So the Settings, use alias_generator=AliasGenerator(lambda s: s.replace('_', '-')) which cause the env_prefix has no effect, the sub_model become sub-model so it won't find any sub_model in env.

Settings1, doe not have the alias generator, so the env options works normally.

Settings2, attempt to use alias_generator=AliasGenerator(validation_alias=lambda field_name: AliasChoices(field_name, field_name.replace("_", "-"))) to allow both --sub-model and env_* options to work. But it raise error without extra="ignore":

pydantic_core._pydantic_core.ValidationError: 1 validation error for Settings2
sub-model
  Extra inputs are not permitted [type=extra_forbidden, input_value={'v1': 'env__no__prefix', 'v2': 'env__no__prefix'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.9/v/extra_forbidden

with extra="ignore", both '--sub_model.v1=cli' and '--sub-model.v1=cli' has no effect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants