Moved the environment assumption from ConfigLoader to `_ProjectSett…

…ings` (#3311) * update release notes Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Remove the default env in config loader - add the env in tests Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * remove unncessary `default_run_env` Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * fix tests Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Added a couple of tests to make sure config loader standalone mode works Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * fix tests Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * lint Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix tests Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * update default Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * fix test coverage Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * fix test and lint Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * rename test to 'without_environment' to reflect the change of defaults Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * fix tests according to comments Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * update test to test arbitary env explicitly Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
kedro-org · Nov 28, 2023 · a0afce8 · a0afce8
1 parent 1bf8dab
commit a0afce8
Show file tree

Hide file tree

Showing 8 changed files with 433 additions and 66 deletions.
diff --git a/RELEASE.md b/RELEASE.md
@@ -8,7 +8,7 @@
 * Added the functionality to choose a merging strategy for config files loaded with `OmegaConfigLoader`.
 * Modified the mechanism of importing datasets, raise more explicit error when dependencies are missing.
 * Added validation for configuration file used to override run commands via the CLI.
-
+* Moved the default environment `base` and `local` from config loader to `_ProjectSettings`. This enables the use of config loader as a standalone class without affecting existing Kedro Framework users.
 ## Bug fixes and other changes
 * Added a new field `add-ons` to `pyproject.toml` when a project is created.
 * Reduced `spaceflights` data to minimise waiting times during tutorial execution.

diff --git a/...st_starter/{{ cookiecutter.repo_name }}/src/{{ cookiecutter.python_package }}/settings.py b/...st_starter/{{ cookiecutter.repo_name }}/src/{{ cookiecutter.python_package }}/settings.py
@@ -26,6 +26,8 @@
 # CONFIG_LOADER_CLASS = OmegaConfigLoader
 # Keyword arguments to pass to the `CONFIG_LOADER_CLASS` constructor.
 # CONFIG_LOADER_ARGS = {
+#       "base_env": "base",
+#       "default_run_env": "local",
 #       "config_patterns": {
 #           "spark" : ["spark*/"],
 #           "parameters": ["parameters*", "parameters*/**", "**/parameters*"],

diff --git a/kedro/config/omegaconf_config.py b/kedro/config/omegaconf_config.py
@@ -80,8 +80,8 @@ def __init__(  # noqa: PLR0913
         runtime_params: dict[str, Any] = None,
         *,
         config_patterns: dict[str, list[str]] = None,
-        base_env: str = "base",
-        default_run_env: str = "local",
+        base_env: str = None,
+        default_run_env: str = None,
         custom_resolvers: dict[str, Callable] = None,
         merge_strategy: dict[str, str] = None,
     ):
@@ -102,8 +102,8 @@ def __init__(  # noqa: PLR0913
             custom_resolvers: A dictionary of custom resolvers to be registered. For more information,
              see here: https://omegaconf.readthedocs.io/en/2.3_branch/custom_resolvers.html#custom-resolvers
         """
-        self.base_env = base_env
-        self.default_run_env = default_run_env
+        self.base_env = base_env or ""
+        self.default_run_env = default_run_env or ""
         self.merge_strategy = merge_strategy or {}
 
         self.config_patterns = {

diff --git a/kedro/framework/project/__init__.py b/kedro/framework/project/__init__.py
@@ -105,7 +105,9 @@ class _ProjectSettings(LazySettings):
         "CONFIG_LOADER_CLASS",
         default=_get_default_class("kedro.config.OmegaConfigLoader"),
     )
-    _CONFIG_LOADER_ARGS = Validator("CONFIG_LOADER_ARGS", default={})
+    _CONFIG_LOADER_ARGS = Validator(
+        "CONFIG_LOADER_ARGS", default={"base_env": "base", "default_run_env": "local"}
+    )
     _DATA_CATALOG_CLASS = _IsSubclassValidator(
         "DATA_CATALOG_CLASS", default=_get_default_class("kedro.io.DataCatalog")
     )

diff --git a/...es/project/{{ cookiecutter.repo_name }}/src/{{ cookiecutter.python_package }}/settings.py b/...es/project/{{ cookiecutter.repo_name }}/src/{{ cookiecutter.python_package }}/settings.py
@@ -29,6 +29,8 @@
 
 # Keyword arguments to pass to the `CONFIG_LOADER_CLASS` constructor.
 # CONFIG_LOADER_ARGS = {
+#       "base_env": "base",
+#       "default_run_env": "local",
 #       "config_patterns": {
 #           "spark" : ["spark*/"],
 #           "parameters": ["parameters*", "parameters*/**", "**/parameters*"],