Skip to content

Commit

Permalink
Add documentation about correct usage of configure_project (#3707)
Browse files Browse the repository at this point in the history
* add docs

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* fix typo

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* whitespace

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>
  • Loading branch information
noklam authored Jul 9, 2024
1 parent 40c73ab commit 87e2f02
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 1 deletion.
4 changes: 3 additions & 1 deletion RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ Many thanks to the following Kedroids for contributing PRs to this release:
## Documentation changes
* Updated the documentation for deploying a Kedro project with Astronomer Airflow.
* Used `kedro-sphinx-theme` for documentation.

* Add mentions about correct usage of `configure_project` with `multiprocessing`.
*
# Release 0.19.4

## Major features and improvements
Expand Down Expand Up @@ -82,6 +83,7 @@ Many thanks to the following Kedroids for contributing PRs to this release:
* Added documentation on best practices for testing nodes and pipelines.
* Clarified docs around using custom resolvers without a full Kedro project.


## Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
* [ondrejzacha](https://github.com/ondrejzacha)
Expand Down
5 changes: 5 additions & 0 deletions docs/source/kedro_project_setup/session.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,8 @@ This function uses `configure_project`, and additionally reads metadata from `py
### `configure_project`

This function reads `settings.py` and `pipeline_registry.py` and registers the configuration before Kedro's run starts. If you have a packaged Kedro project, you only need to run `configure_project` before executing your pipeline.

#### ValueError: Package name not found
> ValueError: Package name not found. Make sure you have configured the project using 'bootstrap_project'. This should happen automatically if you are using Kedro command line interface.
If you are using `multiprocessing`, you need to be careful about this. Depending on your Operating System, you may have [different default](https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods). If the processes are `spawn`, Python will re-import all the modules in each process and thus you need to run `configure_project` again at the start of the new process. For example, this is how Kedro handle this in `ParallelRunner`(https://github.com/kedro-org/kedro/blob/9e883e6a0ba40e3db4497b234dcb3801258e8396/kedro/runner/parallel_runner.py#L84-L85)

0 comments on commit 87e2f02

Please sign in to comment.