Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MVP for "add-ons" flow within kedro new CLI command #2987

Merged
merged 92 commits into from
Oct 10, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
3726ed7
Update prompts.yml
SajidAlamQB Aug 30, 2023
592afb5
Update starters.py
SajidAlamQB Aug 31, 2023
dd27d67
add post_gen_project in cookiecutter hooks
SajidAlamQB Sep 4, 2023
ad127d8
add confirmation message for the options selected
SajidAlamQB Sep 4, 2023
868816c
Merge branch 'develop' into feat/add-ons-cli-flow
SajidAlamQB Sep 5, 2023
52e99dd
Update post_gen_project.py
SajidAlamQB Sep 5, 2023
5108223
changes based on review
SajidAlamQB Sep 5, 2023
6cc090c
Merge branch 'develop' into feat/add-ons-cli-flow
lrcouto Sep 7, 2023
a0fe4a2
Lint
Sep 8, 2023
4fb0671
Remove documentation requirements
Sep 11, 2023
c1a1f7d
Remove testing requirements
Sep 11, 2023
ba01bb2
Remove leftover linting requirements
Sep 11, 2023
0f92782
Lint
Sep 11, 2023
bc0c172
Add add-on requirements when an addon is selected
lrcouto Sep 12, 2023
52ddf73
Correct file path
lrcouto Sep 12, 2023
ded026c
Update tests with new default template files number
Sep 12, 2023
f216847
Update tests with add-ons argument
Sep 12, 2023
ee668fd
Make lint
Sep 12, 2023
a500a7a
Lint
Sep 12, 2023
b88f01c
Make tests use all add-ons by default
Sep 12, 2023
fa379a0
Merge branch 'develop' into feat/add-ons-cli-flow
AhdraMeraliQB Sep 12, 2023
f39ebad
Make unit tests use no add-ons by default
Sep 12, 2023
4a4164d
Add installing project dependencies to e2e tests
Sep 12, 2023
ea40e0e
Add linting requirements, organize code
lrcouto Sep 12, 2023
d56cc31
Refactor test for all add on options
Sep 13, 2023
6208edb
Add test to check parsing add-ons
Sep 13, 2023
66c9f36
Add scaffolding for add-ons tests
Sep 13, 2023
58f211f
Change name of test class
Sep 13, 2023
119465d
Correct test names
Sep 13, 2023
5434858
Correct tests directory
Sep 13, 2023
225f26a
Clean up success message
Sep 13, 2023
2f866f7
Merge branch 'develop' into feat/add-ons-cli-flow
lrcouto Sep 13, 2023
06509fd
Fix logging option
Sep 13, 2023
e73fe7e
Update lint add-on logic
Sep 13, 2023
bb4e358
Ensure add-ons message only shows when add-ons are configurable
Sep 13, 2023
1a52a3c
Add requirement checks to tests
lrcouto Sep 14, 2023
51d0c12
Refactor unit tests
Sep 14, 2023
39761d0
Add validation to add ons in config file
Sep 14, 2023
da14a74
Refactor add-ons flow script
lrcouto Sep 18, 2023
0160bbe
Pass through correct repo name in test
Sep 19, 2023
8609597
Merge branch 'develop' into feat/add-ons-cli-flow
AhdraMeraliQB Sep 19, 2023
603fc93
Clean up and clarify text
Sep 19, 2023
aae48cb
Wrap hook script inside main function
Sep 19, 2023
ca6297a
Revert displayed default
Sep 19, 2023
4a9cdcb
Add range validation
Sep 19, 2023
3711567
Add tests for add-on range validation
Sep 19, 2023
e037441
Apply suggestions from code review
AhdraMeraliQB Sep 19, 2023
08567d3
Update kedro/templates/project/hooks/utils.py
AhdraMeraliQB Sep 19, 2023
7fca19e
Apply suggestions from code review
Sep 19, 2023
47d935e
Remove traceback from add-on validation - review suggestion
Sep 19, 2023
e3c7de5
Output add-on names when selected (via CLI)
Sep 19, 2023
fd0dabd
Merge branch 'develop' into feat/add-ons-cli-flow
AhdraMeraliQB Sep 19, 2023
f8b8920
Merge branch 'develop' into feat/add-ons-cli-flow
lrcouto Sep 20, 2023
57cf29c
Revert 47d935e and fix tests
Sep 21, 2023
951015d
Fix test errors
Sep 21, 2023
d280307
Try remove validation
Sep 21, 2023
8a51b26
Add validation back
Sep 21, 2023
abd467f
Merge branch 'develop' into feat/add-ons-cli-flow
AhdraMeraliQB Sep 22, 2023
47d063f
Add config file input validation
Sep 22, 2023
488eca9
Add /site-packages/ to coverage report omit
Sep 22, 2023
c5e3050
Lint
Sep 22, 2023
472a57c
Remove duplicate error message
Sep 22, 2023
e0e5077
Lint
Sep 22, 2023
e89ed0c
Merge develop into feat/add-ons-cli-flow
Sep 22, 2023
406393d
Remove suppression
Sep 27, 2023
36d72dc
Merge develop into fead/add-ons-flow
Oct 4, 2023
006555e
Merge develop into fead/add-ons-flow
Oct 4, 2023
49b72b0
Prep for merge
Oct 6, 2023
e4138d3
Remove suppression
Sep 27, 2023
a3403de
Merge develop into feat/add-ons-cli-flow
Oct 6, 2023
3579326
Apply changes from code review
Oct 6, 2023
2ad5078
Add clarification to error messages
Oct 9, 2023
608af72
Remove suppression
Sep 27, 2023
78c89e4
Merge develop into feat/add-ons-cli-flow
Oct 9, 2023
01ad5c0
Correct files in starter template
Oct 9, 2023
574cf01
Fix broken link
Oct 9, 2023
902b858
Add project teardown
Oct 9, 2023
c819148
Merge develop into feat/add-ons-cli-flow
Oct 9, 2023
618d4c0
Try be more direct
Oct 9, 2023
1b9c23f
Try be more direct pt 2
Oct 9, 2023
566462c
Try be more direct pt 3
Oct 9, 2023
b5439d0
Try be more direct pt 4
Oct 9, 2023
a8c8f86
Fix clean-up
Oct 9, 2023
b679b6a
Fix type
Oct 9, 2023
90aad98
Apply suggestions from code review
AhdraMeraliQB Oct 9, 2023
4314f23
Lint
Oct 9, 2023
e3800ec
Add docstring
Oct 9, 2023
20a8e61
Add changes to RELEASE.md
Oct 9, 2023
c31a9f4
Merge branch 'develop' into feat/add-ons-cli-flow
AhdraMeraliQB Oct 9, 2023
f7518a1
Merge branch 'develop' into feat/add-ons-cli-flow
AhdraMeraliQB Oct 10, 2023
c292d0d
Merge branch 'develop' into feat/add-ons-cli-flow
AhdraMeraliQB Oct 10, 2023
1332139
Merge branch 'develop' into feat/add-ons-cli-flow
AhdraMeraliQB Oct 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions kedro/framework/cli/starters.py
Original file line number Diff line number Diff line change
Expand Up @@ -363,6 +363,11 @@ def _create_project(template_path: str, cookiecutter_args: dict[str, Any]):
python_package = extra_context.get(
"python_package", project_name.lower().replace(" ", "_").replace("-", "_")
)
add_ons = extra_context.get("add_ons")
click.secho(
f"\nYou have selected the following add-ons: '{add_ons}'"
)

click.secho(
f"\nThe project name '{project_name}' has been applied to: "
f"\n- The project title in {result_path}/README.md "
Expand Down
3 changes: 2 additions & 1 deletion kedro/templates/project/cookiecutter.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,6 @@
"project_name": "New Kedro Project",
"repo_name": "{{ cookiecutter.project_name.strip().replace(' ', '-').replace('_', '-').lower() }}",
"python_package": "{{ cookiecutter.project_name.strip().replace(' ', '_').replace('-', '_').lower() }}",
"kedro_version": "{{ cookiecutter.kedro_version }}"
"kedro_version": "{{ cookiecutter.kedro_version }}",
"add_ons": "none"
}
66 changes: 66 additions & 0 deletions kedro/templates/project/hooks/post_gen_project.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
import json
import os
SajidAlamQB marked this conversation as resolved.
Show resolved Hide resolved
import shutil


def parse_add_ons_input(add_ons_str):
"""Parse the add-ons input string.

Args:
add_ons_str: Input string from prompts.yml.

Returns:
list: List of selected add-ons as strings.
"""
if add_ons_str == "all":
return ["1", "2", "3", "4", "5"]
if add_ons_str == "none":
return []

# Split by comma
parts = add_ons_str.split(",")
SajidAlamQB marked this conversation as resolved.
Show resolved Hide resolved
selected = []

for part in parts:
if "-" in part:
start, end = part.split("-")
selected.extend(str(i) for i in range(int(start), int(end) + 1))
else:
selected.append(part.strip())

return selected


# Get the selected add-ons from cookiecutter
selected_add_ons = "{{ cookiecutter.add_ons }}"

# Parse the add-ons to get a list
selected_add_ons_list = parse_add_ons_input(selected_add_ons)

if "1" not in selected_add_ons_list: # If Linting not selected
setup_cfg_path = os.path.join(os.getcwd(), "setup.cfg")
if os.path.exists(setup_cfg_path):
os.remove(os.path.join(os.getcwd(), "setup.cfg"))
# TODO ADD CODE TO UPDATE SETUP.PY TO REMOVE LINTING DEPENDENCIES

if "2" not in selected_add_ons_list: # If Testing not selected
tests_path = os.path.join(os.getcwd(), "tests")
if os.path.exists(tests_path):
shutil.rmtree(os.path.join(os.getcwd(), "tests"))
# TODO ADD CODE TO UPDATE SETUP.PY TO REMOVE TEST DEPENDENCIES

if "3" not in selected_add_ons_list: # If Logging not selected
logging_yml_path = os.path.join(os.getcwd(), "logging.yml")
if os.path.exists(logging_yml_path):
os.remove(os.path.join(os.getcwd(), "logging.yml"))

if "4" not in selected_add_ons_list: # If Documentation not selected
docs_path = os.path.join(os.getcwd(), "docs")
if os.path.exists(docs_path):
shutil.rmtree(os.path.join(os.getcwd(), "docs"))
# TODO ADD CODE TO UPDATE SETUP.PY TO REMOVE DOCS DEPENDENCIES

if "5" not in selected_add_ons_list: # If Data Structure not selected
data_path = os.path.join(os.getcwd(), "data")
if os.path.exists(data_path):
shutil.rmtree(os.path.join(os.getcwd(), "data"))
19 changes: 19 additions & 0 deletions kedro/templates/project/prompts.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,22 @@
add_ons:
title: "Project Add-Ons"
text: |
Here you can select which add-ons you'd like to include.
Don't worry if you change your mind you can always add/remove these later.
AhdraMeraliQB marked this conversation as resolved.
Show resolved Hide resolved
To read more about these add-ons and what they do visit: kedro.org/{insert-documentation}

Add-Ons
1) Linting : Provides a basic linting set up with Flake8, Black and isort
AhdraMeraliQB marked this conversation as resolved.
Show resolved Hide resolved
2) Testing : Provides basic testing set up with pytest
3) Logging : Provides more logging options, environment specific,
4) Documentation: Provides basic documentations setup with Sphinx
5) Data Structure: Provides a directory structure for storing data

Which add-ons would you like to include in your project? [1-5/1,3/all/none]:
regex_validator: "^(all|none|(\\d(,\\d)*|(\\d-\\d)))$"
error_message: |
Invalid input. Please select valid options for add-ons using comma-separated values, ranges, or 'all/none'.

project_name:
title: "Project Name"
text: |
Expand Down
Loading