Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the SuiteEditNotebookRenderer.add_header to pass env variables #1892

Merged
merged 3 commits into from
Sep 15, 2020
Merged

Conversation

mbakunze
Copy link
Contributor

Pass environment variables to render_with_overwrite method to allow using environment variables in the jinja templates. This allows use cases where depending on environment variables the notebook code generation can change.

We have patched this on our side so we can easily run a profiler on a local/in-memory pandas dataset and then automatically call the great_expectations suite edit command and already have the dataset loaded in pandas (which is otherwise not supported since GE does know where the data lives in this case).

A jinja template using this could look like this:

{% if env["DATASET_FILENAME"] is defined %}
df = pd.read_parquet("{{ env["DATASET_FILENAME"] }}")
{% else %}
df = pd.read_parquet("my_filename") # enter filename here
{% endif %}

Changes proposed in this pull request:

  • [ENHANCEMENT] pass environment variables in the SuiteEditNotebookRenderer.add_header class to allow usage of env variables in jinja templates

Pass environment variables to render_with_overwrite method to allow using environment variables in the jinja templates. This allows use cases where depending on environment variables the notebook code generation can change. 

We have patched this on our side so we can easily run a profiler on a local/in-memory pandas dataset and then automatically call the `great_expectations suite edit` command and already have the dataset loaded in pandas (which is otherwise not supported since GE does know where the data lives in this case).
@eugmandel
Copy link
Contributor

Thank you for submitting, @mbakunze !

@eugmandel eugmandel merged commit 79cf98e into great-expectations:develop Sep 15, 2020
alexsherstinsky pushed a commit to alexsherstinsky/great_expectations that referenced this pull request Feb 19, 2021
)

Pass environment variables to render_with_overwrite method to allow using environment variables in the jinja templates. This allows use cases where depending on environment variables the notebook code generation can change. 

We have patched this on our side so we can easily run a profiler on a local/in-memory pandas dataset and then automatically call the `great_expectations suite edit` command and already have the dataset loaded in pandas (which is otherwise not supported since GE does know where the data lives in this case).

Co-authored-by: Eugene Mandel <eugene.mandel@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants