openlineage: disable running listener if not configured #33120
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
potiuk
approved these changes
Aug 4, 2023
Signed-off-by: Maciej Obuchowski <obuchowski.maciej@gmail.com>
potiuk
force-pushed
the
openlineage-disable-when-not-configured
branch
from
821d5de to
531bb9c
Compare
44 tasks
ephraimbuddy
added
the
changelog:skip
ephraimbuddy
pushed a commit
that referenced
this pull request
Aug 8, 2023
Signed-off-by: Maciej Obuchowski <obuchowski.maciej@gmail.com> (cherry picked from commit 11ff650)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There's a discussion whether to include OpenLineage provider in default Airflow image.
https://apache-airflow.slack.com/archives/C03G9H97MM2/p1691163069712479
In this case, we won't have an assumption that user explicitely enabled OpenLineage or works in an environment where OpenLineage is expected to run. This means logs, generating OpenLineage events and some overhead associated with it would be not an expected behavior from an user.
This PR disables even launching OpenLineage listener if no option required to run it has been configured, so no unexpected behavior exists. Those changes are only relevant in case of it being included by default - they should not be merged otherwise.