feat: auto-inject OpenLineage parent info into TriggerDagRunOperator conf #58672
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TLDR
Automatically inject OpenLineage lineage metadata into triggered DAG run conf when OpenLineage is available and enabled, improving lineage tracking across DAG boundaries.. The implementation never modifies non-OpenLineage parts of conf, gracefully handles all error cases, and preserves original conf if anything goes wrong. Users can opt out by setting
openlineage_inject_parent_info=False. Because it's safe and only works for users with enabled OL we can keep default=True.Built on top of OL feature implemented in #57809
User Impact by Scenario
All Users
openlineage_inject_parent_info=Falseto completely disable this featureUsers with OpenLineage disabled or not installed
Users on older OpenLineage Provider versions (< 2.8.0)
apache-airflow-providers-openlineage>=2.8.0Users on compatible OpenLineage Provider versions (>= 2.8.0)
openlineagekeyopenlineagepart in your conf, they are preserved and merged (parent/root identifiers are not overwritten if they already exist)Testing
Added comprehensive test coverage for both Airflow 2 and Airflow 3:
FalseAll tests follow consistent patterns with parametrization and helper functions for maintainability.
Also saved
trigger_run_idas TriggerDagRunOperator's attribute to be used by OL listener.^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.