Skip to content

Conversation

@Lee-W
Copy link
Member

@Lee-W Lee-W commented Nov 24, 2025

Why

We merge Dag params and task params as task params, and there's no way to distinguish Dag params and task params. It was by design back then. In most cases, users can ignore the Dag level params if they don't need them. But in some cases, such as HITLOperator, we need to distinguish params from different sources, since the param exists only at the Dag level and has no meaning to them, and causes confusion in the "Required Action" tab.

What

  • task-sdk
    • Add source to Param
    • Update source when merging the params
    • Add method filter_params_by_source to ParamsDict
  • standard provider
    • filter only task-level param if ParamsDict.filter_params_by_source is available
  • core
    • serialize Param.source

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

Copy link
Member

@jason810496 jason810496 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! LGTM overall.

@Lee-W Lee-W force-pushed the source-aware-param branch from b475c08 to 9469b6b Compare November 27, 2025 06:39
@Lee-W Lee-W force-pushed the source-aware-param branch from 9469b6b to 13b59eb Compare November 27, 2025 07:52
Copy link
Member

@jason810496 jason810496 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure whether should this kind of typing be included in shared module to avoid DRY?

Though duplicating might still be the better approach as the typing are too small for shared module as long as we are carful enough for the consistency during review.

@Lee-W
Copy link
Member Author

Lee-W commented Nov 27, 2025

Not sure whether should this kind of typing be included in shared module to avoid DRY?

Though duplicating might still be the better approach as the typing are too small for shared module as long as we are carful enough for the consistency during review.

Probably not this one 🤔 We might need to deal with version issues with little gain

@Lee-W Lee-W force-pushed the source-aware-param branch from 20daf37 to 30b5667 Compare November 28, 2025 10:16
@jason810496 jason810496 merged commit 1f6202d into apache:main Nov 28, 2025
118 checks passed
RoyLee1224 pushed a commit to RoyLee1224/airflow that referenced this pull request Dec 3, 2025
Copilot AI pushed a commit to jason810496/airflow that referenced this pull request Dec 5, 2025
itayweb pushed a commit to itayweb/airflow that referenced this pull request Dec 6, 2025
@Lee-W Lee-W deleted the source-aware-param branch December 31, 2025 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants