Skip to content

Conversation

@uranusjr
Copy link
Member

@uranusjr uranusjr commented Feb 4, 2025

Operator link currently supports two signatures, one modern using TaskInstanceKey, the other old using execution_date. The old signature will have a problem after the AIP-83 amendments where None is a possibility. Airflow 3.0 is a good chance to just get rid of this problem entirely.

TODO:

@uranusjr uranusjr added the airflow3.0:breaking Candidates for Airflow 3.0 that contain breaking changes label Feb 4, 2025
@uranusjr uranusjr marked this pull request as ready for review February 5, 2025 06:28
@uranusjr uranusjr requested review from XD-DENG and ashb as code owners February 5, 2025 06:28
@uranusjr uranusjr self-assigned this Feb 5, 2025
@uranusjr
Copy link
Member Author

uranusjr commented Feb 5, 2025

Old signature:

class MyOperatorLink(BaseOperatorLink):
    def get_link(self, operator, dttm):
        ...

New signature:

class MyOperatorLink(BaseOperatorLink):
    def get_link(self, operator, *, ti_key):
        ...

Note that in the old signature, dttm is used by Airflow as a positional argument, so the argument name in the function declaration may not be dttm. In the new signature, however, the argument name must be ti_key (although it may be non-keyword-only; self, operator, ti_key should be considered new signature). So the deprecated signature detection rule should be exactly three positional arguments, the third one not named ti_key.

The migration tool should search files for subclasses of BaseOperatorLink, and check their get_link declaration.

Ideally, the tool should check all entries defined in the operator_extra_links list of a plugin, even if it does not subclass BaseOperatorLink, because Airflow does not actually require you to subclass, as long as the signature match at runtime. But this can be difficult to do, so I think it’s OK to ignore this. We always subclass BaseOperatorLink in documentation, and the non-subclass implementation works only by accident.

Relevant documentation on operator extra link: https://airflow.apache.org/docs/apache-airflow/stable/howto/define-extra-link.html

@uranusjr
Copy link
Member Author

uranusjr commented Feb 5, 2025

Delaying this a bit per discussion with @ashb. The task SDK work may need to change this.

@uranusjr
Copy link
Member Author

Got a go-ahead. We can progress now.

@uranusjr uranusjr force-pushed the remove-get-link-execution-date branch from 0309a51 to 0c1a7ba Compare February 18, 2025 06:17
@uranusjr uranusjr force-pushed the remove-get-link-execution-date branch from 0c1a7ba to a7e177a Compare February 19, 2025 04:15
uranusjr and others added 4 commits February 19, 2025 12:15
Operator link currently supports two signatures, one modern using
TaskInstanceKey, the other old using execution_date. The old signature
will have a problem after the AIP-83 amendments where None is a
possibility. Airflow 3.0 is a good chance to just get rid of this
problem entirely.

Although the signature is documented as deprecated, we currently do not
emit any warnings for this. We'll need to add the warning to 2.11, and
add a migration rule for it.
Co-authored-by: Wei Lee <weilee.rx@gmail.com>
@uranusjr uranusjr force-pushed the remove-get-link-execution-date branch from a7e177a to 018edef Compare February 19, 2025 04:15
@uranusjr uranusjr merged commit 3e13c2f into apache:main Feb 19, 2025
45 checks passed
@uranusjr uranusjr deleted the remove-get-link-execution-date branch February 19, 2025 08:49
ntr pushed a commit to ntr/airflow that referenced this pull request Feb 20, 2025
* Remove deprecated get_link signature support

Operator link currently supports two signatures, one modern using
TaskInstanceKey, the other old using execution_date. The old signature
will have a problem after the AIP-83 amendments where None is a
possibility. Airflow 3.0 is a good chance to just get rid of this
problem entirely.

Although the signature is documented as deprecated, we currently do not
emit any warnings for this. We'll need to add the warning to 2.11, and
add a migration rule for it.

Co-authored-by: Wei Lee <weilee.rx@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

airflow3.0:breaking Candidates for Airflow 3.0 that contain breaking changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants