added cancelled handling in dbt deferred #35597
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
|
| """Execute when the trigger fires - returns immediately.""" | ||
| self.run_id = event["run_id"] | ||
| if event["status"] == "cancelled": | ||
| raise DbtCloudJobRunException(f"Job run {self.run_id} has been cancelled.") |
There was a problem hiding this comment.
Has it been discussed somewhere already as to whether we want to raise an error when the job may have deliberately been cancelled by the user?
Just unsure if we want to raise an error in such scenarios, looking for more thoughts.
There was a problem hiding this comment.
It currently works like this in the non-deferrable mode. I also think it makes sense to prevent the start of any following tasks or DAGs when the dbt job gets cancelled.
|
Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions. |
|
@ccorleis Congrats on your first contribution! 🎉 |
ephraimbuddy
added
the
changelog:skip
4 participants
Hi, this PR fixes an error in the handling of the returned status of the DbtCloudRunJobOperator. When using the deferrable mode, the current implementation does not handle the cancelled status correctly, but marks the task as successful. The new cancelled and error status handling was adjusted according to the handling in the non-deferrable mode.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.