Add support for user defined _retryable_error in databricks provider

[rawwar]

Description

we can let users pass their own _retryable_error method, which will validate if retry is allowed or not.

As of now, databricks operators only accept retry_limit and retry_delay. Adding support for custom retry function and after func should be good enough to handle all custom requirements.

airflow/providers/src/airflow/providers/databricks/hooks/databricks_base.py

Line 127 in 6b090b3

"retry": retry_if_exception(self._retryable_error),

Use case/motivation

It will allow user to set their own retry logics and also not require change to the provider everytime a user wants to retry for a particular type of exception.

Related issues

#43080

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[rawwar]

@potiuk , @pankajkoti , requesting your comments on this feature.

[pankajkoti]

yeah, IMO another option is that we could accept a user supplied list of retryable errors & add those to our _retryable_error validation method. Probably we could also wait to work on this until there is a good demand for this? And users say how they would like it to be exposed as (how much control they would like)?

[rawwar]

yeah, IMO another option is that we could accept a user supplied list of retryable errors & add those to our _retryable_error validation method. Probably we could also wait to work on this until there is a good demand for this? And users say how they would like it to be exposed as (how much control they would like)?

I won't start working on this right away. But, if I see any users requesting this, I'll raise a PR

[lucafurrer]

IMO I think it will be a simpler interface if we pass a list of errors to add instead of a function. In particular since rewriting your own function more or less implies to go check the current implementation to make that you have not forgot any special use case.

[potiuk]

Yes. list of errors look good and we implemented it elsewhere already (i am quite sure I saw it in other operators). Having a method to pass, but if you implement the operator in the way that it has a public "should_retry(response)" method, then you should be able to implement your custom operator deriving from the base one and rewrite only that method - which is the way the operators should be extended with code.