Add DeltaSharing provider and sensor.

[dmoore247]

Description

Using this Issue to add initial support for a DeltaSharing provider.
We would clone an existing provider and modify to monitor for changes in delta share tables via the rest protocol. This is a lightweight mechanism to pause before the entire Delta Share update is processed.

Use case/motivation

Use Case: As a user of Airflow and DeltaSharing to optimally process new data refreshes, I'd like to wait on a new update on a specified Delta Sharing table before progressing the rest of my DAG.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[eladkal]

@dmoore247 are you still working on this issue?

[alexott]

I can pickup this work, starting with sensor (although maybe it makes sense to have a separate issue for a sensor)

[eladkal]

No need for multiple issues :)
Issues are not really required. You can open PRs freely.

[eladkal]

Policy for adding new provider is explained at https://github.com/apache/airflow/blob/main/PROVIDERS.rst

I'm going to close this issue as the process of adding a provider require investment and commitments from the requester thus it's unlikely that someone will just pick up this request and implement it so no reason to keep this issue open.