Add dataset delete feature

[im-perativa]

This PR add a feature to delete dataset from both UI and API to address #36308 with necessary tests and API specification.

delete_dataset_ui_modal.mp4

delete_dataset_api.mp4

closes: #36308

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[im-perativa]

@potiuk re-requesting your review

[bbovenzi]

We should definitely have a confirmation step when deleting anything. Probably using this component in chakra: https://chakra-ui.com/docs/components/alert-dialog/usage

Let me know if you need help here.

[im-perativa]

@bbovenzi added confirmation modal and updated the recording. For now I followed the button spacing as in ActionModal to maintain UI consistency

[bbovenzi]

After looking at this again, I don't like how the delete button is so prevalent for what should be a pretty rare behavior.

So that gives us two options:

1. Only show the delete button when hovering on that row
2. Only show the delete button on the dataset details page

[potiuk]

I think this one should hold before we conclude https://lists.apache.org/thread/q4sd09xymvlskxdgcb1s6gfsqwnvqs2o discussion (so I am doing "Request changes" unti it does - and end up with a VOTE or CONSENSUS).

Whether to create/delete datasets externally is a decision that should be made deliberately, and there are various voices there - some (including myself) think that dataset management should be done internally by Airflow (including possibly reference counting and automatically removing datasets that are not referred by any DAGS) while some others think that there is a value in managing datasets from outside. Until that discussion concludes, I think we should hold on with merging this change.

[potiuk]

I think this one should hold before we conclude https://lists.apache.org/thread/q4sd09xymvlskxdgcb1s6gfsqwnvqs2o discussion (so I am doing "Request changes" unti it does - and end up with a VOTE or CONSENSUS).

Whether to create/delete datasets externally is a decision that should be made deliberately, and there are various voices there - some (including myself) think that dataset management should be done internally by Airflow (including possibly reference counting and automatically removing datasets that are not referred by any DAGS) while some others think that there is a value in managing datasets from outside. Until that discussion concludes, I think we should hold on with merging this change.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.