Upgrade Sphinx to version 7 to improve documentation errors

[potiuk]

Body

We should upgrade Sphinx to version 7

Sphinx 6 has known issues with informing about various errors in our documentation and it becomes REALLY painful. Quite often contributors are mind-boggled by the errors in their documentation after they have done apparently and innocent change. Even super-experienced engineers have hard time figuring out what the error really is and often we have to revert to some woodo-like practices like bisecting in order to narrow down and fix the problems.

Recent examples of such problems which caused us hours of debugging are :

• Update DV360 operators to use API v2 #30326 (comment) - where ` -> ' change cause hours of debugging and was only found by bisecting

• Google provider docstring improvements #31731 (comment) - where lack of empty lines caused the same and also was found by bisecting

What holds us freom migration though is the lack of support for sphinx 7 in the sphinx_rtd-theme which we are using as the base for airflow theme. We build and publish our theme in https://github.com/apache/airflow-site and the sphinx_rtd_theme issue is:

• Described here Add Sphinx 7 support readthedocs/sphinx_rtd_theme#1463
• Proposed PR to fix it here: Add Sphinx 7 support readthedocs/sphinx_rtd_theme#1464

There are three ways how this can be solved:

1. make sphinx_rtd_theme release sphinx7 theme and switch to it
2. vendor in sphinx_rtd_theme into airlfow theme and make necessary changes to support sphinx7
3. switch to another theme.
4. Possibly also switch to sphinx.ext.autodoc from autoapi

If 1) cannot be achieved, seems that 2) might be the easiest one to achieve it.

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.