ci(github-actions): add a script to check significant newsfragments

.github/workflows/news-fragment.yml

@@ -56,28 +56,8 @@ jobs:
           false
           ; }
 
-      - name: Check news fragment contains change types
+      - name: Check news fragment format
         env:
           BASE_REF: ${{ github.base_ref }}
         run: >
-          change_types=(
-            'Dag changes'
-            'Config changes'
-            'API changes'
-            'CLI changes'
-            'Behaviour changes'
-            'Plugin changes'
-            'Dependency changes'
-            'Code interface changes'
-          )
-          news_fragment_content=`git diff origin/${BASE_REF} newsfragments/*.significant.rst`
-
-          for type in "${change_types[@]}"; do
-            if [[ $news_fragment_content != *"$type"* ]]; then
-              printf "\033[1;33mMissing change type '$type' in significant newsfragment for PR labeled with
-              'airflow3.0:breaking'.\nCheck
-              https://github.com/apache/airflow/blob/main/contributing-docs/16_contribution_workflow.rst
-              for guidance.\033[m\n"
-              exit 1
-            fi
-          done
+          uv run scripts/ci/pre_commit/significant_newsfragments_checker.py

.pre-commit-config.yaml

@@ -1129,6 +1129,17 @@ repos:
         # We sometimes won't have newsfragments in the repo, so always run it so `check-hooks-apply` passes
         # This is fast, so not too much downside
         always_run: true
+      - id: check-significant-newsfragments-are-valid
+        name: Check significant newsfragments are valid
+        # Sigifniciant newsfragments follows a special format so that we can group information easily.
+        language: python
+        files: newsfragments/.*\.rst
+        entry: ./scripts/ci/pre_commit/significant_newsfragments_checker.py
+        pass_filenames: false
+        additional_dependencies: ["docutils>=0.21.2", "Pygments>=2.19.1", "Jinja2>=3.1.5"]
+        # We sometimes won't have newsfragments in the repo, so always run it so `check-hooks-apply` passes
+        # This is fast, so not too much downside
+        always_run: true
       - id: update-breeze-cmd-output
         name: Update breeze docs
         description: Update output of breeze commands in Breeze documentation

contributing-docs/08_static_code_checks.rst

@@ -226,6 +226,8 @@ require Breeze Docker image to be built locally.
 +-----------------------------------------------------------+--------------------------------------------------------+---------+
 | check-safe-filter-usage-in-html                           | Don't use safe in templates                            |         |
 +-----------------------------------------------------------+--------------------------------------------------------+---------+
+| check-significant-newsfragments-are-valid                 | Check significant newsfragments are valid              |         |
++-----------------------------------------------------------+--------------------------------------------------------+---------+
 | check-sql-dependency-common-data-structure                | Check dependency of SQL providers                      |         |
 +-----------------------------------------------------------+--------------------------------------------------------+---------+
 | check-start-date-not-used-in-defaults                     | start_date not in default_args                         |         |

dev/breeze/doc/images/output_static-checks.txt

@@ -1 +1 @@
-8b916a8eceeecdae4936ee72ec060491
+bc496446ce0ed673262a2515daf88da9

dev/breeze/src/airflow_breeze/pre_commit_ids.py

@@ -78,6 +78,7 @@
     "check-pyproject-toml-consistency",
     "check-revision-heads-map",
     "check-safe-filter-usage-in-html",
+    "check-significant-newsfragments-are-valid",
     "check-sql-dependency-common-data-structure",
     "check-start-date-not-used-in-defaults",
     "check-system-tests-present",

newsfragments/43774.significant.rst

@@ -16,7 +16,7 @@ For easier change in the future, this function now takes only keyword arguments.
   * [ ] Dependency changes
   * [ ] Code interface changes
 
-* Migrations rules needed
+* Migration rules needed
 
   * ruff
 

newsfragments/43902.significant.rst

@@ -16,7 +16,7 @@ The shift towards ``logical_date`` helps move away from the limitations of ``exe
   * [ ] Dependency changes
   * [ ] Code interface changes
 
-* Migrations rules needed
+* Migration rules needed
 
   * ruff
 

newsfragments/43915.significant.rst

@@ -14,7 +14,7 @@ a hard error on validation failure.
   * [ ] Dependency changes
   * [ ] Code interface changes
 
-* Migrations rules needed
+* Migration rules needed
 
   * ``airflow config lint``
 

newsfragments/43943.significant.rst

@@ -15,7 +15,7 @@ fetching logs to attach to the trace)
   * [ ] Dependency changes
   * [ ] Code interface changes
 
-* Migrations rules needed
+* Migration rules needed
 
   * ``airflow config lint``
 

newsfragments/43975.significant.rst

@@ -18,7 +18,7 @@ backwards compatibility with the previous versions of Airflow.
   * [ ] Dependency changes
   * [ ] Code interface changes
 
-* Migrations rules needed
+* Migration rules needed
 
   * ``airflow config lint``
 

newsfragments/44475.significant.rst

@@ -17,7 +17,7 @@ Remove ``TriggerRule.NONE_FAILED_OR_SKIPPED``
 
 .. List the migration rules needed for this change (see https://github.com/apache/airflow/issues/41641)
 
-* Migrations rules needed
+* Migration rules needed
 
   * ruff
 

newsfragments/aip-66.significant.rst

@@ -2,37 +2,40 @@ Support DAG versioning by introducing DAG Bundles
 
 The following DAG parsing configuration options were moved into the ``dag_processor`` section:
 
-* ``[core] dag_file_processor_timeout`` -> ``[dag_processor] dag_file_processor_timeout``
-* ``[scheduler] parsing_processes`` -> ``[dag_processor] parsing_processes``
-* ``[scheduler] file_parsing_sort_mode`` -> ``[dag_processor] file_parsing_sort_mode``
-* ``[scheduler] max_callbacks_per_loop`` -> ``[dag_processor] max_callbacks_per_loop``
-* ``[scheduler] min_file_process_interval`` -> ``[dag_processor] min_file_process_interval``
-* ``[scheduler] stale_dag_threshold`` -> ``[dag_processor] stale_dag_threshold``
-* ``[scheduler] print_stats_interval`` -> ``[dag_processor] print_stats_interval``
+* ``[core] dag_file_processor_timeout`` → ``[dag_processor] dag_file_processor_timeout``
+* ``[scheduler] parsing_processes`` → ``[dag_processor] parsing_processes``
+* ``[scheduler] file_parsing_sort_mode`` → ``[dag_processor] file_parsing_sort_mode``
+* ``[scheduler] max_callbacks_per_loop`` → ``[dag_processor] max_callbacks_per_loop``
+* ``[scheduler] min_file_process_interval`` → ``[dag_processor] min_file_process_interval``
+* ``[scheduler] stale_dag_threshold`` → ``[dag_processor] stale_dag_threshold``
+* ``[scheduler] print_stats_interval`` → ``[dag_processor] print_stats_interval``
 
 .. Provide additional contextual information
 
 .. Check the type of change that applies to this change
 
 * Types of change
 
-  * [ ] DAG changes
+  * [ ] Dag changes
   * [x] Config changes
   * [ ] API changes
   * [ ] CLI changes
   * [ ] Behaviour changes
   * [ ] Plugin changes
-  * [ ] Dependency change
+  * [ ] Dependency changes
+  * [ ] Code interface changes
 
 .. List the migration rules needed for this change (see https://github.com/apache/airflow/issues/41641)
 
-* [ ] Migration rules needed
+* Migration rules needed
 
-  * ``[core] dag_file_processor_timeout`` -> ``[dag_processor] dag_file_processor_timeout``
-  * ``[scheduler] parsing_processes`` -> ``[dag_processor] parsing_processes``
-  * ``[scheduler] file_parsing_sort_mode`` -> ``[dag_processor] file_parsing_sort_mode``
-  * ``[scheduler] max_callbacks_per_loop`` -> ``[dag_processor] max_callbacks_per_loop``
-  * ``[scheduler] min_file_process_interval`` -> ``[dag_processor] min_file_process_interval``
-  * ``[scheduler] stale_dag_threshold`` -> ``[dag_processor] stale_dag_threshold``
-  * ``[scheduler] print_stats_interval`` -> ``[dag_processor] print_stats_interval``
-  * ``[scheduler] dag_dir_list_interval`` → ``[dag_processor] refresh_interval``
+  * ``airflow config lint``
+
+    * [ ] ``[core] dag_file_processor_timeout`` → ``[dag_processor] dag_file_processor_timeout``
+    * [ ] ``[scheduler] parsing_processes`` → ``[dag_processor] parsing_processes``
+    * [ ] ``[scheduler] file_parsing_sort_mode`` → ``[dag_processor] file_parsing_sort_mode``
+    * [ ] ``[scheduler] max_callbacks_per_loop`` → ``[dag_processor] max_callbacks_per_loop``
+    * [ ] ``[scheduler] min_file_process_interval`` → ``[dag_processor] min_file_process_interval``
+    * [ ] ``[scheduler] stale_dag_threshold`` → ``[dag_processor] stale_dag_threshold``
+    * [ ] ``[scheduler] print_stats_interval`` → ``[dag_processor] print_stats_interval``
+    * [ ] ``[scheduler] dag_dir_list_interval`` → ``[dag_processor] refresh_interval``

newsfragments/aip-72.significant.rst

@@ -54,7 +54,7 @@ As part of this change the following breaking changes have occurred:
   * [ ] Dependency changes
   * [ ] Code interface changes
 
-* Migrations rules needed
+* Migration rules needed
 
   * ``airflow config lint``