Skip to content

Conversation

@vatsrahul1001
Copy link
Contributor

@vatsrahul1001 vatsrahul1001 commented Jan 28, 2026

Add try/except in _get_current_dag(). If deserialization fails, the scheduler logs the error and continues to the next DAG instead of crashing.

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@vatsrahul1001 vatsrahul1001 force-pushed the scheduler-deserialization-guardrail branch from f044371 to db0f75c Compare January 28, 2026 12:30
@boring-cyborg boring-cyborg bot added the area:Scheduler including HA (high availability) scheduler label Jan 28, 2026
@vatsrahul1001 vatsrahul1001 marked this pull request as draft January 28, 2026 12:42
@vatsrahul1001 vatsrahul1001 force-pushed the scheduler-deserialization-guardrail branch from db0f75c to 976a060 Compare January 28, 2026 12:56
@vatsrahul1001 vatsrahul1001 marked this pull request as ready for review January 28, 2026 13:01
@vatsrahul1001 vatsrahul1001 force-pushed the scheduler-deserialization-guardrail branch from 976a060 to f3b642c Compare January 28, 2026 13:09
@vatsrahul1001 vatsrahul1001 force-pushed the scheduler-deserialization-guardrail branch from f3b642c to 5466ffe Compare January 28, 2026 13:47
@vatsrahul1001 vatsrahul1001 added this to the Airflow 3.1.7 milestone Jan 29, 2026
Copy link
Member

@ashb ashb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better than crashing the scheduler hard, though I wonder if will this continue to try and get it very loop around the scheduler leading to log spam and other knock on issues? Since the dag that is failing will still get returned in the DagModel.dags_needing_dagruns list (Ditto for anything asset related too?)

Not sure how best to address this...

@vatsrahul1001
Copy link
Contributor Author

Better than crashing the scheduler hard, though I wonder if will this continue to try and get it very loop around the scheduler leading to log spam and other knock on issues? Since the dag that is failing will still get returned in the DagModel.dags_needing_dagruns list (Ditto for anything asset related too?)

Not sure how best to address this...

Maybe modifying dags_needing_dagruns query to exclude DAGs with certain warning types?

@vatsrahul1001 vatsrahul1001 merged commit d54cf8e into apache:main Jan 29, 2026
71 checks passed
@vatsrahul1001 vatsrahul1001 deleted the scheduler-deserialization-guardrail branch January 29, 2026 11:42
vatsrahul1001 added a commit to astronomer/airflow that referenced this pull request Jan 29, 2026
vatsrahul1001 added a commit to astronomer/airflow that referenced this pull request Jan 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Scheduler including HA (high availability) scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants