Task Runner loads Plugins and fails in DB Connections #52922
Description
Apache Airflow version
main (development) as well as 3.0.3rc3
If "Other Airflow 2 version" selected, which one?
No response
What happened?
When running providers/edge3/src/airflow/providers/edge3/example_dags/integration_test.py Dag with EdgeExecutor for every task it seems the plugin manager is initialized and implicitly when loading plugins a stack trace is generated faailing DB connections. Stack trace in every task execution seen is:
Log message source details: sources=["/root/airflow/logs/dag_id=integration_test/run_id=manual__2025-07-05T19-37-25.606441+00-00/task_id=classic_python/attempt=1.log"]
[2025-07-05, 21:37:53] ERROR - Failed to import plugin edge_executor: source="airflow.plugins_manager"
RuntimeError: Direct database access via the ORM is not allowed in Airflow 3.0
File "/opt/airflow/airflow-core/src/airflow/plugins_manager.py", line 260 in load_entrypoint_plugins
File "/usr/local/lib/python3.12/importlib/metadata/__init__.py", line 205 in load
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90 in import_module
File "<frozen importlib._bootstrap>", line 1387 in _gcd_import
File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935 in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 999 in exec_module
File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
File "/opt/airflow/providers/edge3/src/airflow/providers/edge3/plugins/edge_executor_plugin.py", line 217 in <module>
File "/opt/airflow/providers/edge3/src/airflow/providers/edge3/plugins/edge_executor_plugin.py", line 223 in EdgeExecutorPlugin
File "/opt/airflow/airflow-core/src/airflow/utils/session.py", line 100 in wrapper
File "/usr/local/lib/python3.12/contextlib.py", line 137 in __enter__
File "/opt/airflow/airflow-core/src/airflow/utils/session.py", line 41 in create_session
File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/supervisor.py", line 251 in __init__
Note: This is not happening in LocalExecutor, seems the plugins are initialized still when DB connections are not dropped.
What you think should happen instead?
No stack trace produced as well as I am wondering that plugins manager is initialized at-all because this will eat up a lot of runtime to load all hooks and plugins from all providers.
How to reproduce
Started breeze with breeze start-airflow --python 3.12 --backend postgres --executor EdgeExecutor --load-example-dags
Added integration_test Dag to files/dags
Start a run and any task will have this exception
Operating System
Ubuntu 22.04
Versions of Apache Airflow Providers
All from main
Deployment
Other
Deployment details
Breeze
Anything else?
Is this also related to #51873 ?
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct