API Authentication in Airflow 3.0.0 with FAB Provider

[Jie211]

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

3.0.0

What happened?

I'm currently implementing OAuth authentication for the API server in Airflow 3.0.0 using the FAB provider.
I've configured my setup using AIRFLOW__CORE__AUTH_MANAGER to specify a custom auth manager that extends FabAuthManager.

According to the FAB provider documentation, it should be possible to implement a custom API authentication backend and specify it using AIRFLOW__FAB__AUTH_BACKENDS, similar to how it worked in Airflow 2.x.
The documentation suggests that the requires_authentication decorator in the custom backend should control the authentication process.

However, I've observed that when accessing v2 API endpoints with AIRFLOW__CORE__AUTH_MANAGER configured, the API only verifies tokens through FabAuthManager.get_user_from_token() method. The code decorated with requires_authentication in my custom backend doesn't seem to be executed at all.

Question

Has the requires_authentication decorator in custom API authentication backends become obsolete in Airflow >= 3.0.0?
If so, is the FAB provider documentation outdated?

I'd like to understand the correct way to implement custom API authentication in Airflow 3.0.0:

• Should I focus solely on implementing get_user_from_token() in my custom auth manager?
• Is there any scenario where requires_authentication is still used for API endpoints in Airflow 3.0.0?
• If requires_authentication is indeed obsolete for v2 API endpoints, should the FAB provider documentation be updated to reflect this change?

Additional Context

I've successfully implemented token validation in get_user_from_token(), which works for API v2 endpoints.
However, I initially spent significant time trying to debug why my requires_authentication decorator wasn't being called, only to realize it might be by design in Airflow 3.0.0.

Any clarification on the current authentication flow and documentation accuracy would be greatly appreciated.

What you think should happen instead?

No response

How to reproduce

configuration:

export AIRFLOW__CORE__AUTH_MANAGER="my.custom.FabAuthManager"
export AIRFLOW__FAB__AUTH_BACKENDS="my.custom.api_backend"

Operating System

MacOS

Versions of Apache Airflow Providers

apache-airflow-providers-fab: 2.0.2

Deployment

Other

Deployment details

just set my custom OAUTH configs in webserver_config.py with fab provider
and run script:

export AIRFLOW_HOME=${PROJECT_DIR}/local_airflow/airflow_home
export AIRFLOW__CORE__LOAD_EXAMPLES=False
export AIRFLOW__CORE__DAGS_FOLDER=${PROJECT_DIR}/dags
export AIRFLOW__CORE__AUTH_MANAGER="my.custom.FabAuthManager"
export AIRFLOW__FAB__AUTH_BACKENDS="my.custom.api_backend"
export AIRFLOW__CORE__PLUGINS_FOLDER=${PROJECT_DIR}/src

export AIRFLOW__WEBSERVER__EXPOSE_CONFIG=True
export AIRFLOW__WEBSERVER__RELOAD_ON_PLUGIN_CHANGE=True

export AIRFLOW__CORE__EXECUTE_TASKS_NEW_PYTHON_INTERPRETER=True

export AIRFLOW__LOGGING__FAB_LOGGING_LEVEL=INFO
export AIRFLOW__LOGGING__LOGGING_LEVEL=INFO

rm -rf ${AIRFLOW_HOME:?/tmp/hoge}/*

cp ${PROJECT_DIR}/configs/webserver_config.py ${AIRFLOW_HOME}/webserver_config.py

airflow db migrate && airflow api-server -w 1

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[tirkarthi]

cc: @vincbeck @pierrejeambrun

[Jie211]

Ok, The problem was solved.

Should I focus solely on implementing get_user_from_token() in my custom auth manager?
Is there any scenario where requires_authentication is still used for API endpoints in Airflow 3.0.0?

After upgrading to Airflow 3.0.0 with the FAB provider, all endpoints under the public API (/api/v2/*) use get_user_from_token() to validate tokens.

However, the endpoints provided by the FAB provider itself (for managing users/roles/permissions) still use the requires_authentication decorator from the FAB auth backends module/

If requires_authentication is indeed obsolete for v2 API endpoints, should the FAB provider documentation be updated to reflect this change?

Yes, this information should be noted in the FAB provider's documentation.
Additionally, the path for APIs provided by the FAB provider is directly under /auth/fab/v1/ (like /auth/fab/v1/users, /auth/fab/v1/roles, etc.), which is also not clearly mentioned in the FAB provider's documentation.

[vincbeck]

If you identified some gaps in the documentation, please create a PR. I'll be very happy to review it. It is hard to cover all use cases and scenarios so the best documentation author is the user itself :)

[alpeshv]

@Jie211 I am having similar issues. Do you mind sharing your code for my.custom.FabAuthManage and my.custom.api_backend? You can share it as pseudo/scaffolding if there is anything sensitive.

Thank you.

[Jie211]

@alpeshv
Sorry, my code contains a lot of business logic, so it's hard to share.
If there's something specific you want to know, please let me know and I'll do my best to explain it in detail.

[pierrejeambrun]

As mentioned above the problem is solved. In AF3 you don't use auth_backend to authenticate to the API but use the API to generate a token (based on your AuthManager) and then us this token to call the API.

[alpeshv]

@Jie211 thanks you for your reply. I was interested in only understanding your custom implementation of auth manager but since then I learnt that KeyCloakAuthManager code has been merged into Airflow main branch and just waiting to be released.

[vincbeck]

@Jie211 thanks you for your reply. I was interested in only understanding your custom implementation of auth manager but since then I learnt that KeyCloakAuthManager code has been merged into Airflow main branch and just waiting to be released.

The implementation is not done yet so we cannot release it yet, but it should be very soon. We are very close to be code complete. I'd say, it should be released in a month