Clarify / standardize Airflow API permissions definitions

[zach-overflow]

What do you see as an issue?

(preamble): I'm happy to help out with any doc work or code refactors to address this issue; but, I'm raising this issue partly because I myself do not know the correct answer to the details which I think need to be added to the auth manager / security docs.

Description

This issue pertains to the core auth manager docs (as of Airflow version 3.0.2). While the auth manager doc briefly covers the methods to override when implementing a custom Auth manager, the docs do not clarify any of the following details, which I believe are crucial for a proper implementation of a custom auth manager:

1. What role do the resource identifiers in airflow.security.permissions play in the auth manager?
2. How do the airflow.api_fastapi.auth.managers.models.resource_details class definitions relate to similarly named identifiers in airflow.security.permission?
3. Likewise, the docs do not clarify the implicit hierarchy of resources/permissioning in the DagAccessEntity.
4. Whether the FAB auth manager is slated for deprecation in the near future.
5. Which API-action + resource combinations are possible or not, especially as it relates to DAG sub-entities. (I was only able to understand some of these expected action-resource combinations by digging through the various files under airflow.api_fastapi.core_api.routes.public, but it feels risky to base my assumptions on code which may not be part of the public interface)

Why is this an issue?

1. If the auth manager docs explained these concepts thoroughly, it would benefit both Airflow end-users as well as Airflow core contributors seeking to help out with the remaining auth manager implementations (i.e. KeyCloak, AWS, etc).
2. This also could present unintended security risks if the various authorization-related resource entities remain inconsistently defined as dataclasses, enums, or string constants in various locations, without any documentation recommending the use of one definition over the other.
3. It seems that in general, I've not been the only one who has found the current auth manager docs in need of clarifications, such as in these 2 recent threads: Airflow not using custom API authentication/auth_backends #51362 API Authentication in Airflow 3.0.0 with FAB Provider #50684 . I believe the omissions I listed may help alleviate some (albeit not all of the common sticking points raised in those threads).

Solving the problem

1. Clarify what resource definitions users should rely on for their auth manager implementations:
   • Should people use the constants in airflow.security.permissions, if not, why / what is that module's intended use?
   • What exactly do the various id attributes refer to in the classes defined in airflow.api_fastapi.auth.models.resource_details passed into the auth manager's is_authorized_* methods? How should user's rely on those fields, and what is the recommended approach for when those resource details arguments are None?
2. Maybe add a diagram for the expected resource hierarchies in the API permissions models, or at least describe the potential interactions between the various is_authorized_* methods. For example, it would be nice to have a reference for the expected potential interactions between is_authorized_dag and is_authorized_backfill which is implicitly defined here.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

Absolutely, and totally yes - we should clarify it all and document (and so happy you want to help) . cc: @vincbeck ...

[zach-overflow]

cc @vincbeck , I'd be curious to hear your thoughts as to whether some of the questions here might require auth code adjustments in addition to doc updates. I assumed most of these questions can be answered by documentation, but maybe some of the details here pertain to a need for some kind of unified interface or contract for permissions / resource references?

Either way, I'd be happy to help with that too, but again I'm not totally convinced I have a clear enough understanding of the Airflow 3 auth world to put up any auth-related PRs (docs, or otherwise).

[vincbeck]

Hey @zach-overflow . Thanks for looking into it, I agree with you, these docs can be way better and happy to see you are willing to do it :) Let me try to answer the questions.

What role do the resource identifiers in airflow.security.permissions play in the auth manager?

None. airflow.security.permissions is deprecated and should not be used. The reason why we keep them is they are widely used by users in plugins and it is hard to delete them. But trust, I'd like them to be removed, so please do not use them.

How do the airflow.api_fastapi.auth.managers.models.resource_details class definitions relate to similarly named identifiers in airflow.security.permission?

In resource_details we define 3 kind of classes:

• Details classes (e.g. ConnectionDetails). These classes are used to pass additional information about the resource when calling the is_authorized_* method. Example: is_authorized_connection(method="GET", user=user, details=ConnectionDetails(conn_id="default")) allows to know whether the user has access to read the connection "default"
• AccessView. Is used to provide information which view the user is trying to access. Example: is_authorized_view(access_view=AccessView.PROVIDERS) checks whether the user has accept to the view providers in the UI.
• DagAccessEntity. Gives more information, if any, about the sub DAG entity the user is trying to access. Example: is_authorized_dag(method="GET", user=user, access_entity=DagAccessEntity.TASK_LOGS, details=DagDetails(id="my-dag")) checks whether the user access to read DAG logs from the DAG my-dag. Another example: is_authorized_dag(method="GET", user=user, details=DagDetails(id="my-dag")) checks whether the user has access to the DAG my-dag in general. access_entity is not provided here, which means the user is trying to only access information the DAG and not "sub-entities" of the DAG.

Likewise, the docs do not clarify the implicit hierarchy of resources/permissioning in the DagAccessEntity.

I hope I answered this question in my previous answer.

Whether the FAB auth manager is slated for deprecation in the near future.

Not in the near future, we want to deprecate it at some point but when is unclear now.

Which API-action + resource combinations are possible or not, especially as it relates to DAG sub-entities. (I was only able to understand some of these expected action-resource combinations by digging through the various files under airflow.api_fastapi.core_api.routes.public, but it feels risky to base my assumptions on code which may not be part of the public interface)

Can you provide concrete example of what you might think as a combination does not makes sense? It might be simpler to answer that way :)

Should people use the constants in airflow.security.permissions, if not, why / what is that module's intended use?

No and it is only there to not break plugins. An attempt was made in the past to remove them and many users complained.

What exactly do the various id attributes refer to in the classes defined in airflow.api_fastapi.auth.models.resource_details passed into the auth manager's is_authorized_* methods? How should user's rely on those fields, and what is the recommended approach for when those resource details arguments are None?

As I explained previously, these details attributes/classes are there to provide more information about a resource. If the user tries to access a connection, what is the connection ID. If it is a variable, what is the variable ID. When those resource details are None, depending on the method it means:

• GET. Whether the user has access to see a type of resource in general. It does not mean all of them but some of them. It could be rephrased to: "whether the user is able to list DAGs" (if the resource is DAG, the same applies for the other kind of resources)
• POST. Whether the user has access to create a given type of resource
• PUT. Not sure this combination makes sense
• DELETE. Not sure this combination makes sense
• MENU. This could be potentially dropped from this list. It needs further investigation but it might not makes sense to keep it as ResourceMethod since menu items are filtered by filter_authorized_menu_items

I hope I answered some of your questions and I really hope you'd be able to make some modifications in space.

[zach-overflow]

Thank you @vincbeck, this is a super helpful summary and very much appreciated!

Which API-action + resource combinations are possible or not, especially as it relates to DAG sub-entities. (I was only able to understand some of these expected action-resource combinations by digging through the various files under airflow.api_fastapi.core_api.routes.public, but it feels risky to base my assumptions on code which may not be part of the public interface)

Can you provide concrete example of what you might think as a combination does not makes sense? It might be simpler to answer that way :)

Funny enough, the example I was going to give was related to a DELETE operation on a DAG Run resource, and what that translates to for the DAG resource itself, but the rest of your answer helped clarify this.

Glad to hear the airflow.security.permission module is not recommended, as I was concerned whether I was going down the wrong path with my auth manager implementation. Likewise, I was curious about the MENU API action. To that end, I agree the MENU method might make sense to drop from the list given the filter_authorized_menu_items

I'll be trying to wrap up some work on my end and then I think once I've got some free time in the next week or so I can help draft up some documentation updates for the auth manager. Thanks again for clarifying this. Super helpful!

[zach-overflow]

A few other questions popped up which I forgot to add:

1. The auth manager doc page's section on AuthN-related methods mentions a get_user method, but I don't believe that exists anywhere in the BaseAuthManager or in any of the core/provider-offered implementations. Should that instead be get_user_from_token?
2. As for is_authorized_custom_view, are there any examples of how that actually looks in practice, particularly with regard to how that would look when not using the constants defined in airflow.security.permission? It's hard to follow how that should ideally be used since there's no existing examples of it called as a FastAPI path function dependency (as far as I can tell).

[vincbeck]

The auth manager doc page's section on AuthN-related methods mentions a get_user method, but I don't believe that exists anywhere in the BaseAuthManager or in any of the core/provider-offered implementations. Should that instead be get_user_from_token?

Good catch! when reviewing this page, it is absolutely not up to date! I'll create a PR to fix it. Thanks for catching it.

As for is_authorized_custom_view, are there any examples of how that actually looks in practice, particularly with regard to how that would look when not using the constants defined in airflow.security.permission? It's hard to follow how that should ideally be used since there's no existing examples of it called as a FastAPI path function dependency (as far as I can tell).

I understand this one can be tricky to understand. The auth manager or a plugin exposes a new view, let's say: "View internal documentation". And you want to add access control on this page, you can use this method to check whether the user has access to this given. Following the same example: `is_authorized_custom_view(method="GET", resource_name="INTERNAL_DOC", user=user). Then, in your auth manager implementation, you need to figure whether the user should have access. Does that help?

[vincbeck]

#52586