[Question] How can requests coming from an extension interact with the OpenSearch cluster?

[stephen-crawford]

Problem Statement

This issue tracks specific design questions about Service Accounts.

$\textcolor{orange}{\textsf{Orange text is a header for a decision that is not made }}$
$\textcolor{teal}{\textsf{Teal text is a header for a decision that is made }}$
$\textcolor{violet}{\textsf{Violet text is the recommended option }}$
$\textcolor{lime}{\textsf{Lime text is the chosen option }}$

What type of REST requests are intended for extensions?
How does the Security Plugin interact with REST requests?
What is the current method of signifying things?

Action items decided on this issue:

• How can an admin grant and revoke extension permissions?
• How is an extension prevented from elevating its own permissions?
• How does DLS/FLS work for extensions?

$\textcolor{orange}{\textsf{How can an admin grant and revoke service account permissions?}}$

There are a couple options for handling how an administrator can grant or revoke extension permissions.

1. The first option for managing how an administrator can modify extension permissions is preventing an administrator from making any changes after installation. When an extension is installed, the administrator is expected to grant it privileges based on its configuration file. For instance an extension may specify it requires read access on installation. An administrator would approve this request as part of the installation process. Presumably the extension will have requested all the permissions it requires during installation. As a result, it is not clear why an administrator would later need to modify the granted permissions.

Pros	Cons
Easiest to implement	May restrict extension configuration
Simple rule makes for clear user experience	Incompatible with optional permissions (unless granted on installation)
Solves the issue of an extension elevating its own permissions (they cannot be changed)

2. $\textcolor{lime}{\textsf{Administrators change permissions using the internal user api}}$ The second option is for an administrator to need to use an API to configure permissions in a similar manner to how they would configure user permissions. This option allows an administrator to grant privileges after an extension is installed. The API could use the existing format for modifying user privileges but specify the extension's service account in place of an internal user's name.

Pros	Cons
More configurable	Requires writing an API
Familiar experience for administrators	Does not answer why permissions would need to be changed

$\textcolor{orange}{\textsf{How is an extension prevented from elevating its own permissions?}}$

The two main options for preventing an extension from elevating its own permissions are:

1. By making it impossible for an extension's permissions to be changed after installation, an extension is implicitly prevented from elevating its own permissions. If there is no method of changing an extension's permissions after it is installed, the extension will have no mechanism of escalating its own permissions. Even if the extension were granted administrative privileges, there simply would be no method for changing the permissions. This option makes the most sense if the first option of the previous question is also chosen.

Pros	Cons
Easiest to implement	Requires choosing previous option 1
No loop holes	Not clear or lack of access == enforcement

2. $\textcolor{lime}{\textsf{Service account cannot have permission granting privileges}}$ The alternative approach to preventing an extension from elevating its own permissions is to make the permission grant privileges un-assignable to any service account. This option would require parsing permission grants to extensions and removing any batch grants or individual grants that would provide an extension permission grant privileges. The upside of this option is that it does not prevent an extension from having its permissions modified after installation. An extension could still have its permissions changes but if no extension is ever able to change permissions itself, there will not be a concern that an extension could elevate its own privileges.

Pros	Cons
Allows for changing permissions after installation	Requires parsing & resolving permission assignments
More configurable	How to prove a request is not coming from an extension?

$\textcolor{orange}{\textsf{How does DLS/FLS work for extensions?}}$

There are two main options for handling DLS/FLS for extensions.

1. The first option for handling DLS/FLS is to OR all filters. Normally, DLS/FLS is handled by ORing all of the patterns and then resolving each of them in turn. Employing an approach where the originating user's filters, and the extension's filters were combined and resolved in a similar way may be the most traditional resolution strategy. For instance if the user had patterns A, B and the extension had patterns C, D, then the final request would be treated as if the extension had patterns A, B, C, D.

Pros	Cons
Easy to implement	Information leak into the extension
Matches traditional DLS/FLS resolution	Results could have data the user does not have permission to see

2. $\textcolor{violet}{\textsf{AND filters recursively}}$ The second option for handling DLS/FLS resolution is for the patterns to be AND'd. This option would first apply the pattern of the extension to the query. This would prevent the extension from accessing any information it was not originally allowed to see. After applying the extension's filters, the request would be resolved to the extension level. At that point, the user's filters would be applied to the current result. This would act as an AND operation that would take the intersection of the filters between the extension and the user.

Pros	Cons
More intuitive results for the user	Fewer results since each filter could have aspects excluded
Prevents data leak	Not aligned with traditional DLS/FLS resolution

3. $\textcolor{lime}{\textsf{Service account will support DLS/FLS but scopes/policies will not}}$ Support DLS/FLS for user accounts and on-behalf-of tokens acting as those users. Likewise, we will support DLS/FLS for service accounts and auth tokens representing them when an extension makes an independent action. We won't be P0 supporting filters on the scope of an extensions permissions. Meaning there will be no way to restrict the access of an extension across all use cases. You cannot say an extension can READ for all use cases can't just not this one index. This is the scenario that is complicated to resolve and is going to be left for a later date.

[cwperks]

I know I wrote the blurb originally, but looking at it again this should apply to all extensions and not just out-of-process ones. There are additional considerations for in-process extensions as direct manipulation of immutable areas of the ThreadContext will need to be forbidden.

[cwperks]

When core receives a request originating from an extension it needs to:

1. Lookup the extension's permissions for how it can interact with the cluster
2. If the request is on behalf of a user it also needs to lookup the user's authorizations

When it comes to looking up the extensions permissions, what do you think of the following convention.

The extensions will have a corresponding internal user with a name that matches the extension id. This is a special internal user, a service account, that corresponds to the extension system.

A service account is:

• An account that can only have a single role
• An account that cannot contain backend roles
• A passwordless account - OSD cannot be accessed with a service account

When the request is received at core it will identify the extension, lookup its service account to get the role and then lookup the role definition to get the extension's permissions. By convention, the role name could also match the service account name so that a direct lookup can be done. All of this is in the config cache and should be performant.

[cwperks]

I created an analogy for a hypothetical extension and here are some thoughts:

This analogy assumes that all requests that originate from an extension are governed by the same policy and it would not consider the original user's permissions for the requests that originate from the extension. As long as the user is permitted to use the route that the extension registers then the request will be permitted to flow to the extension and handled from there using the extension's governing policy for how it can interact with the cluster.

UserA makes request to Word Count Extension[TM] to get word count of the word myocardial infarction in index ama_journal_publications. For sake of this exercise imagine the Word Count Extensions adds a REST Action

GET /_extensions/wc/get_word_count?index=<index>&word=<word>

To permit the user to use this action they must be granted:

extensions:wc/get_word_count

Tracing this action out:

We authorize this request initially in the REST layer with the security plugin and ensure user has permission to extensions:wc/get_word_count

2. Assuming permission is granted we create a jwt that minimally could look like:

{
  sub: '<PIT>',
  issuer: '<clustername>',
  audience: 'ext_wc', // extension_id
  nfb: Instance.now(),
  exp: Instance.now() + 3600
} 

3. By convention we could have a role (policy) with the same extension_id that governs how requests originating from the extension interact with the cluster.

ext_wc:
  description: "Word Count Extension Policy"
  cluster_permissions:
    - "cluster_monitor"
  index_permissions:
    - index_patterns:
        - "logs-*"
        - "ama_journal_publications"
      allowed_actions:
        - "read"

# Can include FLS/DLS/Field Masking

4. When Word Count extension sends a SearchRequest to OpenSearch to search for documents that include the word myocardial infarction the security plugin will use this ext_wc role (or w/e we choose to call it) to evaluate whether the request is permitted. Note: At this point it would not consider the user's permissions at all. The extension policy would dictate how all requests originating from the extension can interact with the cluster.

5. The security plugin permits the SearchRequest, the word count extension calculates the official word counts with the documents it received and responds back to the user.

One of the challenges of trying to reconcile the policy that governs how requests coming from an extension can interact with the cluster and the user's permissions is FLS/DLS. By having a single policy in place that makes it clear that any request from an extension is permitted to do action X, Y, Z on indices 1, 2 and 3 then I think it would be easier cognitive burden on the person setting up the policy.

What would be great is if it was possible to compare 2 permission sets.

i.e. If UserA's permissions are a superset of the policy for ExtA then they can do anything the extension can so let the request go through assuming that UserA is allowed to perform the action registered by ExtA.

However, if UserA's permissions are more restricted than the policy for ExtA then don't permit the request - Its tricky to compare permission sets with the current model

---

On installation of this extension, the cluster admin could be presented with multiple prompts:

1. First prompt is detailing how the extension will extend the functionality of OpenSearch

This extension would like to:

- Create REST Handlers (link to details)
- Create a system index
- Create roles

...

Do you want to proceed? (y/n)

2. Next step of the prompt would be to define the policy that governs how requests coming from the extension can interact with the cluster. The extension developer can suggest a default for best usage.

The extension suggests the following permissions for best usage of the extension with the cluster. This is modifiable, but could result in subpar user experience.

ext_wc:
  description: "Word Count Extension Policy"
  cluster_permissions:
    - "cluster_monitor"
  index_permissions:
    - index_patterns:
        - "logs-*"
        - "ama_journal_publications"
      allowed_actions:
        - "read"


Do you want to proceed? (y/n)

3. Next step could be setting up TLS for the cluster and the extension, other final details.

[stephen-crawford]

Hi @cwperks, thank you for your thoughtful reply. The scenario you outlined is definitely something I have been considering as I have looked at designing our permissions system. What you have proposed seems good to me in general though there are a couple issues that I have been mulling over.

First, in your flow, you mention at 3 checking only the permissions of the extension. To me, this seems counter to what we discussed yesterday of carrying the user credentials on the token the entire time. It looks like we have tossed the user credentials and are now only dealing with extension credentials for the request. I assume I am misreading this, but that is how I currently interpret what you outlined.

Second, I am not sure how the policy in you reference after 5. comes into play. If you mean that we should have a single policy to represent the user and extension permissions, I am not sure I see how that would work. This would require a lot of different policies for a single extension and become intractable before too long (we would need one policy per extension per user). Alternatively, if you are referring only to the extension permissions than I certainly agree. I do not think that we would ever want to have multiple policies to resolve the permissions from if we could help it. I would recommend we have a single policy for the user and a single for the extension and look for the intersection of the two to represent the actions that the extension can perform on behalf of the user. For the actions the extension wishes to perform on its own, I would recommend we only looked at the extension's permissions and treated the "user" in this case as the cluster administrator (who should have all permissions). This would mean that when an admin installed the extension they were granting it the ability to execute all actions they gave it permission for without a user.

Otherwise, I think everything you wrote makes a lot of sense. I am certainly in favor of a design very similar if not identical to the shell you outlined here.

[cwperks]

@scrawfor99 yes, this is a bit of a change from yesterday's discussion on considering the user's permissions when an extension makes a request to opensearch on behalf of a user. The primary problem is that it becomes challenging to consider DLS and FLS for these situations. Say for instance that an extension tries to search a table of employees on behalf of a user. The user is restricted to only see employees in the HR department and requests from the extension are only allowed to see employees located in the US. What would you expect the result of the query to be? I would probably expect the intersection of the 2 which would give the employees in the US who have the HR role. Currently roles with DLS are ORed together so it returns all of the documents viewable across all of the roles assigned to a user and its not clear how to implement DLS with AND. Its also a bit confusing to understand how this will work from a cluster admin POV.

In the example above it would be possible for the cluster admin to achieve the user's restrictions of only seeing records with the HR roles, but in order to achieve that the cluster admin would need to have the restriction on the extension's policy for how requests coming from it can interact with OpenSearch and it would apply to all requests coming from the extension.

The other problem is that considering the user's permissions on a request coming from the extension would only provide a veneer of Security. We can't prevent what the extension will do with the data once its been provided to the extension. If one user has DLS/FLS restrictions and another user doesn't and the extension gets all the data, then its already outside the trust boundary and cannot be ensured how it is presented to other users.

IMO there should be a single policy that governs how requests coming from the extension can access data from OpenSearch and that will be used to evaluate whether the request is permitted. Users would be restricted from using an extension based on the user's permissions.

[stephen-crawford]

Hi @cwperks, thank you for the follow-up. I understand better what you are getting at now. I would agree that it would probably make the most sense to provide the AND of the filters as opposed to the typical OR. I was originally thinking of using the AND for such a scenario. In order to do this, we would likely be best off recursively applying the filters if that makes sense. So you would first only look at the HR employees and then only look at the US results in that group. This would give you the AND though it would be expensive. That being said, if we do not want to go this route that is fine.

To your final paragraph, to what extent do you mean users would be restricted from using an extension? Would they need all the permissions that the extension had? I think that we would run into issues if we were to not allow people to use an extension at all just because they lacked one of the permissions for something specific. I may not want every employee to be able to edit the documents in our directory extension, but I probably am fine with all employees being able to see each other. Did you mean something else?

[cwperks]

Hey @scrawfor99 let's take AD for an example which currently defines the following 2 roles:

anomaly_read_access:
  reserved: true
  cluster_permissions:
    - 'cluster:admin/opendistro/ad/detector/info'
    - 'cluster:admin/opendistro/ad/detector/search'
    - 'cluster:admin/opendistro/ad/detectors/get'
    - 'cluster:admin/opendistro/ad/result/search'
    - 'cluster:admin/opendistro/ad/tasks/search'
    - 'cluster:admin/opendistro/ad/detector/validate'
    - 'cluster:admin/opendistro/ad/result/topAnomalies'

# Allows users to use all Anomaly Detection functionality
anomaly_full_access:
  reserved: true
  cluster_permissions:
    - 'cluster_monitor'
    - 'cluster:admin/opendistro/ad/*'
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - 'indices_monitor'
        - 'indices:admin/aliases/get'
        - 'indices:admin/mappings/get'

Notice all of the index permissions that AD provides in its full_access role - that's because in order to use AD to the fullest the user needs to be able to read from all indices. Upon further review, I'm not crazy about this pattern because its also giving full read permissions to the cluster by assigning this role, but let's set that aside for now.

In the extensions world let's break it down by the following:

1. 2 roles intended to be assigned to users similar to above, but without the index permissions

anomaly_read_access:
  cluster_permissions: // could probably nix the cluster_permissions here and keep for backward compatibility sake
    - 'cluster:admin/opendistro/ad/detector/info'
    - 'cluster:admin/opendistro/ad/detector/search'
    - 'cluster:admin/opendistro/ad/detectors/get'
    - 'cluster:admin/opendistro/ad/result/search'
    - 'cluster:admin/opendistro/ad/tasks/search'
    - 'cluster:admin/opendistro/ad/detector/validate'
    - 'cluster:admin/opendistro/ad/result/topAnomalies'

# Allows users to use all Anomaly Detection functionality
anomaly_full_access:
  cluster_permissions:
    - 'cluster:admin/opendistro/ad/*'

2. Then there's the policy that governs how requests coming from the AD extension can interact with the cluster. This would be set upon extension installation and come with a default defined by the extension developer for optimal use, but modifiable by cluster admin.

ext_ad:
  index_permissions:
    - index_patterns:
        - '*'
      allowed_actions:
        - 'indices_monitor'
        - 'indices:admin/aliases/get'
        - 'indices:admin/mappings/get'

---

To recap:

In 1) it defines roles that are mappable to a user. These define what actions they can perform on the extension and they map to endpoints. We will block requests in the REST-layer before forwarded to the extension if the user is not permitted to perform the action.

For 2) this policy governs how requests coming from AD can interact with the cluster. For optimal use, the AD extension is requesting to be able to read from all indices. If a cluster admin wants to apply DLS/FLS they would do so there.

[stephen-crawford]

Hi @cwperks, thank you for your detailed explanation. I think I better understand your suggestion now. I do have one question about your example. Why have we split based on cluster and index permissions? I am not sure why these would be split in such a way. It seems either could be part of the user defined roles (1) or the permissions being granted to the extension itself (2). I don't see a clear difference in the types of permissions outside of the general naming convention we use. Is there something that makes cluster permissions unsuitable to be granted to the extension in 2? Or vice versa?

Thank you again for your thoughtful responses and suggestions.