[Resource Access Control] [Part1] Introduces SPI for resource access control

[DarshitChanpura]

#5016 is being broken down into smaller pieces. This is part 1.

Description

Introduces an SPI to implement Resource Access Control in OpenSearch. This will allow plugins to declare itself as Resource Plugins to leverage the access control methods to be provided by Security plugin.

• Category - New feature
• Why these changes are required?
  • At present, plugins have implemented in-house authorization mechanisms for a lack of centralized framework. This PR introduces a centralized way to offload resource permissions check to security plugin.

Issues Resolved

Related to #4500

Testing

• automated tests

Check List

- [ ] New functionality includes testing
- [ ] New functionality has been documented
~- [ ] New Roles/Permissions have a corresponding security dashboards plugin PR
- [ ] API changes companion pull request created

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.

Project coverage is 71.69%. Comparing base (bfc0f4a) to head (c9e2485).
Report is 1 commits behind head on feature/resource-permissions.

Files with missing lines	Patch %	Lines
.../opensearch/security/OpenSearchSecurityPlugin.java	66.66%	1 Missing ⚠️

Additional details and impacted files

@@                       Coverage Diff                        @@
##           feature/resource-permissions    #5185      +/-   ##
================================================================
- Coverage                         71.71%   71.69%   -0.03%     
================================================================
  Files                               337      337              
  Lines                             22789    22790       +1     
  Branches                           3606     3606              
================================================================
- Hits                              16344    16340       -4     
- Misses                             4649     4653       +4     
- Partials                           1796     1797       +1     

Files with missing lines	Coverage Δ
...g/opensearch/security/support/ConfigConstants.java	95.23% <ø> (ø)
.../opensearch/security/OpenSearchSecurityPlugin.java	83.72% <66.66%> (-0.12%)	⬇️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[cwperks]

One thing to consider is having a method on this interface to assign a service to this extension point that contains methods to interrogate access for this specific type of sharable resource. Could be an alternative to having a client and instead include everything in the SPI.

[DarshitChanpura]

I think client is a better approach as it separates functionalities. SPI is purely intended for providing interfaces that plugins must implement. Client provides a class with APIs that plugin can utilize to implement Access Control.

[cwperks]

The advantage of supplying a Service/Client from a method in the extension point here is that a plugin knows that if it is provided then that already means that security is installed and enabled.

A SPI does provide the extension point, but it can be used like a library as well. I think this should be considered instead of publishing a client that plugins need to depend on as a mandatory dependency.

[DarshitChanpura]

Proposal on path forward for scopes:

To add to the comment here,

Scopes can be thought of as placeholders for action groups. The question here is whether we should include Scopes in this iteration of the feature, at all. From what I see there are 3 paths forward:

Path 1: No Scopes

{
	"share_with": {
		"users": [...],
		"roles": [...],
		"backend_roles": [...]
	}
}

With this structure there is no concept of "scope". The document will be visible on 3 levels: Private, Restricted and Public. Here is what a sample resource-sharing document will look like in each case:

1. Private

• share_with record can be empty or not be present at all.

{
	"resource_id": "id",
	"source_idx": "resource-index",
	"created_by": {
		"user": "x"
	},
	"share_with": { }
}

2. Restricted

• share_with record will contain access information about individual entities.

{
	"resource_id": "id",
	"source_idx": "resource-index",
	"created_by": {
		"user": "x"
	},
	"share_with": {
		"users": [...],
		"roles": [...],
		"backend_roles": [...]
	}
}

3. Public:

• share_with record will contain access information about individual entities.

{
	"resource_id": "id",
	"source_idx": "resource-index",
	"created_by": {
		"user": "x"
	},
	"share_with": {
		"users": ["*"],
		"roles": ["*"],
		"backend_roles": ["*"]
	}
}

Pros:

• No confusion around scopes

Cons:

• No access control of any specific type. It relies on APIs to be defined clearly and users to be mapped correctly.
• Requires another plugin migration when we introduce Resource Authorization Framework as a standalone authorization mechanism

Path 2: Placeholder for Scopes

{
	"share_with": {
		"*" : {
			"users": [...],
			"roles": [...],
			"backend_roles": [...]
		}
	}
}

With this structure we introduce a default scope "" which acts as a place holder for action-groups to be declared when a standalone Resource Authorization framework is in place. "" allows all actions to be matched.

1. Private

• * record inside share_with can be empty or not be present at all

{
	"resource_id": "id",
	"source_idx": "resource-index",
	"created_by": {
		"user": "x"
	},
	"share_with": { 
		"*" : { }
	} 
}

2. Restricted

• * record inside share_with record will contain access information about individual entities.

{
	"resource_id": "id",
	"source_idx": "resource-index",
	"created_by": {
		"user": "x"
	},
	"share_with": { 
		"*": {
			"users": [...],
			"roles": [...],
			"backend_roles": [...]
		}
	}
}

3. Public:

• * record inside share_with record will contain access information about individual entities.

{
	"resource_id": "id",
	"source_idx": "resource-index",
	"created_by": {
		"user": "x"
	},
	"share_with": { 
		"*": {
			"users": ["*"],
			"roles": ["*"],
			"backend_roles": ["*"]
		}
	}
}

Pros:

• No confusion around scopes.
• Will not require migration since future sharing info can be added on top this ("*" still retains its meaning of all actions being allowed, but an additional read_only action-group can be added to allow read-only access)

Cons:

• Any action can be allowed since it is "*".

Path 3: Continue with scopes

This approach allows defining individual access type over each resource. It doesn't rely completely on the APIs. It can be in future expanded to be mapped to action groups when resource authorization framework is implemented.

Example, AD plugin can, at present, define a scope named ad_read_only and in future an action-group can be added by the name of ad_read_only which contains a set of actions to be utilized by Resource Authorization framework.

Here is what an example detector sharing record may look like:

{
	"resource_id": "detector1",
	"source_idx": ".opendistro-anomaly-detectors",
	"created_by": {
		"user": "darshit"
	},
	"share_with": { 
		"ad_read_only" : {
			"users": [...],
			"roles": [...],
			"backend_roles": [...]
		},
		"ad_full_access" : {
			"users": [...],
			"roles": [...],
			"backend_roles": [...]
		}
		
	}
}

or if the detector is intended to be publicly accessible:

{
	"resource_id": "detector1",
	"source_idx": ".opendistro-anomaly-detectors",
	"created_by": {
		"user": "darshit"
	},
	"share_with": { 
		"*" : {
			"users": ["*"],
			"roles": ["*"],
			"backend_roles": ["*"]
		}
	}
}

Pros:

• No future migration

Cons:

• Complex to understand at first.
• Requires plugin to pass scopes, when asking for access verification. What scopes should current user's access be matched against.

@nibix @cwperks @derek-ho Please review this proposal so we can finalize the path-forward for scopes.

I lean towards Path 2 as it is good place between no scopes and complete scopes, and it unblocks this PR. It would require some re-work in following Part 2 and 3 tho.

[DarshitChanpura]

Proposal on path forward for scopes:
@nibix @cwperks @derek-ho Please review this proposal so we can finalize the path-forward for scopes.

I personally am leaning towards Path 2 as it is good place between no scopes and complete scopes.

Path 3 puts us in a good state for future, but I don't want that to be a blocker for this PR. All changes are labelled as @opensearch.experimental so that gives us room to modify things later.

[nibix]

@DarshitChanpura

Proposal on path forward for scopes:

Thank you for the proposals!

Before I am going into the proposals, I'd like to go one step back and try to get a better picture of the situation. Sorry if the following question may sound dumb, but I am possibly missing some details at the moment.

So, my question would be: What's the actual reason we are now discussing these options about the path forward with scopes?

• Is it because there is disagreement about the scopes?
• Is it because the scopes are not sufficiently/thoroughly defined yet?
• Or something else?

I think an answer to this question could help me to contribute better to this discussion.

[cwperks]

Is it because there is disagreement about the scopes?

Not conceptually, more on implementation. @DarshitChanpura has introduced a notion of "scopes" and can elaborate on them and their implementation. At the onset of this project, I had imagined re-using action groups since its an existing mechanism to group an enumerable list of actions together.

Conceptually, a "scope" is a group of actions that can be performed on a resource and the owner of the resource gets to specify the level of access when sharing. For an example, we use a tool for doc reviews that has 4 different access levels when sharing a document. The access levels are "Read Only", "Read + Comment", "Edit" (allows read, comment, edit, but not share) and "Full Access" which includes the ability to share a document. These are examples of what the scopes would be for a plugin.

FYI scopes (or action groups) for resource authorization breaks with the current security model where the cluster administrator is the one assigning permissions.

I think alerting is a good example of a plugin that has multiple levels of access that could benefit from more fine-grained access on sharable resources. They already define 3 default roles to include in the default distribution.

[DarshitChanpura]

Yes, there is some confusion around implementation of scopes.

Scopes do not equate to any action-groups or resolve to any set of actions, at the moment in this PR. They are added to allow resources to be shared with an additional layer of security, meaning even if a user possesses access to, say a GET detector API, they would additionally need READ scope access. This sets us in a good state when Resource Authz framework is implemented as a standalone framework (with Resource Request and ResourcePrivEvaluator). At the moment, Resource Authz works on top of existing authz framework.

To avoid, any sort of confusion, I've put in a proposal for paths forward for this feature. The idea is to at least address ML-commons use case where they declare three types of access to resources: Private, Restricted and Public (as described in Paths 1 and 2 above). This allows us to achieve the final implementation of scopes (or action-groups) when we implement the standalone framework.

I believe Path 2 is a good setup which is forward-looking and also resolves Ml-commons use case.

[nibix]

@cwperks @DarshitChanpura

Thank you for your replies!

I think a part of my personal confusion is caused by the fact that the word "scope" can mean so many things. It could also mean the audience a resource is shared with - that was actually my first intuition. It took me a while to find that this is wrong.

So, IMHO, I'd hope that a lot of the confusion could be resolved by finding another name for scopes, which is more precise and more intuitive. What do you think?

[DarshitChanpura]

I think a part of my personal confusion is caused by the fact that the word "scope" can mean so many things. It could also mean the audience a resource is shared with - that was actually my first intuition. It took me a while to find that this is wrong.

Scope is the level at which the resource is shared.

So, IMHO, I'd hope that a lot of the confusion could be resolved by finding another name for scopes, which is more precise and more intuitive. What do you think?

Yes that can be done. However, the important question here is which path to proceed with, out of the three.

[DarshitChanpura]

I've update this PR and subsequent parts to reflect action-groups change following Path 2 from design proposals put forward here: #5185 (comment)

[derek-ho]

Generally LGTM from a high level, I will take a look at part 2 and 3 to see if there is anything else concerning on this PR

[DarshitChanpura]

@nibix @cwperks I'll be merging this PR into the feature branch. Any further comments can be addressed when I open a PR against main to merge this feature.