[RFC] Migrate the definition of privilege presets from the sample roles.yml to static_action_groups.yml and static_roles.yml #5426
Description
Introduction
Recently, we had quite a few of issues and PRs related to privileges for other plugins defined in roles.yml:
- [BUG]Upgrade APIs not working as expected #5364
- add missing mapping get permission #5412
- add missing cluster:monitor permission #5405
- Add forecast roles and permissions #5386
- Integrate search-relevance functionalities with security plugin #5376
- Adding stop-replication permission to index_management_full_access #5160
- Added roles for ltr read and full access #5067
- add ingest pipeline and indices related permissions for anomaly_full_access role #5065
These follow a practise in OpenSearch, which has IMHO significant downsides, while a better approach would be actually quite easy to achieve. I will explain the downsides and the proposed solution in this issue.
As this is an RFC, I would be very grateful for any comments and opinions on this topic.
The role of the file config/roles.yml
The configuration files in the directory config were initially inteded as a demo or example configuration. This is still evident from the file internal_users.yml, which defines example users with hard-coded insecure passwords. Also, the file roles_mapping.yml contains the comment Demo roles mapping.
By default, OpenSearch is not automatically initialized with these files upon first startup; this can be only activated by setting the config option plugins.security.allow_default_init_securityindex
Still, the file roles.yml inside the config directory has evolved into a kind of storage for pre-defined and reusable role definitions for further OpenSearch plugins.
It contains for example generic roles for anomaly detection and alerting:
Lines 42 to 64 in 224eb2a
It happens quite often that PR are filed against this file to add new privileges either to existing roles or to new roles.
The challenge caused by this practise
Non-automatic updates
While the changes contained in the PRs filed against roles.yml are trivial, these pose a challenge to administrators of running clusters.
One reason is: Any changes to these files are not automatically applied to the already installed configuration on clusters that already existed before. Thus, any bug fix or addition to the roles.yml file is not effective on these clusters.
As a remedy, there is the POST /_plugins/_security/api/_upgrade_perform API:
- https://docs.opensearch.org/docs/latest/api-reference/security/configuration/upgrade-perform/
- [FEATURE] New reserved roles are not sourced to security index on an upgrade #2316
This checks for differences between the active role configuration and the role configuration stored in config/opensearch-security/roles.yml and overwrites any roles in the active configuration that also exist in config/opensearch-security/roles.yml by that predefined version.
This API, however, is not well known. A Google search for the endpoint just delivers the API documentation and the PR in which it was introduced. There are no forum discussions on that API. Also, the documentation about upgrading OpenSearch does not mention the API. A quick company internal survey resulted in zero cases of knowledge about this API.
That means that likely most cluster admins will not use this API and thus will be likely using obsolete versions of the role definitions. This might be not so bad if this just causes missing privileges. If, however, a role needs to be changed because it granted too many privileges, this might be an actual issue.
Default configuration is not installed by default
It requires the explicit enabling of a specific config option to enable installing the OpenSearch default configuration. Thus, we need to expect that there are many clusters present, which actually do not have the roles from config/roles.yml present in their effective configuration.
Actually, using this flag must be considered as bad practise for production clusters, as it will install demo configuration which must be considered insecure and unsuitable for production clusters. I would recommend to any cluster admin to review each file and to only use the mimum configuration necessary for their needs.
For these cases, plugin maintainers cannot expect their predefined roles to be present at all.
The solution is already implemented
The OpenSearch security plugin has already the concept of "static" configuration files. These are separate configuration files which are present in the JAR archive of the deployed plugin. Any entries from these configuration files are automatically added to the effective configuration without being persisted to the index. Thus, any update to these files gets automatically effective on each upgrade of the security plugin.
Excerpts from static_roles.yml:
security/src/main/resources/static_config/static_roles.yml
Lines 79 to 143 in 224eb2a
You can see that this files contains role definitions for Kibana/Dashboards and Logstash in a very similar way as the role definitions for alerting and anomaly detection in config/roles.yml.
Excerpts from static_action_groups.yml:
security/src/main/resources/static_config/static_action_groups.yml
Lines 13 to 28 in 224eb2a
This also defines reusable action groups for Kibana/Dashboards.
Thus, it seems that these files solve exactly the described issue. Thus, I would propose that we use them.
Proposal
I would propose the following way forward:
Convert roles from config/roles.yml into static roles and static action groups.
Making roles static is straight-forward; it is just copying roles over to the other file. One only has to keep in mind that the name of static roles should not be equal to the name of the present non-static roles. If it would be equal, the cluster would log warnings on startup.
Additionally, I would propose to also leverage static action groups as much as possible. Thus, any static roles should not directly refer to individual action names, but only to static action groups. This creates a better reusability. Possibly, there are use cases to use action groups rather than roles for defining plugin privileges.
Example: alerting_read_access
would become the static action group:
"alerting::read_all":
reserved: true
hidden: false
static: true
allowed_actions:
- 'cluster:admin/opendistro/alerting/alerts/get'
- 'cluster:admin/opendistro/alerting/destination/get'
- 'cluster:admin/opendistro/alerting/monitor/get'
- 'cluster:admin/opendistro/alerting/monitor/search'
- 'cluster:admin/opensearch/alerting/comments/search'
- 'cluster:admin/opensearch/alerting/findings/get'
- 'cluster:admin/opensearch/alerting/remote/indexes/get'
- 'cluster:admin/opensearch/alerting/workflow/get'
- 'cluster:admin/opensearch/alerting/workflow_alerts/get'
type: "cluster"
description: "Allows to read all configuration entities used by OpenSearch alerting"
and the static role:
"alerting::read_all":
reserved: true
cluster_permissions:
- 'alerting::read_all'
Example: anomaly_full_access
The role anomaly_full_access is kind of special, as it also gives any user full read access on any index on the cluster. I would argue that we should strive at making this unnecessary, as there are likely very valid scenarios where users can modify the full AD configuration, but should not be able to access all indices.
Thus, we would create the static action group:
"anomaly_detection::manage":
reserved: true
hidden: false
static: true
allowed_actions:
- "cluster:admin/ingest/pipeline/delete"
- "cluster:admin/ingest/pipeline/put"
- 'cluster:admin/opendistro/ad/*'
- 'cluster_monitor'
type: "cluster"
description: "Allows users to use all Anomaly Detection functionality"
and the static role:
"anomaly_detection::manage":
reserved: true
cluster_permissions:
- 'anomaly_detection::manage'
Any cluster admin could give users full index access explicitly by creating a role like this:
"ad_manager":
cluster_permissions:
- 'anomaly_detection::manage'
index_permissions:
- index_patterns:
- "*"
- allowed_actions:
- read
Mark plugin specific roles in config/roles.yml as deprecated
A deprecation notice should be added to config/roles.yml to inform users that they should move to the static roles. This pertains both to cluster admins using these roles and to developers defining the permissions.
Write developer documentation on structuring the privileges
To educate contributers about the proper way of defining privileges, clear and easily discoverable developer docs need to be written on that topic.
Deprecate the _upgrade_perform API
As this API won't be no longer necessary, it shall be deprecated and removed. Possibly, the deprecation notice can log information about the new approach.