[BUG] Policy is not being applied to new indices due to indices:admin/opensearch/ism/managedindex

[franco-caylent]

What is the bug?
When a policy is applied through the API using a custom role it is created but its not applied. Upon modifying the description of the policy as admin using the web UI, I rollover the alias and it works.

The software in use is this one

After the role is created, the following event shows up in the logs:
[2024-07-14T08:27:28,259][INFO ][o.o.s.p.PrivilegesEvaluator] [opensearch01] No index-level perm match for User [name=plugin, backend_roles=[], requestedTenant=null] Resolved [aliases=[], allIndices=[cluster-monitor-000001], types=[*], originalRequested=[cluster-monitor-000001], remoteIndices=[]] [Action [indices:admin/opensearch/ism/managedindex]] [RolesChecked [cluster-monitor, own_index]]
but I cant seem to add those permissions using the UI.

It might be related to #2523

How can one reproduce the bug?
Steps to reproduce the behavior:

1. Deploy the following docker-compose

version: '3'
services:
  cluster-monitor:
    image: registry.gitlab.com/franco-martin/cluster-monitor:2.0.0
    container_name: cluster-monitor
    networks:
      - opensearch-net
    environment:
      - ORIGIN_HOST=opensearch
      - DESTINATION_INDEX=cluster-monitor
      - VERIFY_SSL=false
      - TEMPLATE_REPLICAS=1
      - TEMPLATE_SHARDS=1
      - POLICY_ROLLOVER_SIZE=10
      - POLICY_RETENTION=2
      - ORIGIN_USERNAME=cluster-monitor
      - ORIGIN_PASSWORD=ClusterMonitor2024!
  opensearch: # This is also the hostname of the container within the Docker network (i.e. https://opensearch-node1/)
    image: opensearchproject/opensearch:2.13.0 # Specifying the latest available image - modify if you want a specific version
    container_name: opensearch-node1
    environment:
      - cluster.name=opensearch-cluster # Name the cluster
      - node.name=opensearch-node1 # Name the node that will run in this container
      - discovery.seed_hosts=opensearch-node1 # Nodes to look for when discovering the cluster
      - cluster.initial_cluster_manager_nodes=opensearch-node1 # Nodes eligible to serve as cluster manager
      - bootstrap.memory_lock=true # Disable JVM heap memory swapping
      - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # Set min and max JVM heap sizes to at least 50% of system RAM
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD}    # Sets the demo admin user password when using demo configuration, required for OpenSearch 2.12 and later
    ulimits:
      memlock:
        soft: -1 # Set memlock to unlimited (no soft or hard limit)
        hard: -1
      nofile:
        soft: 65536 # Maximum number of open files for the opensearch user - set to at least 65536
        hard: 65536
    volumes:
      - opensearch-data1:/usr/share/opensearch/data # Creates volume called opensearch-data1 and mounts it to the container
    ports:
      - 9200:9200 # REST API
      - 9600:9600 # Performance Analyzer
    networks:
      - opensearch-net # All of the containers will join the same Docker bridge network
  opensearch-dashboards:
    image: opensearchproject/opensearch-dashboards:2.13.0 # Make sure the version of opensearch-dashboards matches the version of opensearch installed on other nodes
    container_name: opensearch-dashboards
    ports:
      - 5601:5601 # Map host port 5601 to container port 5601
    expose:
      - "5601" # Expose port 5601 for web access to OpenSearch Dashboards
    environment:
      OPENSEARCH_HOSTS: '["https://opensearch-node1:9200","https://opensearch-node2:9200"]' # Define the OpenSearch nodes that OpenSearch Dashboards will query
    networks:
      - opensearch-net

volumes:
  opensearch-data1:
  opensearch-data2:

networks:
  opensearch-net:

2. Create a role with the following configuration

PUT _plugins/_security/api/roles/cluster-monitor
{
  "cluster_permissions": [
          "cluster:monitor/health",
      "cluster:monitor/stats",
      "cluster:monitor/nodes/stats",
      "cluster:monitor/nodes/info",
      "cluster:admin/opendistro/ism/policy/write",
      "indices:admin/index_template/put"
  ],
  "index_permissions": [{
    "index_patterns": [
      "cluster-monitor*"
    ],
    "dls": "",
    "fls": [],
    "masked_fields": [],
    "allowed_actions": [
      "index",
          "create_index"
    ]
  },{
        "index_patterns": [
          "*"
        ],
        "dls": "",
        "fls": [],
        "masked_fields": [],
        "allowed_actions": [
          "indices:admin/aliases/get",
          "indices:admin/aliases",
          "indices:admin/create"
        ]
      }],
  "tenant_permissions": [{
    "tenant_patterns": [
      "human_resources"
    ],
    "allowed_actions": [
      "kibana_all_read"
    ]
  }]
}

3. Create a user "cluster-monitor" and password "ClusterMonitor2024!" and map the role "cluster-monitor" to it.
4. you might need to restart cluster-monitor a couple times until you see an output like. In some scenarios you might need to remove the policy, template and indices created and restart cluster-monitor so it creates its resources properly.

2024-07-14 09:10:07,683 INFO Startup
2024-07-14 09:10:07,684 INFO Validating configurations
------------ Opensearch Cluster Monitor------------
 Origin Cluster Endpoint: opensearch:9200
 Destination Index: opensearch:9200/cluster-monitor
 Frequency: 30
 SSL Verification: False
 ------------ Opensearch Cluster Monitor------------

2024-07-14 09:10:07,692 INFO Setting up index template
2024-07-14 09:10:08,058 INFO Setting up index policy
2024-07-14 09:10:08,209 INFO Setting up alias
2024-07-14 09:10:08,330 INFO Starting Monitor
2024-07-14 09:10:08,405 INFO Logging cluster_health
2024-07-14 09:10:08,567 INFO Logging cluster_stats
2024-07-14 09:10:08,736 INFO Logging node
2024-07-14 09:10:08,898 INFO Logging node_stats

5. Use dashboards to verify that the index policy "cluster-monitor" exists
6. Use dashboards to verify that the index template "cluster-monitor-template" exists
7. Use dashboards to verify that the index "cluster-monitor-000001" has the alias "cluster-monitor" pointed to it.
8. Use dashboards to verify that the index cluster-monitor-000001 is not managed by any policies
9. Review the logs of the opensearch node and check that the following error is displayed.

[2024-07-14T09:18:12,906][INFO ][o.o.s.p.PrivilegesEvaluator] [opensearch-node1] No index-level perm match for User [name=plugin, backend_roles=[], requestedTenant=null] Resolved [aliases=[], allIndices=[cluster-monitor-000001], types=[*], originalRequested=[cluster-monitor-000001], remoteIndices=[]] [Action [indices:admin/opensearch/ism/managedindex]] [RolesChecked [cluster-monitor, own_index]]
[2024-07-14T09:18:12,906][INFO ][o.o.s.p.PrivilegesEvaluator] [opensearch-node1] No permissions for [indices:admin/opensearch/ism/managedindex]

11. Rollover alias "cluster-monitor" and verify that the new index is also not managed by the policy
12. Use dashboards to add a character at the end of the policy description of policy "cluster-monitor".
13. Rollover alias "cluster-monitor" and verify that the new index is now managed by the policy.
14. Verify permission indices:admin/opensearch/ism/managedindex doesn't exist in the UI.

What is the expected behavior?
I should be able to add that permission using the UI.

What is your host/environment?

• OS: tested on ubuntu 24.04 and mac os 14.5
• Version [e.g. 22]
• Plugins

Do you have any screenshots?

Do you have any additional context?
Add any other context about the problem.

[cwperks]

[Triage] Thank you for filing this issue @franco-caylent! I don't see that action listed in the security-dashboards-plugin here which is where the dropdown is being population. The ISM permissions should be added to the dropdown to allow an admin to assign these permissions through a page in OSD.

[franco-caylent]

Thank you for the quick response! Im sorry but I don't understand if there's anything I should be doing or if this will remain open until someone fixes it.
On another subject, why would changing the description of the policy as admin fix the issue?

[cwperks]

@franco-caylent It will stay open until a PR is merged in security-dashboards-plugin to add the missing permissions to the dropdown.

To resolve your issue, can you add the missing permissions to the cluster-monitor role using the API or securityadmin?

[franco-caylent]

Absolutely. Thank you!

…

On Mon, Jul 15, 2024, 12:13 Craig Perkins ***@***.***> wrote: @franco-caylent <https://github.com/franco-caylent> It will stay open until a PR is merged in security-dashboards-plugin <https://github.com/opensearch-project/security-dashboards-plugin> to add the missing permissions to the dropdown. To resolve your issue, can you add the missing permissions to the cluster-monitor role using the API <https://opensearch.org/docs/latest/security/access-control/api/#patch-role> or securityadmin <https://opensearch.org/docs/latest/security/configuration/security-admin/> ? — Reply to this email directly, view it on GitHub <#4559 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AUIZHVPHDY3TWK5RDYJ7RDLZMQNMPAVCNFSM6AAAAABK3A6VVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRZGIYDIMZYGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>