Don't fallback to anonymous user when credentials are present (Basic Authentication)

[AkshayGoyal022]

Elasticsearch Version

8.3.3

Installed Plugins

No response

Java Version

bundled

OS Version

Darwin Kernel Version 23.4.0

Problem Description

We are observing one weird behaviour with anonymous access. Looks like it falls back to anonymous user if credentials are present and user doesn't have required permissions.

I see something similar being fixed in 7.6.0 #51042. Is there a way to disable fallback if credentials are present?

Steps to Reproduce

Create user with role

curl -X POST "localhost:9200/_security/role/new_test_role?pretty" -H 'Content-Type: application/json' -d'
{
  "cluster": ["monitor"],
  "indices": [
    {
      "names": [ "foo*" ],
      "privileges": ["all"]
    }
  ]
}
'

curl -X POST "localhost:9200/_security/user/new_test_user?pretty" -H 'Content-Type: application/json' -d'
{
  "password" : "new_test_password",
  "roles" : [ "new_test_role" ],
  "full_name" : "New Test User",
  "email" : "new_test_user@coupa.com"
}
'

^ Notice this user only has access to indices starting with foo*

cluster's config/elasticsearch.yml settings:

xpack.security.enabled: true
xpack.security.authc:
  anonymous:
    username: anonymous_user
    roles: superuser
    authz_exception: true

Now if i send a request with credentials of this user, i am able to access indices which aren't allowed.
Request:
curl -X GET "http://new_test_user:new_test_password@localhost:9200/bar/_search?pretty"
I even tried with Authorization Header but still same result (`curl -H "Authorization: Basic bmV3X3Rlc3RfdXNlcjpuZXdfdGVzdF9wYXNzd29yZA==" -XGET "http://localhost:9200/bar/_search?pretty")

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-security (Team:Security)

[albertzaharovits]

@AkshayGoyal022 The intention is that the anonymous role is also assigned to all other users that present any credential, even if the credentials they present map to other roles. That's because the anonymous role is assigned to users that don't show any credential, and it makes little sense to have fewer permissions when presenting a credential than when not.
I acknowledge that's not clear from the docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/anonymous-access.html.

[AkshayGoyal022]

@albertzaharovits Not sure if I understand it completely. Do you mean anonymous role takes precedence over the role associated with the user?
From the documentation:

Incoming requests are considered to be anonymous if no authentication token can be extracted from the incoming request

But here we are passing the credentials. Also tried with Authorization Bearer header. So request shouldn't be considered as anonymous

[albertzaharovits]

Do you mean anonymous role takes precedence over the role associated with the user?

I mean that xpack.security.authc.anonymous.roles are appended to the roles that are normally mapped to the user, see

elasticsearch/x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/Subject.java

Line 242 in 589d927

allRoleNames = ArrayUtils.concat(user.roles(), anonymousUser.roles());

.

[albertzaharovits]

We discussed this in our es-security team meeting yesterday, and decided we will not implement this.

We consider it a potential security problem if a client is able to get more or different permissions in case that it omits (eg by mistake) the credentials.

[yaauie]

For posterity:

Restricting an authenticated user's permissions to less than anonymous user's is also fundamentally useless because the default behavior of many HTTP user agents is to not send credentials until they are challenged, so even if a user-agent is configured with credentials it is not guaranteed to transmit them.

A user agent that wishes to authenticate itself with an origin server
   -- usually, but not necessarily, after receiving a 401 (Unauthorized)
   -- can do so by including an Authorization header field with the
   request.

-- Hypertext Transfer Protocol (HTTP/1.1): Authentication | RFC-7235 | § 2.1
HTTP's Auth