[FEATURE] usage of JWKS with JWT (w/o OpenID connect)

[rursprung]

Is your feature request related to a problem?
when using JWT it is cumbersome to manage the public keys / certificates directly in the config file (or as environment variables), making adding/removing issuers and general key rotation a pain.

What solution would you like?
it should be possible to define a JWKS endpoint for JWT-based authentication.

note: it might be that this already works (because JWT is used within OpenID connect), but it's definitely undocumented and so i don't know how we could set this up. so at the very least this would require a documentation update, worst-case it might need some enhancements.

What alternatives have you considered?
see above:

• store the keys in the config file
• store the keys in environment variables which are referenced in config files

both of these also require one auth domain per issuer, which is suboptimal.

Do you have any additional context?

• JWK & JWKS specification: RFC 7517
• this is already fully supported when using OpenID connect: OpenID Connect OpenSearch Security Documentation

[DarshitChanpura]

[Triage] Thank you for filing this feature request.

[cwperks]

@rursprung I was able to get a cluster running locally with keycloak with the config found here: https://github.com/cwperks/osd-oidc-example/tree/master

When running through this example I see that the openid_connect_url contains a link to the jwks_uri in the endpoint's response.

Expand to see example security config:

authc:
      basic_internal_auth_domain:
        description: "Authenticate via HTTP Basic against internal users database"
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: basic
          challenge: false
        authentication_backend:
          type: intern
      openid_auth_domain:
        http_enabled: true
        transport_enabled: true
        order: 1
        http_authenticator:
          type: openid
          challenge: false
          config:
            subject_key: preferred_username
            roles_key: roles
            openid_connect_url: http://localhost:8080/auth/realms/master/.well-known/openid-configuration
        authentication_backend:
          type: noop

The http://localhost:8080/auth/realms/master/.well-known/openid-configuration returns:

{
   "issuer":"http://localhost:8080/auth/realms/master",
   "authorization_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/auth",
   "token_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/token",
   "introspection_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/token/introspect",
   "userinfo_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/userinfo",
   "end_session_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/logout",
   "frontchannel_logout_session_supported":true,
   "frontchannel_logout_supported":true,
   "jwks_uri":"http://localhost:8080/auth/realms/master/protocol/openid-connect/certs",
   "check_session_iframe":"http://localhost:8080/auth/realms/master/protocol/openid-connect/login-status-iframe.html",
   "grant_types_supported":[
      "authorization_code",
      "implicit",
      "refresh_token",
      "password",
      "client_credentials",
      "urn:ietf:params:oauth:grant-type:device_code",
      "urn:openid:params:grant-type:ciba"
   ],
   "acr_values_supported":[
      "0",
      "1"
   ],
   "response_types_supported":[
      "code",
      "none",
      "id_token",
      "token",
      "id_token token",
      "code id_token",
      "code token",
      "code id_token token"
   ],
   "subject_types_supported":[
      "public",
      "pairwise"
   ],
   "id_token_signing_alg_values_supported":[
      "PS384",
      "ES384",
      "RS384",
      "HS256",
      "HS512",
      "ES256",
      "RS256",
      "HS384",
      "ES512",
      "PS256",
      "PS512",
      "RS512"
   ],
   "id_token_encryption_alg_values_supported":[
      "RSA-OAEP",
      "RSA-OAEP-256",
      "RSA1_5"
   ],
   "id_token_encryption_enc_values_supported":[
      "A256GCM",
      "A192GCM",
      "A128GCM",
      "A128CBC-HS256",
      "A192CBC-HS384",
      "A256CBC-HS512"
   ],
   "userinfo_signing_alg_values_supported":[
      "PS384",
      "ES384",
      "RS384",
      "HS256",
      "HS512",
      "ES256",
      "RS256",
      "HS384",
      "ES512",
      "PS256",
      "PS512",
      "RS512",
      "none"
   ],
   "userinfo_encryption_alg_values_supported":[
      "RSA-OAEP",
      "RSA-OAEP-256",
      "RSA1_5"
   ],
   "userinfo_encryption_enc_values_supported":[
      "A256GCM",
      "A192GCM",
      "A128GCM",
      "A128CBC-HS256",
      "A192CBC-HS384",
      "A256CBC-HS512"
   ],
   "request_object_signing_alg_values_supported":[
      "PS384",
      "ES384",
      "RS384",
      "HS256",
      "HS512",
      "ES256",
      "RS256",
      "HS384",
      "ES512",
      "PS256",
      "PS512",
      "RS512",
      "none"
   ],
   "request_object_encryption_alg_values_supported":[
      "RSA-OAEP",
      "RSA-OAEP-256",
      "RSA1_5"
   ],
   "request_object_encryption_enc_values_supported":[
      "A256GCM",
      "A192GCM",
      "A128GCM",
      "A128CBC-HS256",
      "A192CBC-HS384",
      "A256CBC-HS512"
   ],
   "response_modes_supported":[
      "query",
      "fragment",
      "form_post",
      "query.jwt",
      "fragment.jwt",
      "form_post.jwt",
      "jwt"
   ],
   "registration_endpoint":"http://localhost:8080/auth/realms/master/clients-registrations/openid-connect",
   "token_endpoint_auth_methods_supported":[
      "private_key_jwt",
      "client_secret_basic",
      "client_secret_post",
      "tls_client_auth",
      "client_secret_jwt"
   ],
   "token_endpoint_auth_signing_alg_values_supported":[
      "PS384",
      "ES384",
      "RS384",
      "HS256",
      "HS512",
      "ES256",
      "RS256",
      "HS384",
      "ES512",
      "PS256",
      "PS512",
      "RS512"
   ],
   "introspection_endpoint_auth_methods_supported":[
      "private_key_jwt",
      "client_secret_basic",
      "client_secret_post",
      "tls_client_auth",
      "client_secret_jwt"
   ],
   "introspection_endpoint_auth_signing_alg_values_supported":[
      "PS384",
      "ES384",
      "RS384",
      "HS256",
      "HS512",
      "ES256",
      "RS256",
      "HS384",
      "ES512",
      "PS256",
      "PS512",
      "RS512"
   ],
   "authorization_signing_alg_values_supported":[
      "PS384",
      "ES384",
      "RS384",
      "HS256",
      "HS512",
      "ES256",
      "RS256",
      "HS384",
      "ES512",
      "PS256",
      "PS512",
      "RS512"
   ],
   "authorization_encryption_alg_values_supported":[
      "RSA-OAEP",
      "RSA-OAEP-256",
      "RSA1_5"
   ],
   "authorization_encryption_enc_values_supported":[
      "A256GCM",
      "A192GCM",
      "A128GCM",
      "A128CBC-HS256",
      "A192CBC-HS384",
      "A256CBC-HS512"
   ],
   "claims_supported":[
      "aud",
      "sub",
      "iss",
      "auth_time",
      "name",
      "given_name",
      "family_name",
      "preferred_username",
      "email",
      "acr"
   ],
   "claim_types_supported":[
      "normal"
   ],
   "claims_parameter_supported":true,
   "scopes_supported":[
      "openid",
      "microprofile-jwt",
      "email",
      "profile",
      "address",
      "phone",
      "roles",
      "groups",
      "acr",
      "offline_access",
      "web-origins"
   ],
   "request_parameter_supported":true,
   "request_uri_parameter_supported":true,
   "require_request_uri_registration":true,
   "code_challenge_methods_supported":[
      "plain",
      "S256"
   ],
   "tls_client_certificate_bound_access_tokens":true,
   "revocation_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/revoke",
   "revocation_endpoint_auth_methods_supported":[
      "private_key_jwt",
      "client_secret_basic",
      "client_secret_post",
      "tls_client_auth",
      "client_secret_jwt"
   ],
   "revocation_endpoint_auth_signing_alg_values_supported":[
      "PS384",
      "ES384",
      "RS384",
      "HS256",
      "HS512",
      "ES256",
      "RS256",
      "HS384",
      "ES512",
      "PS256",
      "PS512",
      "RS512"
   ],
   "backchannel_logout_supported":true,
   "backchannel_logout_session_supported":true,
   "device_authorization_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/auth/device",
   "backchannel_token_delivery_modes_supported":[
      "poll",
      "ping"
   ],
   "backchannel_authentication_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/ext/ciba/auth",
   "backchannel_authentication_request_signing_alg_values_supported":[
      "PS384",
      "ES384",
      "RS384",
      "ES256",
      "RS256",
      "ES512",
      "PS256",
      "PS512",
      "RS512"
   ],
   "require_pushed_authorization_requests":false,
   "pushed_authorization_request_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/ext/par/request",
   "mtls_endpoint_aliases":{
      "token_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/token",
      "revocation_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/revoke",
      "introspection_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/token/introspect",
      "device_authorization_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/auth/device",
      "registration_endpoint":"http://localhost:8080/auth/realms/master/clients-registrations/openid-connect",
      "userinfo_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/userinfo",
      "pushed_authorization_request_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/ext/par/request",
      "backchannel_authentication_endpoint":"http://localhost:8080/auth/realms/master/protocol/openid-connect/ext/ciba/auth"
   }
}

which contains the jwks_uri that the security plugin uses to obtain certs. See the following lines in the Security repo where the openid_connect_url is fetched and parsed into a JsonWebKeys keySet: https://github.com/opensearch-project/security/blob/main/src/main/java/com/amazon/dlic/auth/http/jwt/keybyoidc/KeySetRetriever.java#L68-L147

Edit: Re-read the initial post and I realize this is a request for w/o OpenID connect. I don't think this is supported at the moment and needs to be implemented.

[cwperks]

Hi @rursprung, I was able to workaround this issue by re-using the openid backend and providing a self-hosted .well-known/configuration file which only contained a link to the jwks uri.

Here is the authc configuration:

authc:
  basic_internal_auth_domain:
    http_enabled: true
    transport_enabled: true
    order: 0
    http_authenticator:
      type: basic
      challenge: false
    authentication_backend:
      type: internal
  openid_auth_domain:
    http_enabled: true
    transport_enabled: true
    order: 1
    http_authenticator:
      type: openid
      challenge: false
      config:
        subject_key: preferred_username
        roles_key: roles
        openid_connect_url: http://host.docker.internal:5604/.well-known-keys
    authentication_backend:
      type: noop

The contents of the http://host.docker.internal:5604/.well-known-keys file:

{
   "jwks_uri": "http://host.docker.internal:5604/keys.json"
}

The contents of keys.json generated using node-jose and the following guide (https://mojoauth.com/blog/jwt-validation-with-jwks-nodejs/)

{
  "keys": [
    {
      "kty": "RSA",
      "kid": "5kungCIMhf0wBnxKRc3jpackKWfnBJq5fiIQqISt6wo",
      "use": "sig",
      "alg": "RS256",
      "e": "AQAB",
      "n": "ojOf8-6mttwgZtFKF_ckGM0RV0ghydu7R-DEGpFqC_Iq-rMqixDiFd80_ZtKlf8tP30jHoyIi76B8jtJhWEu3q25UfRZmLFCEe9yl7ECdEv7iTG9J8ChUBd2RrzifkHdps1mz1KQzXPJip1Q_2iT7J86ZCvg3zXxHPwpg8BO4ZvrAu6JnoptlEkuh-aEjgjnWeH-FlQ-MNbrcaZi8yD22mhUlSyF8eL59g19l0t7pUOT64T64cgcnM0vkljsInwgxM4aTDDohZ6NlfHd_vyb06QgzLajgFkd27OGzbshIPD5jf_mXgzBZYL9guAOQsJoH3l0vMqzf5v09Inz5FA-AQ",
      "d": "Q4lrWI5OLGWLuEP8re3hwdHhHo6SQxKS8GLORKG3njG_Y5OmazEkcib2UYKPk9UkagMY1Gd_k3BHeyzkd_z3HxEG0PEFPWbRM9aXyEGT3AinO4DSWfMDN-4M-Xc8d4XFXOa5oCmo0WMcz_Kg_zf0H9pM3djHUSmMs5zrTEiOj7FKfmacgYsmJWiLnlAKfmg96k8EKbURVfeV4lgZvkzc-DiXa93dp8OIC4fIGV2Cr0o2rQYr3qEsm2dodjkRjvxq3veu4exL_uYLu2tHvzm3kdSQ6UQG8FG9dVKWsDq7YJkQUrxqe0lmNW-P114ACnZ7uaygIQGTFIqCWB1oC6SiYQ",
      "p": "zTtc6Ngnrqpm2jgjsdT6Ld7bZURSKxZtT-IQrP9Bl7rNP04NqtiVmK0_JtlVaUjy7GGNzkhfECH6Y7NIMQuzWpNqJ2tGYTw5VKNCSGwttxTvvlM0XZ41FzE8xG93Jy71SJWzfqZg87sfzhNPC9GbqAnhNRuNE2w_oUSIdI1gM08",
      "q": "ylNN1FVvzr6ER8npBEVz7Z4HSWjGTiPcP_rtVROOqSbCMC3twg3O65zddxQRRwBoA7WGXRzHIfn_ambk0-4zVGImWg8eLuvcwScBgrZLMJap2g09EuH7-DCUhFly7LR5d_hncyDIFWzw-5Xc5FeSnbWZzsZgCRcPj1QlOGgBZa8",
      "dp": "VXtRUoad2KmPF3tkmrBr1-lIFqDjXKEFvM0bRwM2rd0XmjOHi86UWLhoYmUJp0XEMOLdLruEJPMSGK7W9d66wJJAF9UdxmhQNMmfVQ0B6Yzl4q9DU28PC-7tMaB_z6lGbwozbiAcp8gcEjiGpxSSMeaTkAw-sYVX53LlJy6vkA8",
      "dq": "TOlH78yemRgBTwzpIG-KdyyYloZWxAOLyWKI78tkctbCAPCkrhzzqMtTwN2HB9eWsrpBnom_BSJ0vqzolcu6BXzeGWQf1DRjEaXTRC4p_Ql4_eQEcHYpg5psCre6IVMlzb5HDRPMG6DzJqA8fSzxFjSfiGQOw8XZa2HQWFXpGg8",
      "qi": "Jy2XPqpZ5ANHIdZnmWWuDmzN8DuN1XeE4IysPMZ8MCgVs1eSX7reJz9PLx1kRJcnlLjZiROPVlecmIoBk6aGroXqXt1oaCnYSYQNTcWj6kosn48zCzqH-RPkCembuFjRxfw1ol6aAyrloyYCCt29xBSjinUltCt0_a-ZgSiqSds"
    }
  ]
}

.well-known-keys and keys.json are in the same folder and I spun up a simple http server using: python3 -m http.server 5604

I created a token locally using the following script:

const JWKeys = fs.readFileSync("keys.json");

const keyStore = await jose.JWK.asKeyStore(JWKeys.toString());

const [key] = keyStore.all({ use: "sig" });

const opt = { compact: true, jwk: key, fields: { typ: "jwt" } };

const payload = JSON.stringify({
   "sub": "cwperx",
   "preferred_username": "Craig Perkins",
   "roles": "admin,kibanauser",
   "iat": 1516239022
});

const token = await jose.JWS.createSign(opt, key).update(payload).final();

and finally verified Bearer auth by using curl:

curl -XGET 'https://localhost:9200/_plugins/_security/api/account'  --insecure \
--header 'Authorization: Bearer <JWT>'
{
    "user_name": "Craig Perkins",
    "is_reserved": false,
    "is_hidden": false,
    "is_internal_user": false,
    "user_requested_tenant": null,
    "backend_roles": [
        "admin",
        "kibanauser"
    ],
    "custom_attribute_names": [
        "attr.jwt.sub",
        "attr.jwt.preferred_username",
        "attr.jwt.iat",
        "attr.jwt.roles"
    ],
    "tenants": {
        "Craig Perkins": true,
        "global_tenant": true,
        "admin_tenant": true
    },
    "roles": [
        "own_index",
        "kibana_user",
        "all_access"
    ]
}

[rursprung]

sorry, i just realised that i never replied here! that's good news that this would theoretically work, but the workaround with going via another service seems really cumbersome and brittle (there's no guarantee that this will continue to work). so it'd be great if the usage of JWKS could be officially & properly supported with JWT as the auth method (instead of openid).

[cthtrifork]

I think jwks_uri should be part of the type: jwt configuration as an alternativ to the signing_keyin http_authenticator