[Bug]: JWT Auth using Microsoft OAuth does not authorize scopes/roles properly [Enterprise feature]

[domg8man]

What happened?

Bug description:
I was able to request a token from Microsoft and make a successful request against the litellm proxy even though I should not have been authorized to do so, since the requestor did not have the admin_jwt_scope.

Expected response: 401 Unauthorized
Actual response: 200 OK

Logs:

lite-llm-proxy {"@timestamp":"2024-11-18T12:58:17.621Z","log.level":"Info","message":"HTTP Request: GET https://login.microsoftonline.com/{tenant_id_redacted}/discovery/v2.0/keys \"HTTP/1.1 200 OK\"","ecs.version":"1.6.0","log":{"logger":"httpx","origin":{"file":{"line":1786,"name":"_client.py"},"function":"_send_single_request"},"original":"HTTP Request: GET https://login.microsoftonline.com/{tenant_id_redacted}/discovery/v2.0/keys \"HTTP/1.1 200 OK\""},"process":{"name":"MainProcess","pid":1,"thread":{"id":140150023027584,"name":"MainThread"}}}
lite-llm-proxy {"@timestamp":"2024-11-18T12:58:17.629Z","log.level":"Info","message":"HTTP Request: POST http://localhost:39045/ \"HTTP/1.1 200 OK\"","ecs.version":"1.6.0","log":{"logger":"httpx","origin":{"file":{"line":1786,"name":"_client.py"},"function":"_send_single_request"},"original":"HTTP Request: POST http://localhost:39045/ \"HTTP/1.1 200 OK\""},"process":{"name":"MainProcess","pid":1,"thread":{"id":140150023027584,"name":"MainThread"}}}
lite-llm-proxy {"message": "\nLiteLLM completion() model= gpt-4o; provider = azure", "level": "INFO", "timestamp": "2024-11-18T12:58:17.695466"}
lite-llm-proxy {"@timestamp":"2024-11-18T12:58:17.695Z","log.level":"Info","message":"\nLiteLLM completion() model= gpt-4o; provider = azure","ecs.version":"1.6.0","log":{"logger":"LiteLLM","origin":{"file":{"line":2760,"name":"utils.py"},"function":"_check_valid_arg"},"original":"\nLiteLLM completion() model= gpt-4o; provider = azure"},"process":{"name":"MainProcess","pid":1,"thread":{"id":140149811775168,"name":"ThreadPoolExecutor-2_0"}}}
lite-llm-proxy {"@timestamp":"2024-11-18T12:58:18.597Z","log.level":"Info","message":"HTTP Request: POST https://{azure_openai_instance_name_redacted}.openai.azure.com//openai/deployments/gpt-4o/chat/completions?api-version=2023-05-15 \"HTTP/1.1 200 OK\"","ecs.version":"1.6.0","log":{"logger":"httpx","origin":{"file":{"line":1786,"name":"_client.py"},"function":"_send_single_request"},"original":"HTTP Request: POST https://{azure_openai_instance_name_redacted}.openai.azure.com//openai/deployments/gpt-4o/chat/completions?api-version=2023-05-15 \"HTTP/1.1 200 OK\""},"process":{"name":"MainProcess","pid":1,"thread":{"id":140150023027584,"name":"MainThread"}}}
lite-llm-proxy {"message": "litellm.acompletion(model=azure/gpt-4o)\u001b[32m 200 OK\u001b[0m", "level": "INFO", "timestamp": "2024-11-18T12:58:18.599686"}

We can see here in the logs that an OAuth token was checked with Microsoft for validity on the /discovery/v2.0/keys endpoint - so far so good (it probably even checked the audience (aud) - also good). But there does not seem to follow any check of the scopes (scp) or appRoles (roles) fields on the token after token integrity is confirmed. It just authorizes the request to succeed with 200.

Therefore any application within the same Microsoft tenant (without need for scopes or appRoles) can just make a request and get a 200 response from the litellm proxy instance.

How to reproduce:
I have setup our litellm proxy instance following this guide for JWT auth:
https://docs.litellm.ai/docs/proxy/token_auth

We are using Microsoft as an OAuth provider.

These are the steps to reproduce:

1. Set required litellm configuration:

enable_jwt_auth: true
litellm_jwtauth:
  admin_jwt_scope: "litellm_proxy_endpoints_access"
  admin_allowed_routes:
    - openai_routes
    - info_routes
  end_user_id_jwt_field: "appid"
  public_key_ttl: 600

2. Set the required environment variables for litellm proxy:

JWT_PUBLIC_KEY_URL="https://login.microsoftonline.com/{tenant_id}/discovery/v2.0/keys"
JWT_AUDIENCE="api://LiteLLM_Proxy-dev"

3. Create app registration for litellm proxy in Azure for the service (see manifest):

{
  "name": "LiteLLM_Proxy",
  "identifierUris": [
    "api://LiteLLM_Proxy-dev"
  ],
  "api": {
    "oauth2PermissionScopes": [  # JWT scopes
      {
        "adminConsentDescription": "Access LiteLLM Proxy Endpoints",
        "adminConsentDisplayName": "Access LiteLLM Proxy Endpoints",
        "id": "211c4fae-4a66-430a-acbf-3c6cc7cec121",  # a uuidv4 identifier I created for the JWT scope
        "isEnabled": true,
        "type": "User",
        "userConsentDescription": "Access LiteLLM Proxy Endpoints",
        "userConsentDisplayName": "Access LiteLLM Proxy Endpoints",
        "value": "litellm_proxy_endpoints_access"  # this is the scope name we used in the litellm config
      }
    ]
  }
}

4. Create another app registration with app id ($MICROSOFT_CLIENT_ID) for the consumer service that calls litellm proxy - we will call it My_Service for that matter.
5. Create a client secret ($MICROSOFT_CLIENT_SECRET) for My_Service in the azure portal. Note that we do not configure any scopes or appRoles for My_Service.
6. Request an access token for My_Service

curl -s -X POST \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "client_id=$MICROSOFT_CLIENT_ID" \
  -d "scope=api://LiteLLM_Proxy-dev/.default" \
  -d "client_secret=$MICROSOFT_CLIENT_SECRET" \
  -d "grant_type=client_credentials" \
  "https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token"

7. Call litellm proxy using the access token received from Microsoft:

curl 'http://localhost:5000/chat/completions' \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer <access token>" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "hello gpt-4o"
      }
    ]
  }'

8. Receive a 200 OK response.

How to fix this and feedback on how we would like this to work:

This is a diagram of the authorization flow we want to use for our consumers, which are services using OAuth:

Now based on my research and experience from setting our own Python APIs up with Microsoft OAuth, there is a false assumption here:

• scopes are used, which are a delegated type of permission and not returned as fields on the JWT, when requesting an access token from Microsoft using the client_credentials flow (see CURL above in steps to reproduce). Hence there cannot be any checks done based on scopes in the code.

As far as I understand, scopes have another use case (for delegated frontend access), and appRoles would be the correct choice for this use case. There's also the distinction between application and user. There are probably other articles out there, but I found this one quickly:
https://medium.com/azurehelp/roles-and-scopes-in-azure-identity-f201d11e253c

Now it could be that we are using your JWT auth feature wrongly for services and it was intended for users via the UI. But in any case this is a bug - and, if you will, in addition to that, a feature request to make this work properly for services.

My suggestion:

• use appRoles, since appRoles are meant to be used for service to service authorization and will be included as a field (roles) in the token, when performing the client_credentials flow.

Here's an example app registration defining app roles and a consumer app that has been assigned roles (see manifests).
LiteLLM_Proxy:

{
  "name": "LiteLLM_Proxy",
  "identifierUris": [
    "api://LiteLLM_Proxy-dev"
  ],
  "appRoles": [
    {
      "allowedMemberTypes": [
        "Application"
      ],
      "description": "Grants access to make request against the LiteLLM endpoints.",
      "displayName": "Access to LiteLLM Endpoints",
      "isEnabled": true,
      "value": "LiteLLM.Admin"
    }
  ]
}

My_Service:

{
  "name": "My_Service",
  "requiredResourceAccess": [
    {
      "resourceAccess": [
        {
          "id": "LiteLLM.Admin",
          "type": "Role"
        }
      ],
      "resourceAppId": "9e2558fb-ed21-42fe-a4e7-e3d9e3478481"  # azure app id (client id) of litellm proxy
    }
  ]
}

Finally, it would be great to make the JWT auth roles more generic by allowing to provide a list of roles that can be mapped to allowed_routes. Currently everything is prefixed with admin_ which assumes one uses Oauth scopes for users and assumes one uses it for admins. We do neither, we would like to use it for services.

Relevant log output

No response

Twitter / LinkedIn details

No response

[krrishdholakia]

Request an access token for My_Service

can i see what the decoded access token for my_service looks like? @domg8man

[krrishdholakia]

able to repro

[krrishdholakia]

is the expectation here - since user doesn't have admin access they can't shouldn't be able to make LLM API requests? @domg8man

currently it's just checking if it's a valid jwt (which the 2nd one is) and allowing the request

the scopes are used for things like cost tracking etc.

to enforce route by role (non-admin) we can expose a public_routes field on jwt, to block any authenticated but not scoped token from making llm api requests

[krrishdholakia]

allowing to provide a list of roles that can be mapped to allowed_routes

can you share how you'd expect to set this on the config @domg8man