AzureMLChatOnlineEndpoint - outdated endpoint_url validation

[psaks]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The following code:

chat = AzureMLChatOnlineEndpoint(
    endpoint_url="https://Mistral-large-2407-rag.swedencentral.models.ai.azure.com/chat/completions",
    endpoint_api_type=AzureMLEndpointApiType.serverless,
    endpoint_api_key="xxxxxxxxxxxxxxx"
)

Error Message and Stack Trace (if applicable)

Traceback (most recent call last):

File ~/opt/miniconda3/envs/envml/lib/python3.11/site-packages/spyder_kernels/py3compat.py:356 in compat_exec
exec(code, globals, locals)

File ~/Programming/temp/test_mistral_2407.py:26
chat = AzureMLChatOnlineEndpoint(

File ~/opt/miniconda3/envs/envml/lib/python3.11/site-packages/langchain_core/load/serializable.py:113 in init
super().init(*args, **kwargs)

File pydantic/main.py:341 in pydantic.main.BaseModel.init

ValidationError: 1 validation error for AzureMLChatOnlineEndpoint
endpoint_api_type
Endpoints of type serverless should follow the format https://<your-endpoint>.<your_region>.inference.ml.azure.com/v1/chat/completions or https://<your-endpoint>.<your_region>.inference.ml.azure.com/v1/chat/completions (type=value_error)

Description

Azure have updated their endpoint urls to exclude v1 from /v1/chat/completions.

The broken code is in (ln 457);

https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/llms/azureml_endpoint.py

        if field_value == AzureMLEndpointApiType.serverless and not (
            endpoint_url.endswith("/v1/completions")  # type: ignore[union-attr]
            or endpoint_url.endswith("/v1/chat/completions")  # type: ignore[union-attr]
        ):

Easy to fix by removing v1.

System Info

langchain==0.2.14
langchain-aws==0.1.6
langchain-community==0.2.12
langchain-core==0.2.34
langchain-openai==0.1.8
langchain-text-splitters==0.2.1

Python 3.11.4

[ashvin-a]

I have addressed this issue in #25725

[herrmayr]

It seems that the endpoint schema has been changed again, leading to a validation error?

Example for a current deployment of a endpoint url for a serverless llama3.3:70B:
https://ai-w3ujaf023234.services.ai.azure.com/models/chat/completions?api-version=2024-05-01-preview

When using this url, I get a pydantic validation error:

Value error, Endpoints of type serverless should follow the format https://<your-endpoint>.<your_region>.inference.ml.azure.com/chat/completions or https://<your-endpoint>.<your_region>.inference.ml.azure.com/chat/completions [type=value_error, input_value=<AzureMLEndpointApiType.serverless: 'serverless'>, input_type=AzureMLEndpointApiType]

Without the ?api-version=2024-05-01-preview , the endpoint complains of missing parameters:

Error code: 401 - {'statusCode': 401, 'message': 'Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.'}

[ashvin-a]

Hmm.......I'll look into it.

[ashvin-a]

I have updated the endpoints in this PR. The changes will be reflected once its merged, As for the part where you pass the API version, could you try out passing that via model_kwargs as a dictionary with key api_version @herrmayr