[Azure.AI.Inference] Beta 2 (#45757)

Features Added Added new EmbeddingsClient, to provide support for generating text embeddings using supported models. Add support for passing a string file path on disk in order to provide an image for chat completions. Breaking Changes ChatCompletionsClientOptions has been renamed to AzureAIInferenceClientOptions. ChatCompletions response object has been flattened. ChatCompletions.Choices has been removed, and the underlying properties have been bubbled up to be on the ChatCompletions object instead. ChatCompletionsFunctionToolCall has been replaced with ChatCompletionsToolCall. ChatCompletionsFunctionToolDefinition has been replaced with ChatCompletionsToolDefinition. ChatCompletionsToolSelectionPreset has been replaced with ChatCompletionsToolChoicePreset. ChatCompletionsNamedFunctionToolSelection has been replaced with ChatCompletionsNamedToolChoice. ChatCompletionsFunctionToolSelection has been replaced with ChatCompletionsNamedToolChoiceFunction. Bugs Fixed Fixed support for chat completions streaming while using tools. Other Changes Removed the need to manually provide an api-key header when talking to Azure OpenAI. ---------------------------------------------------------- Signed-off-by: Travis Angevine <trangevi@microsoft.com> Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
Azure · Oct 24, 2024 · 3db918d · 3db918d
1 parent 6d72f21
commit 3db918d
Show file tree

Hide file tree

Showing 107 changed files with 5,778 additions and 2,240 deletions.
diff --git a/.vscode/cspell.json b/.vscode/cspell.json
@@ -183,6 +183,12 @@
         "warnaserror"
       ]
     },
+    {
+      "filename": "**/sdk/ai/**/*.cs",
+      "words": [
+        "Ubinary"
+      ]
+    },
     {
       "filename": "**/sdk/analysisservices/**/*.cs",
       "words": [

diff --git a/sdk/ai/Azure.AI.Inference/CHANGELOG.md b/sdk/ai/Azure.AI.Inference/CHANGELOG.md
@@ -1,14 +1,27 @@
 # Release History
 
-## 1.0.0-beta.2 (Unreleased)
+## 1.0.0-beta.2 (2024-10-24)
 
 ### Features Added
+- Added new `EmbeddingsClient`, to provide support for generating text embeddings using supported models.
+- Add support for passing a string file path on disk in order to provide an image for chat completions.
 
 ### Breaking Changes
+- `ChatCompletionsClientOptions` has been renamed to `AzureAIInferenceClientOptions`.
+- `ChatCompletions` response object has been flattened. `ChatCompletions.Choices` has been removed, and the underlying properties have been bubbled up to be on the `ChatCompletions` object instead.
+- `ChatCompletionsFunctionToolCall` has been replaced with `ChatCompletionsToolCall`.
+- `ChatCompletionsFunctionToolDefinition` has been replaced with `ChatCompletionsToolDefinition`.
+- `ChatCompletionsToolSelectionPreset` has been replaced with `ChatCompletionsToolChoicePreset`.
+- `ChatCompletionsNamedFunctionToolSelection` has been replaced with `ChatCompletionsNamedToolChoice`.
+- `ChatCompletionsFunctionToolSelection` has been replaced with `ChatCompletionsNamedToolChoiceFunction`.
+- `StreamingChatCompletionsUpdate.AuthorName` has been removed
+- Removed `extraParams` from the `complete` and `completeAsync` methods. It is now set implicitly if `additionalProperties` is provided in the options object.
 
 ### Bugs Fixed
+- Fixed support for chat completions streaming while using tools.
 
 ### Other Changes
+- Removed the need to manually provide an `api-key` header when talking to Azure OpenAI.
 
 ## 1.0.0-beta.1 (2024-08-06)
 ### Features Added

diff --git a/sdk/ai/Azure.AI.Inference/README.md b/sdk/ai/Azure.AI.Inference/README.md
@@ -53,7 +53,7 @@ The package includes `ChatCompletionsClient` <!-- and `EmbeddingsClient`and `Ima
 var endpoint = new Uri(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_ENDPOINT"));
 var credential = new AzureKeyCredential(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_KEY"));
 
-var client = new ChatCompletionsClient(endpoint, credential, new ChatCompletionsClientOptions());
+var client = new ChatCompletionsClient(endpoint, credential, new AzureAIInferenceClientOptions());
 ```
 
 <!--
@@ -90,7 +90,7 @@ All clients provide a `get_model_info` method to retrive AI model information. T
 var endpoint = new Uri(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_ENDPOINT"));
 var credential = new AzureKeyCredential(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_KEY"));
 
-var client = new ChatCompletionsClient(endpoint, credential, new ChatCompletionsClientOptions());
+var client = new ChatCompletionsClient(endpoint, credential, new AzureAIInferenceClientOptions());
 Response<ModelInfo> modelInfo = client.GetModelInfo();
 
 Console.WriteLine($"Model name: {modelInfo.Value.ModelName}");
@@ -106,13 +106,13 @@ The `ChatCompletionsClient` has a method named `complete`. The method makes a RE
 
 See simple chat completion examples below. More can be found in the [samples](https://aka.ms/azsdk/azure-ai-inference/csharp/samples) folder.
 
-<!--
 ### Text Embeddings
 
-The `EmbeddingsClient` has a method named `embedding`. The method makes a REST API call to the `/embeddings` route on the provided endpoint, as documented in [the REST API reference](https://learn.microsoft.com/azure/ai-studio/reference/reference-model-inference-embeddings).
+The `EmbeddingsClient` has a method named `embed`. The method makes a REST API call to the `/embeddings` route on the provided endpoint, as documented in [the REST API reference](https://learn.microsoft.com/azure/ai-studio/reference/reference-model-inference-embeddings).
 
 See simple text embedding example below. More can be found in the [samples](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/ai/azure-ai-inference/samples) folder.
 
+<!--
 ### Image Embeddings
 
 TODO: Add overview and link to explain image embeddings.
@@ -150,7 +150,7 @@ In the following sections you will find simple examples of:
 * [Chat completions](#chat-completions-example)
 * [Streaming chat completions](#streaming-chat-completions-example)
 * [Chat completions with additional model-specific parameters](#chat-completions-with-additional-model-specific-parameters)
-<!-- * [Text Embeddings](#text-embeddings-example) -->
+* [Text Embeddings](#text-embeddings-example)
 <!-- * [Image Embeddings](#image-embeddings-example) -->
 
 The examples create a client as mentioned in [Create and authenticate a client directly, using key](#create-and-authenticate-a-client-directly-using-key). Only mandatory input settings are shown for simplicity.
@@ -165,7 +165,7 @@ This example demonstrates how to generate a single chat completions, with key au
 var endpoint = new Uri(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_ENDPOINT"));
 var credential = new AzureKeyCredential(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_KEY"));
 
-var client = new ChatCompletionsClient(endpoint, credential, new ChatCompletionsClientOptions());
+var client = new ChatCompletionsClient(endpoint, credential, new AzureAIInferenceClientOptions());
 
 var requestOptions = new ChatCompletionsOptions()
 {
@@ -177,12 +177,12 @@ var requestOptions = new ChatCompletionsOptions()
 };
 
 Response<ChatCompletions> response = client.Complete(requestOptions);
-System.Console.WriteLine(response.Value.Choices[0].Message.Content);
+System.Console.WriteLine(response.Value.Content);
 ```
 
 The following types or messages are supported: `SystemMessage`,`UserMessage`, `AssistantMessage`, `ToolMessage`. See also samples:
 
-* [Sample5_ChatCompletionsWithImageUrl.md](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/ai/Azure.AI.Inference/samples/Sample5_ChatCompletionsWithImageUrl.md) for usage of `UserMessage` that includes sending an image URL.
+* [Sample5_ChatCompletionsWithImages.md](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/ai/Azure.AI.Inference/samples/Sample5_ChatCompletionsWithImages.md) for usage of `UserMessage` that includes sending an image URL or image data from a local file.
 * [Sample7_ChatCompletionsWithTools.md](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/ai/Azure.AI.Inference/samples/Sample7_ChatCompletionsWithTools.md) for usage of `ToolMessage`.
 
 Alternatively, you can read a `BinaryData` object based on a JSON string instead of using the strongly typed classes like `ChatRequestSystemMessage` and `ChatRequestUserMessage`:
@@ -191,7 +191,7 @@ Alternatively, you can read a `BinaryData` object based on a JSON string instead
 var endpoint = new Uri(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_ENDPOINT"));
 var credential = new AzureKeyCredential(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_KEY"));
 
-var client = new ChatCompletionsClient(endpoint, credential, new ChatCompletionsClientOptions());
+var client = new ChatCompletionsClient(endpoint, credential, new AzureAIInferenceClientOptions());
 
 var requestOptions = new ChatCompletionsOptions()
 {
@@ -207,7 +207,7 @@ BinaryData messages = BinaryData.FromString(jsonMessages);
 requestOptions = ModelReaderWriter.Read<ChatCompletionsOptions>(messages);
 
 Response<ChatCompletions> response = client.Complete(requestOptions);
-System.Console.WriteLine(response.Value.Choices[0].Message.Content);
+System.Console.WriteLine(response.Value.Content);
 ```
 
 To generate completions for additional messages, simply call `client.Complete` multiple times using the same `client`.
@@ -220,7 +220,7 @@ This example demonstrates how to generate a single chat completions with streami
 var endpoint = new Uri(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_ENDPOINT"));
 var credential = new AzureKeyCredential(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_KEY"));
 
-var client = new ChatCompletionsClient(endpoint, credential, new ChatCompletionsClientOptions());
+var client = new ChatCompletionsClient(endpoint, credential, new AzureAIInferenceClientOptions());
 
 var requestOptions = new ChatCompletionsOptions()
 {
@@ -255,15 +255,13 @@ In this example, extra JSON elements are inserted at the root of the request bod
 
 Note that by default, the service will reject any request payload that includes unknown parameters (ones that are not defined in the REST API [Request Body table](https://learn.microsoft.com/azure/ai-studio/reference/reference-model-inference-chat-completions#request-body)). In order to change the default service behaviour, when the `Complete` method includes `AdditonalProperties`, the client library will automatically add the HTTP request header `"unknown_params": "pass-through"`.
 
-<!-- The input argument `Additional` is not restricted to chat completions. It is suppored on other client methods as well. -->
-
 Azure_AI_Inference_ChatCompletionsWithAdditionalPropertiesScenario
 
 ```C# Snippet:Azure_AI_Inference_ChatCompletionsWithAdditionalPropertiesScenario
 var endpoint = new Uri(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_ENDPOINT"));
 var credential = new AzureKeyCredential(System.Environment.GetEnvironmentVariable("AZURE_AI_CHAT_KEY"));
 
-var client = new ChatCompletionsClient(endpoint, credential, new ChatCompletionsClientOptions());
+var client = new ChatCompletionsClient(endpoint, credential, new AzureAIInferenceClientOptions());
 
 var requestOptions = new ChatCompletionsOptions()
 {
@@ -278,13 +276,25 @@ Response<ChatCompletions> response = client.Complete(requestOptions);
 System.Console.WriteLine(response.Value.Choices[0].Message.Content);
 ```
 
-<!--
 ### Text Embeddings example
 
 This example demonstrates how to get text embeddings, with key authentication, assuming `endpoint` and `key` are already defined.
 
-```C#
+```C# Snippet:Azure_AI_Inference_BasicEmbedding
+var endpoint = new Uri(System.Environment.GetEnvironmentVariable("AZURE_AI_EMBEDDINGS_ENDPOINT"));
+var credential = new AzureKeyCredential(System.Environment.GetEnvironmentVariable("AZURE_AI_EMBEDDINGS_KEY"));
 
+var client = new EmbeddingsClient(endpoint, credential, new AzureAIInferenceClientOptions());
+
+var input = new List<string> { "King", "Queen", "Jack", "Page" };
+var requestOptions = new EmbeddingsOptions(input);
+
+Response<EmbeddingsResult> response = client.Embed(requestOptions);
+foreach (EmbeddingItem item in response.Value.Data)
+{
+    List<float> embedding = item.Embedding.ToObjectFromJson<List<float>>();
+    Console.WriteLine($"Index: {item.Index}, Embedding: <{string.Join(", ", embedding)}>");
+}
 ```
 
 The length of the embedding vector depends on the model, but you should see something like this:
@@ -296,7 +306,6 @@ data[2]: length=1024, [0.04196167, 0.029083252, ..., -0.0027484894, 0.0073127747
 ```
 
 To generate embeddings for additional phrases, simply call `client.embed` multiple times using the same `client`.
--->
 
 <!--
 ### Image Embeddings example