Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Search] GA Features for API Version 2024-07-01 #44485

Merged
merged 15 commits into from
Jul 17, 2024
Merged
36 changes: 32 additions & 4 deletions sdk/search/Azure.Search.Documents/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,42 @@
# Release History

## 11.6.0-beta.5 (Unreleased)
## 11.6.0 (2024-07-17)

### Features Added
- Added support for `2024-07-01` service version.
- `SemanticSearchOptions` now supports `SemanticQuery`, which allows for specifying a semantic query that is only used
for semantic reranking.
- `VectorQuery` now supports `Oversampling` and `Weight`, which allows for specifying richer configurations on how
vector queries affect search results.
- Added support for `VectorizableTextQuery`, which allows for passing a text-based query that is vectorized service-side
by `VectorSearchVectorizer`s configured on the index so that vectorization doesn't need to happen before querying.
- Added support for "bring your own endpoint" with `VectorSearchVectorizer`, with implementations `AzureOpenAIVectorizer`
and `WebApiVectorizer`. This enables the service to use a user-provided configuration for vectorizing text, rather
than requiring all client-side calls to vectorize before querying, allowing for easier standardization of vectorization.
- Added support for compression with `VectorSearchCompression`, with implementations `BinaryQuantizationCompression`
and `ScalarQuantizationCompression`. This allows for reducing the size of vectors in the index, which can reduce
storage costs and improve querying performance.
- Added support for `VectorEncodingFormat`, which allows for specifying the encoding format of the vector data.
- Added support for `AzureOpenAIEmbeddingSkill`, which is a skill that uses the Azure OpenAI service to create text
embeddings during indexing.
- Added support for index projections with `SearchIndexerIndexProjection`, which allows for specifying how indexed
documents are projected in the index (or indexes).
- Added support for "narrow" types in `SearchFieldDataType`. This allows for specifying smaller types for vector fields
to reduce storage costs and improve querying performance.
- Added support for `SearchIndexerDataIdentity`, which allows for specifying the identity for the data source for the
indexer.
- `SearchField` and `SearchableField` now support `IsStored` and `VectorEncodingFormat` configurations. `IsStored` allows
for specifying behaviors on how the index will retain vector data (enabling the ability to reduce storage costs), and
`VectorEncodingFormat` allows for specifying the encoding format of the vector data.
- `OcrSkill` now supports `LineEnding`, which allows for specifying the line ending character used by the OCR skill.
- `SplitSkill` now supports `MaximumPagesToTake` and `PageOverlapLength`, which allows for specifying how the split
skill behaves when splitting documents into pages.
- `SearchServiceLimits` now supports `MaxStoragePerIndexInBytes`, which shows the maximum storage allowed per index.

### Breaking Changes

### Bugs Fixed

### Other Changes
- All service concepts that have been in preview but not included in the `2024-07-01` GA have been removed. This
includes concepts such as index aliases, normalizers, Azure Machine Learning skills, hybrid search, and more.

## 11.6.0-beta.4 (2024-05-06)

Expand Down

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion sdk/search/Azure.Search.Documents/assets.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@
"AssetsRepo": "Azure/azure-sdk-assets",
"AssetsRepoPrefixPath": "net",
"TagPrefix": "net/search/Azure.Search.Documents",
"Tag": "net/search/Azure.Search.Documents_52c4e21bfa"
"Tag": "net/search/Azure.Search.Documents_cd3c397d05"
}
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,6 @@ SearchIndex index = new SearchIndex("movies")
SearchableField genreField = new SearchableField("genre")
{
AnalyzerName = LexicalAnalyzerName.Values.EnLucene,
NormalizerName = LexicalNormalizerName.Lowercase,
IsFacetable = true,
IsFilterable = true
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ We will create an instace of `SearchIndex` and define `Hotel` fields.
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Reduced_Vector_Search_Index
string vectorSearchProfileName = "my-vector-profile";
string vectorSearchHnswConfig = "my-hsnw-vector-config";
string deploymentId = "my-text-embedding-3-small";
string deploymentName = "my-text-embedding-3-small";
int modelDimensions = 256; // Here's the reduced model dimensions

string indexName = "hotel";
Expand All @@ -34,7 +34,7 @@ SearchIndex searchIndex = new(indexName)
{
new VectorSearchProfile(vectorSearchProfileName, vectorSearchHnswConfig)
{
Vectorizer = "openai"
VectorizerName = "openai"
}
},
Algorithms =
Expand All @@ -45,11 +45,11 @@ SearchIndex searchIndex = new(indexName)
{
new AzureOpenAIVectorizer("openai")
{
AzureOpenAIParameters = new AzureOpenAIParameters()
Parameters = new AzureOpenAIVectorizerParameters()
{
ResourceUri = new Uri(Environment.GetEnvironmentVariable("OPENAI_ENDPOINT")),
ApiKey = Environment.GetEnvironmentVariable("OPENAI_KEY"),
DeploymentId = deploymentId,
DeploymentName = deploymentName,
ModelName = AzureOpenAIModelName.TextEmbedding3Small
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,6 @@ SearchResults<Hotel> response = await searchClient.SearchAsync<Hotel>(
QueryCaption = new(QueryCaptionType.Extractive),
QueryAnswer = new(QueryAnswerType.Extractive)
},
QueryLanguage = QueryLanguage.EnUs,
QueryType = SearchQueryType.Semantic,
});

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ We will create an instace of `SearchIndex` and define `Hotel` fields.
```C# Snippet:Azure_Search_Documents_Tests_Samples_Sample07_Vector_Search_Index_UsingVectorizableTextQuery
string vectorSearchProfileName = "my-vector-profile";
string vectorSearchHnswConfig = "my-hsnw-vector-config";
string deploymentId = "text-embedding-ada-002";
string deploymentName = "text-embedding-ada-002";
int modelDimensions = 1536;

string indexName = "hotel";
Expand All @@ -32,7 +32,7 @@ SearchIndex searchIndex = new(indexName)
{
new VectorSearchProfile(vectorSearchProfileName, vectorSearchHnswConfig)
{
Vectorizer = "openai"
VectorizerName = "openai"
}
},
Algorithms =
Expand All @@ -43,11 +43,11 @@ SearchIndex searchIndex = new(indexName)
{
new AzureOpenAIVectorizer("openai")
{
AzureOpenAIParameters = new AzureOpenAIParameters()
Parameters = new AzureOpenAIVectorizerParameters()
{
ResourceUri = new Uri(Environment.GetEnvironmentVariable("OPENAI_ENDPOINT")),
ApiKey = Environment.GetEnvironmentVariable("OPENAI_KEY"),
DeploymentId = deploymentId,
DeploymentName = deploymentName,
ModelName = AzureOpenAIModelName.TextEmbeddingAda002
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,6 @@ SearchResults<Hotel> response = await searchClient.SearchAsync<Hotel>(
QueryCaption = new(QueryCaptionType.Extractive),
QueryAnswer = new(QueryAnswerType.Extractive)
},
QueryLanguage = QueryLanguage.EnUs,
QueryType = SearchQueryType.Semantic
});

Expand Down Expand Up @@ -177,7 +176,6 @@ SearchResults<Hotel> response = await searchClient.SearchAsync<Hotel>(
QueryAnswer = new(QueryAnswerType.Extractive),
SemanticQuery = "Is there any hotel located on the main commercial artery of the city in the heart of New York?"
},
QueryLanguage = QueryLanguage.EnUs,
});

int count = 0;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<AssemblyTitle>Microsoft Azure.Search.Documents client library</AssemblyTitle>
<Version>11.6.0-beta.5</Version>
<Version>11.6.0</Version>
<!--The ApiCompatVersion is managed automatically and should not generally be modified manually.-->
<ApiCompatVersion>11.5.1</ApiCompatVersion>
<Description>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ public SearchIndexingBufferedSender(
/// </summary>
#pragma warning disable CA1816 // Dispose methods should call SuppressFinalize
void IDisposable.Dispose() =>
DisposeAsync(async: false).EnsureCompleted();
DisposeInternalAsync(async: false).EnsureCompleted();
#pragma warning restore CA1816 // Dispose methods should call SuppressFinalize

/// <summary>
Expand All @@ -155,7 +155,7 @@ void IDisposable.Dispose() =>
/// </returns>
#pragma warning disable CA1816 // Dispose methods should call SuppressFinalize
async ValueTask IAsyncDisposable.DisposeAsync() =>
await DisposeAsync(async: true).ConfigureAwait(false);
await DisposeInternalAsync(async: true).ConfigureAwait(false);
#pragma warning restore CA1816 // Dispose methods should call SuppressFinalize

/// <summary>
Expand All @@ -165,7 +165,7 @@ async ValueTask IAsyncDisposable.DisposeAsync() =>
/// </summary>
/// <param name="async">Whether to call this sync or async.</param>
/// <returns>A Task that will wait until we're disposed.</returns>
internal async Task DisposeAsync(bool async)
internal async Task DisposeInternalAsync(bool async)
{
if (Interlocked.CompareExchange(ref _disposed, 1, 0) == 0)
{
Expand Down
Loading
Loading