Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FTS: Hybrid search query pipeline #42885

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

aayush3011
Copy link
Member

@aayush3011 aayush3011 commented Nov 11, 2024

Description

This PR adds Full text search and hybrid search support.

  • Adds hybrid search query pipeline stage HybridSearchDocumentQueryExecutionContext
  • Adds a new query feature HybridSearch
  • Adds a new query info HybridSearchQueryInfo

Design doc

FTS queries supported now:

  • SELECT TOP 50 c.id, c.abstract, c.title FROM c WHERE FullTextContains(c.abstract, 'quantum')
  • SELECT TOP 50 c.id, c.abstract, c.title FROM c WHERE FullTextContainsAll(c.abstract, 'quantum', 'theory')
  • SELECT TOP 50 c.id, c.abstract, c.title FROM c WHERE FullTextContainsAny(c.abstract, 'quantum', 'theory')
  • SELECT TOP 50 c.id, c.abstract, c.title FROM c ORDER BY RANK FullTextScore(c.abstract, ['quantum', 'theory'])
  • SELECT TOP 50 c.id, c.abstract, c.title FROM c ORDER BY RANK RRF(FullTextScore(c.abstract, ['quantum', 'theory']), FullTextScore(c.abstract, ['model']))
  • SELECT TOP 50 c.id, c.abstract, c.title FROM c ORDER BY RANK RRF(FullTextScore(c.abstract, ['quantum', 'theory']), FullTextScore(c.abstract, ['model']), VectorDistance(c.Embedding, [%s]))",vector)

Hybrid Search queries guide

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

@azure-sdk
Copy link
Collaborator

API change check

APIView has identified API level changes in this PR and created following API reviews.

com.azure:azure-cosmos

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants