Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Es knn index search 5346 #5569

Merged
merged 25 commits into from
Jun 2, 2023

Conversation

jeffvestal
Copy link
Contributor

Create elastic_vector_search.ElasticKnnSearch class

This extends langchain/vectorstores/elastic_vector_search.py by adding a new class ElasticKnnSearch

Features:

  • Allow creating an index with the dense_vector mapping compataible with kNN search
  • Store embeddings in index for use with kNN search (correct mapping creates HNSW data structure)
  • Perform approximate kNN search
  • Perform hybrid BM25 (query{}) + kNN (knn{}) search
  • perform knn search by either providing a query_vector or passing a hosted model_id to use query_vector_builder to automatically generate a query_vector at search time

Connection options

  • Using cloud_id from Elastic Cloud
  • Passing elasticsearch client object

search options

  • query
  • k
  • query_vector
  • model_id
  • size
  • source
  • knn_boost (hybrid search)
  • query_boost (hybrid search)
  • fields

This also adds examples to docs/modules/indexes/vectorstores/examples/elasticsearch.ipynb

Fixes # 5346

cc: @dev2049

-->

@jeffvestal
Copy link
Contributor Author

I'm working on the lint conflicts

@jeffvestal
Copy link
Contributor Author

I fixed most of the lint errors. The couple left seem odd

@dev2049 dev2049 merged commit d1f65d8 into langchain-ai:master Jun 2, 2023
@danielchalef danielchalef mentioned this pull request Jun 5, 2023
Undertone0809 pushed a commit to Undertone0809/langchain that referenced this pull request Jun 19, 2023
# Create elastic_vector_search.ElasticKnnSearch class

This extends `langchain/vectorstores/elastic_vector_search.py` by adding
a new class `ElasticKnnSearch`

Features:
- Allow creating an index with the `dense_vector` mapping compataible
with kNN search
- Store embeddings in index for use with kNN search (correct mapping
creates HNSW data structure)
- Perform approximate kNN search
- Perform hybrid BM25 (`query{}`) + kNN (`knn{}`) search
- perform knn search by either providing a `query_vector` or passing a
hosted `model_id` to use query_vector_builder to automatically generate
a query_vector at search time

Connection options
- Using `cloud_id` from Elastic Cloud
- Passing elasticsearch client object

search options
- query
- k
- query_vector
- model_id
- size
- source
- knn_boost (hybrid search)
- query_boost (hybrid search)
- fields


This also adds examples to
`docs/modules/indexes/vectorstores/examples/elasticsearch.ipynb`


Fixes # [5346](langchain-ai#5346)

cc: @dev2049

 -->

---------

Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
This was referenced Jun 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants