Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancment/qdrant remove default vector name #15078

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

jordanparker6
Copy link
Contributor

Description

Supporting the default vector "" does not add much.

Whether or not the VectorStore is enabled_hybrid=True or not, the DENSE_VECTOR_NAME should be the name of the named vector. We shouldn't rely on the default vector name "" and the DENSE_VECTOR_NAME as this distinction doesn't add anything, it just creates configuration issues.

For example, it makes it difficult to reconfigure the VectorStore with hybrid=False as we are now querying the default name "" instead of the DENSE_VECTOR_NAME. So why not just always use the named vector always and avoid this? I believe this approach is more robust.

Fixes # (issue)

These changes fix this issue to ensure that any dense vector created is named the DENSE_VECTOR_NAME and these configuration issues will be avoided.

Version Bump?

Did I bump the version in the pyproject.toml file of the package I am updating? (Except for the llama-index-core package)

  • [X ] No

Type of Change

  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

This would introduce some backward compatibility issues.

How Has This Been Tested?

  • [ X] I stared at the code and made sure it makes sense

@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Aug 1, 2024
@masci masci self-assigned this Aug 1, 2024
@dosubot dosubot bot added size:S This PR changes 10-29 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Aug 1, 2024
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Aug 1, 2024
@logan-markewich
Copy link
Collaborator

logan-markewich commented Aug 1, 2024

Saw this on discord. I think I agree, but this also feels like a breaking change no?

If I have an existing qdrant index, and install this change, and try to insert more data, it will get inserted under a new vector name?

@jordanparker6
Copy link
Contributor Author

@logan-markewich this is true... Maybe we should have some option to support legacy for a few iterations and then remove it all together.

I have nudged qdrant hoping they add vector name aliases to resolve these issues.

@masci
Copy link
Member

masci commented Aug 5, 2024

@jordanparker6 @logan-markewich Supporting both, even for a short amount of time, would be cumbersome at glance, and probably prone to error.

What if we make this breaking, but offer assistance in migrating? For example, would it be possible at connection time to detect if the database is using the default vector name and emit an error explaining how to upgrade? Maybe we can also provide an utility method to update the existing vector name (to my limited knowledge, qdrant doesn't support it but maybe can be done manually)?

@jordanparker6
Copy link
Contributor Author

@jordanparker6 @logan-markewich Supporting both, even for a short amount of time, would be cumbersome at glance, and probably prone to error.

What if we make this breaking, but offer assistance in migrating? For example, would it be possible at connection time to detect if the database is using the default vector name and emit an error explaining how to upgrade? Maybe we can also provide an utility method to update the existing vector name (to my limited knowledge, qdrant doesn't support it but maybe can be done manually)?

any update on this?

I am in favour of a breaking change.

@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants