Skip to content

Add Float16VectorSimilarityQuery for similarity threshold search over fp16 vectors - #16735

Merged
kaivalnp merged 3 commits into
apache:mainfrom
rcpeken:fp16-vector-similarity-query
Sep 29, 2026
Merged

kaivalnp merged 3 commits into
apache:mainfrom
rcpeken:fp16-vector-similarity-query

Conversation

@rcpeken

@rcpeken rcpeken commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Description

Part of #16725. Adds Float16VectorSimilarityQuery, the fp16 counterpart of
FloatVectorSimilarityQuery / ByteVectorSimilarityQuery, so similarity
threshold search works on fields indexed with KnnFloat16VectorField.

The class mirrors FloatVectorSimilarityQuery on top of the fp16 APIs added in
#16383 (Float16VectorValues, LeafReader#getFloat16VectorValues, the
short[] searchNearestVectors overload, VectorUtil#checkFiniteFloat16).
toString converts the first target component with Float.float16ToFloat so
it prints a readable value rather than the raw bits.

One deliberate difference: only the Adaptive variant is provided. In the
float and byte queries, Explicit is @Deprecated and exists to stay
equivalent to the Lucene 10.4 constructors; fp16 has no such legacy callers, so
adding a new API that is deprecated from day one didn't seem useful. The class
keeps the same sealed shape, so Explicit can be added later if reviewers
prefer parity.

Tests

TestFloat16VectorSimilarityQuery plugs fp16 types into the shared
BaseVectorSimilarityQueryTestCase. Ran it with -Ptests.iters=20, and with
-Dtests.monster=true so testApproximate also exercises the HNSW path; all
pass. tidy and ecjLint are clean.

@rcpeken

rcpeken commented Sep 28, 2026

Copy link
Copy Markdown
Contributor Author

The failing arm CI job is in :lucene:analysis.tests:test, which this PR doesn't touch (the change is limited to lucene/core search). The previous run on the same code passed on all platforms, so this looks like an unrelated flaky test. Could someone re-run the job when convenient?

@kaivalnp kaivalnp left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Edit: I will merge this in a day, unless there are further comments. The test failure does seem unrelated, I opened #16736. The same test failed on a re-run, either we got really (un)lucky or the failure is very common.

@kaivalnp kaivalnp added this to the 11.0.0 milestone Sep 29, 2026
@kaivalnp
kaivalnp merged commit 020802f into apache:main Sep 29, 2026
12 checks passed
@kaivalnp kaivalnp linked an issue Oct 1, 2026 that may be closed by this pull request
2 of 5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Complete FP16 (float16) vector support across Lucene

3 participants