Expand BetaFilter for External SearchStrategy Implementations#793
Expand BetaFilter for External SearchStrategy Implementations#793
Conversation
There was a problem hiding this comment.
Pull request overview
This PR expands the diskann-providers public API for search “layer” composition by re-exporting the internal BetaFilter helper types so external crates can implement SearchStrategy variants without building adapter wrappers.
Changes:
- Re-export
BetaAccessor,BetaComputer, andUnwrapalongsideBetaFilterfrom thelayersmodule.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Quick question: would it make sense to update |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
|
I guess my question is this: you are likely not going to be the only one who runs into this issue. If there is an obvious change to the current |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #793 +/- ##
==========================================
- Coverage 89.16% 89.14% -0.02%
==========================================
Files 431 431
Lines 78895 78895
==========================================
- Hits 70343 70330 -13
- Misses 8552 8565 +13
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Reference Issues/PRs
What does this implement/fix? Briefly explain your changes.
The extra O generic is needed because the original 2-parameter form
SearchStrategy<Provider, T> defaults O to Provider::InternalId, which is u32. That means BetaFilter could only produce search results as u32 (internal IDs).
In diskann-garnet, the unfiltered path uses FullPrecision, which
implements SearchStrategy<Provider, T, O> for any O — so it can output
GarnetId directly into SearchResults. But BetaFilter was
locked to O = u32, forcing diskann-garnet to use the
FilteredSearchResults adapter to convert u32 → GarnetId after the
search.
By adding O to BetaFilter's impl:
impl<Provider, Strategy, T, I, O> SearchStrategy<Provider, T, O> for
BetaFilter<Strategy, I>
where
O: Send,
Strategy: SearchStrategy<Provider, T, O>,
BetaFilter becomes transparent to the output type — it delegates O to
whatever the inner strategy uses. If the inner strategy (FullPrecision)
can output GarnetId, then BetaFilter wrapping it can too. This
eliminates the need for FilteredSearchResults entirely.
The O: Send bound is required because SearchStrategy needs O: Send for
the search output buffer to work across async boundaries.
Any other comments?
Removed previous solution to reimport the beta's internal:
Why these types need to be public:
(u32). Consumers with a different external ID type (like GarnetId) can't use BetaFilter directly with their output
buffer — they're forced to write an adapter (FilteredSearchResults) that accepts u32 and manually converts.
impl<T: VectorRepr> SearchStrategy<GarnetProvider, [T], GarnetId>
for BetaFilter<FullPrecision, u32>
happens in the post-processor — no wrapper needed.