Skip to content

Comments

Expand BetaFilter for External SearchStrategy Implementations#793

Merged
hailangx merged 4 commits intomainfrom
haiyang/exposebeta
Feb 23, 2026
Merged

Expand BetaFilter for External SearchStrategy Implementations#793
hailangx merged 4 commits intomainfrom
haiyang/exposebeta

Conversation

@hailangx
Copy link
Member

@hailangx hailangx commented Feb 22, 2026

  • [ y] Does this PR have a descriptive title that could go in our release notes?
  • [n ] Does this PR add any new dependencies?
  • [y ] Does this PR modify any existing APIs?
  • [ y] Is the change to the API backwards compatible?
  • [n ] Should this result in any changes to our documentation, either updating existing docs or adding new ones?

Reference Issues/PRs

What does this implement/fix? Briefly explain your changes.

The extra O generic is needed because the original 2-parameter form
SearchStrategy<Provider, T> defaults O to Provider::InternalId, which is u32. That means BetaFilter could only produce search results as u32 (internal IDs).

In diskann-garnet, the unfiltered path uses FullPrecision, which
implements SearchStrategy<Provider, T, O> for any O — so it can output
GarnetId directly into SearchResults. But BetaFilter was
locked to O = u32, forcing diskann-garnet to use the
FilteredSearchResults adapter to convert u32 → GarnetId after the
search.

By adding O to BetaFilter's impl:

impl<Provider, Strategy, T, I, O> SearchStrategy<Provider, T, O> for
BetaFilter<Strategy, I>
where
O: Send,
Strategy: SearchStrategy<Provider, T, O>,

BetaFilter becomes transparent to the output type — it delegates O to
whatever the inner strategy uses. If the inner strategy (FullPrecision)
can output GarnetId, then BetaFilter wrapping it can too. This
eliminates the need for FilteredSearchResults entirely.

The O: Send bound is required because SearchStrategy needs O: Send for
the search output buffer to work across async boundaries.

Any other comments?

Removed previous solution to reimport the beta's internal:
Why these types need to be public:

  • BetaFilter's blanket impl (SearchStrategy<Provider, T>) hard-codes the output to the provider's internal ID
    (u32). Consumers with a different external ID type (like GarnetId) can't use BetaFilter directly with their output
    buffer — they're forced to write an adapter (FilteredSearchResults) that accepts u32 and manually converts.
  • If BetaAccessor/BetaComputer/Unwrap are re-exported, consumers can write:
    impl<T: VectorRepr> SearchStrategy<GarnetProvider, [T], GarnetId>
    for BetaFilter<FullPrecision, u32>
  • naming the associated types directly, and the conversion
    happens in the post-processor — no wrapper needed.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands the diskann-providers public API for search “layer” composition by re-exporting the internal BetaFilter helper types so external crates can implement SearchStrategy variants without building adapter wrappers.

Changes:

  • Re-export BetaAccessor, BetaComputer, and Unwrap alongside BetaFilter from the layers module.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@hildebrandmw
Copy link
Contributor

Quick question: would it make sense to update BetaFilter to be friendlier to what you are trying to do? I also think there is potential with Naren's planned post-process changes that we may be able to shim a custom post-process layer to do the id translation.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@hailangx
Copy link
Member Author

Quick question: would it make sense to update BetaFilter to be friendlier to what you are trying to do? I also think there is potential with Naren's planned post-process changes that we may be able to shim a custom post-process layer to do the id translation.
do you mean adding the 3-param impl so BetaFilter is transparent to the external ID type?

@hildebrandmw
Copy link
Contributor

I guess my question is this: you are likely not going to be the only one who runs into this issue. If there is an obvious change to the current BetaFilter to enable this, then I'm all for that. Otherwise, we can make these public as a stopgap until the post-processing change.

@codecov-commenter
Copy link

codecov-commenter commented Feb 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.14%. Comparing base (6f8e8eb) to head (44fd210).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #793      +/-   ##
==========================================
- Coverage   89.16%   89.14%   -0.02%     
==========================================
  Files         431      431              
  Lines       78895    78895              
==========================================
- Hits        70343    70330      -13     
- Misses       8552     8565      +13     
Flag Coverage Δ
miri 89.14% <ø> (-0.02%) ⬇️
unittests 89.14% <ø> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...ders/src/model/graph/provider/layers/betafilter.rs 92.77% <ø> (ø)

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hailangx hailangx enabled auto-merge (squash) February 23, 2026 18:18
@hailangx hailangx changed the title Expose BetaFilter for External SearchStrategy Implementations Expand BetaFilter for External SearchStrategy Implementations Feb 23, 2026
@hailangx hailangx merged commit 35ff1a9 into main Feb 23, 2026
23 checks passed
@hailangx hailangx deleted the haiyang/exposebeta branch February 23, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants