Feature/partner full text search rollout - #4333
Open
Joker666 wants to merge 22 commits into
Open
Conversation
…full-text-search-rollout
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The provider now narrows on the discrete enrollment fields instead of leaving every filter to the database, which applied them to the already truncated candidate list. A broad query with country=US returned 85 rows out of 7,698 real matches; measured on 5,000 seeded partners, filtering inside the query takes that from 20% to 100% of the true set. Metrics stay out on purpose: they move on every click and conversion, so indexing them would make the document churn continuously. Namespace moves to v3 — the new attributes do not exist on v2 rows.
The partner count API was over-counting when a referral filter was present because the database-only fallback did not include `referredByPartnerId`. This adds that filter to the fallback guard and covers it with a search test so partner counts remain accurate when the provider cannot evaluate the filter.
This change avoids expensive Turbopuffer count aggregations for very short final search tokens, where prefix expansion can make the query too slow. The provider now returns null when it should decline counting, and getPartnersCount falls back to the database count in that case. Tests cover both the fallback path and the short-query guard.
countCandidates returned 0 when the response carried no aggregate, which rendered an unanswered count as an exact empty result now that null means the provider cannot answer. The fallback tests mocked a database count larger than the candidate list they were constrained to, which cannot happen under a real id IN clause and hid the fact that every fallback path is capped at the candidate ceiling. They now assert that constraint instead.
…ch' into feature/partner-full-text-search-rollout
package.json moved back to ^1.35.3 without regenerating the lockfile, so CI's frozen-lockfile install failed before building. The resolved version is unchanged — 1.38.2 satisfies the range — so only the specifier moves.
Both described a two-provider world that this branch removed, and the first also described the wrong filtering model. types.ts said filters run after the truncation and called indexing them "out of scope here" — but that is what this branch implemented, and turbopuffer.ts says the opposite three files away. Anyone reading the constant would have understood the model backwards. It now names what the provider filters before the cut (status, group, country, tags) and what the database filters after it (tenant, explicit partner IDs, referral, metric ranges), matching the schema and the databaseOnlyFilters list. The 999 ceiling is described as inherited from the removed Redis provider rather than derived, since nothing enforces it now. The floor beyond it is framed as the accepted trade it is, with what raising it would cost on every query, rather than as a known limitation inviting a fix. shared.ts loses the Redis-versus-Turbopuffer packaging explanation and the "both providers store" phrasing.
…full-text-search-rollout
…full-text-search-rollout # Conflicts: # apps/web/lib/api/partners/search/providers/shared.ts # apps/web/lib/api/partners/search/providers/turbopuffer.ts # apps/web/lib/api/partners/search/types.ts # apps/web/scripts/dev/seed-100k-partners.ts
House style, matching the same pass on the base branch.
The block explaining the ranked query branches sat above buildFilterClauses, which it does not describe. Moved onto buildQueryBranches. The rest is damage from the earlier em-dash removal, which left three sentences with a doubled "so" or "and" and one where the replacement word was simply wrong. Punctuation swapped mechanically, prose not re-read.
The note carried the Redis provider's history and a long defense of the trade-off. Keeps the filter order, the lower-bound behavior, and what raising the ceiling costs.
Joker666
force-pushed
the
feature/partner-full-text-search-rollout
branch
from
August 17, 2026 23:09
ff819a3 to
39dec36
Compare
…full-text-search-rollout
Joker666
force-pushed
the
feature/partner-full-text-search-rollout
branch
from
August 18, 2026 01:54
ac86908 to
0af20a5
Compare
debug-partner-search kept its own copy of the namespace name, which stayed on partner-search-v2 when the filterable-attribute change bumped the provider to v3. Its candidate search went to v3 through the provider while its indexed-text lookup went to v2, so --searchOnly reported empty names and zero tokens instead of failing. Export the constant and import it, so the two cannot drift again.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Partner search rollout
This narrows the proof of concept to something deployable using Turbopuffer.
What changed
Upstash Redis Search removed, along with
PARTNER_SEARCH_PROVIDER,PARTNER_SEARCH_INDEX_NAME, andwaitForIndexing.@upstash/redisrolls back to^1.35.3— the package stays for caching and rate limiting, only the bump the Redis search API needed is reverted. ThePartnerSearchProviderinterface stays; only the multi-provider registry went.TURBOPUFFER_API_KEYis now the feature flag — unset still falls back to database search, so this ships dark.Exact email and partner ID skip the provider and use the indexes the database already has. An email that matches nothing still falls through to search, because that's usually an address being typed (
steven@dub.coon the way tosteven@dub.com). The ID rule ispn_+ 24 characters minimum, from a full scan of all 626,348 production documents — a fixed-length check would have dropped 1 partner in 280.status,groupId,country, andpartnerTagIdsare indexed and filtered before the ranking truncates. This is the main fix. On 5,000 seeded partners with 1,000 real matches, a filtered search returned 201 rows before, 999 after. Metrics stay database-only — they change on every click, and indexing them would make documents churn continuously.Counts are exact, via one aggregation rather than counting the truncated candidate list ("999 partners" when there are twelve thousand). It returns
number | nulland falls back to the old count when it can't answer.Seed script now varies status, groups, and tags, including nulls and empties. Previously every partner was approved, ungrouped, and untagged.
Testing
110 unit tests, covering the paths that fail quietly: email hit/miss routing, partner IDs at 24 and 25 characters and one short, filters reaching every branch with correct exclusion semantics, an empty filter list not becoming
In [], all four count fallbacks, and tags from other programs being dropped.