score: introduce query.Boost to scale score #728
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Couldn't help but do a little hacking this morning. It hasn't been tested yet, but would appreciate feedback on it / the approach/idea. |
keegancsmith
force-pushed
the
k/query-boost
branch
3 times, most recently
from
9cb40ac to
818a9c5
Compare
|
Alright after some experimental testing a boost of 20 works quite well! I updated our e2e ranking tests:
Screenshots below on the same corpus as our e2e ranking tests go run ./cmd/zoekt-webserver -listen 127.0.0.1:6070 -print -index /tmp/zoekt-test-ranking-shards-keegan
|
keegancsmith
force-pushed
the
k/query-boost
branch
3 times, most recently
from
0611e63 to
efdcd9e
Compare
This commit introduces a new primitive Boost to our query language. It allows boosting (or dampening) the contribution to the score a query atoms will match contribute. To achieve this we introduce boostMatchTree which records this weight. We then adjust the visitMatches to take an initial score weight (1.0), and then each time we recurse through a boostMatchTree the score weight is multiplied by the boost weight. Additionally candidateMatch now has a new field, scoreWeight, which records the weight at time of candidate collection. Without boosting in the query this value will always be 1. Finally when scoring a candidateMatch we take the final score for it and multiply it by scoreWeight. Note: we do not expose a way to set this in the query language, only the query API. Test Plan: Manual testing against webserver via the new phrase-boost URL param. Additionally updated ranking tests to use the phrase booster.
keegancsmith
force-pushed
the
k/query-boost
branch
from
efdcd9e to
6ab827f
Compare
stefanhengl
approved these changes
Jan 29, 2024
17 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit introduces a new primitive Boost to our query language. It allows boosting (or dampening) the contribution to the score a query atoms will match contribute.
To achieve this we introduce boostMatchTree which records this weight. We then adjust the visitMatches to take an initial score weight (1.0), and then each time we recurse through a boostMatchTree the score weight is multiplied by the boost weight. Additionally candidateMatch now has a new field, scoreWeight, which records the weight at time of candidate collection. Without boosting in the query this value will always be 1.
Finally when scoring a candidateMatch we take the final score for it and multiply it by scoreWeight.
Note: we do not expose a way to set this in the query language, only the query API.
Test Plan: Manual testing against webserver via the new phrase-boost URL param. Additionally updated ranking tests to use the phrase booster.