Skip to content

Forbid negative scores in function_score query #35709

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion docs/reference/migration/migrate_7_0/search.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -148,4 +148,12 @@ instead which is a more appropriate value for a scenario where scores are not av
==== Negative boosts are not allowed

Setting a negative `boost` in a query, deprecated in 6x, are not allowed in this version.
To deboost a specific query you can use a `boost` comprise between 0 and 1.
To deboost a specific query you can use a `boost` comprise between 0 and 1.

[float]
==== Negative scores are not allowed in Function Score Query

Negative scores in the Function Score Query are deprecated in 6.x, and are
not allowed in this version. If a negative score is produced as a result
of computation (e.g. in `script_score` or `field_value_factor` functions),
an error will be thrown.
6 changes: 6 additions & 0 deletions docs/reference/query-dsl/function-score-query.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,9 @@ GET /_search
// CONSOLE
// TEST[setup:twitter]

NOTE: Scores produced by the `script_score` function must be non-negative,
otherwise an error will be thrown.

On top of the different scripting field values and expression, the
`_score` script parameter can be used to retrieve the score based on the
wrapped query.
Expand Down Expand Up @@ -324,6 +327,9 @@ There are a number of options for the `field_value_factor` function:
values of the field with a range filter to avoid this, or use `log1p` and
`ln1p`.

NOTE: Scores produced by the `field_value_score` function must be non-negative,
otherwise an error will be thrown.

[[function-decay]]
==== Decay functions

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -442,3 +442,41 @@
- match: { error.root_cause.0.reason: "Iterable object is self-referencing itself (ScriptBytesValues value)" }
- match: { error.type: "search_phase_execution_exception" }
- match: { error.reason: "all shards failed" }


---

"Exception on negative score":
- skip:
version: " - 6.99.99"
reason: "check on negative scores was added from 7.0.0 on"

- do:
index:
index: test
type: test
id: 1
body: { "test": "value beck", "num1": 1.0 }
- do:
indices.refresh: {}

- do:
catch: bad_request
search:
index: test
body:
query:
function_score:
query:
term:
test: value
"functions": [{
"script_score": {
"script": {
"lang": "painless",
"source": "doc['num1'].value - 10.0"
}
}
}]
- match: { error.root_cause.0.type: "illegal_argument_exception" }
- match: { error.root_cause.0.reason: "script score function must not produce negative scores, but got: [-9.0]"}
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,10 @@ public double score(int docId, float subQueryScore) throws IOException {
}
double val = value * boostFactor;
double result = modifier.apply(val);
if (result < 0f) {
throw new IllegalArgumentException("field value function must not produce negative scores, but got: [" +
result + "] for field value: [" + value + "]");
}
return result;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,7 @@ public float score() throws IOException {
}
double factor = computeScore(docId, subQueryScore);
float finalScore = scoreCombiner.combine(subQueryScore, factor, maxBoost);
if (finalScore == Float.NEGATIVE_INFINITY || Float.isNaN(finalScore)) {
if (finalScore < 0f || Float.isNaN(finalScore)) {
/*
These scores are invalid for score based {@link org.apache.lucene.search.TopDocsCollector}s.
See {@link org.apache.lucene.search.TopScoreDocCollector} for details.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,9 @@ public double score(int docId, float subQueryScore) throws IOException {
scorer.docid = docId;
scorer.score = subQueryScore;
double result = leafScript.execute();
if (result < 0f) {
throw new IllegalArgumentException("script score function must not produce negative scores, but got: [" + result + "]");
}
return result;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -824,6 +824,20 @@ public void testWithInvalidScores() {
assertThat(exc.getMessage(), containsString("function score query returned an invalid score: " + Float.NEGATIVE_INFINITY));
}


public void testExceptionOnNegativeScores() {
IndexSearcher localSearcher = new IndexSearcher(reader);
TermQuery termQuery = new TermQuery(new Term(FIELD, "out"));

// test that field_value_factor function throws an exception on negative scores
FieldValueFactorFunction.Modifier modifier = FieldValueFactorFunction.Modifier.NONE;
final ScoreFunction fvfFunction = new FieldValueFactorFunction(FIELD, -10, modifier, 1.0, new IndexNumericFieldDataStub());
FunctionScoreQuery fsQuery1 =
new FunctionScoreQuery(termQuery, fvfFunction, CombineFunction.REPLACE, null, Float.POSITIVE_INFINITY);
IllegalArgumentException exc = expectThrows(IllegalArgumentException.class, () -> localSearcher.search(fsQuery1, 1));
assertThat(exc.getMessage(), containsString("field value function must not produce negative scores"));
}

private static class DummyScoreFunction extends ScoreFunction {
protected DummyScoreFunction(CombineFunction scoreCombiner) {
super(scoreCombiner);
Expand Down