Skip to content

Fix inconsistent roundup behavior for date ranges using Temporal object as terms #127739

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/changelog/127739.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
pr: 127739
summary: Fix inconsistent roundup behavior for date ranges using Temporal object as
terms
area: Infra/Core
type: bug
issues:
- 86284
35 changes: 18 additions & 17 deletions server/src/main/java/org/elasticsearch/index/mapper/RangeType.java
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,13 @@
import java.time.ZoneId;
import java.time.ZoneOffset;
import java.time.ZonedDateTime;
import java.time.temporal.Temporal;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Set;
import java.util.function.BiFunction;
import java.util.function.LongSupplier;

/** Enum defining the type of range */
public enum RangeType {
Expand Down Expand Up @@ -293,28 +295,27 @@ public Query rangeQuery(

DateMathParser dateMathParser = (parser == null) ? DateFieldMapper.DEFAULT_DATE_TIME_FORMATTER.toDateMathParser() : parser;
boolean roundUp = includeLower == false; // using "gt" should round lower bound up
Long low = lowerTerm == null
? minValue()
: dateMathParser.parse(
lowerTerm instanceof BytesRef ? ((BytesRef) lowerTerm).utf8ToString() : lowerTerm.toString(),
context::nowInMillis,
roundUp,
zone
).toEpochMilli();
Long low = lowerTerm == null ? minValue() : parseRangeTerm(dateMathParser, lowerTerm, zone, roundUp, context::nowInMillis);

roundUp = includeUpper; // using "lte" should round upper bound up
Long high = upperTerm == null
? maxValue()
: dateMathParser.parse(
upperTerm instanceof BytesRef ? ((BytesRef) upperTerm).utf8ToString() : upperTerm.toString(),
context::nowInMillis,
roundUp,
zone
).toEpochMilli();

Long high = upperTerm == null ? maxValue() : parseRangeTerm(dateMathParser, upperTerm, zone, roundUp, context::nowInMillis);
return createRangeQuery(field, hasDocValues, low, high, includeLower, includeUpper, relation);
}

private static long parseRangeTerm(DateMathParser parser, Object term, ZoneId zone, boolean roundUp, LongSupplier nowInMillis) {
String termString;
if (term instanceof BytesRef bytesRefTerm) {
termString = bytesRefTerm.utf8ToString();
} else {
termString = term.toString();
// `roundUp` may only be used if the term is not a Temporal object. LocalTime#toString() and similar will output
// the shortest possible representation and might omit seconds, milliseconds, etc.
// That interferes badly with the behavior of the roundup parser leading to unexpected results.
roundUp = roundUp && (term instanceof Temporal) == false;
}
return parser.parse(termString, nowInMillis, roundUp, zone).toEpochMilli();
}

@Override
public Query withinQuery(String field, Object from, Object to, boolean includeLower, boolean includeUpper) {
return LONG.withinQuery(field, from, to, includeLower, includeUpper);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,43 @@ public void testRangeQuery() throws Exception {
);
}

public void testDateRangeQueryDoesNotRoundupTemporals() {
type = RangeType.DATE;
SearchExecutionContext context = createContext();
RangeFieldType ft = createDefaultFieldType();
ShapeRelation relation = randomFrom(ShapeRelation.values());

// explicitly not using nextFrom here to provide a concrete, visual example.
// see the behavioral difference compared to testRangeQueryIntersectsAdjacentDates
ZonedDateTime from = ZonedDateTime.parse("2025-05-01T14:10:00.000Z");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little deceptive, the parser used in rangeQuery will call toString on this. I think it would be slightly clearer if the test used Strings instead, which matches what would come from a real query, not a ZonedDateTime object.

I wonder if that is the issue with the original test failures: they're passing in ZonedDateTime objects and subject to their toString behavior, but in production this wouldn't happen, we get strings directly from the request.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't care about ZonedDateTime objects, there's no problem here and the failing tests have simply tested for the wrong thing... the problem is really only the difference between Joda vs Java time behaving differently for the toString (pls read the PR description for the details)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I don't think ZonedDateTime is possible outside tests, see where eg from is set in RangeQueryBuilder, it basically comes down to parsing xcontent, where we only have strings and numbers. The entire point of the date math parser is to turn a string into a ZonedDateTime, so it doesn't make sense that we would try to take an already realized ZonedDateTime, turn it back into a string, and then get a ZonedDateTime.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if that is the issue with the original test failures: they're passing in ZonedDateTime objects and subject to their toString behavior, but in production this wouldn't happen, we get strings directly from the request.

yes, as described in the PR description, that's the cause of the issue
closing this in favor of fixing the test: #127899

ZonedDateTime to = ZonedDateTime.parse("2025-05-01T14:11:00.000Z");

// usage of roundUp parsers is wrong if terms are Temporal objects, otherwise it would arbitrarily change the bounds
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by "Temporal object" don't you mean all dates?

// depending on the string representation
assertEquals(
getExpectedRangeQuery(relation, from, to, false, true),
ft.rangeQuery(from, to, false, true, relation, null, null, context)
);
}

public void testRangeQueryIntersectsAdjacentDates() {
type = RangeType.DATE;
SearchExecutionContext context = createContext();
ShapeRelation relation = randomFrom(ShapeRelation.values());
RangeFieldType ft = createDefaultFieldType();

// explicitly not using nextFrom here to provide a concrete, visual example of the roundUp behavior
String from = "2025-05-01T14:10Z"; // transformed to 2025-05-01T14:10:59.999999999Z by roundUp parser
String to = "2025-05-01T14:11Z";

Query rangeQuery = ft.rangeQuery(from, to, false, false, relation, null, null, context);
assertThat(rangeQuery, instanceOf(IndexOrDocValuesQuery.class));
assertThat(((IndexOrDocValuesQuery) rangeQuery).getIndexQuery(), instanceOf(MatchNoDocsQuery.class));
}

/**
* test the queries are correct if from/to are adjacent and the range is exclusive of those values
*/
@AwaitsFix(bugUrl = "https://github.com/elastic/elasticsearch/issues/86284")
public void testRangeQueryIntersectsAdjacentValues() throws Exception {
SearchExecutionContext context = createContext();
ShapeRelation relation = randomFrom(ShapeRelation.values());
Expand Down Expand Up @@ -143,7 +176,6 @@ public void testRangeQueryIntersectsAdjacentValues() throws Exception {
/**
* check that we catch cases where the user specifies larger "from" than "to" value, not counting the include upper/lower settings
*/
@AwaitsFix(bugUrl = "https://github.com/elastic/elasticsearch/issues/86284")
public void testFromLargerToErrors() throws Exception {
SearchExecutionContext context = createContext();
RangeFieldType ft = createDefaultFieldType();
Expand Down
Loading