Skip to content

Conversation

yuancu
Copy link
Collaborator

@yuancu yuancu commented Oct 9, 2025

Description

Backport #4224 to 2.19-dev

Commit Messages

  • Implement absolute time range in search command

Unit test search with absolute time range

Rephrase timeRange and timeModifier

Switch to earliest and latest udf

Add a convert util

Verify time correctness during coversion

Fix quarter parsing bugs

Fix week snap parsing

Remove old implementation that translates time modifier to time filter

  • Fix anomalyzed test & add a todo for an ignored test

  • Support now() in time range

  • Fix time modifier explain ITs

  • Support unixtimestamp (second) as a time modifier value

  • Update docs for search command with time modifiers

  • Test accessing fields with name earliest and latest in search command

  • Update doctest in condition.rst due to the update in the implementation of earliest and latest conditions

  • Update PPLQueryDataAnonymizerTest.java

  • Update explain ITs to use yaml plan files

  • Update a link to OpenSearch exists

  • Support using timesnaps without quotes

  • Add a unit test for direct format

  • additionally rename parseRelativeTime to resolveTimeModifier
  • Add support to ISO 8601 date format to time modifier, as it is now widely supported in PPL

  • Update syntax by reusing SPANLENGTH definition

  • Update explain IT for search with time modifier

  • Add integration tests for time modifiers

  • Parse timestamp string with multiple parsers in a loop

  • Remove opensearch test dependency from core module

  • Fix unit tests

  • Minor updates to explain limitations in search doc


(cherry picked from commit e468513)

Related Issues

#4135

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@yuancu yuancu force-pushed the backport/backport-4224-to-2.19-dev branch from 6daf5c4 to 37b5c66 Compare October 9, 2025 06:35
* Implement absolute time range in search command

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Unit test search with absolute time range

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Rephrase timeRange and timeModifier

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Switch to earliest and latest udf

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Add a convert util

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Verify time correctness during coversion

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Fix quarter parsing bugs

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Fix week snap parsing

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Remove old implementation that translates time modifier to time filter

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Fix anomalyzed test & add a todo for an ignored test

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Support now() in time range

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Fix time modifier explain ITs

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Support unixtimestamp (second) as a time modifier value

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Update docs for search command with time modifiers

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Test accessing fields with name earliest and latest in search command

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Update doctest in condition.rst due to the update in the implementation of earliest and latest conditions

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Update PPLQueryDataAnonymizerTest.java

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Update explain ITs to use yaml plan files

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Update a link to OpenSearch exists

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Support using timesnaps without quotes

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Add a unit test for direct format

- additionally rename parseRelativeTime to resolveTimeModifier

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Add support to ISO 8601 date format to time modifier, as it is now widely supported in PPL

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Update syntax by reusing SPANLENGTH definition

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Update explain IT for search with time modifier

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Add integration tests for time modifiers

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Parse timestamp string with multiple parsers in a loop

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Remove opensearch test dependency from core module

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Fix unit tests

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Minor updates to explain limitations in search doc

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

---------

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
(cherry picked from commit e468513)
@qianheng-aws qianheng-aws merged commit 5b751ec into opensearch-project:2.19-dev Oct 10, 2025
69 of 70 checks passed
@yuancu yuancu deleted the backport/backport-4224-to-2.19-dev branch October 10, 2025 04:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants