Add The `TO_SECONDS` Function To The SQL Plugin #232

GabeFernandez310 · 2023-02-23T16:21:10Z

Description

Adds the to_seconds function to the SQL plugin. Takes a DATE/TIME/DATETIME/TIMESTAMP/STRING/INTEGER representing a point in time, and converts it to seconds since the year 0. This function is based off of MySQL. For arguments of type TIME, the function takes the current date at the given time. For INTEGER, the integer is parsed as a date (e.g. 950501 == 1995-05-01).

Examples:
SELECT TO_SECONDS(950501); -> 62966505600
SELECT TO_SECONDS('2009-11-29'); -> 63426672000
SELECT TO_SECONDS('2009-11-29 13:43:32'); -> 63426721412

Issues Resolved

#722

Check List

New functionality includes testing.
- All tests pass, including unit test, integration test and doctest
New functionality has been documented.
- New functionality has javadoc added
- New functionality has user manual doc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

codecov · 2023-02-23T16:29:21Z

Codecov Report

Merging #232 (8d2587e) into integ-add-to_seconds-function (259b001) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@                       Coverage Diff                        @@
##             integ-add-to_seconds-function     #232   +/-   ##
================================================================
  Coverage                            98.38%   98.39%           
- Complexity                            3693     3706   +13     
================================================================
  Files                                  343      343           
  Lines                                 9107     9148   +41     
  Branches                               585      587    +2     
================================================================
+ Hits                                  8960     9001   +41     
  Misses                                 142      142           
  Partials                                 5        5

Flag	Coverage Δ
sql-engine	`98.39% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...c/main/java/org/opensearch/sql/expression/DSL.java	`100.00% <100.00%> (ø)`
...arch/sql/expression/datetime/DateTimeFunction.java	`100.00% <100.00%> (ø)`
...h/sql/expression/function/BuiltinFunctionName.java	`100.00% <100.00%> (ø)`
...a/org/opensearch/sql/utils/DateTimeFormatters.java	`100.00% <100.00%> (ø)`

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

core/src/test/java/org/opensearch/sql/expression/datetime/DateTimeFunctionTest.java

docs/user/dql/functions.rst

core/src/test/java/org/opensearch/sql/expression/datetime/DateTimeFunctionTest.java

docs/user/dql/functions.rst

core/src/test/java/org/opensearch/sql/expression/datetime/DateTimeFunctionTest.java

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

core/src/main/java/org/opensearch/sql/utils/DateTimeFormatters.java

core/src/test/java/org/opensearch/sql/expression/datetime/ToSecondsTest.java

GumpacG · 2023-03-02T18:46:16Z

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

+  private ExprValue exprToSecondsForIntType(ExprValue dateExpr) {
+    try {
+      if (dateExpr.longValue() < 0 || dateExpr.longValue() > 99999999) {
+        throw new DateTimeException("Integer argument was out of range");
+      }
+
+      try {
+        LocalDate date = LocalDate.parse(String.valueOf(dateExpr.longValue()),
+            DATE_FORMATTER_LONG_YEAR);
+
+        return new ExprLongValue(
+            date.toEpochSecond(LocalTime.MIN, ZoneOffset.UTC)
+                + DAYS_0000_TO_1970 * SECONDS_PER_DAY);
+      } catch (DateTimeParseException ignored) {
+        //ignore parse exception and try next format
+      }
+
+      try {
+        LocalDate date = LocalDate.parse(String.valueOf(dateExpr.longValue()),
+            DATE_FORMATTER_SHORT_YEAR);
+
+        return new ExprLongValue(
+            date.toEpochSecond(LocalTime.MIN, ZoneOffset.UTC)
+                + DAYS_0000_TO_1970 * SECONDS_PER_DAY);
+      } catch (DateTimeParseException ignored) {
+        //ignore parse exception and try next format
+      }
+
+      try {
+        LocalDate date = LocalDate.parse(String.valueOf(dateExpr.longValue()),
+            DATE_FORMATTER_SINGLE_DIGIT_YEAR);
+
+        return new ExprLongValue(
+            date.toEpochSecond(LocalTime.MIN, ZoneOffset.UTC)
+                + DAYS_0000_TO_1970 * SECONDS_PER_DAY);
+      } catch (DateTimeParseException ignored) {
+        //ignore parse exception and try next format
+      }
+
+      try {
+        LocalDate date = LocalDate.parse(String.valueOf(dateExpr.longValue()),
+            DATE_FORMATTER_NO_YEAR);
+
+        return new ExprLongValue(
+            date.toEpochSecond(LocalTime.MIN, ZoneOffset.UTC)
+                + DAYS_0000_TO_1970 * SECONDS_PER_DAY);
+      } catch (DateTimeParseException ignored) {
+        //ignore parse exception and try next format
+      }
+
+      LocalDate date = LocalDate.parse(String.valueOf(dateExpr.longValue()),
+          DATE_FORMATTER_SINGLE_DIGIT_MONTH);
+
+      return new ExprLongValue(
+          date.toEpochSecond(LocalTime.MIN, ZoneOffset.UTC)
+              + DAYS_0000_TO_1970 * SECONDS_PER_DAY);
+    } catch (DateTimeException e) {
+      return ExprNullValue.of();
+    }


I think this can be refactored to reduce duplication of try catches

Addressed in ec908ad. Github formatted the diff strangely so I would suggest just looking at the file.

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

acarbonetto · 2023-03-08T00:13:58Z

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

+   * @return is a DateTimeFormatter that can parse the input.
+   */
+  private DateTimeFormatter getFormatter(int dateAsInt) {
+    if (dateAsInt < 0 || dateAsInt > 99999999) {


99999999 is a magic number. Make this a static constant.

nit: that being said, you're trying to length of the number. Why not convert it to a string and get the string length? I think that makes more sense - although less efficient

This is the main reason I did it this way, for efficiency. Actually, using the length of a string might not be too bad though because it enables me to change the multiple if-statements below into a switch/case block. It would also probably be clearer for people who come back to this and read it later... I'll just try to change it.

Fixed in 2b37b0d

acarbonetto · 2023-03-08T00:14:08Z

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

+    }
+
+    //Check if dateAsInt is at least 6 digits long
+    if (dateAsInt - 99999 > 0) {


99999 is a magic number

Fixed in 2b37b0d

acarbonetto · 2023-03-08T00:14:21Z

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

+    }
+
+    //Check if dateAsInt is at least 4 digits long
+    if (dateAsInt - 999 > 0) {


999 magic number

Fixed in 2b37b0d

acarbonetto · 2023-03-08T00:14:27Z

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java

+    }
+
+    //Check if dateAsInt is at least 3 digits long
+    if (dateAsInt - 99 > 0) {


magic number

Fixed in 2b37b0d

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

core/src/main/java/org/opensearch/sql/utils/DateTimeFormatters.java

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

) * Add The `TO_SECONDS` Function To The SQL Plugin (#232) * Added Basic Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added IT Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed Integration Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Test For Time Type Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Documentation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Addressed PR Comments Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Docs and Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Checkstyle Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed DateTimeFormatter Priority Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added More Formatters Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Updated Docs Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Reworked Implementation For Formatters Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Cleanup Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Implementation And Code Coverage Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed getFormatter Function Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Comments Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Removed Unneeded Code Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit d38a6ec)

) * Add The `TO_SECONDS` Function To The SQL Plugin (#232) * Added Basic Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added IT Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed Integration Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Test For Time Type Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Documentation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Addressed PR Comments Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Docs and Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Checkstyle Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed DateTimeFormatter Priority Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added More Formatters Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Updated Docs Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Reworked Implementation For Formatters Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Cleanup Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Implementation And Code Coverage Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed getFormatter Function Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Comments Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Removed Unneeded Code Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

) (opensearch-project#1447) * Add The `TO_SECONDS` Function To The SQL Plugin (#232) * Added Basic Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added IT Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed Integration Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Test For Time Type Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Documentation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Addressed PR Comments Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Docs and Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Checkstyle Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed DateTimeFormatter Priority Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added More Formatters Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Updated Docs Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Reworked Implementation For Formatters Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Cleanup Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Implementation And Code Coverage Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed getFormatter Function Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Comments Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Removed Unneeded Code Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit d38a6ec)

GabeFernandez310 added 6 commits February 20, 2023 22:02

Added Basic Tests

a94cdef

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Added IT Test

ab66d15

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Added Implementation

3af206e

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Changed Integration Tests

cad2e47

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Added Test For Time Type

51f2410

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Added Documentation

07e0b60

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Yury-Fridlyand requested changes Feb 23, 2023

View reviewed changes

Yury-Fridlyand reviewed Feb 24, 2023

View reviewed changes

core/src/test/java/org/opensearch/sql/expression/datetime/DateTimeFunctionTest.java Outdated Show resolved Hide resolved

Addressed PR Comments

9a2377a

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

GabeFernandez310 commented Feb 28, 2023

View reviewed changes

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java Outdated Show resolved Hide resolved

GabeFernandez310 commented Feb 28, 2023

View reviewed changes

core/src/main/java/org/opensearch/sql/expression/datetime/DateTimeFunction.java Outdated Show resolved Hide resolved

GabeFernandez310 added 5 commits February 28, 2023 16:03

Fixed Docs and Implementation

a2045b9

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Fixed Checkstyle

26f4cda

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Changed DateTimeFormatter Priority

0dcd3ce

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Added More Formatters

d4344d3

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Updated Docs

12b981c

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

GabeFernandez310 marked this pull request as ready for review March 1, 2023 23:36

Yury-Fridlyand approved these changes Mar 2, 2023

View reviewed changes

GumpacG reviewed Mar 2, 2023

View reviewed changes

GabeFernandez310 added 3 commits March 2, 2023 11:20

Reworked Implementation For Formatters

ec908ad

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Cleanup

8388705

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Added Test

fb3c19b

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

matthewryanwells approved these changes Mar 6, 2023

View reviewed changes

GabeFernandez310 added 2 commits March 7, 2023 08:19

Merge branch 'opensearch-project:main' into dev-add-to_seconds-function

c12849e

Fixed Implementation And Code Coverage

d30f749

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

acarbonetto approved these changes Mar 8, 2023

View reviewed changes

GumpacG approved these changes Mar 8, 2023

View reviewed changes

Changed getFormatter Function

2b37b0d

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Yury-Fridlyand approved these changes Mar 8, 2023

View reviewed changes

core/src/main/java/org/opensearch/sql/utils/DateTimeFormatters.java Show resolved Hide resolved

Added Comments

8d2587e

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

GabeFernandez310 merged commit b366b76 into integ-add-to_seconds-function Mar 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add The `TO_SECONDS` Function To The SQL Plugin #232

Add The `TO_SECONDS` Function To The SQL Plugin #232

GabeFernandez310 commented Feb 23, 2023 •

edited

Loading

codecov bot commented Feb 23, 2023 •

edited

Loading

GumpacG Mar 2, 2023

GabeFernandez310 Mar 2, 2023 •

edited

Loading

acarbonetto Mar 8, 2023

acarbonetto Mar 8, 2023

GabeFernandez310 Mar 8, 2023

GabeFernandez310 Mar 8, 2023

acarbonetto Mar 8, 2023

GabeFernandez310 Mar 8, 2023

acarbonetto Mar 8, 2023

GabeFernandez310 Mar 8, 2023

acarbonetto Mar 8, 2023

GabeFernandez310 Mar 8, 2023

Add The TO_SECONDS Function To The SQL Plugin #232

Add The TO_SECONDS Function To The SQL Plugin #232

Conversation

GabeFernandez310 commented Feb 23, 2023 • edited Loading

Description

Issues Resolved

Check List

codecov bot commented Feb 23, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

GabeFernandez310 Mar 2, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Add The `TO_SECONDS` Function To The SQL Plugin #232

Add The `TO_SECONDS` Function To The SQL Plugin #232

GabeFernandez310 commented Feb 23, 2023 •

edited

Loading

codecov bot commented Feb 23, 2023 •

edited

Loading

GabeFernandez310 Mar 2, 2023 •

edited

Loading