[Backport 2.x] Refactor SparkQueryDispatcher (#2636) #2668

ykmr1224 · 2024-05-13T21:11:20Z

Description

Backport Refactor SparkQueryDispatcher #2636
Automatic backport failed because of unit test failure due to change to the same class in different PR (Add option to use LakeFormation in S3Glue data source. #2624)
The issue was solved in Fix test failure in SparkQueryDispatcherTest due to merge #2645 for main branch

Issues Resolved

N/A

Check List

New functionality includes testing.
- All tests pass, including unit test, integration test and doctest
New functionality has been documented.
- New functionality has javadoc added
- New functionality has user manual doc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

… (opensearch-project#1349) Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit 672c72f) Co-authored-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

…search-project#1351) Added WeekOfYear As An Alias To Week Function Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit 114e02a) Co-authored-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

…1344) (opensearch-project#1350) Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit d9114f5) Co-authored-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

…1353) (opensearch-project#1355) * Capture IndexNotFoundException and re-throw directly Signed-off-by: Chen Dai <daichen@amazon.com> * Add IT Signed-off-by: Chen Dai <daichen@amazon.com> --------- Signed-off-by: Chen Dai <daichen@amazon.com> (cherry picked from commit 76a8d29) Co-authored-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Guian Gumpac <guian.gumpac@improving.com> (cherry picked from commit de40f42)

…ject#1369) * Add release notes for 2.6.0 Signed-off-by: Rupal Mahajan <maharup@amazon.com> * Remove incorrect links Signed-off-by: Rupal Mahajan <maharup@amazon.com> --------- Signed-off-by: Rupal Mahajan <maharup@amazon.com> (cherry picked from commit 73305e9) Co-authored-by: Rupal Mahajan <maharup@amazon.com>

* Integ Test Refactoring Signed-off-by: Vamsi Manohar <reddyvam@amazon.com>

…pensearch-project#1389) * Refactor AWSSigV4 auth to support different AWSCredentialProviders Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> * Added unit tests for sts assume role credentials provider Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> --------- Signed-off-by: Vamsi Manohar <reddyvam@amazon.com>

…pensearch-project#1380) (opensearch-project#1381) Signed-off-by: Chen Dai <daichen@amazon.com> (cherry picked from commit 71d8604) Co-authored-by: Chen Dai <daichen@amazon.com>

Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com>

…h-project#1404) Signed-off-by: MaxKsyunz <maxk@bitquilltech.com> (cherry picked from commit 6108ca1) Co-authored-by: Max Ksyunz <maxk@bitquilltech.com>

… (opensearch-project#1410) Signed-off-by: Peng Huo <penghuo@gmail.com> (cherry picked from commit 0a02009)

…rface (opensearch-project#1408) (opensearch-project#1414) * Added new interface methods in DataSourceService and DataSourceMetadataStorage Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> (cherry picked from commit 98fb969) Co-authored-by: vamsi-amazon <reddyvam@amazon.com>

Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>

…opensearch-project#1354) (opensearch-project#1424) Signed-off-by: Peng Huo <penghuo@gmail.com> (cherry picked from commit bcfda37) Co-authored-by: Peng Huo <penghuo@gmail.com>

…-project#1434) (opensearch-project#1437) * Updated SINH function to V2 engine, added documentation, unit/IT tests Signed-off-by: Matthew Wells <matthew.wells@improving.com> (cherry picked from commit c9f96fb) Co-authored-by: Matthew Wells <matthew.wells@improving.com>

…-project#1433) (opensearch-project#1439) Added RINT to V2 engine, updated documentation, added unit and IT tests Signed-off-by: Matthew Wells <matthew.wells@improving.com> (cherry picked from commit 7220cfe) Co-authored-by: Matthew Wells <matthew.wells@improving.com>

…pensearch-project#1438) * Added Tests * Added Implementation And Documentation * Added Integration Test And Fixed Checkstyle * Addressed PR Comments * Fixed Checkstyle * Added Tests And Modified Docs * Fixed formatNanos Helper Function * Temporarily Removed Failing Tests * Fixed Nanoseconds Float/Double Imprecision Issues * Fixed Checkstyle * Added Tests * Altered Implementation To Use BigDecimal --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit aa7f90e) Co-authored-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

…earch-project#1440) * Added Tests * Added Implementation * Added Documentation * Fixed IT Test * Adjusted IT test * Fixed Implementation for Time And Unit Tests * Addressed PR Comments * Cleaned WeekdayTest File * Fixed Docs * Fixed Doctest * Removed Unneeded Function --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit 4f9f5ca) Co-authored-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

…pensearch-project#1445) * Add `YEARWEEK` Function To OpenSearch SQL * Added Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Documentation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Addressed PR Comments Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Cleaned YearweekTest File Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Tests and Fixed Docs Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added And Modified Unit Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Jacoco In Core Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Reworked convertWeekModeFromMySqlToJava Function Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Refactored To Remove Extra Function Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Magic Number Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit 03e4f97)

) (opensearch-project#1443) * Add `EXTRACT` Function To OpenSearch SQL Plugin * Added Basic Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Lexer And Parser Language Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Implementation And Fixed Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Documentation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Modified Implementation And Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Checkstyle Issues Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Javadoc comments and Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed A Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Integration Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Refactored Extract Tests Into New File Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Refactored Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Renamed Rule In Parser Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed AstExpressionBuilder Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Removed Unneeded Code And Added Parser Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit 87018c6)

opensearch-project#1444) * Add `STR_TO_DATE` Function To The SQL Plugin. * Added Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Empty Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Partially Working Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Cleaned Up Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Modified Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Modified IT Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Documentation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Unit Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Altered Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Reworked Implementation To Always Return Datetime Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Addressed PR Comments Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Some Checkstyle Issues Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Cleaned Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Supported Function Properties Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Checkstyle Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Removed Unneeded Function Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Cleaned Implementation and Added IT Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Code Coverage and Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Cleaned Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit 40336d4)

) (opensearch-project#1447) * Add The `TO_SECONDS` Function To The SQL Plugin (opensearch-project#232) * Added Basic Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added IT Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed Integration Tests Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Test For Time Type Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Documentation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Addressed PR Comments Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Docs and Implementation Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Checkstyle Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed DateTimeFormatter Priority Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added More Formatters Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Updated Docs Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Reworked Implementation For Formatters Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Cleanup Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Test Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Implementation And Code Coverage Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Changed getFormatter Function Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Added Comments Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Removed Unneeded Code Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit d38a6ec)

…ensearch-project#1448) * Updated Arithmetic functions from old engine to new engine (opensearch-project#235) * Updated ADD, SUBTRACT, MULTIPLY, DIVIDE, MODULUS to V2 engine Signed-off-by: Matthew Wells <matthew.wells@improving.com> (cherry picked from commit bc39346) Co-authored-by: Matthew Wells <matthew.wells@improving.com>

…rch-project#1442) * Added SIGNUM function to V2 engine (opensearch-project#241) * Added SIGNUM function to V2 engine, updated documentation, added unit/IT tests Signed-off-by: Matthew Wells <matthew.wells@improving.com> * fixed merge mistake Signed-off-by: Matthew Wells <matthew.wells@improving.com> * fixed merging error Signed-off-by: Matthew Wells <matthew.wells@improving.com> --------- Signed-off-by: Matthew Wells <matthew.wells@improving.com> (cherry picked from commit 6496692) Co-authored-by: Matthew Wells <matthew.wells@improving.com>

…ect#1451) (opensearch-project#1453) * Add `TIMESTAMPADD` Function To OpenSearch SQL Plugin Added Testing And Implementation for TIMESTAMPADD Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Fixed Formatting Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit 7b83e50) Co-authored-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

…ion (opensearch-project#1314) (opensearch-project#1455) * Rework on `OpenSearchDataType`: parse, store and use mapping information (opensearch-project#180) Rework on `OpenSearchDataType`: * Add data types for those classes are not defined in `ExprCoreType`. * Address Bit-Quill#180 (comment) * Remove `TextKeywordValue`. * Add changes according to the PR review. Bit-Quill#180 * Update `IndexMapping::parseMapping` function. * Add `OpenSearchDataType::resolve` function. * Add new constructor for `OpenSearchTextType`. * Make `fields` and `properties` in `OpenSearchDataType` readonly. Update tests and mapping parser. * Move `getFields` from `OpenSearchDataType` to `OpenSearchTextType`. Update tests. * Rewrite `traverseAndFlatten` according to Bit-Quill#180 (comment) * Minor comment fix. * A fix to avoid breaking changes. * `typeName` and `legacyTypeName` to return different type names. * Change `typeof` function and corresponding tests. * Move `convertTextToKeyword` from `ScriptUtils` to `OpenSearchTextType`. Update tests. * Update UT for `typeof` function. * Make all instances of `OpenSearchDataType` and of derived types singletones as much as possible. * Make string representations of all `ExprType`s uppercase. * Remove functions from `IndexMapping` used in tests only. Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com> Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com> (cherry picked from commit d44cd39) Co-authored-by: Yury-Fridlyand <yury.fridlyand@improving.com>

…search-project#247) (opensearch-project#1457) (opensearch-project#1459) * Changed pow and power to return null when an invalid response would be returned and added missing integration tests for the pow and power function. Signed-off-by: Matthew Wells <matthew.wells@improving.com> (cherry picked from commit a35f963) Co-authored-by: Matthew Wells <matthew.wells@improving.com>

…) (opensearch-project#1472) * Add `Timestampdiff` Function To OpenSearch SQL Added Tests And Implementation For Timestampdiff --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> * Removed Import Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> --------- Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit 31148da) Co-authored-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

) Signed-off-by: vamsi-amazon <reddyvam@amazon.com>

…pensearch-project#2560) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> (cherry picked from commit d11a268)

…h-project#2562) * Add vacuum operation and IT * Add index state doc delete and more IT * Refactor IT * Fix bytebuddy version conflict * Fix broken IT * Fix broken IT * Fix jacoco failure with new IT * Fix code format * Fix jacoco test coverage --------- (cherry picked from commit 8374cb6) Signed-off-by: Chen Dai <daichen@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…rch-project#2563) * Change aysnc_query default setting * fix doctest --------- (cherry picked from commit b375a28) Signed-off-by: Peng Huo <penghuo@gmail.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…earch-project#2566) * percent encode opensearch index name * spec test vacuum * spotlessApply --------- (cherry picked from commit e17962f) Signed-off-by: Sean Kao <seankao@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Refactor query param * Reduce scope of changes --------- (cherry picked from commit ee2dbd5) Signed-off-by: Louis Chu <clingzhi@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…opensearch-project#2567) (cherry picked from commit b3fc1ca) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…ch-project#2569) (cherry picked from commit 405068b) Signed-off-by: Louis Chu <clingzhi@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…opensearch-project#2575) (cherry picked from commit 284a0be) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…nsearch-project#2571) (cherry picked from commit 4dc83b7) Signed-off-by: Louis Chu <clingzhi@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…h-project#2580) (cherry picked from commit a67de21) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…earch-project#2582) (opensearch-project#2583) (cherry picked from commit 85dae6f) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…#2577) (opensearch-project#2589) (cherry picked from commit bfcaedf) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

) (opensearch-project#2592) * escape query in spark submit parameter Signed-off-by: Sean Kao <seankao@amazon.com> * spotless Signed-off-by: Sean Kao <seankao@amazon.com> * test case for special character Signed-off-by: Sean Kao <seankao@amazon.com> --------- Signed-off-by: Sean Kao <seankao@amazon.com> (cherry picked from commit 2bcf0b8)

…arch-project#2595) (cherry picked from commit e153609) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…h-project#2607) (cherry picked from commit 36b423c) Signed-off-by: Chen Dai <daichen@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…ensearch-project#2615) (opensearch-project#2618) (cherry picked from commit 204c7da) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…oject#2613) (cherry picked from commit 2649200) Signed-off-by: Rupal Mahajan <maharup@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…oject#2627) (cherry picked from commit 0d8341f) Signed-off-by: Louis Chu <clingzhi@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…-project#2441) (opensearch-project#2629) (cherry picked from commit 3f53904) Signed-off-by: Andreas Kulicke <a.kulicke@gmx.de> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…opensearch-project#2608) (cherry picked from commit 39c0222) Signed-off-by: Adi Suresh <adsuresh@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…search-project#2633) * update sql grammar files from upstream * resolve semicolon cause incorrect parsing * spotlessApply --------- (cherry picked from commit 7f8fbe9) Signed-off-by: Sean Kao <seankao@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…error (opensearch-project#2626) (opensearch-project#2634) (cherry picked from commit 5464bfc) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com>

…ensearch-project#2635) Instead of downloading the JAR from Maven, the JAR in the EMR serverless root file system can be used. (cherry picked from commit e578a57) Signed-off-by: Adi Suresh <adsuresh@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…ect#2648) (cherry picked from commit b454a2c) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…oject#2624) (opensearch-project#2639) * Add option to use LakeFormation in S3Glue data source. * Update s3glue_connector.rst corrected formatting issue. --------- (cherry picked from commit ea08c8f) Signed-off-by: Adi Suresh <adsuresh@amazon.com> Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Vamsi Manohar <reddyvam@amazon.com>

…#2664) (cherry picked from commit cfd222c) Signed-off-by: Sean Kao <seankao@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…nsearch-project#2667) * Delete Spark datasource (opensearch-project#2638) * Delete Spark datasource Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Fix build error Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Delete spark_connector.rst Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Add missing test Signed-off-by: Tomoyuki Morita <moritato@amazon.com> --------- Signed-off-by: Tomoyuki Morita <moritato@amazon.com> (cherry picked from commit de7b367) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Fix test failure in DefaultSparkSqlFunctionResponseHandleTest Signed-off-by: Tomoyuki Morita <moritato@amazon.com> * Reformat Signed-off-by: Tomoyuki Morita <moritato@amazon.com> --------- Signed-off-by: Tomoyuki Morita <moritato@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

opensearch-trigger-bot bot and others added 30 commits February 16, 2023 09:07

Added Support For More Formats For Get_Format (opensearch-project#1343)…

4afe013

… (opensearch-project#1349) Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit 672c72f) Co-authored-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Add last_day Function To OpenSearch SQL Plugin (opensearch-project#…

cc2a768

…1344) (opensearch-project#1350) Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com> (cherry picked from commit d9114f5) Co-authored-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Modified returning NaN to NULL (opensearch-project#1347)

a32bcbe

Signed-off-by: Guian Gumpac <guian.gumpac@improving.com> (cherry picked from commit de40f42)

Integ Test Refactoring (opensearch-project#1383)

8f6793b

* Integ Test Refactoring Signed-off-by: Vamsi Manohar <reddyvam@amazon.com>

Exclude .opensearch index when cleanup and ignore exception in case (o…

4749176

…pensearch-project#1380) (opensearch-project#1381) Signed-off-by: Chen Dai <daichen@amazon.com> (cherry picked from commit 71d8604) Co-authored-by: Chen Dai <daichen@amazon.com>

Increment version to 2.7.0-SNAPSHOT (opensearch-project#1368)

74b802e

Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com>

Update usage of Strings.toString (opensearch-project#1309) (opensearc…

652e771

…h-project#1404) Signed-off-by: MaxKsyunz <maxk@bitquilltech.com> (cherry picked from commit 6108ca1) Co-authored-by: Max Ksyunz <maxk@bitquilltech.com>

Deprecated Spring IoC and using Guice instead (opensearch-project#1177)…

fcce603

… (opensearch-project#1410) Signed-off-by: Peng Huo <penghuo@gmail.com> (cherry picked from commit 0a02009)

Bump backport version. (opensearch-project#1009)

81e5569

Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>

Resolve table function based StorageEngine provided function resolver (…

0b6e581

…opensearch-project#1354) (opensearch-project#1424) Signed-off-by: Peng Huo <penghuo@gmail.com> (cherry picked from commit bcfda37) Co-authored-by: Peng Huo <penghuo@gmail.com>

Create datasource API (opensearch-project#1458) (opensearch-project#1479

1aa1861

) Signed-off-by: vamsi-amazon <reddyvam@amazon.com>

vamsimanohar and others added 28 commits March 18, 2024 17:45

Handle ALTER Index Queries in SQL Plugin (opensearch-project#2554) (o…

8e25dd9

…pensearch-project#2560) Signed-off-by: Vamsi Manohar <reddyvam@amazon.com> (cherry picked from commit d11a268)

Increment version to 2.14.0-SNAPSHOT (opensearch-project#2585)

294566f

Signed-off-by: opensearch-ci-bot <opensearch-infra@amazon.com> Co-authored-by: opensearch-ci-bot <opensearch-infra@amazon.com>

ykmr1224 closed this May 13, 2024

ykmr1224 deleted the backport-2636 branch May 18, 2024 00:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Backport 2.x] Refactor SparkQueryDispatcher (#2636) #2668

[Backport 2.x] Refactor SparkQueryDispatcher (#2636) #2668

Uh oh!

ykmr1224 commented May 13, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

17 participants

[Backport 2.x] Refactor SparkQueryDispatcher (#2636) #2668

[Backport 2.x] Refactor SparkQueryDispatcher (#2636) #2668

Uh oh!

Conversation

ykmr1224 commented May 13, 2024

Description

Issues Resolved

Check List

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

17 participants