Skip to content

Conversation

@yuancu
Copy link
Collaborator

@yuancu yuancu commented Jan 29, 2026

Summary

  • Moved Calcite-only command tests from CrossClusterSearchIT to CalciteCrossClusterSearchIT
  • The following tests were using Calcite-only commands but Calcite was not enabled in CrossClusterSearchIT:
    • testCrossClusterAddTotals - uses addtotals command
    • testCrossClusterAddColTotals - uses addcoltotals command
    • testCrossClusterTranspose - uses transpose command
    • testCrossClusterAppend - uses append command
    • testCrossClusterMvcombine - uses mvcombine command

Issues Resolved

Fixes #5084

Check List

  • New functionality includes testing.
  • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
  • New functionality has javadoc added
  • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…citeCrossClusterSearchIT

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jan 29, 2026

📝 Walkthrough

Walkthrough

Relocates five cross-cluster search tests that use Calcite-only commands (addtotals, addcoltotals, transpose, append, mvcombine) from CrossClusterSearchIT into CalciteCrossClusterSearchIT, and updates docs to reference either test class for cross-cluster PPL commands.

Changes

Cohort / File(s) Summary
Cross-Cluster Calcite Tests Addition
integ-test/src/test/java/org/opensearch/sql/security/CalciteCrossClusterSearchIT.java
Added five test methods exercising Calcite-only cross-cluster features: testCrossClusterAddTotals, testCrossClusterAddColTotals, testCrossClusterTranspose, testCrossClusterAppend, testCrossClusterMvcombine.
Cross-Cluster Search Tests Removal
integ-test/src/test/java/org/opensearch/sql/security/CrossClusterSearchIT.java
Removed the same five test methods from the non-Calcite test class to avoid running Calcite-only commands when Calcite is disabled.
Docs Update
docs/dev/ppl-commands.md
Adjusted cross-cluster tests mention to allow CrossClusterSearchIT or CalciteCrossClusterSearchIT (Calcite-required cases).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

Suggested labels

testing, PPL

Suggested reviewers

  • LantaoJin
  • penghuo
  • anirudha
  • Swiddis
  • ykmr1224
🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: moving Calcite-only tests from one test class to another.
Linked Issues check ✅ Passed The PR successfully addresses issue #5084 by moving all five Calcite-only tests (addtotals, addcoltotals, transpose, append, mvcombine) from CrossClusterSearchIT to CalciteCrossClusterSearchIT where Calcite is enabled.
Out of Scope Changes check ✅ Passed All changes are directly related to relocating tests and updating documentation. No unrelated modifications were introduced.
Description check ✅ Passed The PR description clearly explains the changes: moving five Calcite-only command tests from CrossClusterSearchIT to CalciteCrossClusterSearchIT due to Calcite not being enabled in the source class.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings

Comment @coderabbitai help to get the list of available commands and usage tips.

@yuancu yuancu added the bugFix label Jan 29, 2026
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In
`@integ-test/src/test/java/org/opensearch/sql/security/CalciteCrossClusterSearchIT.java`:
- Around line 354-361: Update misleading copy-paste comments in the
CalciteCrossClusterSearchIT test methods so they accurately describe the test
action instead of saying "Test query_string without fields parameter";
specifically edit the comment above testCrossClusterAddTotals to mention testing
the addtotals command on a remote cluster, the comment above
testCrossClusterAddColTotals to mention testing addcoltotals, and the comment
above testCrossClusterTranspose to mention testing transpose; locate these
comments in the CalciteCrossClusterSearchIT class near the methods
testCrossClusterAddTotals, testCrossClusterAddColTotals, and
testCrossClusterTranspose and either remove the incorrect lines or replace them
with concise, accurate descriptions referencing the exact command being
verified.

…ommands

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In
`@integ-test/src/test/java/org/opensearch/sql/security/CalciteCrossClusterSearchIT.java`:
- Around line 353-421: Add null/invalid/boundary and multi-document coverage for
the new cross-cluster Calcite tests by extending the existing test methods (or
adding new tests) for testCrossClusterAddTotals, testCrossClusterAddColTotals,
testCrossClusterTranspose, testCrossClusterAppend, and
testCrossClusterMvcombine: include cases with null field values, empty result
sets, out-of-range/boundary numeric values, and malformed/invalid input strings;
verify results using the same helpers (verifyDataRows, verifyColumn, rows,
columnName) and add multi-document scenarios where the operator is per-document
(e.g., multiple docs with/without target field) and negative tests asserting
proper error messages or empty outputs for invalid inputs. Ensure each new test
asserts NULL handling, boundary numeric sums/concats, and at least one
multi-document aggregation scenario for each operator.

Comment on lines +353 to +421
@Test
public void testCrossClusterAddTotals() throws IOException {
JSONObject result =
executeQuery(
String.format(
"search source=%s| sort 1 age | fields firstname, age | addtotals age",
TEST_INDEX_BANK_REMOTE));
verifyDataRows(result, rows("Nanette", 28, 28));
}

/** CrossClusterSearchIT Test for addcoltotals. */
@Test
public void testCrossClusterAddColTotals() throws IOException {
JSONObject result =
executeQuery(
String.format(
"search source=%s | where firstname='Hattie' or firstname ='Nanette'|fields"
+ " firstname,age,balance | addcoltotals age balance",
TEST_INDEX_BANK_REMOTE));
verifyDataRows(
result, rows("Hattie", 36, 5686), rows("Nanette", 28, 32838), rows(null, 64, 38524));
}

@Test
public void testCrossClusterTranspose() throws IOException {
JSONObject result =
executeQuery(
String.format(
"search source=%s | where firstname='Hattie' or firstname ='Nanette' or"
+ " firstname='Dale'|sort firstname desc |fields firstname,age,balance |"
+ " transpose 3 column_name='column_names'",
TEST_INDEX_BANK_REMOTE));

verifyDataRows(
result,
rows("firstname", "Nanette", "Hattie", "Dale"),
rows("balance", "32838", "5686", "4180"),
rows("age", "28", "36", "33"));
}

@Test
public void testCrossClusterAppend() throws IOException {
JSONObject result =
executeQuery(
String.format(
"search source=%s | stats count() as cnt by gender | append [ search source=%s |"
+ " stats count() as cnt ]",
TEST_INDEX_BANK_REMOTE, TEST_INDEX_BANK_REMOTE));
verifyDataRows(result, rows(3, "F"), rows(4, "M"), rows(7, null));
}

/** CrossClusterSearchIT Test for mvcombine. */
@Test
public void testCrossClusterMvcombine() throws IOException {

JSONObject result =
executeQuery(
String.format(
"search source=%s | where firstname='Hattie' or firstname='Nanette' "
+ "| fields firstname, age | mvcombine age",
TEST_INDEX_BANK_REMOTE));

verifyColumn(result, columnName("firstname"), columnName("age"));

verifyDataRows(
result,
rows("Hattie", new org.json.JSONArray().put(36)),
rows("Nanette", new org.json.JSONArray().put(28)));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add null/invalid/boundary coverage (or point to existing coverage) for these new Calcite cross-cluster tests.
These additions are happy-path only. Please add null/invalid/boundary cases for addtotals/addcoltotals/transpose/append/mvcombine and ensure multi-document coverage where the command is per-document, or link to existing tests that already cover these scenarios.

As per coding guidelines: “NULL input tests must be included for all new functions”, “Include boundary condition tests (min/max values, empty inputs) for all new functions”, “Include error condition tests (invalid inputs, exceptions) for all new functions”, and “Include multi-document tests for per-document operations”.

🤖 Prompt for AI Agents
In
`@integ-test/src/test/java/org/opensearch/sql/security/CalciteCrossClusterSearchIT.java`
around lines 353 - 421, Add null/invalid/boundary and multi-document coverage
for the new cross-cluster Calcite tests by extending the existing test methods
(or adding new tests) for testCrossClusterAddTotals,
testCrossClusterAddColTotals, testCrossClusterTranspose, testCrossClusterAppend,
and testCrossClusterMvcombine: include cases with null field values, empty
result sets, out-of-range/boundary numeric values, and malformed/invalid input
strings; verify results using the same helpers (verifyDataRows, verifyColumn,
rows, columnName) and add multi-document scenarios where the operator is
per-document (e.g., multiple docs with/without target field) and negative tests
asserting proper error messages or empty outputs for invalid inputs. Ensure each
new test asserts NULL handling, boundary numeric sums/concats, and at least one
multi-document aggregation scenario for each operator.

@qianheng-aws qianheng-aws merged commit 4b785b0 into opensearch-project:main Jan 29, 2026
38 of 39 checks passed
@yuancu yuancu deleted the issues/5084 branch January 29, 2026 08:50
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.19-dev failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/sql/backport-2.19-dev 2.19-dev
# Navigate to the new working tree
pushd ../.worktrees/sql/backport-2.19-dev
# Create a new branch
git switch --create backport/backport-5085-to-2.19-dev
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 4b785b0cd4ad081129b4ee130d96a0e74fab0b8e
# Push it to GitHub
git push --set-upstream origin backport/backport-5085-to-2.19-dev
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/sql/backport-2.19-dev

Then, create a pull request where the base branch is 2.19-dev and the compare/head branch is backport/backport-5085-to-2.19-dev.

yuancu added a commit that referenced this pull request Jan 29, 2026
…terSearchIT (#5085)

* Move tests for calcite-only commands from CrossClusterSearchIT to CalciteCrossClusterSearchIT

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

* Ask to use CalciteCrossClusterSearchIT when developing calcite-only commands

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

---------

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
(cherry picked from commit 4b785b0)
penghuo pushed a commit that referenced this pull request Jan 29, 2026
…terSearchIT (#5085) (#5087)

* Move tests for calcite-only commands from CrossClusterSearchIT to CalciteCrossClusterSearchIT



* Ask to use CalciteCrossClusterSearchIT when developing calcite-only commands



---------


(cherry picked from commit 4b785b0)

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
@LantaoJin LantaoJin added the backport-manually Filed a PR to backport manually. label Jan 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Flaky tests in CrossClusterSearchIT due to Calcite-only features

3 participants