Add wildcard support for rename command

[ritvibhatt]

Description

• Added wildcard support for the rename command in PPL queries
• Implemented wildcard pattern matching using * to rename multiple fields at once
• Allow space delimiting

Changes

• Added WildcardRenameUtils utility class with pattern matching and transformation
  logic
• Updated CalciteRelNodeVisitor.java to support wildcard renaming for Calcite engine and change behavior for renaming nonexisting fields/renaming to existing field
• Modified PPL parser to accept * in wildcard expressions and delimit with space

Field Rename Behavior:

• Renaming a non-existent field to a non-existent field: No change occurs to the result set.
• Renaming a non-existent field to an existing field: The existing target field is removed from the result set.
• Renaming an existing field to an existing field: The existing target field is removed, then the source field is renamed to the target.

Examples

• source=accounts | rename *name as *_name -- firstname → first_name

• source=accounts | rename * as old_* -- all fields get "old_" prefix

• source=data | rename old_* as * -- removes "old_ "prefix from matching fields

• source and target field names much have matching number of wildcards

• * matches all fields

Performance testing:
Baseline:
source=big5 | head <size>
baseline 10: Avg = 16ms, P90 = 17ms
baseline 100: Avg = 23ms, P90 = 24ms
baseline 1K: Avg = 95ms, P90 = 97ms
baseline 10K: Avg = 816ms, P90 = 821ms
baseline 100K: Avg = 812ms, P90 = 815ms

source=big5 | rename @timestamp as ts | head <size>
10: Avg = 16ms, P90 = 17ms
100: Avg = 24ms, P90 = 24ms
1K: Avg = 95ms, P90 = 97ms
10K: Avg = 816ms, P90 = 820ms
100K: Avg = 813ms, P90 = 817ms

source=big5 | rename @timestamp as ts, agent.type as agent_type | head <size>
10: Avg = 17ms, P90 = 18ms
100: Avg = 24ms, P90 = 25ms
1K: Avg = 97ms, P90 = 99ms
10K: Avg = 832ms, P90 = 835ms
100K: Avg = 828ms, P90 = 833ms

source=big5 | rename agent* as new_* | head <size>
10: Avg = 16ms, P90 = 17ms
100: Avg = 23ms, P90 = 24ms
1K: Avg = 95ms, P90 = 97ms
10K: Avg = 815ms, P90 = 818ms
100K: Avg = 813ms, P90 = 819ms

source=big5 | rename *name as *_name | head <size>
10: Avg = 16ms, P90 = 17ms
100: Avg = 23ms, P90 = 24ms
1K: Avg = 95ms, P90 = 97ms
10K: Avg = 814ms, P90 = 822ms
100K: Avg = 811ms, P90 = 815ms

Related Issues

Resolves #[Issue number to be closed when this PR is merged]
#4008

Check List

• New functionality includes testing.
• New functionality has been documented.
• New functionality has javadoc added.
• New functionality has a user manual doc added.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[penghuo]

Does rename f*r*t* as F*R*T* supported?

[ritvibhatt]

Yes, added test for partial match to CalcitePPLRenameIT

[penghuo]

What happen if origin field name include speicial * character, e.g.

rename `field*` as `myfield*`

[ritvibhatt]

Can't match/replace wildcard characters in field names, this is SPL behavior as well. Added a note in rename.rst

[penghuo]

Is it expected?

###
POST {{baseUrl}}/tttt/_bulk
Content-Type: application/x-ndjson

{ "index": { "_id": 1 } }
{ "value*": 10, "id123": 1 }


###
POST {{baseUrl}}/_plugins/_ppl/
Content-Type: application/x-ndjson

{
  "query": "source=tttt "
}

###
POST {{baseUrl}}/_plugins/_ppl
Content-Type: application/x-ndjson

{
  "query": "source=tttt | rename `value*` as cde*"
}

### Results
{
  "schema": [
    {
      "name": "cde*",
      "type": "bigint"
    },
    {
      "name": "id123",
      "type": "bigint"
    }
  ],
  "datarows": [
    [
      10,
      1
    ]
  ],
  "total": 1,
  "size": 1
}

[ritvibhatt]

Yes, the * will match the * in the other field name but it can't be replaced in the rename, so trying to rename value* as cde won't work

[penghuo]

If match failed, query should works, but no field renamed, right?

[ritvibhatt]

Yes, updated to just not rename anything if wildcard doesn't match any fields

[penghuo]

Why chagne Analyzer? It is for calcite.enable=false? If yes, it is not required.

[ritvibhatt]

Removed changes for v2 engine support

[penghuo]

does cascading rename use case supported?, e.g. e.g. rename *name as *_name, *_name as *@name

[ritvibhatt]

Updated to support cascading rename for both wildcard and no wildcard and added test in CalcitePPLRenameIT

[ritvibhatt]

Changed to update field names after each rename clause so next rename can see changes instead of all at once after all renames. Added testCascadingRename and testCascadingRenameWithWildcard in CalcitePPLRenameIT

[penghuo]

Add since 3.3 to indicate this is a new feature

[penghuo]

Add IT for failure case, and assert error message.

[penghuo]

Add IT for failure case, and assert error message.

[ykmr1224]

nit&optional: It might be simpler to put this logic into validatePatternCompatibility and throw exception from there. (simply call WildcardRenameUtils.validatePatternCompatibility from this method)

[opensearch-trigger-bot]

The backport to 2.19-dev failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/sql/backport-2.19-dev 2.19-dev
# Navigate to the new working tree
pushd ../.worktrees/sql/backport-2.19-dev
# Create a new branch
git switch --create backport/backport-4019-to-2.19-dev
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 ab5f21a9eb82d511dff8adec383930d135930295
# Push it to GitHub
git push --set-upstream origin backport/backport-4019-to-2.19-dev
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/sql/backport-2.19-dev

Then, create a pull request where the base branch is 2.19-dev and the compare/head branch is backport/backport-4019-to-2.19-dev.

[LantaoJin]

Backporing is required manually in this PR. @ykmr1224

[ritvibhatt]

@LantaoJin I raised the manual backport (#4250), trying to figure out test failures