generated from amazon-archives/__template_Custom
-
Notifications
You must be signed in to change notification settings - Fork 176
Add wildcard support for rename command #4019
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
ykmr1224
merged 26 commits into
opensearch-project:main
from
ritvibhatt:rename-wildcard-support
Sep 8, 2025
Merged
Changes from all commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
106d5d9
add wildcard support for rename
ritvibhatt 6d39673
fix calcite wildcard support and add tests
ritvibhatt 0235b29
fix formatting
ritvibhatt 90ae92a
add check to analyzer
ritvibhatt 9a893fa
update doc formatting
ritvibhatt a7b77db
remove v2 engine wildcard support
ritvibhatt 5b1c62b
update doc
ritvibhatt 60c8713
fix formatting
ritvibhatt d01927b
support cascading rename
ritvibhatt 610485b
update formatting
ritvibhatt a3f2667
add cross cluster test
ritvibhatt a2ebe12
add test for cascading rename
ritvibhatt 86eff61
fix formatting
ritvibhatt 775bd85
add test for cascading rename
ritvibhatt 845820f
change behavior for renaming existing fields
ritvibhatt a4c08e1
add tests and update docs
ritvibhatt 6c51ffd
update docs
ritvibhatt 5801542
update docs
ritvibhatt 2d3a38b
fix renaming to same name
ritvibhatt b6fb2ba
fix behavior for consecutive wildcards/address comments
ritvibhatt f8f2a6a
add back import
ritvibhatt 6f4d4f2
fix doc
ritvibhatt a1e1002
Merge branch 'main' into rename-wildcard-support
ritvibhatt 555c457
fix doc
ritvibhatt 2f1c08d
Merge branch 'main' into rename-wildcard-support
ritvibhatt e8e71f1
fix formatting
ritvibhatt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -133,6 +133,7 @@ | |
import org.opensearch.sql.expression.function.PPLFuncImpTable; | ||
import org.opensearch.sql.expression.parse.RegexCommonUtils; | ||
import org.opensearch.sql.utils.ParseUtils; | ||
import org.opensearch.sql.utils.WildcardRenameUtils; | ||
|
||
public class CalciteRelNodeVisitor extends AbstractNodeVisitor<RelNode, CalcitePlanContext> { | ||
|
||
|
@@ -465,25 +466,52 @@ public RelNode visitRename(Rename node, CalcitePlanContext context) { | |
visitChildren(node, context); | ||
List<String> originalNames = context.relBuilder.peek().getRowType().getFieldNames(); | ||
List<String> newNames = new ArrayList<>(originalNames); | ||
|
||
for (org.opensearch.sql.ast.expression.Map renameMap : node.getRenameList()) { | ||
if (renameMap.getTarget() instanceof Field t) { | ||
String newName = t.getField().toString(); | ||
RexNode check = rexVisitor.analyze(renameMap.getOrigin(), context); | ||
if (check instanceof RexInputRef ref) { | ||
newNames.set(ref.getIndex(), newName); | ||
} else { | ||
throw new SemanticCheckException( | ||
String.format("the original field %s cannot be resolved", renameMap.getOrigin())); | ||
} | ||
} else { | ||
if (!(renameMap.getTarget() instanceof Field)) { | ||
throw new SemanticCheckException( | ||
String.format("the target expected to be field, but is %s", renameMap.getTarget())); | ||
} | ||
|
||
String sourcePattern = ((Field) renameMap.getOrigin()).getField().toString(); | ||
String targetPattern = ((Field) renameMap.getTarget()).getField().toString(); | ||
|
||
if (WildcardRenameUtils.isWildcardPattern(sourcePattern) | ||
&& !WildcardRenameUtils.validatePatternCompatibility(sourcePattern, targetPattern)) { | ||
throw new SemanticCheckException( | ||
"Source and target patterns have different wildcard counts"); | ||
} | ||
Comment on lines
+479
to
+483
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit&optional: It might be simpler to put this logic into validatePatternCompatibility and throw exception from there. (simply call |
||
|
||
List<String> matchingFields = WildcardRenameUtils.matchFieldNames(sourcePattern, newNames); | ||
|
||
for (String fieldName : matchingFields) { | ||
String newName = | ||
WildcardRenameUtils.applyWildcardTransformation( | ||
sourcePattern, targetPattern, fieldName); | ||
if (newNames.contains(newName) && !newName.equals(fieldName)) { | ||
removeFieldIfExists(newName, newNames, context); | ||
} | ||
int fieldIndex = newNames.indexOf(fieldName); | ||
if (fieldIndex != -1) { | ||
newNames.set(fieldIndex, newName); | ||
} | ||
} | ||
|
||
if (matchingFields.isEmpty() && newNames.contains(targetPattern)) { | ||
removeFieldIfExists(targetPattern, newNames, context); | ||
context.relBuilder.rename(newNames); | ||
} | ||
} | ||
context.relBuilder.rename(newNames); | ||
return context.relBuilder.peek(); | ||
} | ||
|
||
private void removeFieldIfExists( | ||
String fieldName, List<String> newNames, CalcitePlanContext context) { | ||
newNames.remove(fieldName); | ||
context.relBuilder.projectExcept(context.relBuilder.field(fieldName)); | ||
} | ||
|
||
@Override | ||
public RelNode visitSort(Sort node, CalcitePlanContext context) { | ||
visitChildren(node, context); | ||
|
141 changes: 141 additions & 0 deletions
141
core/src/main/java/org/opensearch/sql/utils/WildcardRenameUtils.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,141 @@ | ||
/* | ||
* Copyright OpenSearch Contributors | ||
* SPDX-License-Identifier: Apache-2.0 | ||
*/ | ||
|
||
package org.opensearch.sql.utils; | ||
|
||
import java.util.ArrayList; | ||
import java.util.Arrays; | ||
import java.util.Collection; | ||
import java.util.List; | ||
import java.util.regex.Matcher; | ||
import java.util.regex.Pattern; | ||
import java.util.stream.Collectors; | ||
|
||
/** Utility class for handling wildcard patterns in rename operations. */ | ||
public class WildcardRenameUtils { | ||
|
||
/** | ||
* Check if pattern contains any supported wildcards. | ||
* | ||
* @param pattern the pattern to check | ||
* @return true if pattern contains * wildcards | ||
*/ | ||
public static boolean isWildcardPattern(String pattern) { | ||
return pattern.contains("*"); | ||
} | ||
|
||
/** | ||
* Check if pattern is only wildcards that matches all fields. | ||
* | ||
* @param pattern the pattern to check | ||
* @return true if pattern is only made up of wildcards "*" | ||
*/ | ||
public static boolean isFullWildcardPattern(String pattern) { | ||
return pattern.matches("\\*+"); | ||
} | ||
|
||
/** | ||
* Convert wildcard pattern to regex. | ||
* | ||
* @param pattern the wildcard pattern | ||
* @return regex pattern with capture groups | ||
*/ | ||
public static String wildcardToRegex(String pattern) { | ||
String[] parts = pattern.split("\\*", -1); | ||
return Arrays.stream(parts).map(Pattern::quote).collect(Collectors.joining("(.*)")); | ||
} | ||
|
||
/** | ||
* Match field names against wildcard pattern. | ||
* | ||
* @param wildcardPattern the pattern to match against | ||
* @param availableFields collection of available field names | ||
* @return list of matching field names | ||
*/ | ||
public static List<String> matchFieldNames( | ||
String wildcardPattern, Collection<String> availableFields) { | ||
// Single wildcard matches all available fields | ||
if (isFullWildcardPattern(wildcardPattern)) { | ||
return new ArrayList<>(availableFields); | ||
} | ||
|
||
String regexPattern = "^" + wildcardToRegex(wildcardPattern) + "$"; | ||
Pattern pattern = Pattern.compile(regexPattern); | ||
|
||
return availableFields.stream() | ||
.filter(field -> pattern.matcher(field).matches()) | ||
.collect(Collectors.toList()); | ||
} | ||
|
||
/** | ||
* Apply wildcard transformation to get new field name. | ||
* | ||
* @param sourcePattern the source wildcard pattern | ||
* @param targetPattern the target wildcard pattern | ||
* @param actualFieldName the actual field name to transform | ||
* @return transformed field name | ||
* @throws IllegalArgumentException if patterns don't match or are invalid | ||
*/ | ||
public static String applyWildcardTransformation( | ||
String sourcePattern, String targetPattern, String actualFieldName) { | ||
|
||
if (sourcePattern.equals(targetPattern)) { | ||
return actualFieldName; | ||
} | ||
|
||
if (!isFullWildcardPattern(sourcePattern) || !isFullWildcardPattern(targetPattern)) { | ||
if (sourcePattern.matches(".*\\*{2,}.*") || targetPattern.matches(".*\\*{2,}.*")) { | ||
throw new IllegalArgumentException("Consecutive wildcards in pattern are not supported"); | ||
} | ||
} | ||
|
||
String sourceRegex = "^" + wildcardToRegex(sourcePattern) + "$"; | ||
Matcher matcher = Pattern.compile(sourceRegex).matcher(actualFieldName); | ||
|
||
if (!matcher.matches()) { | ||
throw new IllegalArgumentException( | ||
String.format("Field '%s' does not match pattern '%s'", actualFieldName, sourcePattern)); | ||
} | ||
|
||
String result = targetPattern; | ||
|
||
for (int i = 1; i <= matcher.groupCount(); i++) { | ||
String capturedValue = matcher.group(i); | ||
|
||
int index = result.indexOf("*"); | ||
if (index >= 0) { | ||
ykmr1224 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
result = result.substring(0, index) + capturedValue + result.substring(index + 1); | ||
} else { | ||
throw new IllegalArgumentException( | ||
"Target pattern has fewer wildcards than source pattern"); | ||
} | ||
} | ||
|
||
return result; | ||
} | ||
|
||
/** | ||
* Validate that source and target patterns have matching wildcard counts. | ||
* | ||
* @param sourcePattern the source pattern | ||
* @param targetPattern the target pattern | ||
* @return true if patterns are compatible | ||
*/ | ||
public static boolean validatePatternCompatibility(String sourcePattern, String targetPattern) { | ||
int sourceWildcards = countWildcards(sourcePattern); | ||
int targetWildcards = countWildcards(targetPattern); | ||
return sourceWildcards == targetWildcards; | ||
} | ||
|
||
/** | ||
* Count the number of wildcards in a pattern. | ||
* | ||
* @param pattern the pattern to analyze | ||
* @return number of wildcard characters | ||
*/ | ||
private static int countWildcards(String pattern) { | ||
return (int) pattern.chars().filter(ch -> ch == '*').count(); | ||
} | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does cascading rename use case supported?, e.g.
e.g. rename *name as *_name, *_name as *@name
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated to support cascading rename for both wildcard and no wildcard and added test in
CalcitePPLRenameIT
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed to update field names after each rename clause so next rename can see changes instead of all at once after all renames. Added
testCascadingRename
andtestCascadingRenameWithWildcard
inCalcitePPLRenameIT