Skip to content

[SPARK-29680][SQL] Remove ALTER TABLE CHANGE COLUMN syntax #26338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 10 commits into from
Closed
Original file line number Diff line number Diff line change
Expand Up @@ -149,11 +149,8 @@ statement
| ALTER (TABLE | VIEW) multipartIdentifier
UNSET TBLPROPERTIES (IF EXISTS)? tablePropertyList #unsetTableProperties
| ALTER TABLE multipartIdentifier
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can distinguish the two multipartIdentifier

ALTER TABLE table=multipartIdentifier ... COLUMN? column=multipartIdentifier

You can fix it in your followup

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok.

(ALTER | CHANGE) COLUMN? qualifiedName
(ALTER | CHANGE) COLUMN? multipartIdentifier
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously we use qualifiedName: identifier ('.' identifier)* to capture column name.

This conflicts a test in ErrorParserSuite that test-col is not allowed in ALTER TABLE t CHANGE COLUMN test-col TYPE BIGINT.

The column name should be multiple errorCapturingIdentifier. So I changed it to multipartIdentifier:

multipartIdentifier
    : parts+=errorCapturingIdentifier ('.' parts+=errorCapturingIdentifier)*
    ;

@cloud-fan

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like we should replace qualifiedName with multipartIdentifier in all other places. We can do it in a followup.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. will make a followup later.

(TYPE dataType)? (COMMENT comment=STRING)? colPosition? #alterTableColumn
| ALTER TABLE tableIdentifier partitionSpec?
CHANGE COLUMN?
colName=errorCapturingIdentifier colType colPosition? #changeColumn
| ALTER TABLE multipartIdentifier (partitionSpec)?
SET SERDE STRING (WITH SERDEPROPERTIES tablePropertyList)? #setTableSerDe
| ALTER TABLE multipartIdentifier (partitionSpec)?
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2677,9 +2677,12 @@ class AstBuilder(conf: SQLConf) extends SqlBaseBaseVisitor[AnyRef] with Logging
operationNotAllowed(s"ALTER TABLE table $verb COLUMN requires a TYPE or a COMMENT", ctx)
}

val tableIdentifier = ctx.multipartIdentifier(0)
val qualifiedColumn = ctx.multipartIdentifier(1)

AlterTableAlterColumnStatement(
visitMultipartIdentifier(ctx.multipartIdentifier),
typedVisit[Seq[String]](ctx.qualifiedName),
visitMultipartIdentifier(tableIdentifier),
typedVisit[Seq[String]](qualifiedColumn),
Option(ctx.dataType).map(typedVisit[DataType]),
Option(ctx.comment).map(string))
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ class ErrorParserSuite extends AnalysisTest {
"""
|ALTER TABLE t
|CHANGE COLUMN
|test-col BIGINT
|test-col TYPE BIGINT
""".stripMargin, 4, 4, 5, msg + " test-col")
intercept("CREATE TABLE test (attri-bute INT)", 1, 24, 25, msg + " attri-bute")
intercept(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -408,33 +408,6 @@ class SparkSqlAstBuilder(conf: SQLConf) extends AstBuilder(conf) {
ctx.VIEW != null)
}

/**
* Create a [[AlterTableChangeColumnCommand]] command.
*
* For example:
* {{{
* ALTER TABLE table [PARTITION partition_spec]
* CHANGE [COLUMN] column_old_name column_new_name column_dataType [COMMENT column_comment]
* [FIRST | AFTER column_name];
* }}}
*/
override def visitChangeColumn(ctx: ChangeColumnContext): LogicalPlan = withOrigin(ctx) {
if (ctx.partitionSpec != null) {
operationNotAllowed("ALTER TABLE table PARTITION partition_spec CHANGE COLUMN", ctx)
}

if (ctx.colPosition != null) {
operationNotAllowed(
"ALTER TABLE table [PARTITION partition_spec] CHANGE COLUMN ... FIRST | AFTER otherCol",
ctx)
}

AlterTableChangeColumnCommand(
tableName = visitTableIdentifier(ctx.tableIdentifier),
columnName = ctx.colName.getText,
newColumn = visitColType(ctx.colType))
}

/**
* Convert a nested constants list into a sequence of string sequences.
*/
Expand Down
45 changes: 16 additions & 29 deletions sql/core/src/test/resources/sql-tests/inputs/change-column.sql
Original file line number Diff line number Diff line change
Expand Up @@ -2,56 +2,43 @@
CREATE TABLE test_change(a INT, b STRING, c INT) using parquet;
DESC test_change;

-- Change column name (not supported yet)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have a test file for RENAME COLUMN? If not can we add some tests here to keep the test coverage?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok.

ALTER TABLE test_change CHANGE a a1 INT;
-- ALTER TABLE CHANGE COLUMN must change either type or comment
ALTER TABLE test_change CHANGE a;
DESC test_change;

-- Change column name (not supported on v1 table)
ALTER TABLE test_change RENAME COLUMN a TO a1;
DESC test_change;

-- Change column dataType (not supported yet)
ALTER TABLE test_change CHANGE a a STRING;
ALTER TABLE test_change CHANGE a TYPE STRING;
DESC test_change;

-- Change column position (not supported yet)
ALTER TABLE test_change CHANGE a a INT AFTER b;
ALTER TABLE test_change CHANGE b b STRING FIRST;
ALTER TABLE test_change CHANGE a TYPE INT AFTER b;
ALTER TABLE test_change CHANGE b TYPE STRING FIRST;
DESC test_change;

-- Change column comment
ALTER TABLE test_change CHANGE a a INT COMMENT 'this is column a';
ALTER TABLE test_change CHANGE b b STRING COMMENT '#*02?`';
ALTER TABLE test_change CHANGE c c INT COMMENT '';
ALTER TABLE test_change CHANGE a TYPE INT COMMENT 'this is column a';
ALTER TABLE test_change CHANGE b TYPE STRING COMMENT '#*02?`';
ALTER TABLE test_change CHANGE c TYPE INT COMMENT '';
DESC test_change;

-- Don't change anything.
ALTER TABLE test_change CHANGE a a INT COMMENT 'this is column a';
ALTER TABLE test_change CHANGE a TYPE INT COMMENT 'this is column a';
DESC test_change;

-- Change a invalid column
ALTER TABLE test_change CHANGE invalid_col invalid_col INT;
DESC test_change;

-- Change column name/dataType/position/comment together (not supported yet)
ALTER TABLE test_change CHANGE a a1 STRING COMMENT 'this is column a1' AFTER b;
DESC test_change;

-- Check the behavior with different values of CASE_SENSITIVE
SET spark.sql.caseSensitive=false;
ALTER TABLE test_change CHANGE a A INT COMMENT 'this is column A';
SET spark.sql.caseSensitive=true;
ALTER TABLE test_change CHANGE a A INT COMMENT 'this is column A1';
ALTER TABLE test_change CHANGE invalid_col TYPE INT;
DESC test_change;

-- Change column can't apply to a temporary/global_temporary view
CREATE TEMPORARY VIEW temp_view(a, b) AS SELECT 1, "one";
ALTER TABLE temp_view CHANGE a a INT COMMENT 'this is column a';
ALTER TABLE temp_view CHANGE a TYPE INT COMMENT 'this is column a';
CREATE GLOBAL TEMPORARY VIEW global_temp_view(a, b) AS SELECT 1, "one";
ALTER TABLE global_temp.global_temp_view CHANGE a a INT COMMENT 'this is column a';

-- Change column in partition spec (not supported yet)
CREATE TABLE partition_table(a INT, b STRING, c INT, d STRING) USING parquet PARTITIONED BY (c, d);
ALTER TABLE partition_table PARTITION (c = 1) CHANGE COLUMN a new_a INT;
ALTER TABLE partition_table CHANGE COLUMN c c INT COMMENT 'this is column C';
ALTER TABLE global_temp.global_temp_view CHANGE a TYPE INT COMMENT 'this is column a';

-- DROP TEST TABLE
DROP TABLE test_change;
DROP TABLE partition_table;
DROP VIEW global_temp.global_temp_view;
Loading