Skip to content

[SPARK-52475][TESTS] Remove deprecated table.schema method from tests #51173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 5 commits into from
Closed

[SPARK-52475][TESTS] Remove deprecated table.schema method from tests #51173

wants to merge 5 commits into from

Conversation

szehon-ho
Copy link
Member

What changes were proposed in this pull request?

Fix many compile warnings like

[warn] /Users/szehon.ho/repos/apache-spark/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/CatalogSuite.scala:282:18: method schema in trait Table is deprecated (since 3.4.0)
[warn] Applicable -Wconf / @nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.sql.connector.catalog.CatalogSuite, origin=org.apache.spark.sql.connector.catalog.Table.schema, version=3.4.0

Why are the changes needed?

Reduce compiler warnings

Does this PR introduce any user-facing change?

No

How was this patch tested?

Test only

Was this patch authored or co-authored using generative AI tooling?

No

@github-actions github-actions bot added the SQL label Jun 13, 2025
@LuciferYang
Copy link
Contributor

Here are a huge number of compilation warnings related to "method schema in trait Table is deprecated". Can we fix them all at once? Or, could we resolve all of them in tests in one go? Fixing them one by one would lead to a multitude of prs.

@LuciferYang
Copy link
Contributor

And I think it's worth creating a JIRA ticket for this.

@szehon-ho
Copy link
Member Author

szehon-ho commented Jun 13, 2025 •
edited
Loading

Yea its a large number and a bit time consuming, so I was thinking to at least make progress and avoid conflict risk by reasonable size batches. Let me look at removing from all the tests

@szehon-ho szehon-ho changed the title [MINOR][SQL] Fix CatalogSuite and AlterTableTests deprecation warnings [SPARK-52475][SQL] Remove deprecated table.schema method from tests Jun 14, 2025
@szehon-ho
Copy link
Member Author

szehon-ho commented Jun 14, 2025 •
edited
Loading

Updated to remove from all tests and created a JIRA

@szehon-ho szehon-ho changed the title [SPARK-52475][SQL] Remove deprecated table.schema method from tests [SPARK-52475][Tests] Remove deprecated table.schema method from tests Jun 14, 2025
@szehon-ho szehon-ho force-pushed the fix_warnings_other_tests branch from c8a5ed0 to 2194ea1 Compare June 14, 2025 02:47
@LuciferYang
Copy link
Contributor

@szehon-ho Thank you for your work. However, it seems there are some compilation errors that need to be fixed first.

@LuciferYang LuciferYang changed the title [SPARK-52475][Tests] Remove deprecated table.schema method from tests [SPARK-52475][TESTS] Remove deprecated table.schema method from tests Jun 14, 2025
@LuciferYang
Copy link
Contributor

sql-others has test failures:

  • org.apache.spark.sql.execution.datasources.v2.V2SessionCatalogTableSuite

In addition,

./build/mvn scalafmt:format -Dscalafmt.skip=false -Dscalafmt.validateOnly=false -Dscalafmt.changedOnly=false -pl sql/api -pl sql/connect/common -pl sql/connect/server -pl sql/connect/client/jvm

should be executed. @szehon-ho

@szehon-ho
Copy link
Member Author

Sorry there's a lot of tests and I missed running some locally first, thanks for your help

dongjoon-hyun
dongjoon-hyun approved these changes Jun 16, 2025
View reviewed changes
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @szehon-ho and all.

@dongjoon-hyun
Copy link
Member

Merged to master for Apache Spark 4.1.0.

@LuciferYang
Copy link
Contributor

late LGTM, thanks @szehon-ho

haoyangeng-db pushed a commit to haoyangeng-db/apache-spark that referenced this pull request Jun 25, 2025
@szehon-ho @haoyangeng-db
[SPARK-52475][TESTS] Remove deprecated table.schema method from tests
12652f3 
### What changes were proposed in this pull request?
Fix many compile warnings like
```
[warn] /Users/szehon.ho/repos/apache-spark/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/CatalogSuite.scala:282:18: method schema in trait Table is deprecated (since 3.4.0)
[warn] Applicable -Wconf / nowarn filters for this warning: msg=<part of the message>, cat=deprecation, site=org.apache.spark.sql.connector.catalog.CatalogSuite, origin=org.apache.spark.sql.connector.catalog.Table.schema, version=3.4.0
```

### Why are the changes needed?
Reduce compiler warnings

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Test only

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#51173 from szehon-ho/fix_warnings_other_tests.

Authored-by: Szehon Ho <szehon.apache@gmail.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HyukjinKwon HyukjinKwon HyukjinKwon approved these changes

@dongjoon-hyun dongjoon-hyun dongjoon-hyun approved these changes

Assignees
No one assigned
Labels
CONNECT SQL STRUCTURED STREAMING
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@szehon-ho @LuciferYang @dongjoon-hyun @HyukjinKwon