Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[#1530] feat(spark-connector): support specifying scala version to build and test Gravitino #1582

Merged
merged 4 commits into from
Jan 22, 2024
Merged

[#1530] feat(spark-connector): support specifying scala version to build and test Gravitino #1582

merged 4 commits into from
Jan 22, 2024

Conversation

FANNG1
Copy link
Contributor

@FANNG1 FANNG1 commented Jan 17, 2024 •
edited
Loading

What changes were proposed in this pull request?

support specify scala versions, by two methods:

  1. gradle build -PscalaVersion=xx
  2. set defaultScalaVersion in gradle.properties

method2 will be overwrite by method1 if both existing,
and supports scala 2.12 for now.

Why are the changes needed?

spark hive connector just supports scala 2.12

Fix: #1530

Does this PR introduce any user-facing change?

no

How was this patch tested?

existing UTs

@qqqttt123
Copy link
Contributor

Could you provide different profiles for different version scala referring to the solution of Spark?

@FANNG1
Copy link
Contributor Author

FANNG1 commented Jan 18, 2024

Could you provide different profiles for different version scala referring to the solution of Spark?

I planed to do it latter on #1574 after spark-connector is almost ready, because this will introduce complex, maybe we should avoid it in the initial stage.

@FANNG1
Copy link
Contributor Author

FANNG1 commented Jan 18, 2024

@jerryshao , please help to review

qqqttt123
qqqttt123 previously approved these changes Jan 18, 2024
View reviewed changes
Copy link
Contributor

@qqqttt123 qqqttt123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@qqqttt123 qqqttt123 dismissed their stale review January 18, 2024 09:55

Sorry for cancel my approval.

@qqqttt123
Copy link
Contributor

Another issue is to ensure compatible issue. It seems that there is no compatible issue.

@FANNG1
Copy link
Contributor Author

FANNG1 commented Jan 18, 2024

sorry there is a mistake, libs.versions.toml doesn't support placeholders, the changed scala just infect integrate-test which failed in the CI and I just tested gradle build locally which not build integrate tests. so I use 2.12 to specify the version in the latest pr @qqqttt123

@FANNG1 FANNG1 changed the title [#1530] feat(spark-connector): downgrade scala version from 2.13 to 2.12 [#1530] feat(spark-connector): support specifing scala version to build and test Gravitino Jan 22, 2024
@FANNG1 FANNG1 changed the title [#1530] feat(spark-connector): support specifing scala version to build and test Gravitino [#1530] feat(spark-connector): support specifying scala version to build and test Gravitino Jan 22, 2024
@FANNG1
Copy link
Contributor Author

FANNG1 commented Jan 22, 2024

support specify scala versions, by two methods:

  1. gradle build -PscalaVersion=xx
  2. set defaultScalaVersion in gradle.properties

@qqqttt123 please help to review again

qqqttt123
qqqttt123 approved these changes Jan 22, 2024
View reviewed changes
Copy link
Contributor

@qqqttt123 qqqttt123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @FANNG1

@FANNG1 FANNG1 merged commit 27ce25b into apache:branch-spark-connector-dev Jan 22, 2024
12 checks passed
@FANNG1
Copy link
Contributor Author

FANNG1 commented Jan 22, 2024

thanks @qqqttt123 for your advise

@FANNG1 FANNG1 mentioned this pull request Feb 2, 2024
Closed
FANNG1 added a commit to FANNG1/gravitino that referenced this pull request Feb 6, 2024
@FANNG1
[apache#1530] feat(spark-connector): support specifying scala version…
62b96c0 
… to build and test Gravitino (apache#1582)

### What changes were proposed in this pull request?
support specify scala versions, by two methods:
1. `gradle build -PscalaVersion=xx`
2. set `defaultScalaVersion` in gradle.properties

method2 will be overwrite  by method1 if both existing, 
and supports scala 2.12 for now.

### Why are the changes needed?
spark hive connector just supports scala 2.12

Fix: apache#1530 

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
existing UTs
FANNG1 added a commit to FANNG1/gravitino that referenced this pull request Feb 7, 2024
@FANNG1
[apache#1530] feat(spark-connector): support specifying scala version…
68a1318 
… to build and test Gravitino (apache#1582)

### What changes were proposed in this pull request?
support specify scala versions, by two methods:
1. `gradle build -PscalaVersion=xx`
2. set `defaultScalaVersion` in gradle.properties

method2 will be overwrite  by method1 if both existing, 
and supports scala 2.12 for now.

### Why are the changes needed?
spark hive connector just supports scala 2.12

Fix: apache#1530 

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
existing UTs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@qqqttt123 qqqttt123 qqqttt123 approved these changes

@jerryshao jerryshao Awaiting requested review from jerryshao

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@FANNG1 @qqqttt123