Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Subtask] integrate pytest with gradle to run python test with CICD #2518

Closed
Tracked by #2113
shaofengshi opened this issue Mar 13, 2024 · 4 comments · Fixed by #2684 or #2753
Closed
Tracked by #2113

[Subtask] integrate pytest with gradle to run python test with CICD #2518

shaofengshi opened this issue Mar 13, 2024 · 4 comments · Fixed by #2684 or #2753
Assignees
@xunliu @coolderli
Labels
good first issue Good for newcomers subtask Subtasks of umbrella issue
Milestone
Gravitino 0.5.0

Comments

@shaofengshi
Copy link
Contributor

shaofengshi commented Mar 13, 2024 •
edited
Loading

Describe the subtask

As discussed in #2280, integrate pytest with gradle, so that we can use gradle to build/test the python client module

This might be a good reference: https://attx-project.github.io/Building-with-Gradle.html

Parent issue

#2229
@shaofengshi shaofengshi added the subtask Subtasks of umbrella issue label Mar 13, 2024
@shaofengshi shaofengshi mentioned this issue Mar 13, 2024
Merged
3 tasks
@shaofengshi shaofengshi added the good first issue Good for newcomers label Mar 14, 2024
@coolderli
Copy link
Collaborator

coolderli commented Mar 18, 2024 •
edited
Loading

@shaofengshi Hi, Why do we use gradle to integrate pytest. We can use the Github CI to finish it. If we develop the python module, we can use pytest to have a test locally.

@coolderli coolderli mentioned this issue Mar 18, 2024
Closed
@shaofengshi
Copy link
Contributor Author

shaofengshi commented Mar 19, 2024 •
edited
Loading

@shaofengshi Hi, Why do we use gradle to integrate pytest. We can use the Github CI to finish it. If we develop the python module, we can use pytest to have a test locally.

@coolderli , with gradle integration, user can run the build in local, without dependency on Github. Is that reasonable? Just let me know if that will bring potential issue.

@coolderli
Copy link
Collaborator

@shaofengshi Hi, Why do we use gradle to integrate pytest. We can use the Github CI to finish it. If we develop the python module, we can use pytest to have a test locally.

@coolderli , with gradle integration, user can run the build in local, without dependency on Github. Is that reasonable? Just let me know if that will bring potential issue.

@shaofengshi Users can use pytest to have a test in local env. Some Python users may not be familiar with gradle. I'd like to use some Python tools to build and test the Python project.

@shaofengshi
Copy link
Contributor Author

@shaofengshi Hi, Why do we use gradle to integrate pytest. We can use the Github CI to finish it. If we develop the python module, we can use pytest to have a test locally.

@coolderli , with gradle integration, user can run the build in local, without dependency on Github. Is that reasonable? Just let me know if that will bring potential issue.

@shaofengshi Users can use pytest to have a test in local env. Some Python users may not be familiar with gradle. I'd like to use some Python tools to build and test the Python project.

Hi Peidian, I believe python user still use pytest to run the test. What Jerry suggests is, when run a "gradle build" command, it will call the "pytest" command in the client-python module. Not sure whether this solves your concern. @coolderli

@coolderli coolderli mentioned this issue Mar 26, 2024
Merged
jerryshao pushed a commit that referenced this issue Mar 27, 2024
@coolderli
[#2518] feat(build): support Github CI for client-python (#2684)
004fe89 
### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: #2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
shaofengshi added a commit that referenced this issue Mar 28, 2024
@shaofengshi
Revert "[#2518] feat(build): support Github CI for client-python (#2684…
df25534 
…)"

This reverts commit 004fe89.
yuqi1129 pushed a commit that referenced this issue Mar 28, 2024
@shaofengshi
Revert "[#2518] feat(build): support Github CI for client-python" (#2724
40eef72 
)

Reverts #2684 , as it blocks development.
@shaofengshi shaofengshi reopened this Mar 29, 2024
xunliu pushed a commit to xunliu/gravitino that referenced this issue Mar 30, 2024
@coolderli @xunliu
[apache#2518] feat(build): support Github CI for client-python (apach…
b0677c4 
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
@jerryshao jerryshao added this to the Gravitino 0.5.0 milestone Apr 1, 2024
xunliu pushed a commit to xunliu/gravitino that referenced this issue Apr 1, 2024
@coolderli @xunliu
[apache#2518] feat(build): support Github CI for client-python (apach…
6ddefb3 
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
This was referenced Apr 1, 2024
Merged
Open
xunliu pushed a commit to xunliu/gravitino that referenced this issue Apr 2, 2024
@coolderli @xunliu
[apache#2518] feat(build): support Github CI for client-python (apach…
d5534de 
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
xunliu pushed a commit to xunliu/gravitino that referenced this issue Apr 3, 2024
@coolderli @xunliu
[apache#2518] feat(build): support Github CI for client-python (apach…
e5e2139 
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
xunliu pushed a commit to xunliu/gravitino that referenced this issue Apr 3, 2024
@coolderli @xunliu
[apache#2518] feat(build): support Github CI for client-python (apach…
932eb65 
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
xunliu pushed a commit to xunliu/gravitino that referenced this issue Apr 3, 2024
@coolderli @xunliu
[apache#2518] feat(build): support Github CI for client-python (apach…
012831a 
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
xunliu added a commit that referenced this issue Apr 3, 2024
@xunliu @coolderli
[#2518] feat(PyClient): Integrate pytest with Gradle to run python te…
f7f12d8 
…st with CICD (#2753)

### What changes were proposed in this pull request?

1. Use Gradle python plugin to manage python module
2. Automatic download conda and create python env in
`gravitino/.gradle/python` directory
3. Use multiple python version `3.8`, `3.9`, `3.10`,and `3.11` test ITs
in Github Action
4. Support `gradlew clean` & `gradlew build` & `gradlew
:client:client-python:test -PpythonVersion=3.8`

### Why are the changes needed?

Use Gradle manager and development Python client module.

Fix: #2518 

### Does this PR introduce _any_ user-facing change?

1. Use `pythonVersion` Gradle parameter setting Python version to
compile and test Python client, `gradlew :client:client-python:test
-PpythonVersion=3.8`
2. Use `skipPythonITs` Gradle parameter to control if skip Python test
cases.

### How was this patch tested?

CI Passed.

---------

Co-authored-by: Peidian li <38486782+coolderli@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@xunliu xunliu

@coolderli coolderli

Labels
good first issue Good for newcomers subtask Subtasks of umbrella issue
Projects
None yet
Milestone
Gravitino 0.5.0
4 participants
@jerryshao @xunliu @shaofengshi @coolderli