Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TASK][EASY] Make flaky test spnego batch rest client reliable #5617

Closed
2 of 3 tasks
yaooqinn opened this issue Nov 3, 2023 · 0 comments
Closed
2 of 3 tasks

[TASK][EASY] Make flaky test spnego batch rest client reliable #5617

yaooqinn opened this issue Nov 3, 2023 · 0 comments
Assignees
@cxzl25
Labels
hacktoberfest

Comments

@yaooqinn
Copy link
Member

yaooqinn commented Nov 3, 2023

What's the level of this task?

EASY

Code of Conduct

Search before creating

  • I have searched in the task list and found no similar tasks.

Mentor

  • I have sufficient expertise on this task, and I volunteer to be a mentor of this task to guide contributors through the task.

Skill requirements

  • scala

Background and Goals

BatchCliSuite:

  • basic batch rest client
  • spnego batch rest client *** FAILED ***
    "{"success":false,"msg":"APPLICATION_NOT_FOUND"}
    Error: Failed to delete batch 97c52484-1719-44c7-b0cc-320e8dad32c4, its current state is RUNNING" did not contain ""success":true" (TestPrematureExit.scala:72)

Implementation steps

Additional log can be found at https://github.com/apache/kyuubi/actions/runs/6742150635/job/18327786655?pr=5596

Additional context

Introduction of 2023 Kyuubi Code Contribution Program
@cxzl25 cxzl25 mentioned this issue Nov 17, 2023
Closed
18 tasks
@cxzl25 cxzl25 closed this as completed in 71b0376 Nov 20, 2023
cxzl25 added a commit that referenced this issue Nov 20, 2023
@cxzl25
[KYUUBI #5617][TEST] Make flaky test spnego batch rest client reliable
e70bfc5 
# 🔍 Description
## Issue References 🔗

This pull request fixes #5617

## Describe Your Solution 🔧

`batch_long_time` job sleep is changed from 10 seconds to 120 seconds to avoid spark executing and exiting when deleting the batch.
When pulling the spark running log, make sure the spark driver is already running.

## Types of changes 🔖

- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

#### Behavior Without This Pull Request ⚰️
```
- basic batch rest client *** FAILED ***
  "{"success":false,"msg":"APPLICATION_NOT_FOUND"}
  Warning: Batch 1f3af615-721b-45dd-9f0f-cb976bfef988 is already in terminal state ERROR." did not contain ""success":true" (TestPrematureExit.scala:72)
- spnego batch rest client *** FAILED ***
  "{"success":false,"msg":"APPLICATION_NOT_FOUND"}
  Warning: Batch 30b1e3d6-9996-4409-8b28-eead5258fc3b is already in terminal state ERROR." did not contain ""success":true" (TestPrematureExit.scala:72)
```

```
- spnego batch rest client *** FAILED ***
  "{"success":false,"msg":"APPLICATION_NOT_FOUND"}
  Error: Failed to delete batch f20c041c-3246-4fca-9814-c4892cd83220, its current state is RUNNING" did not contain ""success":true" (TestPrematureExit.scala:72)
```

#### Behavior With This Pull Request 🎉
GA pass

#### Related Unit Tests
BatchCliSuite.scala  "basic batch rest client" "spnego batch rest client"

---

# Checklists
## 📝 Author Self Checklist

- [x] My code follows the [style guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html) of this project
- [x] I have performed a self-review
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

## 📝 Committer Pre-Merge Checklist

- [x] Pull request title is okay.
- [x] No license issues.
- [x] Milestone correctly set?
- [x] Test coverage is ok
- [x] Assignees are selected.
- [x] Minimum number of approvals
- [x] No changes are requested

**Be nice. Be informative.**

Closes #5722 from cxzl25/KYUUBI-5617.

Closes #5617

ba2ca7b [sychen] invalidCount
fb0586c [sychen] retry
2d07457 [sychen] sleep 1 min

Authored-by: sychen <sychen@ctrip.com>
Signed-off-by: Shaoyun Chen <csy@apache.org>
(cherry picked from commit 71b0376)
pan3793 added a commit that referenced this issue Feb 22, 2024
@pan3793
[KYUUBI #6074] Add a script to simplify the process of creating relea…
2c70c67 
…se notes

# 🔍 Description
## Issue References 🔗

Currently, we use a rather primitive way to manually write release notes from scratch, and some of the mechanical and repetitive work can be simplified by the scripts.

## Describe Your Solution 🔧

Adds a script to simplify the process of creating release notes.

Note: it just simplifies some processes, the release manager still needs to tune the outputs by hand.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

```
RELEASE_TAG=v1.8.1 PREVIOUS_RELEASE_TAG=v1.8.0 build/release/pre_gen_release_notes.py
```

```
$ head build/release/commits-v1.8.1.txt
[KYUUBI #5981] Deploy Spark Hive connector with Scala 2.13 to Maven Central
[KYUUBI #6058] Make Jetty server stop timeout configurable
[KYUUBI #5952][1.8] Disconnect connections without running operations after engine maxlife time graceful period
[KYUUBI #6048] Assign serviceNode and add volatile for variables
[KYUUBI #5991] Error on reading Atlas properties composed of multi values
[KYUUBI #6045] [REST] Sync the AdminRestApi with the AdminResource Apis
[KYUUBI #6047] [CI] Free up disk space
[KYUUBI #6036] JDBC driver conditional sets fetchSize on opening session
[KYUUBI #6028] Exited spark-submit process should not block batch submit queue
[KYUUBI #6018] Speed up GetTables operation for Spark session catalog
```

```
$ head build/release/contributors-v1.8.1.txt
* Shaoyun Chen        -- [KYUUBI #5857][KYUUBI #5720][KYUUBI #5785][KYUUBI #5617]
* Chao Chen           -- [KYUUBI #5750]
* Flyangz             -- [KYUUBI #5832]
* Pengqi Li           -- [KYUUBI #5713]
* Bowen Liang         -- [KYUUBI #5730][KYUUBI #5802][KYUUBI #5767][KYUUBI #5831][KYUUBI #5801][KYUUBI #5754][KYUUBI #5626][KYUUBI #5811][KYUUBI #5853][KYUUBI #5765]
* Paul Lin            -- [KYUUBI #5799][KYUUBI #5814]
* Senmiao Liu         -- [KYUUBI #5969][KYUUBI #5244]
* Xiao Liu            -- [KYUUBI #5962]
* Peiyue Liu          -- [KYUUBI #5331]
* Junjie Ma           -- [KYUUBI #5789]
```
---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6074 from pan3793/release-script.

Closes #6074

3d5ec20 [Cheng Pan] credits
1765279 [Cheng Pan] Add a script to simplify the process of creating release notes

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
zhaohehuhu pushed a commit to zhaohehuhu/incubator-kyuubi that referenced this issue Mar 21, 2024
@pan3793 @zhaohehuhu
[KYUUBI apache#6074] Add a script to simplify the process of creating…
9f3ab85 
… release notes

# 🔍 Description
## Issue References 🔗

Currently, we use a rather primitive way to manually write release notes from scratch, and some of the mechanical and repetitive work can be simplified by the scripts.

## Describe Your Solution 🔧

Adds a script to simplify the process of creating release notes.

Note: it just simplifies some processes, the release manager still needs to tune the outputs by hand.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

```
RELEASE_TAG=v1.8.1 PREVIOUS_RELEASE_TAG=v1.8.0 build/release/pre_gen_release_notes.py
```

```
$ head build/release/commits-v1.8.1.txt
[KYUUBI apache#5981] Deploy Spark Hive connector with Scala 2.13 to Maven Central
[KYUUBI apache#6058] Make Jetty server stop timeout configurable
[KYUUBI apache#5952][1.8] Disconnect connections without running operations after engine maxlife time graceful period
[KYUUBI apache#6048] Assign serviceNode and add volatile for variables
[KYUUBI apache#5991] Error on reading Atlas properties composed of multi values
[KYUUBI apache#6045] [REST] Sync the AdminRestApi with the AdminResource Apis
[KYUUBI apache#6047] [CI] Free up disk space
[KYUUBI apache#6036] JDBC driver conditional sets fetchSize on opening session
[KYUUBI apache#6028] Exited spark-submit process should not block batch submit queue
[KYUUBI apache#6018] Speed up GetTables operation for Spark session catalog
```

```
$ head build/release/contributors-v1.8.1.txt
* Shaoyun Chen        -- [KYUUBI apache#5857][KYUUBI apache#5720][KYUUBI apache#5785][KYUUBI apache#5617]
* Chao Chen           -- [KYUUBI apache#5750]
* Flyangz             -- [KYUUBI apache#5832]
* Pengqi Li           -- [KYUUBI apache#5713]
* Bowen Liang         -- [KYUUBI apache#5730][KYUUBI apache#5802][KYUUBI apache#5767][KYUUBI apache#5831][KYUUBI apache#5801][KYUUBI apache#5754][KYUUBI apache#5626][KYUUBI apache#5811][KYUUBI apache#5853][KYUUBI apache#5765]
* Paul Lin            -- [KYUUBI apache#5799][KYUUBI apache#5814]
* Senmiao Liu         -- [KYUUBI apache#5969][KYUUBI apache#5244]
* Xiao Liu            -- [KYUUBI apache#5962]
* Peiyue Liu          -- [KYUUBI apache#5331]
* Junjie Ma           -- [KYUUBI apache#5789]
```
---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes apache#6074 from pan3793/release-script.

Closes apache#6074

3d5ec20 [Cheng Pan] credits
1765279 [Cheng Pan] Add a script to simplify the process of creating release notes

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cxzl25 cxzl25

Labels
hacktoberfest
Projects
No open projects
2023 Kyuubi Code Contribution Program
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[KYUUBI #5617][TEST] Make flaky test spnego batch rest client reliable cxzl25/kyuubi
2 participants
@cxzl25 @yaooqinn