Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase nightly ci ephemeral-storage #7498

Conversation

LoveEachDay
Copy link
Contributor

Originally the ephemeral-storage for nightly-ci krte pod is 50Gi. The pod is get evicted because of storage usage exceeded the quota.
image

Sep 06 11:45:43 test-worker6 kubelet[1877]: I0906 11:45:43.963190    1877 eviction_manager.go:578] eviction manager: pod milvus-e2e-test-kind-nightly-546tr-cmn0v_kube-ops(1d81845d-0eaf-11ec-ba94-1c1b0d568bc0) is evicted successfully
Sep 06 11:45:43 test-worker6 kubelet[1877]: I0906 11:45:43.963213    1877 eviction_manager.go:191] eviction manager: pods milvus-e2e-test-kind-nightly-546tr-cmn0v_kube-ops(1d81845d-0eaf-11ec-ba94-1c1b0d568bc0) evicted, waiting for pod to be cleaned up

Increase the ephemeral-storage to resolve the pod the eviction.
Resolves: #7494

Signed-off-by: Edward Zeng jie.zeng@zilliz.com

/cc @zwd1208
/cc @yanliang567

Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
@sre-ci-robot sre-ci-robot added the size/XS Denotes a PR that changes 0-9 lines. label Sep 6, 2021
@zwd1208
Copy link

zwd1208 commented Sep 6, 2021

/lgtm

@yanliang567
Copy link
Contributor

/approve

@sre-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: LoveEachDay, yanliang567

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@yanliang567 yanliang567 added area/test ci-passed kind/improvement Changes related to something improve, likes ut and code refactor labels Sep 6, 2021
@yanliang567 yanliang567 added this to the 2.0-Backlog milestone Sep 6, 2021
@sre-ci-robot sre-ci-robot merged commit 90e3072 into milvus-io:master Sep 6, 2021
@LoveEachDay LoveEachDay deleted the increase-nightly-ci-ephemeral-storage branch September 7, 2021 05:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved area/test ci-passed kind/improvement Changes related to something improve, likes ut and code refactor lgtm size/XS Denotes a PR that changes 0-9 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Nightly ci Pod crashed due to ephemeral storage limit
4 participants