Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: add integration test cases for minikube image build #15163

Merged
merged 1 commit into from
Jan 17, 2023
Merged

test: add integration test cases for minikube image build #15163

merged 1 commit into from
Jan 17, 2023

Conversation

ComradeProgrammer
Copy link
Member

@ComradeProgrammer ComradeProgrammer commented Oct 18, 2022 •
edited
Loading

test: add 5 integration test cases for "minikube image build"
part of #12436
test 1: normal test case for minikube image build, with -t parameter
test 2: normal test case for minikube image build, with -t and -f parameter
test 3: build with --build-opt, test case for bug in #12384
test 4: build with --build-env, for bug in #12431
test 5: build with .dockerignore

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Oct 18, 2022
@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Oct 18, 2022
@k8s-ci-robot
Copy link
Contributor

Hi @ComradeProgrammer. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 18, 2022
@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

spowelljr
spowelljr requested changes Oct 20, 2022
View reviewed changes
test/integration/image_test.go Outdated Show resolved Hide resolved
test/integration/image_test.go Show resolved Hide resolved
test/integration/image_test.go Outdated Show resolved Hide resolved
test/integration/image_test.go Outdated Show resolved Hide resolved
test/integration/image_test.go Show resolved Hide resolved
test/integration/image_test.go Outdated Show resolved Hide resolved
test/integration/testdata/image-build/test-arg/Dockerfile Outdated Show resolved Hide resolved
test/integration/image_test.go Outdated Show resolved Hide resolved
test/integration/image_test.go Outdated Show resolved Hide resolved
test/integration/image_test.go Outdated Show resolved Hide resolved
@ComradeProgrammer
Copy link
Member Author

Thank you for the suggestions given in the review. All suggestions have been taken, and revisions have been made accordingly.

@spowelljr
Copy link
Member

Hi @ComradeProgrammer, can you take a look at the lint errors. It looks like you're using the append func but you're only passing it one param, so you can remove the append part and just set the variable to the array.

@ComradeProgrammer
Copy link
Member Author

Hi @ComradeProgrammer, can you take a look at the lint errors. It looks like you're using the append func but you're only passing it one param, so you can remove the append part and just set the variable to the array.

Thanks a lot, lint errors and comments has been fixed

spowelljr
spowelljr requested changes Oct 26, 2022
View reviewed changes
Copy link
Member

@spowelljr spowelljr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, could you just run make generate-docs and push up the documentation changes, thanks

@ComradeProgrammer
Copy link
Member Author

ComradeProgrammer commented Oct 30, 2022 •
edited
Loading

Looks good, could you just run make generate-docs and push up the documentation changes, thanks

docs has been added

@spowelljr
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 31, 2022
@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15163) |
+----------------+----------+---------------------+
| minikube start | 54.9s    | 54.9s               |
| enable ingress | 27.1s    | 27.6s               |
+----------------+----------+---------------------+

Times for minikube ingress: 28.2s 24.7s 24.7s 28.7s 29.2s
Times for minikube (PR 15163) ingress: 25.7s 29.1s 28.7s 29.2s 25.2s

Times for minikube start: 55.6s 55.6s 55.1s 54.6s 53.3s
Times for minikube (PR 15163) start: 56.8s 53.9s 54.0s 56.3s 53.5s

docker driver with docker runtime
error collecting results for docker driver: timing run 0 with minikube: timing cmd: [out/minikube addons enable ingress]: waiting for minikube: exit status 10
docker driver with containerd runtime
error downloading artifacts: artifact download start: exit status 90

@ComradeProgrammer
Copy link
Member Author

/ok-to-test

I saw there are so many errors in the test, are these failures caused by my pr?

@ComradeProgrammer
Copy link
Member Author

Is there anything that I should do next? @spowelljr

spowelljr
spowelljr requested changes Nov 16, 2022
View reviewed changes
test/integration/image_test.go Outdated Show resolved Hide resolved
test/integration/image_test.go Outdated Show resolved Hide resolved
test/integration/testdata/image-build/test-normal/Dockerfile Show resolved Hide resolved
@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15163) |
+----------------+----------+---------------------+
| minikube start | 55.4s    | 54.5s               |
| enable ingress | 27.7s    | 28.6s               |
+----------------+----------+---------------------+

Times for minikube (PR 15163) start: 53.5s 55.8s 53.9s 55.3s 54.2s
Times for minikube start: 54.0s 54.9s 55.8s 57.0s 55.1s

Times for minikube ingress: 27.8s 28.8s 28.2s 28.3s 25.2s
Times for minikube (PR 15163) ingress: 28.8s 25.7s 29.3s 29.3s 29.7s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15163) |
+----------------+----------+---------------------+
| minikube start | 27.3s    | 27.7s               |
| enable ingress | 32.8s    | 20.4s               |
+----------------+----------+---------------------+

Times for minikube ingress: 81.0s 20.5s 21.5s 22.0s 19.0s
Times for minikube (PR 15163) ingress: 21.0s 19.5s 21.0s 19.5s 21.0s

Times for minikube start: 25.4s 25.7s 29.7s 27.2s 28.5s
Times for minikube (PR 15163) start: 25.1s 25.9s 27.2s 28.7s 31.4s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15163) |
+----------------+----------+---------------------+
| minikube start | 23.8s    | 23.7s               |
| enable ingress | 26.3s    | 26.1s               |
+----------------+----------+---------------------+

Times for minikube start: 25.4s 26.0s 22.7s 22.4s 22.4s
Times for minikube (PR 15163) start: 22.6s 23.0s 23.1s 25.4s 24.5s

Times for minikube ingress: 26.5s 26.0s 26.0s 26.0s 27.0s
Times for minikube (PR 15163) ingress: 26.0s 26.0s 26.0s 26.5s 26.0s

@minikube-pr-bot
Copy link

These are the flake rates of all failed tests.

Environment Failed Tests Flake Rate (%)
Docker_Linux_containerd TestImageBuild/serial/BuildWithBuildArg (gopogh) n/a
KVM_Linux_containerd TestImageBuild/serial/BuildWithBuildArg (gopogh) n/a
Hyper-V_Windows TestNetworkPlugins/group/flannel/NetCatPod (gopogh) 0.83 (chart)
Docker_Linux_containerd TestStartStop/group/no-preload/serial/DeployApp (gopogh) 11.01 (chart)
Docker_Linux_containerd TestStartStop/group/no-preload/serial/FirstStart (gopogh) 11.01 (chart)
Docker_Linux_containerd TestStartStop/group/no-preload/serial/UserAppExistsAfterStop (gopogh) 11.01 (chart)
Docker_Linux_containerd TestStartStop/group/no-preload/serial/SecondStart (gopogh) 13.39 (chart)
Hyper-V_Windows TestNetworkPlugins/group/bridge/Start (gopogh) 20.57 (chart)
Docker_macOS TestMultiNode/serial/RestartMultiNode (gopogh) 26.17 (chart)
Hyper-V_Windows TestNetworkPlugins/group/kubenet/Start (gopogh) 27.66 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/FirstStart (gopogh) 43.12 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/UserAppExistsAfterStop (gopogh) 43.12 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/DeployApp (gopogh) 43.64 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/SecondStart (gopogh) 44.14 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/SecondStart (gopogh) 50.46 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/UserAppExistsAfterStop (gopogh) 50.46 (chart)
Docker_macOS TestMultiNode/serial/RestartKeepsNodes (gopogh) 51.01 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/DeployApp (gopogh) 51.35 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/FirstStart (gopogh) 51.79 (chart)
Docker_Linux_containerd TestNetworkPlugins/group/enable-default-cni/DNS (gopogh) 76.52 (chart)
Docker_Linux_containerd TestNetworkPlugins/group/bridge/DNS (gopogh) 81.30 (chart)
Hyper-V_Windows TestNoKubernetes/serial/StartWithStopK8s (gopogh) 96.74 (chart)
Docker_Linux_containerd TestNetworkPlugins/group/calico/Start (gopogh) 99.08 (chart)
Docker_Linux_containerd TestKubernetesUpgrade (gopogh) 100.00 (chart)
Docker_Linux_containerd TestPreload (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressAddonActivation (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressAddons (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressDNSAddonActivation (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/StartLegacyK8sCluster (gopogh) 100.00 (chart)
Docker_macOS TestKubernetesUpgrade (gopogh) 100.00 (chart)
More tests... Continued...

Too many tests failed - See test logs for more details.

To see the flake rates of all tests by environment, click here.

@ComradeProgrammer
Copy link
Member Author

please trigger the test again. Just limited the test on docker engines(not containered). Thanks. @spowelljr

@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15163) |
+----------------+----------+---------------------+
| minikube start | 54.6s    | 54.5s               |
| enable ingress | 27.8s    | 25.8s               |
+----------------+----------+---------------------+

Times for minikube start: 55.0s 54.6s 54.6s 54.6s 54.0s
Times for minikube (PR 15163) start: 54.7s 53.7s 54.0s 56.8s 53.3s

Times for minikube ingress: 24.2s 29.6s 28.2s 28.7s 28.1s
Times for minikube (PR 15163) ingress: 25.1s 25.2s 24.2s 26.7s 27.7s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15163) |
+----------------+----------+---------------------+
| minikube start | 25.6s    | 25.8s               |
| enable ingress | 26.9s    | 26.7s               |
+----------------+----------+---------------------+

Times for minikube start: 27.2s 24.8s 25.4s 25.8s 24.6s
Times for minikube (PR 15163) start: 25.4s 28.4s 24.4s 26.2s 24.8s

Times for minikube ingress: 50.0s 21.9s 21.0s 19.5s 21.9s
Times for minikube (PR 15163) ingress: 20.9s 20.9s 50.9s 19.4s 21.4s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15163) |
+----------------+----------+---------------------+
| minikube start | 24.8s    | 21.9s               |
| enable ingress | 26.5s    | 26.4s               |
+----------------+----------+---------------------+

Times for minikube (PR 15163) start: 21.1s 22.1s 21.6s 22.5s 22.2s
Times for minikube start: 24.8s 32.8s 22.3s 21.6s 22.2s

Times for minikube (PR 15163) ingress: 26.4s 26.4s 26.4s 26.4s 26.4s
Times for minikube ingress: 26.4s 27.4s 26.4s 26.5s 25.9s

@minikube-pr-bot
Copy link

These are the flake rates of all failed tests.

Environment Failed Tests Flake Rate (%)
Docker_Windows TestErrorSpam/setup (gopogh) 0.00 (chart)
Docker_Windows TestKicCustomNetwork/create_custom_network (gopogh) 0.00 (chart)
Docker_Windows TestKicCustomSubnet (gopogh) 0.00 (chart)
Docker_Windows TestKicExistingNetwork (gopogh) 0.00 (chart)
Docker_Windows TestMultiNode/serial/StartAfterStop (gopogh) 4.90 (chart)
Hyperkit_macOS TestPause/serial/SecondStartNoReconfiguration (gopogh) 23.42 (chart)
Docker_Windows TestPause/serial/SecondStartNoReconfiguration (gopogh) 24.48 (chart)
Docker_macOS TestMultiNode/serial/RestartMultiNode (gopogh) 27.22 (chart)
Hyper-V_Windows TestNoKubernetes/serial/StartWithK8s (gopogh) 36.67 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/FirstStart (gopogh) 41.03 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/UserAppExistsAfterStop (gopogh) 41.03 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/DeployApp (gopogh) 41.53 (chart)
Docker_Linux_containerd TestStartStop/group/old-k8s-version/serial/SecondStart (gopogh) 42.02 (chart)
Docker_Windows TestNetworkPlugins/group/kubenet/DNS (gopogh) 42.66 (chart)
Docker_Linux_containerd TestStartStop/group/embed-certs/serial/UserAppExistsAfterStop (gopogh) 44.44 (chart)
Docker_Linux_containerd TestStartStop/group/embed-certs/serial/FirstStart (gopogh) 44.92 (chart)
Docker_Linux_containerd TestStartStop/group/embed-certs/serial/DeployApp (gopogh) 45.83 (chart)
Docker_Linux_containerd TestStartStop/group/embed-certs/serial/SecondStart (gopogh) 45.83 (chart)
KVM_Linux TestMultiNode/serial/RestartMultiNode (gopogh) 46.91 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/SecondStart (gopogh) 47.86 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/UserAppExistsAfterStop (gopogh) 47.86 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/DeployApp (gopogh) 48.74 (chart)
Docker_Linux_containerd TestStartStop/group/default-k8s-diff-port/serial/FirstStart (gopogh) 49.17 (chart)
Docker_Windows TestNetworkPlugins/group/enable-default-cni/DNS (gopogh) 49.30 (chart)
Docker_Linux TestNetworkPlugins/group/calico/Start (gopogh) 80.26 (chart)
Docker_Linux TestNetworkPlugins/group/false/DNS (gopogh) 83.55 (chart)
Docker_Linux TestNetworkPlugins/group/bridge/DNS (gopogh) 84.87 (chart)
Docker_Linux TestNetworkPlugins/group/enable-default-cni/DNS (gopogh) 84.87 (chart)
Docker_Linux TestNetworkPlugins/group/kubenet/DNS (gopogh) 87.50 (chart)
Docker_Linux_containerd TestNetworkPlugins/group/calico/Start (gopogh) 98.29 (chart)
More tests... Continued...

Too many tests failed - See test logs for more details.

To see the flake rates of all tests by environment, click here.

@ComradeProgrammer
Copy link
Member Author

ComradeProgrammer commented Nov 29, 2022 •
edited
Loading

Well I think it seems that everything is fine now, is there any other suggestion?

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 19, 2022
@ComradeProgrammer
Copy link
Member Author

Anything I need to do next?
@medyagh @spowelljr @afbjorklund

@spowelljr spowelljr merged commit 0d0a51d into kubernetes:master Jan 17, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ComradeProgrammer, medyagh, spowelljr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@spowelljr spowelljr spowelljr approved these changes

@medyagh medyagh medyagh approved these changes

@prezha prezha Awaiting requested review from prezha

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@ComradeProgrammer @k8s-ci-robot @minikube-bot @spowelljr @minikube-pr-bot @medyagh