Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

remove retry start from integeration tests #5150

Merged
merged 19 commits into from
Aug 22, 2019
Merged

remove retry start from integeration tests #5150

merged 19 commits into from
Aug 22, 2019

Conversation

medyagh
Copy link
Member

@medyagh medyagh commented Aug 20, 2019 •
edited
Loading

  • removing all retrying on minikube start in intergration tests
  • added StartWithFail helper func for integraiton tests
  • better and unified logging for all minikube start fails in integration tests with including global args and etc.
  • added more duraition metric logging
  • fixed a dumb string replace mistake (introduced by myself)
  • make download-only tests to check if download files exist
  • less retry on minikube (no more retry on minikube stop) expect to open more cans of worm.
  • new helpers that fail on default, MustStart, MustRun

hopefully improve a lot of defunct process, some TestVersionUpgrade flakes

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 20, 2019
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Aug 20, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: medyagh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 20, 2019
@medyagh
Copy link
Member Author

medyagh commented Aug 21, 2019 •
edited
Loading

Ah ! I think I opened a canned of worm !

on the KVM TestVersionUpgrade I see the second minikube start, starts in Virtualbox !!!!

16:34:25  23:34:25 | === PAUSE TestStartStop/group
16:34:25 RunWithContext: [/home/jenkins/workspace/KVM_Linux_integration/out/minikube-linux-amd64  -p=TestPersistence start --vm-driver=kvm2  --wait-timeout=13m --wait=false --v=10 --logtostderr  ]
16:34:25 === CONT  TestVersionUpgrade
16:34:25  23:34:25 | RunWithContext: [/home/jenkins/workspace/KVM_Linux_integration/testdata/minikube-linux-amd64-latest-stable  -p=TestVersionUpgrade start mkRelease.StartArgs --v=10 --logtostderr  --kubernetes-version=v1.10.13]

it seems like a minikube runner bug !!!

   version_upgrade_test.go:88: TestVersionUpgrade minikube start failed : exit status 70
16:34:41         stdout: * [TestVersionUpgrade] minikube v1.3.1 on Debian 9.9
16:34:41         * Creating virtualbox VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
16:34:41         Downloading /home/jenkins/minikube-integration/linux-amd64-kvm2-5150-8053-27bc3b23a0a3d5c48450606726c8070d4d3dbc0a/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/linux-amd64-kvm2-5150-8053-27bc3b23a0a3d5c48450606726c8070d4d3dbc0a/.minikube/cache/iso/minikube-v1.3.0.iso...
16:34:41         Creating VirtualBox VM...
16:34:41         Creating SSH key...
16:34:41         Starting the VM...
16:34:41         Check network to re-create if needed...
16:34:41         
16:34:41         stderr: I0820 23:34:25.136992   11222 notify.go:124] Checking for updates...
16:34:41         I0820 23:34:25.295032   11222 start.go:224] hostinfo: {"hostname":"kvm-integration-slave","uptime":621777,"bootTime":1565722288,"procs":218,"os":"linux","platform":"debian","platformFamily":"debian","platformVersion":"9.9","kernelVersion":"4.9.0-9-amd64","virtualizationSystem":"kvm","virtualizationRole":"host","hostid":"ae41e7f6-8b8e-4d40-b77d-1ebb5a2d5fdb"}

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 21, 2019
tstromberg
tstromberg suggested changes Aug 21, 2019
View reviewed changes
test/integration/start_stop_delete_test.go Outdated Show resolved Hide resolved
test/integration/start_stop_delete_test.go Outdated Show resolved Hide resolved
test/integration/util/minikube_runner.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 21, 2019
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 21, 2019
@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 21, 2019
@tstromberg
Copy link
Contributor 

make: *** Waiting for unfinished jobs....
# k8s.io/minikube/test/integration [k8s.io/minikube/test/integration.test]
test/integration/config_test.go:68:18: assignment mismatch: 2 variables but mk.RunCommand returns 3 values
Makefile:154: recipe for target 'out/e2e-windows-amd64' failed

@medyagh
Copy link
Member Author

medyagh commented Aug 21, 2019 •
edited
Loading

@tstromberg the TestVersionUpgrade fails consistently
but it fails on the Latest Released minikube binary (not the HEAD)

--- FAIL: TestVersionUpgrade (411.87s)
    minikube_runner.go:231: TestVersionUpgrade Failed to start minikube With error: exit status 70 
             begin Start log block ------------> 
             With Profile: TestVersionUpgrade 
             With Args: --kubernetes-version=v1.10.13 
             With Global Args: --v=10 --logtostderr   
             With Driver Args: --vm-driver=virtualbox    
             With STDOUT:

that makes me believe that our pre-HEAD minikube time out was too bad and it wont make during parallel testing.

Related: #5161

@medyagh medyagh mentioned this pull request Aug 21, 2019
Closed
@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 21, 2019
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 21, 2019
@TravisBuddy
Copy link

Travis tests have failed

Hey @medyagh,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

1st Build

View build log

make test 
which go-bindata || GO111MODULE=off GOBIN=/home/travis/gopath/bin go get github.com/jteeuwen/go-bindata/...
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.12.9.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.0/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/assets/assets.go -pkg assets deploy/addons/...
gofmt -s -w pkg/minikube/assets/assets.go
which go-bindata || GO111MODULE=off GOBIN=/home/travis/gopath/bin go get github.com/jteeuwen/go-bindata/...
/home/travis/gopath/bin/go-bindata
PATH="/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.12.9.linux.amd64/bin:/home/travis/bin:/home/travis/bin:/home/travis/.local/bin:/usr/local/lib/jvm/openjdk11/bin:/opt/pyenv/shims:/home/travis/.phpenv/shims:/home/travis/perl5/perlbrew/bin:/home/travis/.nvm/versions/node/v8.12.0/bin:/home/travis/.rvm/gems/ruby-2.5.3/bin:/home/travis/.rvm/gems/ruby-2.5.3@global/bin:/home/travis/.rvm/rubies/ruby-2.5.3/bin:/home/travis/gopath/bin:/home/travis/.gimme/versions/go1.11.1.linux.amd64/bin:/usr/local/maven-3.6.0/bin:/usr/local/cmake-3.12.4/bin:/usr/local/clang-7.0.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/home/travis/.rvm/bin:/home/travis/.phpenv/bin:/opt/pyenv/bin:/home/travis/.yarn/bin:/home/travis/gopath/bin" go-bindata -nomemcopy -o pkg/minikube/translate/translations.go -pkg translate translations/...
gofmt -s -w pkg/minikube/translate/translations.go
./test.sh
= go mod ================================================================
ok
= make lint =============================================================
golangci/golangci-lint info checking GitHub for tag 'v1.17.1'
golangci/golangci-lint info found version: 1.17.1 for v1.17.1/linux/amd64
golangci/golangci-lint info installed out/linters/golangci-lint
ok
= boilerplate ===========================================================
ok
= schema_check ==========================================================
ok
= go test ===============================================================
ok  	k8s.io/minikube/cmd/minikube/cmd	0.075s	coverage: 15.7% of statements
ok  	k8s.io/minikube/cmd/minikube/cmd/config	0.044s	coverage: 18.9% of statements
ok  	k8s.io/minikube/pkg/drivers	0.009s	coverage: 19.6% of statements
ok  	k8s.io/minikube/pkg/drivers/kvm	0.038s	coverage: 2.3% of statements
ok  	k8s.io/minikube/pkg/minikube/assets	0.028s	coverage: 61.7% of statements
ok  	k8s.io/minikube/pkg/minikube/bootstrapper	2.685s	coverage: 72.9% of statements
ERROR: logging before flag.Parse: I0821 23:10:48.237775   11605 vm_assets.go:203] Created asset deploy/addons/addon-manager.yaml.tmpl with 1452 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.238102   11605 vm_assets.go:203] Created asset deploy/addons/dashboard/dashboard-dp.yaml.tmpl with 1605 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.238229   11605 vm_assets.go:203] Created asset deploy/addons/dashboard/dashboard-svc.yaml.tmpl with 979 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.238300   11605 vm_assets.go:203] Created asset deploy/addons/storageclass/storageclass.yaml.tmpl with 271 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.238531   11605 vm_assets.go:203] Created asset deploy/addons/storage-provisioner/storage-provisioner.yaml.tmpl with 1755 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.238641   11605 vm_assets.go:203] Created asset deploy/addons/storage-provisioner-gluster/storage-gluster-ns.yaml.tmpl with 230 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.238780   11605 vm_assets.go:203] Created asset deploy/addons/storage-provisioner-gluster/glusterfs-daemonset.yaml.tmpl with 3888 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.238923   11605 vm_assets.go:203] Created asset deploy/addons/storage-provisioner-gluster/heketi-deployment.yaml.tmpl with 3932 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.239064   11605 vm_assets.go:203] Created asset deploy/addons/storage-provisioner-gluster/storage-provisioner-glusterfile.yaml.tmpl with 3145 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.239416   11605 vm_assets.go:203] Created asset deploy/addons/heapster/influx-grafana-rc.yaml.tmpl with 2540 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.239579   11605 vm_assets.go:203] Created asset deploy/addons/heapster/grafana-svc.yaml.tmpl with 1085 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.239667   11605 vm_assets.go:203] Created asset deploy/addons/heapster/influxdb-svc.yaml.tmpl with 1048 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.239925   11605 vm_assets.go:203] Created asset deploy/addons/heapster/heapster-rc.yaml.tmpl with 1651 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.240044   11605 vm_assets.go:203] Created asset deploy/addons/heapster/heapster-svc.yaml.tmpl with 1006 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.240248   11605 vm_assets.go:203] Created asset deploy/addons/efk/elasticsearch-rc.yaml.tmpl with 2209 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.240377   11605 vm_assets.go:203] Created asset deploy/addons/efk/elasticsearch-svc.yaml.tmpl with 947 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.240590   11605 vm_assets.go:203] Created asset deploy/addons/efk/fluentd-es-rc.yaml.tmpl with 2096 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.240811   11605 vm_assets.go:203] Created asset deploy/addons/efk/fluentd-es-configmap.yaml.tmpl with 9863 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.240933   11605 vm_assets.go:203] Created asset deploy/addons/efk/kibana-rc.yaml.tmpl with 1606 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.241099   11605 vm_assets.go:203] Created asset deploy/addons/efk/kibana-svc.yaml.tmpl with 1011 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.241235   11605 vm_assets.go:203] Created asset deploy/addons/ingress/ingress-configmap.yaml.tmpl with 1281 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.241353   11605 vm_assets.go:203] Created asset deploy/addons/ingress/ingress-rbac.yaml.tmpl with 2575 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.241632   11605 vm_assets.go:203] Created asset deploy/addons/ingress/ingress-dp.yaml.tmpl with 3056 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.241765   11605 vm_assets.go:203] Created asset deploy/addons/metrics-server/metrics-apiservice.yaml.tmpl with 401 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.241990   11605 vm_assets.go:203] Created asset deploy/addons/metrics-server/metrics-server-deployment.yaml.tmpl with 671 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.242126   11605 vm_assets.go:203] Created asset deploy/addons/metrics-server/metrics-server-service.yaml.tmpl with 401 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.242244   11605 vm_assets.go:203] Created asset deploy/addons/registry/registry-rc.yaml.tmpl with 748 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.242314   11605 vm_assets.go:203] Created asset deploy/addons/registry/registry-svc.yaml.tmpl with 333 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.242388   11605 vm_assets.go:203] Created asset deploy/addons/registry/registry-proxy.yaml.tmpl with 754 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.242506   11605 vm_assets.go:203] Created asset deploy/addons/registry-creds/registry-creds-rc.yaml.tmpl with 2868 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.245580   11605 vm_assets.go:203] Created asset deploy/addons/freshpod/freshpod-rc.yaml.tmpl with 1437 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.245980   11605 vm_assets.go:203] Created asset deploy/addons/gpu/nvidia-driver-installer.yaml.tmpl with 2487 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.246146   11605 vm_assets.go:203] Created asset deploy/addons/gpu/nvidia-gpu-device-plugin.yaml.tmpl with 2111 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.246226   11605 vm_assets.go:203] Created asset deploy/addons/logviewer/logviewer-dp-and-svc.yaml.tmpl with 1942 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.246296   11605 vm_assets.go:203] Created asset deploy/addons/logviewer/logviewer-rbac.yaml.tmpl with 1064 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.246430   11605 vm_assets.go:203] Created asset deploy/addons/gvisor/gvisor-pod.yaml.tmpl with 1944 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.246585   11605 vm_assets.go:203] Created asset deploy/addons/gvisor/gvisor-config.toml with 1789 bytes
ERROR: logging before flag.Parse: I0821 23:10:48.246647   11605 vm_assets.go:203] Created asset deploy/addons/gvisor/gvisor-containerd-shim.toml with 90 bytes
--- FAIL: TestGenerateConfig (0.01s)
    --- FAIL: TestGenerateConfig/default_v1.16 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.16/default.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd_v1.16 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.16/containerd.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio_v1.16 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.16/crio.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/options_v1.16 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.16/options.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio-options-gates_v1.16 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.16/crio-options-gates.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-api-port_v1.16 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.16/containerd-api-port.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-pod-network-cidr_v1.16 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.16/containerd-pod-network-cidr.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/image-repository_v1.16 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.16/image-repository.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/default_v1.15 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.15/default.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd_v1.15 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.15/containerd.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio_v1.15 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.15/crio.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/options_v1.15 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.15/options.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio-options-gates_v1.15 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.15/crio-options-gates.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-api-port_v1.15 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.15/containerd-api-port.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-pod-network-cidr_v1.15 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.15/containerd-pod-network-cidr.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/image-repository_v1.15 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.15/image-repository.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/default_v1.14 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.14/default.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd_v1.14 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.14/containerd.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio_v1.14 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.14/crio.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/options_v1.14 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.14/options.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio-options-gates_v1.14 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.14/crio-options-gates.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-api-port_v1.14 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.14/containerd-api-port.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-pod-network-cidr_v1.14 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.14/containerd-pod-network-cidr.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/image-repository_v1.14 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.14/image-repository.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/default_v1.13 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.13/default.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd_v1.13 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.13/containerd.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio_v1.13 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.13/crio.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/options_v1.13 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.13/options.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio-options-gates_v1.13 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.13/crio-options-gates.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-api-port_v1.13 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.13/containerd-api-port.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-pod-network-cidr_v1.13 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.13/containerd-pod-network-cidr.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/image-repository_v1.13 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.13/image-repository.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/default_v1.12 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.12/default.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd_v1.12 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.12/containerd.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio_v1.12 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.12/crio.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/options_v1.12 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.12/options.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio-options-gates_v1.12 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.12/crio-options-gates.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-api-port_v1.12 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.12/containerd-api-port.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-pod-network-cidr_v1.12 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.12/containerd-pod-network-cidr.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/image-repository_v1.12 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.12/image-repository.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/default_v1.11 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.11/default.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd_v1.11 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.11/containerd.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio_v1.11 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.11/crio.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/options_v1.11 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.11/options.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio-options-gates_v1.11 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.11/crio-options-gates.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-api-port_v1.11 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.11/containerd-api-port.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-pod-network-cidr_v1.11 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.11/containerd-pod-network-cidr.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/image-repository_v1.11 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.11/image-repository.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/default_v1.10 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.10/default.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd_v1.10 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.10/containerd.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio_v1.10 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.10/crio.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/options_v1.10 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.10/options.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/crio-options-gates_v1.10 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.10/crio-options-gates.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-api-port_v1.10 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.10/containerd-api-port.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/containerd-pod-network-cidr_v1.10 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.10/containerd-pod-network-cidr.yaml: no such file or directory
    --- FAIL: TestGenerateConfig/image-repository_v1.10 (0.00s)
        kubeadm_test.go:265: unable to read testdata: open testdata/v1.10/image-repository.yaml: no such file or directory
FAIL
coverage: 32.4% of statements
FAIL	k8s.io/minikube/pkg/minikube/bootstrapper/kubeadm	0.062s
ok  	k8s.io/minikube/pkg/minikube/cluster	0.664s	coverage: 54.2% of statements
ok  	k8s.io/minikube/pkg/minikube/config	0.033s	coverage: 76.0% of statements
ok  	k8s.io/minikube/pkg/minikube/cruntime	0.010s	coverage: 62.4% of statements
ok  	k8s.io/minikube/pkg/minikube/extract	0.008s	coverage: 56.7% of statements
ok  	k8s.io/minikube/pkg/minikube/kubeconfig	0.038s	coverage: 75.6% of statements
ok  	k8s.io/minikube/pkg/minikube/logs	0.023s	coverage: 1.5% of statements
ok  	k8s.io/minikube/pkg/minikube/machine	0.045s	coverage: 12.0% of statements
ok  	k8s.io/minikube/pkg/minikube/notify	0.036s	coverage: 77.8% of statements
ok  	k8s.io/minikube/pkg/minikube/out	0.008s	coverage: 70.3% of statements
ok  	k8s.io/minikube/pkg/minikube/problem	0.020s	coverage: 42.9% of statements
ok  	k8s.io/minikube/pkg/minikube/proxy	0.010s	coverage: 67.3% of statements
ok  	k8s.io/minikube/pkg/minikube/registry	0.021s	coverage: 81.8% of statements
ok  	k8s.io/minikube/pkg/minikube/service	0.038s	coverage: 35.9% of statements
ok  	k8s.io/minikube/pkg/minikube/sshutil	0.210s	coverage: 75.0% of statements
ok  	k8s.io/minikube/pkg/minikube/translate	0.005s	coverage: 8.4% of statements
Makefile:229: recipe for target 'test' failed
make: *** [test] Error 16
TravisBuddy Request Identifier: fba1d240-c468-11e9-9094-9f1a962f5d6e

@medyagh
Copy link
Member Author

medyagh commented Aug 22, 2019

thanks @TravisBuddy :) you are helpful !

@medyagh
Copy link
Member Author

medyagh commented Aug 22, 2019 •
edited
Loading

Current Flakes with latest commit of this PR:

  • TestVersionUpgrade Both : MacOs Virtualbox and MacOs Hyperkit
        X Wait failed: waiting for component=etcd: timed out waiting for the condition

https://storage.googleapis.com/minikube-builds/logs/5150/VirtualBox_macOS.txt
and
https://storage.googleapis.com/minikube-builds/logs/5150/HyperKit_macOS.txt

See "systemctl status docker.service" and "journalctl -xe" for details.
        : Process exited with status 1
        * 
        X Failed to enable container runtime: command failed: sudo systemctl start docker

--- FAIL: TestDocker (145.01s)
    minikube_runner.go:231: TestDocker Failed to start minikube With error: exit status 70 
        	 begin Start log block ------------> 
        	 With Profile: TestDocker 
        	 With Args: --docker-env=FOO=BAR --docker-env=BAZ=BAT --docker-opt=debug  --docker-opt=icc=true 
        	 With Global Args: --v=10 --logtostderr   
        	 With Driver Args: --vm-driver=virtualbox  --wait-timeout=13m --wait=false 
        	 With STDOUT: 
         	 * [TestDocker] minikube v1.3.1 on Debian 9.9
        * Creating virtualbox VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
        Downloading /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/boot2docker.iso from file:///home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/iso/minikube-v1.3.0.iso...
        Creating VirtualBox VM...
        Creating SSH key...
        Starting the VM...
        Check network to re-create if needed...
        Waiting for an IP...
        Setting Docker configuration on the remote daemon...
        Waiting for SSH to be available...
        Detecting the provisioner...

    	 With STDERR: 
     	 I0821 18:49:47.241885   20691 notify.go:123] Checking for updates...
    I0821 18:49:47.436317   20691 start.go:226] hostinfo: {"hostname":"kvm-integration-slave4","uptime":1973763,"bootTime":1564464824,"procs":211,"os":"linux","platform":"debian","platformFamily":"debian","platformVersion":"9.9","kernelVersion":"4.9.0-9-amd64","virtualizationSystem":"kvm","virtualizationRole":"host","hostid":"ae41e7f6-8b8e-4d40-b77d-1ebb5a2d5fdb"}
    I0821 18:49:47.437391   20691 start.go:236] virtualization: kvm host
    I0821 18:49:47.437796   20691 downloader.go:59] Not caching ISO, using https://storage.googleapis.com/minikube/iso/minikube-v1.3.0.iso
    I0821 18:49:47.438095   20691 cache_images.go:288] Attempting to cache image: gcr.io/k8s-minikube/storage-provisioner:v1.8.1 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/gcr.io/k8s-minikube/storage-provisioner_v1.8.1
    I0821 18:49:47.438113   20691 profile.go:65] Saving config:
    {
        "MachineConfig": {
            "KeepContext": false,
            "EmbedCerts": false,
            "MinikubeISO": "https://storage.googleapis.com/minikube/iso/minikube-v1.3.0.iso",
            "Memory": 2000,
            "CPUs": 2,
            "DiskSize": 20000,
            "VMDriver": "virtualbox",
            "ContainerRuntime": "docker",
            "HyperkitVpnKitSock": "",
            "HyperkitVSockPorts": [],
            "DockerEnv": [
                "FOO=BAR",
                "BAZ=BAT"
            ],
            "InsecureRegistry": null,
            "RegistryMirror": null,
            "HostOnlyCIDR": "192.168.99.1/24",
            "HypervVirtualSwitch": "",
            "KVMNetwork": "default",
            "KVMQemuURI": "qemu:///system",
            "KVMGPU": false,
            "KVMHidden": false,
            "DockerOpt": [
                "debug",
                "icc=true"
            ],
            "DisableDriverMounts": false,
            "NFSShare": [],
            "NFSSharesRoot": "/nfsshares",
            "UUID": "",
            "NoVTXCheck": false,
            "DNSProxy": false,
            "HostDNSResolver": true
        },
        "KubernetesConfig": {
            "KubernetesVersion": "v1.15.2",
            "NodeIP": "",
            "NodePort": 8443,
            "NodeName": "minikube",
            "APIServerName": "minikubeCA",
            "APIServerNames": null,
            "APIServerIPs": null,
            "DNSDomain": "cluster.local",
            "ContainerRuntime": "docker",
            "CRISocket": "",
            "NetworkPlugin": "",
            "FeatureGates": "",
            "ServiceCIDR": "10.96.0.0/12",
            "ImageRepository": "",
            "ExtraOptions": null,
            "ShouldLoadCachedImages": true,
            "EnableDefaultCNI": false
        }
    }
    I0821 18:49:47.438160   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/kube-proxy:v1.15.2 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/kube-proxy_v1.15.2
    I0821 18:49:47.438186   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/kube-scheduler:v1.15.2 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/kube-scheduler_v1.15.2
    I0821 18:49:47.438216   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/kube-controller-manager:v1.15.2 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/kube-controller-manager_v1.15.2
    I0821 18:49:47.438234   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/kube-apiserver:v1.15.2 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/kube-apiserver_v1.15.2
    I0821 18:49:47.438261   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/pause:3.1 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/pause_3.1
    I0821 18:49:47.438253   20691 lock.go:33] attempting to write to file "/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/profiles/TestDocker/config.json" with filemode -rw-------
    I0821 18:49:47.438281   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/k8s-dns-kube-dns-amd64_1.14.13
    I0821 18:49:47.438333   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.13 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64_1.14.13
    I0821 18:49:47.438353   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.13 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/k8s-dns-sidecar-amd64_1.14.13
    I0821 18:49:47.438379   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/etcd:3.3.10 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/etcd_3.3.10
    I0821 18:49:47.438398   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/coredns:1.3.1 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/coredns_1.3.1
    I0821 18:49:47.438425   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/kubernetes-dashboard-amd64_v1.10.1
    I0821 18:49:47.438450   20691 cache_images.go:288] Attempting to cache image: k8s.gcr.io/kube-addon-manager:v9.0 at /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/cache/images/k8s.gcr.io/kube-addon-manager_v9.0
    I0821 18:49:47.438465   20691 cache_images.go:85] Successfully cached all images.
    I0821 18:49:47.440072   20691 cluster.go:93] Machine does not exist... provisioning new machine
    I0821 18:49:47.440105   20691 cluster.go:94] Provisioning machine with config: {KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube/iso/minikube-v1.3.0.iso Memory:2000 CPUs:2 DiskSize:20000 VMDriver:virtualbox ContainerRuntime:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[FOO=BAR BAZ=BAT] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false Downloader:{} DockerOpt:[debug icc=true] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true}
    Reading certificate data from /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/certs/ca.pem
    Decoding PEM data...
    Parsing certificate...
    Reading certificate data from /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/certs/cert.pem
    Decoding PEM data...
    Parsing certificate...
    COMMAND: /usr/bin/VBoxManage --version
    STDOUT:
    {
    5.2.32r132073
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage list hostonlyifs
    STDOUT:
    {
    Name:            vboxnet0
    GUID:            786f6276-656e-4074-8000-0a0027000000
    DHCP:            Disabled
    IPAddress:       192.168.99.1
    NetworkMask:     255.255.255.0
    IPV6Address:     
    IPV6NetworkMaskPrefixLength: 0
    HardwareAddress: 0a:00:27:00:00:00
    MediumType:      Ethernet
    Wireless:        No
    Status:          Up
    VBoxNetworkName: HostInterfaceNetworking-vboxnet0
    
    }
    STDERR:
    {
    }
    Creating disk image...
    Creating 20000 MB hard disk image...
    Writing magic tar header
    Writing SSH key tar header
    Calling inner createDiskImage
    &{/usr/bin/VBoxManage [/usr/bin/VBoxManage convertfromraw stdin /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk 20971520000 --format VMDK] []  <nil> <nil> <nil> [] <nil> <nil> <nil> <nil> <nil> false [] [] [] [] <nil> <nil>}
    Starting command
    Copying to stdin
    Filling zeroes
    Closing STDIN
    Waiting on cmd
    COMMAND: /usr/bin/VBoxManage createvm --basefolder /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker --name TestDocker --register
    STDOUT:
    {
    Virtual machine 'TestDocker' is created and registered.
    UUID: 072db02c-c80c-4e8d-a76a-01aded656cd2
    Settings file: '/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.vbox'
    }
    STDERR:
    {
    }
    VM CPUS: 2
    VM Memory: 2000
    COMMAND: /usr/bin/VBoxManage modifyvm TestDocker --firmware bios --bioslogofadein off --bioslogofadeout off --bioslogodisplaytime 0 --biosbootmenu disabled --ostype Linux26_64 --cpus 2 --memory 2000 --acpi on --ioapic on --rtcuseutc on --natdnshostresolver1 on --natdnsproxy1 off --cpuhotplug off --pae on --hpet on --hwvirtex on --nestedpaging on --largepages on --vtxvpid on --accelerate3d off --boot1 dvd
    STDOUT:
    {
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage modifyvm TestDocker --nic1 nat --nictype1 virtio --cableconnected1 on
    STDOUT:
    {
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage storagectl TestDocker --name SATA --add sata --hostiocache on
    STDOUT:
    {
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage storageattach TestDocker --storagectl SATA --port 0 --device 0 --type dvddrive --medium /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/boot2docker.iso
    STDOUT:
    {
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage storageattach TestDocker --storagectl SATA --port 1 --device 0 --type hdd --medium /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk
    STDOUT:
    {
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage guestproperty set TestDocker /VirtualBox/GuestAdd/SharedFolders/MountPrefix /
    STDOUT:
    {
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage guestproperty set TestDocker /VirtualBox/GuestAdd/SharedFolders/MountDir /
    STDOUT:
    {
    }
    STDERR:
    {
    }
    setting up shareDir '/home' -> 'hosthome'
    COMMAND: /usr/bin/VBoxManage sharedfolder add TestDocker --name hosthome --hostpath /home --automount
    STDOUT:
    {
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage setextradata TestDocker VBoxInternal2/SharedFoldersEnableSymlinksCreate/hosthome 1
    STDOUT:
    {
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage showvminfo TestDocker --machinereadable
    STDOUT:
    {
    name="TestDocker"
    groups="/"
    ostype="Linux 2.6 / 3.x / 4.x (64-bit)"
    UUID="072db02c-c80c-4e8d-a76a-01aded656cd2"
    CfgFile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.vbox"
    SnapFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Snapshots"
    LogFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Logs"
    hardwareuuid="072db02c-c80c-4e8d-a76a-01aded656cd2"
    memory=2000
    pagefusion="off"
    vram=8
    cpuexecutioncap=100
    hpet="on"
    chipset="piix3"
    firmware="BIOS"
    cpus=2
    pae="on"
    longmode="on"
    triplefaultreset="off"
    apic="on"
    x2apic="off"
    cpuid-portability-level=0
    bootmenu="disabled"
    boot1="dvd"
    boot2="dvd"
    boot3="disk"
    boot4="none"
    acpi="on"
    ioapic="on"
    biosapic="apic"
    biossystemtimeoffset=0
    rtcuseutc="on"
    hwvirtex="on"
    nestedpaging="on"
    largepages="on"
    vtxvpid="on"
    vtxux="on"
    paravirtprovider="default"
    effparavirtprovider="kvm"
    VMState="poweroff"
    VMStateChangeTime="2019-08-22T01:49:59.765000000"
    monitorcount=1
    accelerate3d="off"
    accelerate2dvideo="off"
    teleporterenabled="off"
    teleporterport=0
    teleporteraddress=""
    teleporterpassword=""
    tracing-enabled="off"
    tracing-allow-vm-access="off"
    tracing-config=""
    autostart-enabled="off"
    autostart-delay=0
    defaultfrontend=""
    storagecontrollername0="SATA"
    storagecontrollertype0="IntelAhci"
    storagecontrollerinstance0="0"
    storagecontrollermaxportcount0="30"
    storagecontrollerportcount0="30"
    storagecontrollerbootable0="on"
    "SATA-0-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/boot2docker.iso"
    "SATA-ImageUUID-0-0"="d0a429ea-f72a-423a-aaef-613177125889"
    "SATA-tempeject"="off"
    "SATA-IsEjected"="off"
    "SATA-1-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk"
    "SATA-ImageUUID-1-0"="4d2808a7-3d0e-4f46-b9f1-9e07b433a953"
    "SATA-2-0"="none"
    "SATA-3-0"="none"
    "SATA-4-0"="none"
    "SATA-5-0"="none"
    "SATA-6-0"="none"
    "SATA-7-0"="none"
    "SATA-8-0"="none"
    "SATA-9-0"="none"
    "SATA-10-0"="none"
    "SATA-11-0"="none"
    "SATA-12-0"="none"
    "SATA-13-0"="none"
    "SATA-14-0"="none"
    "SATA-15-0"="none"
    "SATA-16-0"="none"
    "SATA-17-0"="none"
    "SATA-18-0"="none"
    "SATA-19-0"="none"
    "SATA-20-0"="none"
    "SATA-21-0"="none"
    "SATA-22-0"="none"
    "SATA-23-0"="none"
    "SATA-24-0"="none"
    "SATA-25-0"="none"
    "SATA-26-0"="none"
    "SATA-27-0"="none"
    "SATA-28-0"="none"
    "SATA-29-0"="none"
    natnet1="nat"
    macaddress1="080027EEE7A1"
    cableconnected1="on"
    nic1="nat"
    nictype1="virtio"
    nicspeed1="0"
    mtu="0"
    sockSnd="64"
    sockRcv="64"
    tcpWndSnd="64"
    tcpWndRcv="64"
    nic2="none"
    nic3="none"
    nic4="none"
    nic5="none"
    nic6="none"
    nic7="none"
    nic8="none"
    hidpointing="ps2mouse"
    hidkeyboard="ps2kbd"
    uart1="off"
    uart2="off"
    uart3="off"
    uart4="off"
    lpt1="off"
    lpt2="off"
    audio="alsa"
    audio_in="false"
    audio_out="false"
    clipboard="disabled"
    draganddrop="disabled"
    vrde="off"
    usb="off"
    ehci="off"
    xhci="off"
    SharedFolderNameMachineMapping1="hosthome"
    SharedFolderPathMachineMapping1="/home"
    videocap="off"
    videocap_audio="off"
    videocapscreens=0
    videocapfile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.webm"
    videocapres=1024x768
    videocaprate=512
    videocapfps=25
    videocapopts=
    GuestMemoryBalloon=0
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage list hostonlyifs
    STDOUT:
    {
    Name:            vboxnet0
    GUID:            786f6276-656e-4074-8000-0a0027000000
    DHCP:            Disabled
    IPAddress:       192.168.99.1
    NetworkMask:     255.255.255.0
    IPV6Address:     
    IPV6NetworkMaskPrefixLength: 0
    HardwareAddress: 0a:00:27:00:00:00
    MediumType:      Ethernet
    Wireless:        No
    Status:          Up
    VBoxNetworkName: HostInterfaceNetworking-vboxnet0
    
    }
    STDERR:
    {
    }
    Searching for hostonly interface for IPv4: 192.168.99.1 and Mask: ffffff00
    Found: vboxnet0
    COMMAND: /usr/bin/VBoxManage list dhcpservers
    STDOUT:
    {
    NetworkName:    HostInterfaceNetworking-vboxnet0
    IP:             192.168.99.7
    NetworkMask:    255.255.255.0
    lowerIPAddress: 192.168.99.100
    upperIPAddress: 192.168.99.254
    Enabled:        Yes
    
    }
    STDERR:
    {
    }
    Removing orphan DHCP servers...
    COMMAND: /usr/bin/VBoxManage list hostonlyifs
    STDOUT:
    {
    Name:            vboxnet0
    GUID:            786f6276-656e-4074-8000-0a0027000000
    DHCP:            Disabled
    IPAddress:       192.168.99.1
    NetworkMask:     255.255.255.0
    IPV6Address:     
    IPV6NetworkMaskPrefixLength: 0
    HardwareAddress: 0a:00:27:00:00:00
    MediumType:      Ethernet
    Wireless:        No
    Status:          Up
    VBoxNetworkName: HostInterfaceNetworking-vboxnet0
    
    }
    STDERR:
    {
    }
    Adding/Modifying DHCP server "192.168.99.11" with address range "192.168.99.100" - "192.168.99.254"...
    COMMAND: /usr/bin/VBoxManage list dhcpservers
    STDOUT:
    {
    NetworkName:    HostInterfaceNetworking-vboxnet0
    IP:             192.168.99.10
    NetworkMask:    255.255.255.0
    lowerIPAddress: 192.168.99.100
    upperIPAddress: 192.168.99.254
    Enabled:        Yes
    
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage dhcpserver modify --netname HostInterfaceNetworking-vboxnet0 --ip 192.168.99.11 --netmask 255.255.255.0 --lowerip 192.168.99.100 --upperip 192.168.99.254 --enable
    STDOUT:
    {
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage modifyvm TestDocker --nic2 hostonly --nictype2 virtio --nicpromisc2 deny --hostonlyadapter2 vboxnet0 --cableconnected2 on
    STDOUT:
    {
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage modifyvm TestDocker --natpf1 delete ssh
    STDOUT:
    {
    }
    STDERR:
    {
    VBoxManage: error: Code NS_ERROR_INVALID_ARG (0x80070057) - Invalid argument value (extended info not available)
    VBoxManage: error: Context: "RemoveRedirect(Bstr(ValueUnion.psz).raw())" at line 1884 of file VBoxManageModifyVM.cpp
    }
    COMMAND: /usr/bin/VBoxManage modifyvm TestDocker --natpf1 ssh,tcp,127.0.0.1,38133,,22
    STDOUT:
    {
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage startvm TestDocker --type headless
    STDOUT:
    {
    Waiting for VM "TestDocker" to power on...
    VM "TestDocker" has been successfully started.
    }
    STDERR:
    {
    }
    Checking vm logs: /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Logs/VBox.log
    Getting to WaitForSSH function...
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    exit 0
    Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:50780->127.0.0.1:38133: read: connection reset by peer
    SSH cmd err, output: <nil>: 
    COMMAND: /usr/bin/VBoxManage showvminfo TestDocker --machinereadable
    STDOUT:
    {
    name="TestDocker"
    groups="/"
    ostype="Linux 2.6 / 3.x / 4.x (64-bit)"
    UUID="072db02c-c80c-4e8d-a76a-01aded656cd2"
    CfgFile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.vbox"
    SnapFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Snapshots"
    LogFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Logs"
    hardwareuuid="072db02c-c80c-4e8d-a76a-01aded656cd2"
    memory=2000
    pagefusion="off"
    vram=8
    cpuexecutioncap=100
    hpet="on"
    chipset="piix3"
    firmware="BIOS"
    cpus=2
    pae="on"
    longmode="on"
    triplefaultreset="off"
    apic="on"
    x2apic="off"
    cpuid-portability-level=0
    bootmenu="disabled"
    boot1="dvd"
    boot2="dvd"
    boot3="disk"
    boot4="none"
    acpi="on"
    ioapic="on"
    biosapic="apic"
    biossystemtimeoffset=0
    rtcuseutc="on"
    hwvirtex="on"
    nestedpaging="on"
    largepages="on"
    vtxvpid="on"
    vtxux="on"
    paravirtprovider="default"
    effparavirtprovider="kvm"
    VMState="running"
    VMStateChangeTime="2019-08-22T01:50:01.750000000"
    monitorcount=1
    accelerate3d="off"
    accelerate2dvideo="off"
    teleporterenabled="off"
    teleporterport=0
    teleporteraddress=""
    teleporterpassword=""
    tracing-enabled="off"
    tracing-allow-vm-access="off"
    tracing-config=""
    autostart-enabled="off"
    autostart-delay=0
    defaultfrontend=""
    storagecontrollername0="SATA"
    storagecontrollertype0="IntelAhci"
    storagecontrollerinstance0="0"
    storagecontrollermaxportcount0="30"
    storagecontrollerportcount0="30"
    storagecontrollerbootable0="on"
    "SATA-0-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/boot2docker.iso"
    "SATA-ImageUUID-0-0"="d0a429ea-f72a-423a-aaef-613177125889"
    "SATA-tempeject"="off"
    "SATA-IsEjected"="off"
    "SATA-1-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk"
    "SATA-ImageUUID-1-0"="4d2808a7-3d0e-4f46-b9f1-9e07b433a953"
    "SATA-2-0"="none"
    "SATA-3-0"="none"
    "SATA-4-0"="none"
    "SATA-5-0"="none"
    "SATA-6-0"="none"
    "SATA-7-0"="none"
    "SATA-8-0"="none"
    "SATA-9-0"="none"
    "SATA-10-0"="none"
    "SATA-11-0"="none"
    "SATA-12-0"="none"
    "SATA-13-0"="none"
    "SATA-14-0"="none"
    "SATA-15-0"="none"
    "SATA-16-0"="none"
    "SATA-17-0"="none"
    "SATA-18-0"="none"
    "SATA-19-0"="none"
    "SATA-20-0"="none"
    "SATA-21-0"="none"
    "SATA-22-0"="none"
    "SATA-23-0"="none"
    "SATA-24-0"="none"
    "SATA-25-0"="none"
    "SATA-26-0"="none"
    "SATA-27-0"="none"
    "SATA-28-0"="none"
    "SATA-29-0"="none"
    natnet1="nat"
    macaddress1="080027EEE7A1"
    cableconnected1="on"
    nic1="nat"
    nictype1="virtio"
    nicspeed1="0"
    mtu="0"
    sockSnd="64"
    sockRcv="64"
    tcpWndSnd="64"
    tcpWndRcv="64"
    Forwarding(0)="ssh,tcp,127.0.0.1,38133,,22"
    hostonlyadapter2="vboxnet0"
    macaddress2="080027B647FD"
    cableconnected2="on"
    nic2="hostonly"
    nictype2="virtio"
    nicspeed2="0"
    nic3="none"
    nic4="none"
    nic5="none"
    nic6="none"
    nic7="none"
    nic8="none"
    hidpointing="ps2mouse"
    hidkeyboard="ps2kbd"
    uart1="off"
    uart2="off"
    uart3="off"
    uart4="off"
    lpt1="off"
    lpt2="off"
    audio="alsa"
    audio_in="false"
    audio_out="false"
    clipboard="disabled"
    draganddrop="disabled"
    SessionName="headless"
    VideoMode="720,400,0"@0,0 1
    vrde="off"
    usb="off"
    ehci="off"
    xhci="off"
    SharedFolderNameMachineMapping1="hosthome"
    SharedFolderPathMachineMapping1="/home"
    VRDEActiveConnection="off"
    VRDEClients=0
    videocap="off"
    videocap_audio="off"
    videocapscreens=0
    videocapfile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.webm"
    videocapres=1024x768
    videocaprate=512
    videocapfps=25
    videocapopts=
    GuestMemoryBalloon=0
    GuestOSType="Linux26_64"
    GuestAdditionsRunLevel=2
    GuestAdditionsVersion="5.1.38 r122592"
    GuestAdditionsFacility_VirtualBox Base Driver=50,1566438711915
    GuestAdditionsFacility_VirtualBox System Service=50,1566438714740
    GuestAdditionsFacility_Seamless Mode=0,1566438711858
    GuestAdditionsFacility_Graphics Mode=0,1566438711857
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage showvminfo TestDocker --machinereadable
    STDOUT:
    {
    name="TestDocker"
    groups="/"
    ostype="Linux 2.6 / 3.x / 4.x (64-bit)"
    UUID="072db02c-c80c-4e8d-a76a-01aded656cd2"
    CfgFile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.vbox"
    SnapFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Snapshots"
    LogFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Logs"
    hardwareuuid="072db02c-c80c-4e8d-a76a-01aded656cd2"
    memory=2000
    pagefusion="off"
    vram=8
    cpuexecutioncap=100
    hpet="on"
    chipset="piix3"
    firmware="BIOS"
    cpus=2
    pae="on"
    longmode="on"
    triplefaultreset="off"
    apic="on"
    x2apic="off"
    cpuid-portability-level=0
    bootmenu="disabled"
    boot1="dvd"
    boot2="dvd"
    boot3="disk"
    boot4="none"
    acpi="on"
    ioapic="on"
    biosapic="apic"
    biossystemtimeoffset=0
    rtcuseutc="on"
    hwvirtex="on"
    nestedpaging="on"
    largepages="on"
    vtxvpid="on"
    vtxux="on"
    paravirtprovider="default"
    effparavirtprovider="kvm"
    VMState="running"
    VMStateChangeTime="2019-08-22T01:50:01.750000000"
    monitorcount=1
    accelerate3d="off"
    accelerate2dvideo="off"
    teleporterenabled="off"
    teleporterport=0
    teleporteraddress=""
    teleporterpassword=""
    tracing-enabled="off"
    tracing-allow-vm-access="off"
    tracing-config=""
    autostart-enabled="off"
    autostart-delay=0
    defaultfrontend=""
    storagecontrollername0="SATA"
    storagecontrollertype0="IntelAhci"
    storagecontrollerinstance0="0"
    storagecontrollermaxportcount0="30"
    storagecontrollerportcount0="30"
    storagecontrollerbootable0="on"
    "SATA-0-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/boot2docker.iso"
    "SATA-ImageUUID-0-0"="d0a429ea-f72a-423a-aaef-613177125889"
    "SATA-tempeject"="off"
    "SATA-IsEjected"="off"
    "SATA-1-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk"
    "SATA-ImageUUID-1-0"="4d2808a7-3d0e-4f46-b9f1-9e07b433a953"
    "SATA-2-0"="none"
    "SATA-3-0"="none"
    "SATA-4-0"="none"
    "SATA-5-0"="none"
    "SATA-6-0"="none"
    "SATA-7-0"="none"
    "SATA-8-0"="none"
    "SATA-9-0"="none"
    "SATA-10-0"="none"
    "SATA-11-0"="none"
    "SATA-12-0"="none"
    "SATA-13-0"="none"
    "SATA-14-0"="none"
    "SATA-15-0"="none"
    "SATA-16-0"="none"
    "SATA-17-0"="none"
    "SATA-18-0"="none"
    "SATA-19-0"="none"
    "SATA-20-0"="none"
    "SATA-21-0"="none"
    "SATA-22-0"="none"
    "SATA-23-0"="none"
    "SATA-24-0"="none"
    "SATA-25-0"="none"
    "SATA-26-0"="none"
    "SATA-27-0"="none"
    "SATA-28-0"="none"
    "SATA-29-0"="none"
    natnet1="nat"
    macaddress1="080027EEE7A1"
    cableconnected1="on"
    nic1="nat"
    nictype1="virtio"
    nicspeed1="0"
    mtu="0"
    sockSnd="64"
    sockRcv="64"
    tcpWndSnd="64"
    tcpWndRcv="64"
    Forwarding(0)="ssh,tcp,127.0.0.1,38133,,22"
    hostonlyadapter2="vboxnet0"
    macaddress2="080027B647FD"
    cableconnected2="on"
    nic2="hostonly"
    nictype2="virtio"
    nicspeed2="0"
    nic3="none"
    nic4="none"
    nic5="none"
    nic6="none"
    nic7="none"
    nic8="none"
    hidpointing="ps2mouse"
    hidkeyboard="ps2kbd"
    uart1="off"
    uart2="off"
    uart3="off"
    uart4="off"
    lpt1="off"
    lpt2="off"
    audio="alsa"
    audio_in="false"
    audio_out="false"
    clipboard="disabled"
    draganddrop="disabled"
    SessionName="headless"
    VideoMode="720,400,0"@0,0 1
    vrde="off"
    usb="off"
    ehci="off"
    xhci="off"
    SharedFolderNameMachineMapping1="hosthome"
    SharedFolderPathMachineMapping1="/home"
    VRDEActiveConnection="off"
    VRDEClients=0
    videocap="off"
    videocap_audio="off"
    videocapscreens=0
    videocapfile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.webm"
    videocapres=1024x768
    videocaprate=512
    videocapfps=25
    videocapopts=
    GuestMemoryBalloon=0
    GuestOSType="Linux26_64"
    GuestAdditionsRunLevel=2
    GuestAdditionsVersion="5.1.38 r122592"
    GuestAdditionsFacility_VirtualBox Base Driver=50,1566438711915
    GuestAdditionsFacility_VirtualBox System Service=50,1566438714740
    GuestAdditionsFacility_Seamless Mode=0,1566438711858
    GuestAdditionsFacility_Graphics Mode=0,1566438711857
    }
    STDERR:
    {
    }
    Host-only MAC: 080027b647fd
    
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    ip addr show
    SSH cmd err, output: <nil>: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host 
           valid_lft forever preferred_lft forever
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:ee:e7:a1 brd ff:ff:ff:ff:ff:ff
        inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
           valid_lft 86389sec preferred_lft 86389sec
        inet6 fe80::a00:27ff:feee:e7a1/64 scope link 
           valid_lft forever preferred_lft forever
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:b6:47:fd brd ff:ff:ff:ff:ff:ff
        inet 192.168.99.101/24 brd 192.168.99.255 scope global dynamic eth1
           valid_lft 1189sec preferred_lft 1189sec
        inet6 fe80::a00:27ff:feb6:47fd/64 scope link 
           valid_lft forever preferred_lft forever
    4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
        link/sit 0.0.0.0 brd 0.0.0.0
    
    SSH returned: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host 
           valid_lft forever preferred_lft forever
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:ee:e7:a1 brd ff:ff:ff:ff:ff:ff
        inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
           valid_lft 86389sec preferred_lft 86389sec
        inet6 fe80::a00:27ff:feee:e7a1/64 scope link 
           valid_lft forever preferred_lft forever
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:b6:47:fd brd ff:ff:ff:ff:ff:ff
        inet 192.168.99.101/24 brd 192.168.99.255 scope global dynamic eth1
           valid_lft 1189sec preferred_lft 1189sec
        inet6 fe80::a00:27ff:feb6:47fd/64 scope link 
           valid_lft forever preferred_lft forever
    4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
        link/sit 0.0.0.0 brd 0.0.0.0
    
    END SSH
    
    IP is 192.168.99.101
    COMMAND: /usr/bin/VBoxManage showvminfo TestDocker --machinereadable
    STDOUT:
    {
    name="TestDocker"
    groups="/"
    ostype="Linux 2.6 / 3.x / 4.x (64-bit)"
    UUID="072db02c-c80c-4e8d-a76a-01aded656cd2"
    CfgFile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.vbox"
    SnapFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Snapshots"
    LogFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Logs"
    hardwareuuid="072db02c-c80c-4e8d-a76a-01aded656cd2"
    memory=2000
    pagefusion="off"
    vram=8
    cpuexecutioncap=100
    hpet="on"
    chipset="piix3"
    firmware="BIOS"
    cpus=2
    pae="on"
    longmode="on"
    triplefaultreset="off"
    apic="on"
    x2apic="off"
    cpuid-portability-level=0
    bootmenu="disabled"
    boot1="dvd"
    boot2="dvd"
    boot3="disk"
    boot4="none"
    acpi="on"
    ioapic="on"
    biosapic="apic"
    biossystemtimeoffset=0
    rtcuseutc="on"
    hwvirtex="on"
    nestedpaging="on"
    largepages="on"
    vtxvpid="on"
    vtxux="on"
    paravirtprovider="default"
    effparavirtprovider="kvm"
    VMState="running"
    VMStateChangeTime="2019-08-22T01:50:01.750000000"
    monitorcount=1
    accelerate3d="off"
    accelerate2dvideo="off"
    teleporterenabled="off"
    teleporterport=0
    teleporteraddress=""
    teleporterpassword=""
    tracing-enabled="off"
    tracing-allow-vm-access="off"
    tracing-config=""
    autostart-enabled="off"
    autostart-delay=0
    defaultfrontend=""
    storagecontrollername0="SATA"
    storagecontrollertype0="IntelAhci"
    storagecontrollerinstance0="0"
    storagecontrollermaxportcount0="30"
    storagecontrollerportcount0="30"
    storagecontrollerbootable0="on"
    "SATA-0-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/boot2docker.iso"
    "SATA-ImageUUID-0-0"="d0a429ea-f72a-423a-aaef-613177125889"
    "SATA-tempeject"="off"
    "SATA-IsEjected"="off"
    "SATA-1-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk"
    "SATA-ImageUUID-1-0"="4d2808a7-3d0e-4f46-b9f1-9e07b433a953"
    "SATA-2-0"="none"
    "SATA-3-0"="none"
    "SATA-4-0"="none"
    "SATA-5-0"="none"
    "SATA-6-0"="none"
    "SATA-7-0"="none"
    "SATA-8-0"="none"
    "SATA-9-0"="none"
    "SATA-10-0"="none"
    "SATA-11-0"="none"
    "SATA-12-0"="none"
    "SATA-13-0"="none"
    "SATA-14-0"="none"
    "SATA-15-0"="none"
    "SATA-16-0"="none"
    "SATA-17-0"="none"
    "SATA-18-0"="none"
    "SATA-19-0"="none"
    "SATA-20-0"="none"
    "SATA-21-0"="none"
    "SATA-22-0"="none"
    "SATA-23-0"="none"
    "SATA-24-0"="none"
    "SATA-25-0"="none"
    "SATA-26-0"="none"
    "SATA-27-0"="none"
    "SATA-28-0"="none"
    "SATA-29-0"="none"
    natnet1="nat"
    macaddress1="080027EEE7A1"
    cableconnected1="on"
    nic1="nat"
    nictype1="virtio"
    nicspeed1="0"
    mtu="0"
    sockSnd="64"
    sockRcv="64"
    tcpWndSnd="64"
    tcpWndRcv="64"
    Forwarding(0)="ssh,tcp,127.0.0.1,38133,,22"
    hostonlyadapter2="vboxnet0"
    macaddress2="080027B647FD"
    cableconnected2="on"
    nic2="hostonly"
    nictype2="virtio"
    nicspeed2="0"
    nic3="none"
    nic4="none"
    nic5="none"
    nic6="none"
    nic7="none"
    nic8="none"
    hidpointing="ps2mouse"
    hidkeyboard="ps2kbd"
    uart1="off"
    uart2="off"
    uart3="off"
    uart4="off"
    lpt1="off"
    lpt2="off"
    audio="alsa"
    audio_in="false"
    audio_out="false"
    clipboard="disabled"
    draganddrop="disabled"
    SessionName="headless"
    VideoMode="720,400,0"@0,0 1
    vrde="off"
    usb="off"
    ehci="off"
    xhci="off"
    SharedFolderNameMachineMapping1="hosthome"
    SharedFolderPathMachineMapping1="/home"
    VRDEActiveConnection="off"
    VRDEClients=0
    videocap="off"
    videocap_audio="off"
    videocapscreens=0
    videocapfile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.webm"
    videocapres=1024x768
    videocaprate=512
    videocapfps=25
    videocapopts=
    GuestMemoryBalloon=0
    GuestOSType="Linux26_64"
    GuestAdditionsRunLevel=2
    GuestAdditionsVersion="5.1.38 r122592"
    GuestAdditionsFacility_VirtualBox Base Driver=50,1566438711915
    GuestAdditionsFacility_VirtualBox System Service=50,1566438714740
    GuestAdditionsFacility_Seamless Mode=0,1566438711858
    GuestAdditionsFacility_Graphics Mode=0,1566438711857
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage showvminfo TestDocker --machinereadable
    STDOUT:
    {
    name="TestDocker"
    groups="/"
    ostype="Linux 2.6 / 3.x / 4.x (64-bit)"
    UUID="072db02c-c80c-4e8d-a76a-01aded656cd2"
    CfgFile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.vbox"
    SnapFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Snapshots"
    LogFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Logs"
    hardwareuuid="072db02c-c80c-4e8d-a76a-01aded656cd2"
    memory=2000
    pagefusion="off"
    vram=8
    cpuexecutioncap=100
    hpet="on"
    chipset="piix3"
    firmware="BIOS"
    cpus=2
    pae="on"
    longmode="on"
    triplefaultreset="off"
    apic="on"
    x2apic="off"
    cpuid-portability-level=0
    bootmenu="disabled"
    boot1="dvd"
    boot2="dvd"
    boot3="disk"
    boot4="none"
    acpi="on"
    ioapic="on"
    biosapic="apic"
    biossystemtimeoffset=0
    rtcuseutc="on"
    hwvirtex="on"
    nestedpaging="on"
    largepages="on"
    vtxvpid="on"
    vtxux="on"
    paravirtprovider="default"
    effparavirtprovider="kvm"
    VMState="running"
    VMStateChangeTime="2019-08-22T01:50:01.750000000"
    monitorcount=1
    accelerate3d="off"
    accelerate2dvideo="off"
    teleporterenabled="off"
    teleporterport=0
    teleporteraddress=""
    teleporterpassword=""
    tracing-enabled="off"
    tracing-allow-vm-access="off"
    tracing-config=""
    autostart-enabled="off"
    autostart-delay=0
    defaultfrontend=""
    storagecontrollername0="SATA"
    storagecontrollertype0="IntelAhci"
    storagecontrollerinstance0="0"
    storagecontrollermaxportcount0="30"
    storagecontrollerportcount0="30"
    storagecontrollerbootable0="on"
    "SATA-0-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/boot2docker.iso"
    "SATA-ImageUUID-0-0"="d0a429ea-f72a-423a-aaef-613177125889"
    "SATA-tempeject"="off"
    "SATA-IsEjected"="off"
    "SATA-1-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk"
    "SATA-ImageUUID-1-0"="4d2808a7-3d0e-4f46-b9f1-9e07b433a953"
    "SATA-2-0"="none"
    "SATA-3-0"="none"
    "SATA-4-0"="none"
    "SATA-5-0"="none"
    "SATA-6-0"="none"
    "SATA-7-0"="none"
    "SATA-8-0"="none"
    "SATA-9-0"="none"
    "SATA-10-0"="none"
    "SATA-11-0"="none"
    "SATA-12-0"="none"
    "SATA-13-0"="none"
    "SATA-14-0"="none"
    "SATA-15-0"="none"
    "SATA-16-0"="none"
    "SATA-17-0"="none"
    "SATA-18-0"="none"
    "SATA-19-0"="none"
    "SATA-20-0"="none"
    "SATA-21-0"="none"
    "SATA-22-0"="none"
    "SATA-23-0"="none"
    "SATA-24-0"="none"
    "SATA-25-0"="none"
    "SATA-26-0"="none"
    "SATA-27-0"="none"
    "SATA-28-0"="none"
    "SATA-29-0"="none"
    natnet1="nat"
    macaddress1="080027EEE7A1"
    cableconnected1="on"
    nic1="nat"
    nictype1="virtio"
    nicspeed1="0"
    mtu="0"
    sockSnd="64"
    sockRcv="64"
    tcpWndSnd="64"
    tcpWndRcv="64"
    Forwarding(0)="ssh,tcp,127.0.0.1,38133,,22"
    hostonlyadapter2="vboxnet0"
    macaddress2="080027B647FD"
    cableconnected2="on"
    nic2="hostonly"
    nictype2="virtio"
    nicspeed2="0"
    nic3="none"
    nic4="none"
    nic5="none"
    nic6="none"
    nic7="none"
    nic8="none"
    hidpointing="ps2mouse"
    hidkeyboard="ps2kbd"
    uart1="off"
    uart2="off"
    uart3="off"
    uart4="off"
    lpt1="off"
    lpt2="off"
    audio="alsa"
    audio_in="false"
    audio_out="false"
    clipboard="disabled"
    draganddrop="disabled"
    SessionName="headless"
    VideoMode="720,400,0"@0,0 1
    vrde="off"
    usb="off"
    ehci="off"
    xhci="off"
    SharedFolderNameMachineMapping1="hosthome"
    SharedFolderPathMachineMapping1="/home"
    VRDEActiveConnection="off"
    VRDEClients=0
    videocap="off"
    videocap_audio="off"
    videocapscreens=0
    videocapfile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.webm"
    videocapres=1024x768
    videocaprate=512
    videocapfps=25
    videocapopts=
    GuestMemoryBalloon=0
    GuestOSType="Linux26_64"
    GuestAdditionsRunLevel=2
    GuestAdditionsVersion="5.1.38 r122592"
    GuestAdditionsFacility_VirtualBox Base Driver=50,1566438711915
    GuestAdditionsFacility_VirtualBox System Service=50,1566438714740
    GuestAdditionsFacility_Seamless Mode=0,1566438711858
    GuestAdditionsFacility_Graphics Mode=0,1566438711857
    }
    STDERR:
    {
    }
    Host-only MAC: 080027b647fd
    
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    ip addr show
    SSH cmd err, output: <nil>: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host 
           valid_lft forever preferred_lft forever
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:ee:e7:a1 brd ff:ff:ff:ff:ff:ff
        inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
           valid_lft 86388sec preferred_lft 86388sec
        inet6 fe80::a00:27ff:feee:e7a1/64 scope link 
           valid_lft forever preferred_lft forever
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:b6:47:fd brd ff:ff:ff:ff:ff:ff
        inet 192.168.99.101/24 brd 192.168.99.255 scope global dynamic eth1
           valid_lft 1189sec preferred_lft 1189sec
        inet6 fe80::a00:27ff:feb6:47fd/64 scope link 
           valid_lft forever preferred_lft forever
    4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
        link/sit 0.0.0.0 brd 0.0.0.0
    
    SSH returned: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host 
           valid_lft forever preferred_lft forever
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:ee:e7:a1 brd ff:ff:ff:ff:ff:ff
        inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
           valid_lft 86388sec preferred_lft 86388sec
        inet6 fe80::a00:27ff:feee:e7a1/64 scope link 
           valid_lft forever preferred_lft forever
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:b6:47:fd brd ff:ff:ff:ff:ff:ff
        inet 192.168.99.101/24 brd 192.168.99.255 scope global dynamic eth1
           valid_lft 1189sec preferred_lft 1189sec
        inet6 fe80::a00:27ff:feb6:47fd/64 scope link 
           valid_lft forever preferred_lft forever
    4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
        link/sit 0.0.0.0 brd 0.0.0.0
    
    END SSH
    
    COMMAND: /usr/bin/VBoxManage list hostonlyifs
    STDOUT:
    {
    Name:            vboxnet0
    GUID:            786f6276-656e-4074-8000-0a0027000000
    DHCP:            Disabled
    IPAddress:       192.168.99.1
    NetworkMask:     255.255.255.0
    IPV6Address:     
    IPV6NetworkMaskPrefixLength: 0
    HardwareAddress: 0a:00:27:00:00:00
    MediumType:      Ethernet
    Wireless:        No
    Status:          Up
    VBoxNetworkName: HostInterfaceNetworking-vboxnet0
    
    }
    STDERR:
    {
    }
    Found: vboxnet0
    COMMAND: /usr/bin/VBoxManage showvminfo TestDocker --machinereadable
    STDOUT:
    {
    name="TestDocker"
    groups="/"
    ostype="Linux 2.6 / 3.x / 4.x (64-bit)"
    UUID="072db02c-c80c-4e8d-a76a-01aded656cd2"
    CfgFile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.vbox"
    SnapFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Snapshots"
    LogFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Logs"
    hardwareuuid="072db02c-c80c-4e8d-a76a-01aded656cd2"
    memory=2000
    pagefusion="off"
    vram=8
    cpuexecutioncap=100
    hpet="on"
    chipset="piix3"
    firmware="BIOS"
    cpus=2
    pae="on"
    longmode="on"
    triplefaultreset="off"
    apic="on"
    x2apic="off"
    cpuid-portability-level=0
    bootmenu="disabled"
    boot1="dvd"
    boot2="dvd"
    boot3="disk"
    boot4="none"
    acpi="on"
    ioapic="on"
    biosapic="apic"
    biossystemtimeoffset=0
    rtcuseutc="on"
    hwvirtex="on"
    nestedpaging="on"
    largepages="on"
    vtxvpid="on"
    vtxux="on"
    paravirtprovider="default"
    effparavirtprovider="kvm"
    VMState="running"
    VMStateChangeTime="2019-08-22T01:50:01.750000000"
    monitorcount=1
    accelerate3d="off"
    accelerate2dvideo="off"
    teleporterenabled="off"
    teleporterport=0
    teleporteraddress=""
    teleporterpassword=""
    tracing-enabled="off"
    tracing-allow-vm-access="off"
    tracing-config=""
    autostart-enabled="off"
    autostart-delay=0
    defaultfrontend=""
    storagecontrollername0="SATA"
    storagecontrollertype0="IntelAhci"
    storagecontrollerinstance0="0"
    storagecontrollermaxportcount0="30"
    storagecontrollerportcount0="30"
    storagecontrollerbootable0="on"
    "SATA-0-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/boot2docker.iso"
    "SATA-ImageUUID-0-0"="d0a429ea-f72a-423a-aaef-613177125889"
    "SATA-tempeject"="off"
    "SATA-IsEjected"="off"
    "SATA-1-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk"
    "SATA-ImageUUID-1-0"="4d2808a7-3d0e-4f46-b9f1-9e07b433a953"
    "SATA-2-0"="none"
    "SATA-3-0"="none"
    "SATA-4-0"="none"
    "SATA-5-0"="none"
    "SATA-6-0"="none"
    "SATA-7-0"="none"
    "SATA-8-0"="none"
    "SATA-9-0"="none"
    "SATA-10-0"="none"
    "SATA-11-0"="none"
    "SATA-12-0"="none"
    "SATA-13-0"="none"
    "SATA-14-0"="none"
    "SATA-15-0"="none"
    "SATA-16-0"="none"
    "SATA-17-0"="none"
    "SATA-18-0"="none"
    "SATA-19-0"="none"
    "SATA-20-0"="none"
    "SATA-21-0"="none"
    "SATA-22-0"="none"
    "SATA-23-0"="none"
    "SATA-24-0"="none"
    "SATA-25-0"="none"
    "SATA-26-0"="none"
    "SATA-27-0"="none"
    "SATA-28-0"="none"
    "SATA-29-0"="none"
    natnet1="nat"
    macaddress1="080027EEE7A1"
    cableconnected1="on"
    nic1="nat"
    nictype1="virtio"
    nicspeed1="0"
    mtu="0"
    sockSnd="64"
    sockRcv="64"
    tcpWndSnd="64"
    tcpWndRcv="64"
    Forwarding(0)="ssh,tcp,127.0.0.1,38133,,22"
    hostonlyadapter2="vboxnet0"
    macaddress2="080027B647FD"
    cableconnected2="on"
    nic2="hostonly"
    nictype2="virtio"
    nicspeed2="0"
    nic3="none"
    nic4="none"
    nic5="none"
    nic6="none"
    nic7="none"
    nic8="none"
    hidpointing="ps2mouse"
    hidkeyboard="ps2kbd"
    uart1="off"
    uart2="off"
    uart3="off"
    uart4="off"
    lpt1="off"
    lpt2="off"
    audio="alsa"
    audio_in="false"
    audio_out="false"
    clipboard="disabled"
    draganddrop="disabled"
    SessionName="headless"
    VideoMode="720,400,0"@0,0 1
    vrde="off"
    usb="off"
    ehci="off"
    xhci="off"
    SharedFolderNameMachineMapping1="hosthome"
    SharedFolderPathMachineMapping1="/home"
    VRDEActiveConnection="off"
    VRDEClients=0
    videocap="off"
    videocap_audio="off"
    videocapscreens=0
    videocapfile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.webm"
    videocapres=1024x768
    videocaprate=512
    videocapfps=25
    videocapopts=
    GuestMemoryBalloon=0
    GuestOSType="Linux26_64"
    GuestAdditionsRunLevel=2
    GuestAdditionsVersion="5.1.38 r122592"
    GuestAdditionsFacility_VirtualBox Base Driver=50,1566438711915
    GuestAdditionsFacility_VirtualBox System Service=50,1566438714740
    GuestAdditionsFacility_Seamless Mode=0,1566438711858
    GuestAdditionsFacility_Graphics Mode=0,1566438711857
    }
    STDERR:
    {
    }
    setting hostname "TestDocker"
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    sudo hostname TestDocker && echo "TestDocker" | sudo tee /etc/hostname
    SSH cmd err, output: <nil>: TestDocker
    
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    
    		if ! grep -xq '.*\sTestDocker' /etc/hosts; then
    			if grep -xq '127.0.1.1\s.*' /etc/hosts; then
    				sudo sed -i 's/^127.0.1.1\s.*/127.0.1.1 TestDocker/g' /etc/hosts;
    			else 
    				echo '127.0.1.1 TestDocker' | sudo tee -a /etc/hosts; 
    			fi
    		fi
    SSH cmd err, output: <nil>: 
    set auth options {CertDir:/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube CaCertPath:/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/certs/ca.pem CaPrivateKeyPath:/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/certs/ca-key.pem CaCertRemotePath:/etc/docker/ca.pem ServerCertPath:/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/server.pem ServerKeyPath:/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/server-key.pem ClientKeyPath:/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/certs/key.pem ServerCertRemotePath:/etc/docker/server.pem ServerKeyRemotePath:/etc/docker/server-key.pem ClientCertPath:/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/certs/cert.pem ServerCertSANs:[] StorePath:/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube}
    setting up certificates
    COMMAND: /usr/bin/VBoxManage showvminfo TestDocker --machinereadable
    STDOUT:
    {
    name="TestDocker"
    groups="/"
    ostype="Linux 2.6 / 3.x / 4.x (64-bit)"
    UUID="072db02c-c80c-4e8d-a76a-01aded656cd2"
    CfgFile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.vbox"
    SnapFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Snapshots"
    LogFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Logs"
    hardwareuuid="072db02c-c80c-4e8d-a76a-01aded656cd2"
    memory=2000
    pagefusion="off"
    vram=8
    cpuexecutioncap=100
    hpet="on"
    chipset="piix3"
    firmware="BIOS"
    cpus=2
    pae="on"
    longmode="on"
    triplefaultreset="off"
    apic="on"
    x2apic="off"
    cpuid-portability-level=0
    bootmenu="disabled"
    boot1="dvd"
    boot2="dvd"
    boot3="disk"
    boot4="none"
    acpi="on"
    ioapic="on"
    biosapic="apic"
    biossystemtimeoffset=0
    rtcuseutc="on"
    hwvirtex="on"
    nestedpaging="on"
    largepages="on"
    vtxvpid="on"
    vtxux="on"
    paravirtprovider="default"
    effparavirtprovider="kvm"
    VMState="running"
    VMStateChangeTime="2019-08-22T01:50:01.750000000"
    monitorcount=1
    accelerate3d="off"
    accelerate2dvideo="off"
    teleporterenabled="off"
    teleporterport=0
    teleporteraddress=""
    teleporterpassword=""
    tracing-enabled="off"
    tracing-allow-vm-access="off"
    tracing-config=""
    autostart-enabled="off"
    autostart-delay=0
    defaultfrontend=""
    storagecontrollername0="SATA"
    storagecontrollertype0="IntelAhci"
    storagecontrollerinstance0="0"
    storagecontrollermaxportcount0="30"
    storagecontrollerportcount0="30"
    storagecontrollerbootable0="on"
    "SATA-0-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/boot2docker.iso"
    "SATA-ImageUUID-0-0"="d0a429ea-f72a-423a-aaef-613177125889"
    "SATA-tempeject"="off"
    "SATA-IsEjected"="off"
    "SATA-1-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk"
    "SATA-ImageUUID-1-0"="4d2808a7-3d0e-4f46-b9f1-9e07b433a953"
    "SATA-2-0"="none"
    "SATA-3-0"="none"
    "SATA-4-0"="none"
    "SATA-5-0"="none"
    "SATA-6-0"="none"
    "SATA-7-0"="none"
    "SATA-8-0"="none"
    "SATA-9-0"="none"
    "SATA-10-0"="none"
    "SATA-11-0"="none"
    "SATA-12-0"="none"
    "SATA-13-0"="none"
    "SATA-14-0"="none"
    "SATA-15-0"="none"
    "SATA-16-0"="none"
    "SATA-17-0"="none"
    "SATA-18-0"="none"
    "SATA-19-0"="none"
    "SATA-20-0"="none"
    "SATA-21-0"="none"
    "SATA-22-0"="none"
    "SATA-23-0"="none"
    "SATA-24-0"="none"
    "SATA-25-0"="none"
    "SATA-26-0"="none"
    "SATA-27-0"="none"
    "SATA-28-0"="none"
    "SATA-29-0"="none"
    natnet1="nat"
    macaddress1="080027EEE7A1"
    cableconnected1="on"
    nic1="nat"
    nictype1="virtio"
    nicspeed1="0"
    mtu="0"
    sockSnd="64"
    sockRcv="64"
    tcpWndSnd="64"
    tcpWndRcv="64"
    Forwarding(0)="ssh,tcp,127.0.0.1,38133,,22"
    hostonlyadapter2="vboxnet0"
    macaddress2="080027B647FD"
    cableconnected2="on"
    nic2="hostonly"
    nictype2="virtio"
    nicspeed2="0"
    nic3="none"
    nic4="none"
    nic5="none"
    nic6="none"
    nic7="none"
    nic8="none"
    hidpointing="ps2mouse"
    hidkeyboard="ps2kbd"
    uart1="off"
    uart2="off"
    uart3="off"
    uart4="off"
    lpt1="off"
    lpt2="off"
    audio="alsa"
    audio_in="false"
    audio_out="false"
    clipboard="disabled"
    draganddrop="disabled"
    SessionName="headless"
    VideoMode="720,400,0"@0,0 1
    vrde="off"
    usb="off"
    ehci="off"
    xhci="off"
    SharedFolderNameMachineMapping1="hosthome"
    SharedFolderPathMachineMapping1="/home"
    VRDEActiveConnection="off"
    VRDEClients=0
    videocap="off"
    videocap_audio="off"
    videocapscreens=0
    videocapfile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.webm"
    videocapres=1024x768
    videocaprate=512
    videocapfps=25
    videocapopts=
    GuestMemoryBalloon=0
    GuestOSType="Linux26_64"
    GuestAdditionsRunLevel=2
    GuestAdditionsVersion="5.1.38 r122592"
    GuestAdditionsFacility_VirtualBox Base Driver=50,1566438711915
    GuestAdditionsFacility_VirtualBox System Service=50,1566438714740
    GuestAdditionsFacility_Seamless Mode=0,1566438711858
    GuestAdditionsFacility_Graphics Mode=0,1566438711857
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage showvminfo TestDocker --machinereadable
    STDOUT:
    {
    name="TestDocker"
    groups="/"
    ostype="Linux 2.6 / 3.x / 4.x (64-bit)"
    UUID="072db02c-c80c-4e8d-a76a-01aded656cd2"
    CfgFile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.vbox"
    SnapFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Snapshots"
    LogFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Logs"
    hardwareuuid="072db02c-c80c-4e8d-a76a-01aded656cd2"
    memory=2000
    pagefusion="off"
    vram=8
    cpuexecutioncap=100
    hpet="on"
    chipset="piix3"
    firmware="BIOS"
    cpus=2
    pae="on"
    longmode="on"
    triplefaultreset="off"
    apic="on"
    x2apic="off"
    cpuid-portability-level=0
    bootmenu="disabled"
    boot1="dvd"
    boot2="dvd"
    boot3="disk"
    boot4="none"
    acpi="on"
    ioapic="on"
    biosapic="apic"
    biossystemtimeoffset=0
    rtcuseutc="on"
    hwvirtex="on"
    nestedpaging="on"
    largepages="on"
    vtxvpid="on"
    vtxux="on"
    paravirtprovider="default"
    effparavirtprovider="kvm"
    VMState="running"
    VMStateChangeTime="2019-08-22T01:50:01.750000000"
    monitorcount=1
    accelerate3d="off"
    accelerate2dvideo="off"
    teleporterenabled="off"
    teleporterport=0
    teleporteraddress=""
    teleporterpassword=""
    tracing-enabled="off"
    tracing-allow-vm-access="off"
    tracing-config=""
    autostart-enabled="off"
    autostart-delay=0
    defaultfrontend=""
    storagecontrollername0="SATA"
    storagecontrollertype0="IntelAhci"
    storagecontrollerinstance0="0"
    storagecontrollermaxportcount0="30"
    storagecontrollerportcount0="30"
    storagecontrollerbootable0="on"
    "SATA-0-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/boot2docker.iso"
    "SATA-ImageUUID-0-0"="d0a429ea-f72a-423a-aaef-613177125889"
    "SATA-tempeject"="off"
    "SATA-IsEjected"="off"
    "SATA-1-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk"
    "SATA-ImageUUID-1-0"="4d2808a7-3d0e-4f46-b9f1-9e07b433a953"
    "SATA-2-0"="none"
    "SATA-3-0"="none"
    "SATA-4-0"="none"
    "SATA-5-0"="none"
    "SATA-6-0"="none"
    "SATA-7-0"="none"
    "SATA-8-0"="none"
    "SATA-9-0"="none"
    "SATA-10-0"="none"
    "SATA-11-0"="none"
    "SATA-12-0"="none"
    "SATA-13-0"="none"
    "SATA-14-0"="none"
    "SATA-15-0"="none"
    "SATA-16-0"="none"
    "SATA-17-0"="none"
    "SATA-18-0"="none"
    "SATA-19-0"="none"
    "SATA-20-0"="none"
    "SATA-21-0"="none"
    "SATA-22-0"="none"
    "SATA-23-0"="none"
    "SATA-24-0"="none"
    "SATA-25-0"="none"
    "SATA-26-0"="none"
    "SATA-27-0"="none"
    "SATA-28-0"="none"
    "SATA-29-0"="none"
    natnet1="nat"
    macaddress1="080027EEE7A1"
    cableconnected1="on"
    nic1="nat"
    nictype1="virtio"
    nicspeed1="0"
    mtu="0"
    sockSnd="64"
    sockRcv="64"
    tcpWndSnd="64"
    tcpWndRcv="64"
    Forwarding(0)="ssh,tcp,127.0.0.1,38133,,22"
    hostonlyadapter2="vboxnet0"
    macaddress2="080027B647FD"
    cableconnected2="on"
    nic2="hostonly"
    nictype2="virtio"
    nicspeed2="0"
    nic3="none"
    nic4="none"
    nic5="none"
    nic6="none"
    nic7="none"
    nic8="none"
    hidpointing="ps2mouse"
    hidkeyboard="ps2kbd"
    uart1="off"
    uart2="off"
    uart3="off"
    uart4="off"
    lpt1="off"
    lpt2="off"
    audio="alsa"
    audio_in="false"
    audio_out="false"
    clipboard="disabled"
    draganddrop="disabled"
    SessionName="headless"
    VideoMode="720,400,0"@0,0 1
    vrde="off"
    usb="off"
    ehci="off"
    xhci="off"
    SharedFolderNameMachineMapping1="hosthome"
    SharedFolderPathMachineMapping1="/home"
    VRDEActiveConnection="off"
    VRDEClients=0
    videocap="off"
    videocap_audio="off"
    videocapscreens=0
    videocapfile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.webm"
    videocapres=1024x768
    videocaprate=512
    videocapfps=25
    videocapopts=
    GuestMemoryBalloon=0
    GuestOSType="Linux26_64"
    GuestAdditionsRunLevel=2
    GuestAdditionsVersion="5.1.38 r122592"
    GuestAdditionsFacility_VirtualBox Base Driver=50,1566438711915
    GuestAdditionsFacility_VirtualBox System Service=50,1566438714740
    GuestAdditionsFacility_Seamless Mode=0,1566438711858
    GuestAdditionsFacility_Graphics Mode=0,1566438711857
    }
    STDERR:
    {
    }
    Host-only MAC: 080027b647fd
    
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    ip addr show
    SSH cmd err, output: <nil>: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host 
           valid_lft forever preferred_lft forever
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:ee:e7:a1 brd ff:ff:ff:ff:ff:ff
        inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
           valid_lft 86387sec preferred_lft 86387sec
        inet6 fe80::a00:27ff:feee:e7a1/64 scope link 
           valid_lft forever preferred_lft forever
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:b6:47:fd brd ff:ff:ff:ff:ff:ff
        inet 192.168.99.101/24 brd 192.168.99.255 scope global dynamic eth1
           valid_lft 1187sec preferred_lft 1187sec
        inet6 fe80::a00:27ff:feb6:47fd/64 scope link 
           valid_lft forever preferred_lft forever
    4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
        link/sit 0.0.0.0 brd 0.0.0.0
    
    SSH returned: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host 
           valid_lft forever preferred_lft forever
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:ee:e7:a1 brd ff:ff:ff:ff:ff:ff
        inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
           valid_lft 86387sec preferred_lft 86387sec
        inet6 fe80::a00:27ff:feee:e7a1/64 scope link 
           valid_lft forever preferred_lft forever
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:b6:47:fd brd ff:ff:ff:ff:ff:ff
        inet 192.168.99.101/24 brd 192.168.99.255 scope global dynamic eth1
           valid_lft 1187sec preferred_lft 1187sec
        inet6 fe80::a00:27ff:feb6:47fd/64 scope link 
           valid_lft forever preferred_lft forever
    4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
        link/sit 0.0.0.0 brd 0.0.0.0
    
    END SSH
    
    generating server cert: /home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/server.pem ca-key=/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/certs/ca.pem private-key=/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/certs/ca-key.pem org=jenkins.TestDocker san=[192.168.99.101 localhost]
    I0821 18:52:06.246592   20691 ssh_runner.go:101] SSH: sudo rm -f /etc/docker/ca.pem
    I0821 18:52:06.298006   20691 ssh_runner.go:101] SSH: sudo mkdir -p /etc/docker
    I0821 18:52:06.341788   20691 ssh_runner.go:182] Transferring 1038 bytes to ca.pem
    I0821 18:52:06.344495   20691 ssh_runner.go:195] ca.pem: copied 1038 bytes
    I0821 18:52:06.371871   20691 ssh_runner.go:101] SSH: sudo rm -f /etc/docker/server.pem
    I0821 18:52:06.427692   20691 ssh_runner.go:101] SSH: sudo mkdir -p /etc/docker
    I0821 18:52:06.483287   20691 ssh_runner.go:182] Transferring 1115 bytes to server.pem
    I0821 18:52:06.487764   20691 ssh_runner.go:195] server.pem: copied 1115 bytes
    I0821 18:52:06.519109   20691 ssh_runner.go:101] SSH: sudo rm -f /etc/docker/server-key.pem
    I0821 18:52:06.561468   20691 ssh_runner.go:101] SSH: sudo mkdir -p /etc/docker
    I0821 18:52:06.590351   20691 ssh_runner.go:182] Transferring 1679 bytes to server-key.pem
    I0821 18:52:06.594211   20691 ssh_runner.go:195] server-key.pem: copied 1679 bytes
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    sudo mkdir -p /lib/systemd/system && printf %s "[Unit]
    Description=Docker Application Container Engine
    Documentation=https://docs.docker.com
    After=network.target  minikube-automount.service docker.socket
    Requires= minikube-automount.service docker.socket 
    
    [Service]
    Type=notify
    
    # DOCKER_RAMDISK disables pivot_root in Docker, using MS_MOVE instead.
    Environment=DOCKER_RAMDISK=yes
    Environment="FOO=BAR"
    Environment="BAZ=BAT"
    
    
    # This file is a systemd drop-in unit that inherits from the base dockerd configuration.
    # The base configuration already specifies an 'ExecStart=...' command. The first directive
    # here is to clear out that command inherited from the base configuration. Without this,
    # the command from the base configuration and the command specified here are treated as
    # a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
    # will catch this invalid input and refuse to start the service with an error like:
    #  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
    ExecStart=
    ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=virtualbox --insecure-registry 10.96.0.0/12 --debug --icc=true 
    ExecReload=/bin/kill -s HUP $MAINPID
    
    # Having non-zero Limit*s causes performance problems due to accounting overhead
    # in the kernel. We recommend using cgroups to do container-local accounting.
    LimitNOFILE=infinity
    LimitNPROC=infinity
    LimitCORE=infinity
    
    # Uncomment TasksMax if your systemd version supports it.
    # Only systemd 226 and above support this version.
    TasksMax=infinity
    TimeoutStartSec=0
    
    # set delegate yes so that systemd does not reset the cgroups of docker containers
    Delegate=yes
    
    # kill only the docker process, not all processes in the cgroup
    KillMode=process
    
    [Install]
    WantedBy=multi-user.target
    " | sudo tee /lib/systemd/system/docker.service
    SSH cmd err, output: <nil>: [Unit]
    Description=Docker Application Container Engine
    Documentation=https://docs.docker.com
    After=network.target  minikube-automount.service docker.socket
    Requires= minikube-automount.service docker.socket 
    
    [Service]
    Type=notify
    
    # DOCKER_RAMDISK disables pivot_root in Docker, using MS_MOVE instead.
    Environment=DOCKER_RAMDISK=yes
    Environment=FOO=BAR
    Environment=BAZ=BAT
    
    
    # This file is a systemd drop-in unit that inherits from the base dockerd configuration.
    # The base configuration already specifies an 'ExecStart=...' command. The first directive
    # here is to clear out that command inherited from the base configuration. Without this,
    # the command from the base configuration and the command specified here are treated as
    # a sequence of commands, which is not the desired behavior, nor is it valid -- systemd
    # will catch this invalid input and refuse to start the service with an error like:
    #  Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services.
    ExecStart=
    ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock --tlsverify --tlscacert /etc/docker/ca.pem --tlscert /etc/docker/server.pem --tlskey /etc/docker/server-key.pem --label provider=virtualbox --insecure-registry 10.96.0.0/12 --debug --icc=true 
    ExecReload=/bin/kill -s HUP 
    
    # Having non-zero Limit*s causes performance problems due to accounting overhead
    # in the kernel. We recommend using cgroups to do container-local accounting.
    LimitNOFILE=infinity
    LimitNPROC=infinity
    LimitCORE=infinity
    
    # Uncomment TasksMax if your systemd version supports it.
    # Only systemd 226 and above support this version.
    TasksMax=infinity
    TimeoutStartSec=0
    
    # set delegate yes so that systemd does not reset the cgroups of docker containers
    Delegate=yes
    
    # kill only the docker process, not all processes in the cgroup
    KillMode=process
    
    [Install]
    WantedBy=multi-user.target
    
    setting minikube options for container-runtime
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    sudo mkdir -p /etc/sysconfig && printf %s "
    CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
    " | sudo tee /etc/sysconfig/crio.minikube
    SSH cmd err, output: <nil>: 
    CRIO_MINIKUBE_OPTIONS='--insecure-registry 10.96.0.0/12 '
    
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    sudo systemctl daemon-reload
    SSH cmd err, output: <nil>: 
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    sudo systemctl -f restart crio
    SSH cmd err, output: <nil>: 
    Getting to WaitForSSH function...
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    exit 0
    SSH cmd err, output: <nil>: 
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    cat /etc/os-release
    SSH cmd err, output: <nil>: NAME=Buildroot
    VERSION=2018.05.3
    ID=buildroot
    VERSION_ID=2018.05.3
    PRETTY_NAME="Buildroot 2018.05.3"
    
    found compatible host: buildroot
    I0821 18:52:10.381060   20691 cluster.go:411] Provisioned with Buildroot 2018.05.3
    COMMAND: /usr/bin/VBoxManage showvminfo TestDocker --machinereadable
    STDOUT:
    {
    name="TestDocker"
    groups="/"
    ostype="Linux 2.6 / 3.x / 4.x (64-bit)"
    UUID="072db02c-c80c-4e8d-a76a-01aded656cd2"
    CfgFile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.vbox"
    SnapFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Snapshots"
    LogFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Logs"
    hardwareuuid="072db02c-c80c-4e8d-a76a-01aded656cd2"
    memory=2000
    pagefusion="off"
    vram=8
    cpuexecutioncap=100
    hpet="on"
    chipset="piix3"
    firmware="BIOS"
    cpus=2
    pae="on"
    longmode="on"
    triplefaultreset="off"
    apic="on"
    x2apic="off"
    cpuid-portability-level=0
    bootmenu="disabled"
    boot1="dvd"
    boot2="dvd"
    boot3="disk"
    boot4="none"
    acpi="on"
    ioapic="on"
    biosapic="apic"
    biossystemtimeoffset=0
    rtcuseutc="on"
    hwvirtex="on"
    nestedpaging="on"
    largepages="on"
    vtxvpid="on"
    vtxux="on"
    paravirtprovider="default"
    effparavirtprovider="kvm"
    VMState="running"
    VMStateChangeTime="2019-08-22T01:50:01.750000000"
    monitorcount=1
    accelerate3d="off"
    accelerate2dvideo="off"
    teleporterenabled="off"
    teleporterport=0
    teleporteraddress=""
    teleporterpassword=""
    tracing-enabled="off"
    tracing-allow-vm-access="off"
    tracing-config=""
    autostart-enabled="off"
    autostart-delay=0
    defaultfrontend=""
    storagecontrollername0="SATA"
    storagecontrollertype0="IntelAhci"
    storagecontrollerinstance0="0"
    storagecontrollermaxportcount0="30"
    storagecontrollerportcount0="30"
    storagecontrollerbootable0="on"
    "SATA-0-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/boot2docker.iso"
    "SATA-ImageUUID-0-0"="d0a429ea-f72a-423a-aaef-613177125889"
    "SATA-tempeject"="off"
    "SATA-IsEjected"="off"
    "SATA-1-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk"
    "SATA-ImageUUID-1-0"="4d2808a7-3d0e-4f46-b9f1-9e07b433a953"
    "SATA-2-0"="none"
    "SATA-3-0"="none"
    "SATA-4-0"="none"
    "SATA-5-0"="none"
    "SATA-6-0"="none"
    "SATA-7-0"="none"
    "SATA-8-0"="none"
    "SATA-9-0"="none"
    "SATA-10-0"="none"
    "SATA-11-0"="none"
    "SATA-12-0"="none"
    "SATA-13-0"="none"
    "SATA-14-0"="none"
    "SATA-15-0"="none"
    "SATA-16-0"="none"
    "SATA-17-0"="none"
    "SATA-18-0"="none"
    "SATA-19-0"="none"
    "SATA-20-0"="none"
    "SATA-21-0"="none"
    "SATA-22-0"="none"
    "SATA-23-0"="none"
    "SATA-24-0"="none"
    "SATA-25-0"="none"
    "SATA-26-0"="none"
    "SATA-27-0"="none"
    "SATA-28-0"="none"
    "SATA-29-0"="none"
    natnet1="nat"
    macaddress1="080027EEE7A1"
    cableconnected1="on"
    nic1="nat"
    nictype1="virtio"
    nicspeed1="0"
    mtu="0"
    sockSnd="64"
    sockRcv="64"
    tcpWndSnd="64"
    tcpWndRcv="64"
    Forwarding(0)="ssh,tcp,127.0.0.1,38133,,22"
    hostonlyadapter2="vboxnet0"
    macaddress2="080027B647FD"
    cableconnected2="on"
    nic2="hostonly"
    nictype2="virtio"
    nicspeed2="0"
    nic3="none"
    nic4="none"
    nic5="none"
    nic6="none"
    nic7="none"
    nic8="none"
    hidpointing="ps2mouse"
    hidkeyboard="ps2kbd"
    uart1="off"
    uart2="off"
    uart3="off"
    uart4="off"
    lpt1="off"
    lpt2="off"
    audio="alsa"
    audio_in="false"
    audio_out="false"
    clipboard="disabled"
    draganddrop="disabled"
    SessionName="headless"
    VideoMode="720,400,0"@0,0 1
    vrde="off"
    usb="off"
    ehci="off"
    xhci="off"
    SharedFolderNameMachineMapping1="hosthome"
    SharedFolderPathMachineMapping1="/home"
    VRDEActiveConnection="off"
    VRDEClients=0
    videocap="off"
    videocap_audio="off"
    videocapscreens=0
    videocapfile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.webm"
    videocapres=1024x768
    videocaprate=512
    videocapfps=25
    videocapopts=
    GuestMemoryBalloon=0
    GuestOSType="Linux26_64"
    GuestAdditionsRunLevel=2
    GuestAdditionsVersion="5.1.38 r122592"
    GuestAdditionsFacility_VirtualBox Base Driver=50,1566438711915
    GuestAdditionsFacility_VirtualBox System Service=50,1566438714740
    GuestAdditionsFacility_Seamless Mode=0,1566438711858
    GuestAdditionsFacility_Graphics Mode=0,1566438711857
    }
    STDERR:
    {
    }
    COMMAND: /usr/bin/VBoxManage showvminfo TestDocker --machinereadable
    STDOUT:
    {
    name="TestDocker"
    groups="/"
    ostype="Linux 2.6 / 3.x / 4.x (64-bit)"
    UUID="072db02c-c80c-4e8d-a76a-01aded656cd2"
    CfgFile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.vbox"
    SnapFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Snapshots"
    LogFldr="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/Logs"
    hardwareuuid="072db02c-c80c-4e8d-a76a-01aded656cd2"
    memory=2000
    pagefusion="off"
    vram=8
    cpuexecutioncap=100
    hpet="on"
    chipset="piix3"
    firmware="BIOS"
    cpus=2
    pae="on"
    longmode="on"
    triplefaultreset="off"
    apic="on"
    x2apic="off"
    cpuid-portability-level=0
    bootmenu="disabled"
    boot1="dvd"
    boot2="dvd"
    boot3="disk"
    boot4="none"
    acpi="on"
    ioapic="on"
    biosapic="apic"
    biossystemtimeoffset=0
    rtcuseutc="on"
    hwvirtex="on"
    nestedpaging="on"
    largepages="on"
    vtxvpid="on"
    vtxux="on"
    paravirtprovider="default"
    effparavirtprovider="kvm"
    VMState="running"
    VMStateChangeTime="2019-08-22T01:50:01.750000000"
    monitorcount=1
    accelerate3d="off"
    accelerate2dvideo="off"
    teleporterenabled="off"
    teleporterport=0
    teleporteraddress=""
    teleporterpassword=""
    tracing-enabled="off"
    tracing-allow-vm-access="off"
    tracing-config=""
    autostart-enabled="off"
    autostart-delay=0
    defaultfrontend=""
    storagecontrollername0="SATA"
    storagecontrollertype0="IntelAhci"
    storagecontrollerinstance0="0"
    storagecontrollermaxportcount0="30"
    storagecontrollerportcount0="30"
    storagecontrollerbootable0="on"
    "SATA-0-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/boot2docker.iso"
    "SATA-ImageUUID-0-0"="d0a429ea-f72a-423a-aaef-613177125889"
    "SATA-tempeject"="off"
    "SATA-IsEjected"="off"
    "SATA-1-0"="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/disk.vmdk"
    "SATA-ImageUUID-1-0"="4d2808a7-3d0e-4f46-b9f1-9e07b433a953"
    "SATA-2-0"="none"
    "SATA-3-0"="none"
    "SATA-4-0"="none"
    "SATA-5-0"="none"
    "SATA-6-0"="none"
    "SATA-7-0"="none"
    "SATA-8-0"="none"
    "SATA-9-0"="none"
    "SATA-10-0"="none"
    "SATA-11-0"="none"
    "SATA-12-0"="none"
    "SATA-13-0"="none"
    "SATA-14-0"="none"
    "SATA-15-0"="none"
    "SATA-16-0"="none"
    "SATA-17-0"="none"
    "SATA-18-0"="none"
    "SATA-19-0"="none"
    "SATA-20-0"="none"
    "SATA-21-0"="none"
    "SATA-22-0"="none"
    "SATA-23-0"="none"
    "SATA-24-0"="none"
    "SATA-25-0"="none"
    "SATA-26-0"="none"
    "SATA-27-0"="none"
    "SATA-28-0"="none"
    "SATA-29-0"="none"
    natnet1="nat"
    macaddress1="080027EEE7A1"
    cableconnected1="on"
    nic1="nat"
    nictype1="virtio"
    nicspeed1="0"
    mtu="0"
    sockSnd="64"
    sockRcv="64"
    tcpWndSnd="64"
    tcpWndRcv="64"
    Forwarding(0)="ssh,tcp,127.0.0.1,38133,,22"
    hostonlyadapter2="vboxnet0"
    macaddress2="080027B647FD"
    cableconnected2="on"
    nic2="hostonly"
    nictype2="virtio"
    nicspeed2="0"
    nic3="none"
    nic4="none"
    nic5="none"
    nic6="none"
    nic7="none"
    nic8="none"
    hidpointing="ps2mouse"
    hidkeyboard="ps2kbd"
    uart1="off"
    uart2="off"
    uart3="off"
    uart4="off"
    lpt1="off"
    lpt2="off"
    audio="alsa"
    audio_in="false"
    audio_out="false"
    clipboard="disabled"
    draganddrop="disabled"
    SessionName="headless"
    VideoMode="720,400,0"@0,0 1
    vrde="off"
    usb="off"
    ehci="off"
    xhci="off"
    SharedFolderNameMachineMapping1="hosthome"
    SharedFolderPathMachineMapping1="/home"
    VRDEActiveConnection="off"
    VRDEClients=0
    videocap="off"
    videocap_audio="off"
    videocapscreens=0
    videocapfile="/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/machines/TestDocker/TestDocker/TestDocker.webm"
    videocapres=1024x768
    videocaprate=512
    videocapfps=25
    videocapopts=
    GuestMemoryBalloon=0
    GuestOSType="Linux26_64"
    GuestAdditionsRunLevel=2
    GuestAdditionsVersion="5.1.38 r122592"
    GuestAdditionsFacility_VirtualBox Base Driver=50,1566438711915
    GuestAdditionsFacility_VirtualBox System Service=50,1566438714740
    GuestAdditionsFacility_Seamless Mode=0,1566438711858
    GuestAdditionsFacility_Graphics Mode=0,1566438711857
    }
    STDERR:
    {
    }
    Host-only MAC: 080027b647fd
    
    Using SSH client type: native
    &{{{<nil> 0 [] [] []} docker [0xe9b410] 0xe9b3e0 <nil>  [] 0s} 127.0.0.1 38133 <nil> <nil>}
    About to run SSH command:
    ip addr show
    SSH cmd err, output: <nil>: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host 
           valid_lft forever preferred_lft forever
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:ee:e7:a1 brd ff:ff:ff:ff:ff:ff
        inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
           valid_lft 86382sec preferred_lft 86382sec
        inet6 fe80::a00:27ff:feee:e7a1/64 scope link 
           valid_lft forever preferred_lft forever
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:b6:47:fd brd ff:ff:ff:ff:ff:ff
        inet 192.168.99.101/24 brd 192.168.99.255 scope global dynamic eth1
           valid_lft 1182sec preferred_lft 1182sec
        inet6 fe80::a00:27ff:feb6:47fd/64 scope link 
           valid_lft forever preferred_lft forever
    4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
        link/sit 0.0.0.0 brd 0.0.0.0
    
    SSH returned: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
        inet 127.0.0.1/8 scope host lo
           valid_lft forever preferred_lft forever
        inet6 ::1/128 scope host 
           valid_lft forever preferred_lft forever
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:ee:e7:a1 brd ff:ff:ff:ff:ff:ff
        inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic eth0
           valid_lft 86382sec preferred_lft 86382sec
        inet6 fe80::a00:27ff:feee:e7a1/64 scope link 
           valid_lft forever preferred_lft forever
    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
        link/ether 08:00:27:b6:47:fd brd ff:ff:ff:ff:ff:ff
        inet 192.168.99.101/24 brd 192.168.99.255 scope global dynamic eth1
           valid_lft 1182sec preferred_lft 1182sec
        inet6 fe80::a00:27ff:feb6:47fd/64 scope link 
           valid_lft forever preferred_lft forever
    4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
        link/sit 0.0.0.0 brd 0.0.0.0
    
    END SSH
    
    I0821 18:52:11.071598   20691 profile.go:65] Saving config:
    {
        "MachineConfig": {
            "KeepContext": false,
            "EmbedCerts": false,
            "MinikubeISO": "https://storage.googleapis.com/minikube/iso/minikube-v1.3.0.iso",
            "Memory": 2000,
            "CPUs": 2,
            "DiskSize": 20000,
            "VMDriver": "virtualbox",
            "ContainerRuntime": "docker",
            "HyperkitVpnKitSock": "",
            "HyperkitVSockPorts": [],
            "DockerEnv": [
                "FOO=BAR",
                "BAZ=BAT"
            ],
            "InsecureRegistry": null,
            "RegistryMirror": null,
            "HostOnlyCIDR": "192.168.99.1/24",
            "HypervVirtualSwitch": "",
            "KVMNetwork": "default",
            "KVMQemuURI": "qemu:///system",
            "KVMGPU": false,
            "KVMHidden": false,
            "DockerOpt": [
                "debug",
                "icc=true"
            ],
            "DisableDriverMounts": false,
            "NFSShare": [],
            "NFSSharesRoot": "/nfsshares",
            "UUID": "",
            "NoVTXCheck": false,
            "DNSProxy": false,
            "HostDNSResolver": true
        },
        "KubernetesConfig": {
            "KubernetesVersion": "v1.15.2",
            "NodeIP": "192.168.99.101",
            "NodePort": 8443,
            "NodeName": "minikube",
            "APIServerName": "minikubeCA",
            "APIServerNames": null,
            "APIServerIPs": null,
            "DNSDomain": "cluster.local",
            "ContainerRuntime": "docker",
            "CRISocket": "",
            "NetworkPlugin": "",
            "FeatureGates": "",
            "ServiceCIDR": "10.96.0.0/12",
            "ImageRepository": "",
            "ExtraOptions": null,
            "ShouldLoadCachedImages": true,
            "EnableDefaultCNI": false
        }
    }
    I0821 18:52:11.071762   20691 lock.go:33] attempting to write to file "/home/jenkins/minikube-integration/linux-amd64-virtualbox-5150-15375-7c9c4ac0f950def8ee4d4a04a369661b081478c5/.minikube/profiles/TestDocker/config.json.tmp715451592" with filemode -rw-------
    I0821 18:52:11.192303   20691 ssh_runner.go:101] SSH: systemctl is-active --quiet service containerd
    I0821 18:52:11.269669   20691 ssh_runner.go:101] SSH: systemctl is-active --quiet service crio
    I0821 18:52:11.323829   20691 ssh_runner.go:101] SSH: sudo systemctl stop crio
    I0821 18:52:11.468022   20691 ssh_runner.go:101] SSH: systemctl is-active --quiet service crio
    I0821 18:52:11.515910   20691 ssh_runner.go:101] SSH: sudo systemctl start docker
    I0821 18:52:11.983797   20691 utils.go:167] ! Job for docker.service failed because the control process exited with error code.
    I0821 18:52:11.983884   20691 utils.go:167] ! See "systemctl status docker.service" and "journalctl -xe" for details.
    W0821 18:52:11.989283   20691 exit.go:99] Failed to enable container runtime: command failed: sudo systemctl start docker
    stdout: 
    stderr: Job for docker.service failed because the control process exited with error code.
    See "systemctl status docker.service" and "journalctl -xe" for details.
    : Process exited with status 1
    * 
    X Failed to enable container runtime: command failed: sudo systemctl start docker
    stdout: 
    stderr: Job for docker.service failed because the control process exited with error code.
    See "systemctl status docker.service" and "journalctl -xe" for details.
    : Process exited with status 1
    * 
    * Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
      - https://github.com/kubernetes/minikube/issues/new/choose
     
    	 <------------ End of Start (TestDocker) log block

@medyagh medyagh merged commit 73c0d9c into kubernetes:master Aug 22, 2019
@medyagh medyagh mentioned this pull request Aug 22, 2019
Closed
@medyagh
Copy link
Member Author

medyagh commented Aug 22, 2019

flake is documented here #5174

@medyagh medyagh deleted the intg_avoid_retry_start branch November 6, 2019 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tstromberg tstromberg tstromberg approved these changes

@sharifelgamal sharifelgamal Awaiting requested review from sharifelgamal

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@medyagh @k8s-ci-robot @tstromberg @TravisBuddy