Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error starting cluster: timed out waiting to elevate kube-system RBAC privileges: Temporary Error: creating clusterrolebinding #3747

Closed
michaeljeffrey opened this issue Feb 25, 2019 · 4 comments
Closed

Error starting cluster: timed out waiting to elevate kube-system RBAC privileges: Temporary Error: creating clusterrolebinding #3747

michaeljeffrey opened this issue Feb 25, 2019 · 4 comments

Comments

@michaeljeffrey
Copy link

I'm getting the following error on a Mac (Mojave - 10.14.3).

I tried looking at folks reported the same error, but their fixes didn't work for me.

$ minikube start
😄  minikube v0.34.1 on darwin (amd64)
🔥  Creating virtualbox VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...
💿  Downloading Minikube ISO ...
 184.30 MB / 184.30 MB [============================================] 100.00% 0s
📶  "minikube" IP address is 192.168.99.112
🐳  Configuring Docker as the container runtime ...
✨  Preparing Kubernetes environment ...
💾  Downloading kubeadm v1.13.3
💾  Downloading kubelet v1.13.3
🚜  Pulling images required by Kubernetes v1.13.3 ...
🚀  Launching Kubernetes v1.13.3 using kubeadm ...
🔑  Configuring cluster permissions ...
💣  Error starting cluster: timed out waiting to elevate kube-system RBAC privileges: Temporary Error: creating clusterrolebinding: Post https://192.168.99.112:8443/apis/rbac.authorization.k8s.io/v1beta1/clusterrolebindings?timeout=1m0s: dial tcp 192.168.99.112:8443: i/o timeout
creating clusterrolebinding: Post https://192.168.99.112:8443/apis/rbac.authorization.k8s.io/v1beta1/clusterrolebindings?timeout=1m0s: dial tcp 192.168.99.112:8443: connect: connection refused

😿  Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉  https://github.com/kubernetes/minikube/issues/new
$

I've tried reinstalling minikube and virtualbox. I've tried 'minikube delete' too. Even running the start as sudo failed.

Logs show this:

$ minikube logs
==> k8s_coredns_coredns <==
E0225 04:55:52.989978       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:311: Failed to list *v1.Service: Get https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: connect: connection refused
E0225 04:55:52.990054       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:318: Failed to list *v1.Namespace: Get https://10.96.0.1:443/api/v1/namespaces?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: connect: connection refused
E0225 04:55:52.989982       1 reflector.go:205] github.com/coredns/coredns/plugin/kubernetes/controller.go:313: Failed to list *v1.Endpoints: Get https://10.96.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0: dial tcp 10.96.0.1:443: connect: connection refused
.:53
2019-02-25T04:55:54.12Z [INFO] CoreDNS-1.2.6
2019-02-25T04:55:54.12Z [INFO] linux/amd64, go1.11.2, 756749c
CoreDNS-1.2.6
linux/amd64, go1.11.2, 756749c
 [INFO] plugin/reload: Running configuration MD5 = f65c4821c8a9b7b5eb30fa4fbc167769
==> k8s_kube-apiserver <==
I0225 04:55:41.787184       1 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:replicaset-controller
I0225 04:55:41.826903       1 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:replication-controller
I0225 04:55:41.875706       1 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:resourcequota-controller
I0225 04:55:41.907197       1 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:route-controller
I0225 04:55:41.947158       1 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:service-account-controller
I0225 04:55:41.987656       1 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:service-controller
I0225 04:55:42.027422       1 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:statefulset-controller
I0225 04:55:42.067832       1 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:ttl-controller
I0225 04:55:42.108297       1 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:certificate-controller
I0225 04:55:42.148392       1 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:pvc-protection-controller
I0225 04:55:42.188597       1 storage_rbac.go:215] created clusterrolebinding.rbac.authorization.k8s.io/system:controller:pv-protection-controller
I0225 04:55:42.224799       1 controller.go:608] quota admission added evaluator for: roles.rbac.authorization.k8s.io
I0225 04:55:42.230947       1 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-public
I0225 04:55:42.249274       1 controller.go:608] quota admission added evaluator for: endpoints
I0225 04:55:42.267954       1 storage_rbac.go:246] created role.rbac.authorization.k8s.io/extension-apiserver-authentication-reader in kube-system
I0225 04:55:42.309608       1 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-system
I0225 04:55:42.347266       1 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system:controller:cloud-provider in kube-system
I0225 04:55:42.388173       1 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system:controller:token-cleaner in kube-system
I0225 04:55:42.426848       1 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system::leader-locking-kube-controller-manager in kube-system
I0225 04:55:42.469705       1 storage_rbac.go:246] created role.rbac.authorization.k8s.io/system::leader-locking-kube-scheduler in kube-system
I0225 04:55:42.508355       1 controller.go:608] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
I0225 04:55:42.515682       1 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-controller-manager in kube-system
I0225 04:55:42.546726       1 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-scheduler in kube-system
I0225 04:55:42.587473       1 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-system
I0225 04:55:42.627837       1 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system:controller:cloud-provider in kube-system
I0225 04:55:42.667538       1 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system:controller:token-cleaner in kube-system
I0225 04:55:42.708103       1 storage_rbac.go:276] created rolebinding.rbac.authorization.k8s.io/system:controller:bootstrap-signer in kube-public
I0225 04:55:43.522330       1 trace.go:76] Trace[106423851]: "Create /api/v1/namespaces/default/services" (started: 2019-02-25 04:55:39.513268616 +0000 UTC m=+13.915867861) (total time: 4.009030613s):
Trace[106423851]: [4.002967585s] [4.00290333s] About to store object in database
W0225 04:55:43.529638       1 lease.go:222] Resetting endpoints for master service "kubernetes" to [192.168.99.112]
I0225 04:55:43.570409       1 trace.go:76] Trace[617848521]: "Create /apis/apiregistration.k8s.io/v1/apiservices" (started: 2019-02-25 04:55:39.566052557 +0000 UTC m=+13.968651791) (total time: 4.004325836s):
Trace[617848521]: [4.001990817s] [4.001953663s] About to store object in database
I0225 04:55:43.575630       1 trace.go:76] Trace[1729277520]: "Create /apis/apiregistration.k8s.io/v1/apiservices" (started: 2019-02-25 04:55:39.570341989 +0000 UTC m=+13.972941223) (total time: 4.005269356s):
Trace[1729277520]: [4.00129373s] [4.001269375s] About to store object in database
I0225 04:55:43.576008       1 trace.go:76] Trace[575056517]: "Create /apis/apiregistration.k8s.io/v1/apiservices" (started: 2019-02-25 04:55:39.569990865 +0000 UTC m=+13.972590100) (total time: 4.006004124s):
Trace[575056517]: [4.002108957s] [4.002063156s] About to store object in database
I0225 04:55:43.576174       1 trace.go:76] Trace[228325463]: "Create /apis/apiregistration.k8s.io/v1/apiservices" (started: 2019-02-25 04:55:39.570429412 +0000 UTC m=+13.973028646) (total time: 4.005734081s):
Trace[228325463]: [4.001538793s] [4.001515463s] About to store object in database
I0225 04:55:43.580637       1 trace.go:76] Trace[22586870]: "Create /apis/apiregistration.k8s.io/v1/apiservices" (started: 2019-02-25 04:55:39.570247355 +0000 UTC m=+13.972846589) (total time: 4.01037251s):
Trace[22586870]: [4.00158269s] [4.00155818s] About to store object in database
I0225 04:55:43.649169       1 trace.go:76] Trace[1804269007]: "Create /api/v1/namespaces/default/events" (started: 2019-02-25 04:55:39.644856897 +0000 UTC m=+14.047456130) (total time: 4.004281814s):
Trace[1804269007]: [4.001018731s] [4.000746174s] About to store object in database
I0225 04:55:44.233655       1 trace.go:76] Trace[1196739634]: "Create /api/v1/namespaces" (started: 2019-02-25 04:55:40.226551947 +0000 UTC m=+14.629151180) (total time: 4.007077972s):
Trace[1196739634]: [4.003077383s] [4.002307263s] About to store object in database
I0225 04:55:44.293448       1 controller.go:608] quota admission added evaluator for: serviceaccounts
I0225 04:55:44.836050       1 controller.go:608] quota admission added evaluator for: deployments.apps
I0225 04:55:44.880423       1 controller.go:608] quota admission added evaluator for: daemonsets.apps
I0225 04:55:50.832265       1 controller.go:608] quota admission added evaluator for: replicasets.apps
I0225 04:55:51.186295       1 controller.go:608] quota admission added evaluator for: controllerrevisions.apps
I0225 04:55:54.438026       1 controller.go:608] quota admission added evaluator for: namespaces
==> k8s_kube-scheduler <==
E0225 04:55:36.499445       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0225 04:55:36.501008       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0225 04:55:36.501034       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0225 04:55:36.501053       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0225 04:55:36.501072       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0225 04:55:36.501924       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0225 04:55:37.481575       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0225 04:55:37.488260       1 reflector.go:134] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:232: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0225 04:55:37.494887       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0225 04:55:37.494934       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0225 04:55:37.500643       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0225 04:55:37.503827       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0225 04:55:37.503877       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0225 04:55:37.506119       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0225 04:55:37.509040       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0225 04:55:37.509966       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0225 04:55:38.483134       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0225 04:55:38.489566       1 reflector.go:134] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:232: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0225 04:55:38.497104       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0225 04:55:38.497579       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0225 04:55:38.501979       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0225 04:55:38.504754       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0225 04:55:38.506272       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0225 04:55:38.508186       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0225 04:55:38.509917       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0225 04:55:38.510834       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0225 04:55:39.484304       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0225 04:55:39.490488       1 reflector.go:134] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:232: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0225 04:55:39.498049       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0225 04:55:39.499316       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0225 04:55:39.502607       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0225 04:55:39.505400       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0225 04:55:39.507276       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0225 04:55:39.510003       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0225 04:55:39.510631       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0225 04:55:39.512077       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
E0225 04:55:40.485362       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.ReplicationController: replicationcontrollers is forbidden: User "system:kube-scheduler" cannot list resource "replicationcontrollers" in API group "" at the cluster scope
E0225 04:55:40.491660       1 reflector.go:134] k8s.io/kubernetes/cmd/kube-scheduler/app/server.go:232: Failed to list *v1.Pod: pods is forbidden: User "system:kube-scheduler" cannot list resource "pods" in API group "" at the cluster scope
E0225 04:55:40.500273       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope
E0225 04:55:40.501152       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.ReplicaSet: replicasets.apps is forbidden: User "system:kube-scheduler" cannot list resource "replicasets" in API group "apps" at the cluster scope
E0225 04:55:40.503930       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolume: persistentvolumes is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumes" in API group "" at the cluster scope
E0225 04:55:40.508263       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StorageClass: storageclasses.storage.k8s.io is forbidden: User "system:kube-scheduler" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope
E0225 04:55:40.509231       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.StatefulSet: statefulsets.apps is forbidden: User "system:kube-scheduler" cannot list resource "statefulsets" in API group "apps" at the cluster scope
E0225 04:55:40.512375       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.PersistentVolumeClaim: persistentvolumeclaims is forbidden: User "system:kube-scheduler" cannot list resource "persistentvolumeclaims" in API group "" at the cluster scope
E0225 04:55:40.512523       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope
E0225 04:55:40.513427       1 reflector.go:134] k8s.io/client-go/informers/factory.go:132: Failed to list *v1beta1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User "system:kube-scheduler" cannot list resource "poddisruptionbudgets" in API group "policy" at the cluster scope
I0225 04:55:42.371961       1 controller_utils.go:1027] Waiting for caches to sync for scheduler controller
I0225 04:55:42.472205       1 controller_utils.go:1034] Caches are synced for scheduler controller
I0225 04:55:42.472241       1 leaderelection.go:205] attempting to acquire leader lease  kube-system/kube-scheduler...
I0225 04:55:42.482765       1 leaderelection.go:214] successfully acquired lease kube-system/kube-scheduler
==> kubelet <==
-- Logs begin at Mon 2019-02-25 04:47:06 UTC, end at Mon 2019-02-25 05:03:29 UTC. --
Feb 25 04:55:36 minikube kubelet[3332]: E0225 04:55:36.508280    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:36 minikube kubelet[3332]: E0225 04:55:36.608723    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:36 minikube kubelet[3332]: E0225 04:55:36.708899    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:36 minikube kubelet[3332]: E0225 04:55:36.809402    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:36 minikube kubelet[3332]: E0225 04:55:36.900673    3332 event.go:203] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"minikube.1586826c8d8874c7", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"minikube", UID:"minikube", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node minikube status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"minikube"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f18ef1cc7, ext:193043398, loc:(*time.Location)(0x71d3440)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f1af5fbbe, ext:227048134, loc:(*time.Location)(0x71d3440)}}, Count:3, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Feb 25 04:55:36 minikube kubelet[3332]: E0225 04:55:36.909746    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.009985    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.110388    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.212213    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.299168    3332 event.go:203] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"minikube.1586826c8d8874c7", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"minikube", UID:"minikube", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node minikube status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"minikube"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f18ef1cc7, ext:193043398, loc:(*time.Location)(0x71d3440)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f1b30d9d0, ext:230906065, loc:(*time.Location)(0x71d3440)}}, Count:4, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.312534    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.412702    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.513645    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.616157    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.698226    3332 event.go:203] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"minikube.1586826c8d8857c0", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"minikube", UID:"minikube", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node minikube status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"minikube"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f18eeffc0, ext:193035974, loc:(*time.Location)(0x71d3440)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f1b30bc69, ext:230898537, loc:(*time.Location)(0x71d3440)}}, Count:4, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.716650    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.817412    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:37 minikube kubelet[3332]: E0225 04:55:37.922847    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.023305    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.099471    3332 event.go:203] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"minikube.1586826c8d886a2f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"minikube", UID:"minikube", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node minikube status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"minikube"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f18ef122f, ext:193040686, loc:(*time.Location)(0x71d3440)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f1b30ca18, ext:230902041, loc:(*time.Location)(0x71d3440)}}, Count:4, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.124296    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.224899    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.325149    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.425386    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.498780    3332 event.go:203] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"minikube.1586826c8d886a2f", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"minikube", UID:"minikube", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasNoDiskPressure", Message:"Node minikube status is now: NodeHasNoDiskPressure", Source:v1.EventSource{Component:"kubelet", Host:"minikube"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f18ef122f, ext:193040686, loc:(*time.Location)(0x71d3440)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f1b28bda9, ext:230374574, loc:(*time.Location)(0x71d3440)}}, Count:5, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.526470    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.627301    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.727799    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.828183    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.897630    3332 event.go:203] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"minikube.1586826c8d8857c0", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"minikube", UID:"minikube", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientMemory", Message:"Node minikube status is now: NodeHasSufficientMemory", Source:v1.EventSource{Component:"kubelet", Host:"minikube"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f18eeffc0, ext:193035974, loc:(*time.Location)(0x71d3440)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f1b28aabb, ext:230369725, loc:(*time.Location)(0x71d3440)}}, Count:5, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Feb 25 04:55:38 minikube kubelet[3332]: E0225 04:55:38.928397    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:39 minikube kubelet[3332]: E0225 04:55:39.028604    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:39 minikube kubelet[3332]: E0225 04:55:39.128817    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:39 minikube kubelet[3332]: E0225 04:55:39.229038    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:39 minikube kubelet[3332]: E0225 04:55:39.298849    3332 event.go:203] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"minikube.1586826c8d8874c7", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"", Name:"minikube", UID:"minikube", APIVersion:"", ResourceVersion:"", FieldPath:""}, Reason:"NodeHasSufficientPID", Message:"Node minikube status is now: NodeHasSufficientPID", Source:v1.EventSource{Component:"kubelet", Host:"minikube"}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f18ef1cc7, ext:193043398, loc:(*time.Location)(0x71d3440)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbf14fb2f1b28c798, ext:230377115, loc:(*time.Location)(0x71d3440)}}, Count:5, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'namespaces "default" not found' (will not retry!)
Feb 25 04:55:39 minikube kubelet[3332]: E0225 04:55:39.329547    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:39 minikube kubelet[3332]: E0225 04:55:39.430117    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:39 minikube kubelet[3332]: E0225 04:55:39.534430    3332 kubelet.go:2266] node "minikube" not found
Feb 25 04:55:39 minikube kubelet[3332]: I0225 04:55:39.583897    3332 kubelet_node_status.go:75] Successfully registered node minikube
Feb 25 04:55:51 minikube kubelet[3332]: I0225 04:55:51.170935    3332 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "coredns-token-mq99d" (UniqueName: "kubernetes.io/secret/9f93e408-38b9-11e9-848d-08002770ffd4-coredns-token-mq99d") pod "coredns-86c58d9df4-8z28z" (UID: "9f93e408-38b9-11e9-848d-08002770ffd4")
Feb 25 04:55:51 minikube kubelet[3332]: I0225 04:55:51.170974    3332 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "config-volume" (UniqueName: "kubernetes.io/configmap/9f93e408-38b9-11e9-848d-08002770ffd4-config-volume") pod "coredns-86c58d9df4-8z28z" (UID: "9f93e408-38b9-11e9-848d-08002770ffd4")
Feb 25 04:55:51 minikube kubelet[3332]: I0225 04:55:51.171079    3332 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "config-volume" (UniqueName: "kubernetes.io/configmap/9f92b016-38b9-11e9-848d-08002770ffd4-config-volume") pod "coredns-86c58d9df4-2nzfp" (UID: "9f92b016-38b9-11e9-848d-08002770ffd4")
Feb 25 04:55:51 minikube kubelet[3332]: I0225 04:55:51.171100    3332 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "coredns-token-mq99d" (UniqueName: "kubernetes.io/secret/9f92b016-38b9-11e9-848d-08002770ffd4-coredns-token-mq99d") pod "coredns-86c58d9df4-2nzfp" (UID: "9f92b016-38b9-11e9-848d-08002770ffd4")
Feb 25 04:55:51 minikube kubelet[3332]: I0225 04:55:51.271789    3332 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "kube-proxy-token-pktlv" (UniqueName: "kubernetes.io/secret/9fc2099a-38b9-11e9-848d-08002770ffd4-kube-proxy-token-pktlv") pod "kube-proxy-ff5cr" (UID: "9fc2099a-38b9-11e9-848d-08002770ffd4")
Feb 25 04:55:51 minikube kubelet[3332]: I0225 04:55:51.271940    3332 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "kube-proxy" (UniqueName: "kubernetes.io/configmap/9fc2099a-38b9-11e9-848d-08002770ffd4-kube-proxy") pod "kube-proxy-ff5cr" (UID: "9fc2099a-38b9-11e9-848d-08002770ffd4")
Feb 25 04:55:51 minikube kubelet[3332]: I0225 04:55:51.272022    3332 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "lib-modules" (UniqueName: "kubernetes.io/host-path/9fc2099a-38b9-11e9-848d-08002770ffd4-lib-modules") pod "kube-proxy-ff5cr" (UID: "9fc2099a-38b9-11e9-848d-08002770ffd4")
Feb 25 04:55:51 minikube kubelet[3332]: I0225 04:55:51.272194    3332 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "xtables-lock" (UniqueName: "kubernetes.io/host-path/9fc2099a-38b9-11e9-848d-08002770ffd4-xtables-lock") pod "kube-proxy-ff5cr" (UID: "9fc2099a-38b9-11e9-848d-08002770ffd4")
Feb 25 04:55:51 minikube kubelet[3332]: W0225 04:55:51.383040    3332 raw.go:87] Error while processing event ("/sys/fs/cgroup/devices/system.slice/run-r57a5bfd67ae34098a21a7cc05688f323.scope": 0x40000100 == IN_CREATE|IN_ISDIR): inotify_add_watch /sys/fs/cgroup/devices/system.slice/run-r57a5bfd67ae34098a21a7cc05688f323.scope: no such file or directory
Feb 25 04:55:52 minikube kubelet[3332]: I0225 04:55:52.593280    3332 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "tmp" (UniqueName: "kubernetes.io/host-path/a088234b-38b9-11e9-848d-08002770ffd4-tmp") pod "storage-provisioner" (UID: "a088234b-38b9-11e9-848d-08002770ffd4")
Feb 25 04:55:52 minikube kubelet[3332]: I0225 04:55:52.593886    3332 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "storage-provisioner-token-c27fx" (UniqueName: "kubernetes.io/secret/a088234b-38b9-11e9-848d-08002770ffd4-storage-provisioner-token-c27fx") pod "storage-provisioner" (UID: "a088234b-38b9-11e9-848d-08002770ffd4")
$

I've been trying to figure this out, but to no avail. Any ideas?
@michaeljeffrey
Copy link
Author

BTW, I'm running VirtualBox version 6.0.4.

@michaeljeffrey
Copy link
Author

Also, when I run 'minikube status', it yields:

host: Running
kubelet: Running
apiserver: Stopped

@michaeljeffrey
Copy link
Author

Update: this worked fine at work.

So there's something borking this when I'm at home. I did try toggling VPN off/on to no avail at home, so it's not that by itself.

Any ideas?

@michaeljeffrey
Copy link
Author

Googling around more I see some people have issues the Cisco AnyConnect VPN client.

When I run this before starting my cluster, it seems to work okay (at home):

sudo route add 192.168.99.0/24 -iface vboxnet0

I hope that helps someone.

@nwyatt nwyatt mentioned this issue May 24, 2019
Closed
@jfrabetti jfrabetti mentioned this issue Aug 2, 2019
Closed
@calltosantosh calltosantosh mentioned this issue Nov 11, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@michaeljeffrey