Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cri-containerd: hostpath mounted read only #59

Closed
yvespp opened this issue Feb 11, 2018 · 1 comment
Closed

cri-containerd: hostpath mounted read only #59

yvespp opened this issue Feb 11, 2018 · 1 comment

Comments

@yvespp
Copy link

yvespp commented Feb 11, 2018

Description
Tried the cri-containerd runtime but the master node never becomes ready. In the logs of weave I can see that it can't write its configuration to disk because the file system is mounted read only.

Steps to reproduce the issue:
Used the master branch. Build the kube master like this: KUBE_RUNTIME=cri-containerd make master. Used ./boot.sh to boot it and then run kubeadm-init.sh
I'm on Mac OS 10.13.

Describe the results you received:
Weave tries to write it's configuration to a host path but fails:

linuxkit-0800279ca819:/# kubectl -n kube-system logs -f weave-net-h7gv9  weave
...
cp: can't create '/host/opt/cni/bin/weave-plugin-2.1.3': Read-only file system
/home/weave/weave: line 1576: can't create /host/etc/cni/net.d/10-weave.conf: Read-only file system
INFO: 2018/02/11 12:28:03.586451 Discovered local MAC 4a:7a:5a:07:dc:d1
INFO: 2018/02/11 12:28:04.427532 Weave version 2.2.0 is available; please update at https://github.com/weaveworks/weave/releases/download/v2.2.0/weave

The master node never becomes ready:

linuxkit-0800279ca819:/# kubectl describe nodes
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  Ready            False   Sun, 11 Feb 2018 13:12:00 +0000   Sun, 11 Feb 2018 12:23:48 +0000   KubeletNotReady              runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni config uninitialized

In the weave container I can see that the hostPath ist mounted ro:

linuxkit-0800279ca819:/# kubectl -n kube-system exec -it  weave-net-h7gv9  -c weave sh
/home/weave # mount | grep host
rootfs on /host/opt type tmpfs (ro,relatime)
rootfs on /host/home type tmpfs (ro,relatime)
rootfs on /host/etc type tmpfs (ro,relatime)
/dev/sda1 on /host/var/lib/dbus type ext4 (rw,relatime,data=ordered)
/dev/sda1 on /etc/hosts type ext4 (rw,relatime,data=ordered)
@yvespp yvespp changed the title containerd: hostpath mounted read only cri-containerd: hostpath mounted read only Feb 11, 2018
@ijc
Copy link
Contributor

ijc commented Feb 12, 2018

I spotted this when writing/running #58 too. I was testing a fix on Friday and hope to raise a PR today.

ijc pushed a commit to ijc/linuxkit-kubernetes that referenced this issue Feb 12, 2018
This reverts commit 81e8953.

These mounts are apparently not as redundant as they first seemed. The
`runtimes.mounts` paths are all wrt the host mount namespace, so these are in
fact binding the (writeable) `/var/lib/cni` paths over the (readonly) canonical
host CNI paths. The `binds` will then bind those host paths into the kubelet
container.

Fixes linuxkit#59, I also saw this during development of linuxkit#58.

Signed-off-by: Ian Campbell <ijc@docker.com>
@ijc ijc closed this as completed in #60 Feb 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants