Permission denied when try to create a pvc

[danilorsilva]

I am following the steps to install/configure local-path-provider.

I have a local-path-provisioner running.

I have created pvc and status is pending (waiting for the pod).

I am trying to create a nginx pod like the example and i am facing this problem:

When i check create-pvc-33e6692e.... pod i have this error:
mkdir: can't create directory '/data/pvc-33e6692e-a32d-11e9-85ac-42010a8e0036': Permission denied

My local path is already configured like root.root and 777 permissions.

Anyone can help me?

[yasker]

Is there selinux or anything similar running on your node? That's mostly like the reason.

[danilorsilva]

I think that is not the problem.

When i configure my deployment using hostpath directly i don't have any problem to mount the path and write file on the path.

For hostpath i followed this paper: https://kubernetes.io/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#before-you-begin

[warmchang]

How to reproduce this issue?

[danilorsilva]

Just following the steps on the main page.

[davidwindell]

@danilorsilva are you still experiencing this problem, I'm having similar issues

[usrbinkat]

resolved with steps here k3s-io/k3s#1823

[identitymonk]

Hi,

I got it resolved by disabling SELinux... this is not optimum. But it was a main difference between an instance I got it working on and one I got it in error. (In my case, it was the installation of Hashicorp/Consul as an Helm chart).

The Audit log for the action related to the permission denied was:

type=AVC msg=audit(1591333700.327:32518): avc: denied { create } for pid=20562 comm="consul" name="node-id" scontext=system_u:system_r:container_t:s0:c148,c168 tcontext=system_u:object_r:container_var_lib_t:s0 tclass=file permissive=0
type=SYSCALL msg=audit(1591333700.327:32518): arch=c000003e syscall=257 success=no exit=-13 a0=ffffffffffffff9c a1=c0008701e0 a2=80241 a3=180 items=0 ppid=13756 pid=20562 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="consul" exe="/bin/consul" subj=system_u:system_r:container_t:s0:c148,c168 key=(null)
type=PROCTITLE msg=audit(1591333700.327:32518): proctitle=2F62696E2F636F6E73756C006167656E74002D6164766572746973653D3139322E3136382E35362E313431002D62696E643D302E302E302E30002D626F6F7473747261702D6578706563743D31002D636C69656E743D302E302E302E30002D636F6E6669672D6469723D2F636F6E73756C2F636F6E666967002D646174616365

I will try to dig it more

[usrbinkat]

Confirmed Findings

1. Deploy CentOS 8
2. Deploy k3s from scratch
3. Disable selinux
4. Deploy example pvc backed nginx demo per docs
5. observed the issue is resolved
   Refrence Docs: 1 2

root@one .ccio$ ssh ccio@ensign k3s --version
k3s version v1.18.3+k3s1 (96653e8d)
root@one .ccio$ kubectl get node -o wide
NAME     STATUS   ROLES    AGE   VERSION        INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION                CONTAINER-RUNTIME                                                                                           
ensign   Ready    master   37m   v1.18.3+k3s1   10.180.0.44   <none>        CentOS Linux 8 (Core)   4.18.0-147.8.1.el8_1.x86_64   containerd://1.3.3-k3s2
root@one .ccio$ ssh ccio@ensign                                                          
Last login: Wed Jun 10 19:56:54 2020                                                            
[ccio@ensign ~]$ sudo setenforce 0                                                                                    
[ccio@ensign ~]$ sudo getenforce                                                                                      
Permissive                                                                                                            
[ccio@ensign ~]$ exit                                                                                                 
logout                                                                                                                
Connection to ensign closed. 
root@one .ccio$ cat <<EOF | kubectl apply -f -                                                                       
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: local-path-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: local-path
  resources:
    requests:
      storage: 2Gi
EOF
persistentvolumeclaim/local-path-pvc created
root@one .ccio$ cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: volume-test
  namespace: default
spec:
  containers:
  - name: volume-test
    image: nginx:stable-alpine
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - name: volv
      mountPath: /data
    ports:
    - containerPort: 80
  volumes:
  - name: volv
    persistentVolumeClaim:
      claimName: local-path-pvc
EOF
pod/volume-test created
root@one .ccio$ kubectl get pvc
NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE    
local-path-pvc   Bound    pvc-e4f32a10-b858-4edb-8713-df64ff4cbdfe   2Gi        RWO            local-path     36s

[identitymonk]

refer to K3s Issue 1821