gefyra-bridge: bridging fails when pod uses (privileged) user id

[crkurz]

What happened?

Problem description

bridging does not work when pod uses (privileged) user id e.g. 12 as opposed to 1000 used as default in gefyra-carrier-image. This prevents setroute.sh to write the ngx config file.

RCA

Carrier._configure_carrier() uses k8s-exec with setroute.sh to configure Carrier container. In this k8s-exec call no user-id is specified. So default user will be used. The unspoken assumption here is that default user (in our case privileged userid 12) is able to execute setroute.sh and is able to update file /tmp/nginx.conf, which is not the case.

Notes/Questions:

• docker images have root user removed
• docker images use userid 12; probably configured by USER-cmd in dockerfile
• different pods/containers may use different (privileged) user ids.
• would it be enough to enable any user to write to /tmp/nginx.conf and to execute setroute.sh and setprobe.sh ?

work-around (not really)

change carrier/Dockerfile from

COPY --chown=1000:1000 gefyra-carrier.conf /tmp/nginx.conf
COPY setroute.sh setroute.sh
COPY setprobe.sh setprobe.sh

to

COPY --chown=12:12 gefyra-carrier.conf /tmp/nginx.conf
COPY --chown=12:12 setroute.sh setroute.sh
COPY --chown=12:12 setprobe.sh setprobe.sh

What did you expect to happen?

gefyra-bridge to succeed

Please provide the output of gefyra check.

.

How can we reproduce it (as minimally and precisely as possible)?

untested: use bridging on a pod with docker image, which has default user set to 12 (via USER-cmd in Dockerfile)

What Kubernetes setup are you working with?

.

OS version

.

Anything else we need to know?

in Carrier._configure_carrier() we call set_route.sh. Even though setroute.sh fails with

sed: can't move '/tmp/nginx.confpnajIJ' to '/tmp/nginx.conf': Operation not permitted

the error is ignored: nothing reported in operator logs neither failure of gefyra-bridge command. Fixing this may require two things:

• Carrier._configure_carrier() must fail on error (not swallow exception)
• setroute.sh may need to enable strict bash exception handling: set -eo pipefail