Provide binaries without CAP_NET_BIND_SERVICE

[jkroepke]

I would like to request the deprecate/drop setting capabilities OR provide a copy of the binaries inside that image without the capability.

ingress-nginx/rootfs/Dockerfile

Lines 70 to 75 in 9398c7e

	&& setcap cap_net_bind_service=+ep /nginx-ingress-controller \
	&& setcap -v cap_net_bind_service=+ep /nginx-ingress-controller \
	&& setcap cap_net_bind_service=+ep /usr/local/nginx/sbin/nginx \
	&& setcap -v cap_net_bind_service=+ep /usr/local/nginx/sbin/nginx \
	&& setcap cap_net_bind_service=+ep /usr/bin/dumb-init \
	&& setcap -v cap_net_bind_service=+ep /usr/bin/dumb-init \

The goal here is to get one step closer for support #9212. At the moment, its not possible to run the container with

   securityContext:
      runAsUser: 101
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities:
        drop:
          - ALL

exec /usr/bin/dumb-init: operation not permitted

I would like to run ingress-nginx on port 8080, 8443 which does not require. However, since the binaries having the cap, they are not allowed to executed in that environment.

In general, setcap is not required anymore. Since Kubernetes 1.22 it's possible to define the sysctl net.ipv4.ip_unprivileged_port_start through securityContext.

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[rikatz]

@jkroepke sounds good :D Do you think you can take a look/try implementing this on the Makefiles? Can be an env that doesn't sets the capabilities, and we can test if we cna run without it

[jkroepke]

Whats the end result? 2 docker images? or 1 docker images contains both both binaries?

Or removing setcap by default with an breaking change?

[github-actions]

This is stale, but we won't close it automatically, just bare in mind the maintainers may be busy with other tasks and will reach your issue ASAP. If you have any question or request to prioritize this, please reach #ingress-nginx-dev on Kubernetes Slack.

[schoentoon]

It seems that if you configure nosuid as mountopt for your container runtime (only tested this with crio).. This works just fine?.. I completely ran into this by accident while debugging a completely different issue I had with ingress-nginx. Resulting in us moving to adding --http-port=8080 --https-port=8443 and thus wanting to remove the NET_BIND_SERVICE capability. Which seemed to work on one node in our testing cluster, but gave an instant crash on the others. I later realized this was the node where I once set nosuid as a mountopt as a test for something completely different. Upon adding this to a node where ingress-nginx crashed due to a missing NET_BIND_SERVICE it started working.
Ideally would still see this capability requirement removed from the files directly of course.

As per capabilities(7)

Note: during the capability transitions described above, file
       capabilities may be ignored (treated as empty) for the same
       reasons that the set-user-ID and set-group-ID bits are ignored;
       see execve(2).  File capabilities are similarly ignored if the
       kernel was booted with the no_file_caps option.

execve(2) obviously mentioning the nosuid mount option on the topic.. Today I Learned

[longwuyuan]

This has not been worked on for a long time and now the project is in shortage of resources bad enough to deprecate features due to lack of developer time.

So there is hardly any chance that this will get worked on. Focus is to use whatever resources for security & Gateway-API. If some developer commits in future to this, then we can re-open this. But I am closing this now as this is adding to the tally of open issues without tracking any action item on the project.

/close

[k8s-ci-robot]

@longwuyuan: Closing this issue.

In response to this:

This has not been worked on for a long time and now the project is in shortage of resources bad enough to deprecate features due to lack of developer time.

So there is hardly any chance that this will get worked on. Focus is to use whatever resources for security & Gateway-API. If some developer commits in future to this, then we can re-open this. But I am closing this now as this is adding to the tally of open issues without tracking any action item on the project.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.