Networking improvements to support Docker Swarm / VXLAN #5062
Conversation
| @@ -1018,9 +1018,9 @@ CONFIG_NETFILTER_FAMILY_BRIDGE=y | |||
| # CONFIG_NETFILTER_NETLINK_QUEUE is not set | |||
| CONFIG_NETFILTER_NETLINK_LOG=m | |||
| # CONFIG_NETFILTER_NETLINK_OSF is not set | |||
| CONFIG_NF_CONNTRACK=m | |||
There was a problem hiding this comment.
This one was originally a Module, but later complained it needed to be compiled into the kernel, probably caused by some other compiled netfilter module.
| # end of Core Netfilter Configuration | ||
|
|
||
| # CONFIG_IP_SET is not set | ||
| # CONFIG_IP_VS is not set |
There was a problem hiding this comment.
This list of added modules is needed for round robin virtual server loadbalancing in netfilter, used by Docker Swarm to load balance incoming connections over multiple worker nodes, on each of those nodes.
| @@ -1866,7 +1871,7 @@ CONFIG_WIREGUARD=m | |||
| CONFIG_MACVLAN=m | |||
| # CONFIG_MACVTAP is not set | |||
| # CONFIG_IPVLAN is not set | |||
| # CONFIG_VXLAN is not set | |||
There was a problem hiding this comment.
And this is how it started, enabling VXLAN overlay networking support in the kernel.
|
Oh, and the original feature request I made: |
heitbaum
added
GENERIC
IMPROVEMENT
KERNEL
LE 10.0
FEATURE
and removed
IMPROVEMENT
labels
|
Not saying we should not, but vlanx is really enterprise tech, so basically no common router/switch or for example pfsense supports it. |
Thats' correct, it's not something you'd run at home normally speaking and also not to create a network "manual" this way. It's purely needed for Docker Swarm. And I guess, the majority of users are not even running Docker containers and if they do it's doubtful they'd run a cluster between the docker servers at home. I realise I'm "special" haha. Maybe look at the usage numbers of the Docker plugin to get a feel for how many people could potentially benefit? VXLAN is part of the default kernel config I think, since the only references I found were special distributions and hosters that prevented people from running things like Swarm or Kubernetes. Anyway, like I wrote in the feature request: I do realise I didn't contribute something for the masses :) I just wanted to hare my work back and it's up to you guys if it's useful for other users. |
|
Just as an info, I also compiled this on the master branch with the following image and kernel sizes as a result. My estimation is that kernel size increase is similar to the 9.2.6 build I made earlier, the image size increase is the same as well. Compared to 9.2.6 I didn't have to bump iptables and libnft, they were already included... 260131368 Jan 30 11:05 LibreELEC-Generic.x86_64-9.80-devel-20210130110233-d7a5aa0.img.gz |
|
@rvleij I am closing this because its not really something that fits into LE by default Nevertheless, tx for the pr ! |
|
@rvleij : for information, I've managed to make k3s (lightweight Kubernetes) run on LibreElec, even if it's still using dirty hacks. See k3s-io/k3s#4859 |
|
@mossroy Hi, thanks for the pointer! I've actually built a new "home server"and I'm running vanilla Ubuntu now, with Swarm still. I was looking at k3s (or whatever Ubuntu has as light-weight K8S) but decided to leave things as they are right now. :) Thanks for the heads up though! Maybe one day I'll add a LibreElec to the game again and use it just as an extra node. |
Why
In order to support the Docker Add-On in LE to run in a Docker Swarm setup, overlay networking (VXLAN) support is required, which was not part of the included kernel in 9.2.6. Docker Swarm allows the LE box to run in a cluster, automatically load balancing workloads (containers) over the LE box and (for example) a Pi4.
How
This PR includes the required kernel modules (VXLAN as well as netfilter (iptables) modules and IP_VS modules required for the built-in load balancing and iptables filtering and matching rules used with Docker Swarm. The required version bump of iptables and it's libraries is already included in the master branch (originally needed the same bump on 9.2.6).
Testing
To test I ran multiple test builds (on 9.2.6) and installed them on amd64, until I got this running without errors. Docker now starts without errors and provides the full cluster / overlay networking functionality out of the box. Testing and size comparisons are on 9.2.6, the image included the new Iptables and Libnft versions that are already part of master branch. Size increase is limited:
Size Increase numbers
New image:
256001312 Jan 26 22:30 LibreELEC-Generic.x86_64-9.2-devel-20210126212510-6bd7e98.img.gz
16655232 Jan 26 22:30 LibreELEC-Generic.x86_64-9.2-devel-20210126212510-6bd7e98.kernel
257443840 Jan 26 22:31 LibreELEC-Generic.x86_64-9.2-devel-20210126212510-6bd7e98.ova
242139136 Jan 26 22:30 LibreELEC-Generic.x86_64-9.2-devel-20210126212510-6bd7e98.system
259123200 Jan 26 22:30 LibreELEC-Generic.x86_64-9.2-devel-20210126212510-6bd7e98.tar
Original 9.2.6 image from website:
16577408 Oct 25 11:03 KERNEL (increase 77824 B)
255879664 Jan 29 21:18 LibreELEC-Generic.x86_64-9.2.6.img.gz (increase 121648 B)
I haven't made changes to the other projects/archs, since I can't test.
The individual components (iptables, netfilter libs and kernel modules) are running on other architectures (arm etc) normally.