Add builder for Flatcar #131

jepio · 2022-04-01T13:50:49Z

Example output

# Requires the builderimage to be updated, so replicating this might need also a --builderimage argument
./_output/bin/driverkit docker -l debug -t flatcar --kernelrelease 3255.0.0  --output-module=/tmp/falco.ko --driverversion master

DEBU running without a configuration file         
DEBU running with options                          arch=amd64 driverversion=master kernelrelease=3255.0.0 kernelversion=1 output-module=/tmp/falco.ko target=flatcar
INFO driver building, it will take a few seconds   processor=docker
DEBU doing a new docker build                     
DEBU kernel header url found                       url="https://alpha.release.flatcar-linux.net/amd64-usr/3255.0.0/flatcar_production_image_packages.txt"
DEBU kernel header url found                       url="https://alpha.release.flatcar-linux.net/amd64-usr/3255.0.0/flatcar_production_image_kernel_config.txt"
DEBU kernel header url found                       url="https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.43.tar.xz"
...
DEBU make[1]: Leaving directory '/tmp/kernel' 
DEBU + mv falco.ko /tmp/driver/module.ko  
DEBU + strip -g /tmp/driver/module.ko     
DEBU + modinfo /tmp/driver/module.ko      
DEBU filename:       /tmp/driver/module.ko 
DEBU schema_version: 1.0.0                        
DEBU api_version:    1.0.0                        
DEBU build_commit:   master                       
DEBU version:        master                       
DEBU author:         the Falco authors            
DEBU license:        GPL                          
DEBU srcversion:     F2DE132E5F90A47892185E9      
DEBU depends:                                     
DEBU retpoline:      Y                            
DEBU name:           falco                        
DEBU vermagic:       5.15.43-flatcar SMP mod_unload  
DEBU parm:           max_consumers:Maximum number of consumers that can simultaneously open the devices (uint) 
DEBU parm:           verbose:Enable verbose logging (bool) 
DEBU log pipe close                                error=EOF
INFO kernel module available                       path=/tmp/falco.ko
DEBU context canceled

What type of PR is this?
/kind feature

Any specific area of the project related to this PR?
/area build

What this PR does / why we need it:
This PR introduces a builder for Flatcar. There have been several reports of vanilla builder not working well for Flatcar (https://github.com/falcosecurity/driverkit/issues?q=flatcar), we also have reports on our own issue tracker of the stock Falco build process not working since we shipped a newer GLIBC (flatcar/Flatcar#534). In slack there are occasional questions in slack regarding the kernel module build for Flatcar. So we know that there are plenty of users that would like this to work.

Which issue(s) this PR fixes:

Example output:

Fixes #

Special notes for your reviewer:
I tried to not introduce an extra "osrelease" or "osversion" field but that's actually what we would need. I snuck the "osrelease" into the "kernelrelease" field instead, hope that's ok. Also: any thoughts on getting a GCC newer than 8 into the builder container?

Does this PR introduce a user-facing change?:

Add support for Flatcar Container Linux

poiana · 2022-04-01T13:50:57Z

Welcome @jepio! It looks like this is your first PR to falcosecurity/driverkit 🎉

dwindsor · 2022-04-06T20:43:52Z

pkg/driverbuilder/builder/flatcar.go

+curl --silent -o /tmp/kernel.config -SL {{ .KernelConfigURL }}
+
+cd /tmp/kernel
+make KCONFIG_CONFIG=/tmp/kernel.config oldconfig


I wonder if these calls to make oldconfig and modules_prepare are necessary? I don't actually see them in any of the other builders, but this may be required for flatcar?

I copied this part from the vanilla builder. Our own "kernel-devel" package equivalent is compiled against newer glibc, such that it doesn't work with the old debian based builder image. Here we're pulling generic kernel sources (we don't patch our sources) and need to "prepare" those.

Makes sense, thanks!

jepio · 2022-04-20T11:39:35Z

@leodido any chance you could have a look at this?

dwindsor · 2022-05-31T15:55:03Z

Hi! This looks fine to me. kernel-crawler already has support for Flatcar, so it makes sense to add this.

It's been a while since this PR was originally submitted, so not sure if the vendor has changed its upstream package naming format and/or location (I assume not, it's been only a month).

jepio · 2022-06-01T12:38:17Z

Hi! This looks fine to me. kernel-crawler already has support for Flatcar, so it makes sense to add this.

It's been a while since this PR was originally submitted, so not sure if the vendor has changed its upstream package naming format and/or location (I assume not, it's been only a month).

Thanks for the link to kernel-crawler, I checked what output it produces, and decided to drop the -flatcar from the kernelrelease value to be consistent with it. None of the package naming/formats/locations have changed, but I did catch 2 minor issues that I fixed: I was missing setting EXTRAVERSION correctly and we need dwarves/pahole in the builder container.

With this, I would be happy to get lgtm's and/or make any necessary changes.

dwindsor · 2022-06-03T12:24:49Z

Thanks for the link to kernel-crawler, I checked what output it produces, and decided to drop the -flatcar from the kernelrelease value to be consistent with it. None of the package naming/formats/locations have changed, but I did catch 2 minor issues that I fixed: I was missing setting EXTRAVERSION correctly and we need dwarves/pahole in the builder container.

Nice! Thanks for the updates. Have you had a chance to try to load the built drivers/probes on Flatcar yet?

jepio · 2022-06-03T12:32:40Z

Yes, that's how I came across the issues 2 issues I found. The module now loads correctly on flatcar.

dwindsor · 2022-06-03T17:40:55Z

Could you update the description with an example command-line invocation building a flatcar driver with successful results as in PR130:

On Flatcar, a kernel version alone is not enough to determine the kernel config and which GCC version was used. But a Flatcar release version number can will give us this information, including the kernel version. So this build works the following way: the kernel release field has to contain a flatcar version number followed by `-flatcar`, such as 3185.0.0-flatcar. Based on this, the correct "channel" will be found (there is 1-1 mapping between flatcar version and channel), and we will fetch kernel version, GCC version and kernel config. Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

…version This was an arbitrary requirement that served no purpose. Kernel-crawler generates configs with the kernelrelease set to the Flatcar version number without the '-flatcar' suffix, so let's be consistent with that. Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

This is required to build Flatcar kernel compatible modules, as otherwise the modules have the wrong version magic and can't be probed. Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

Flatcar kernels are built with BTF type info, which requires pahole when configuring the kernel. If pahole is missing, then running `make modules_prepare` removes entries from the kernel config and results in built modules failing to probe with the error: module: x86/modules: Skipping invalid relocation target, existing value is nonzero for type 1, loc 0000000090295d6a, val ffffffffc0b5ae8d Buster provides pahole 1.12, but CONFIG_PAHOLE_HAS_SPLIT_BTF requires 1.19 which, luckily, is available in the buster-packports repo. Add that package to the container image. Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

jepio · 2022-06-07T14:00:39Z

I've added an example command line invocation of this builder and the output that generates (truncated most of the verbose shell output).

Also rebased the series to add a commit to support the architecture flag that was merged to master.

Pre-built kernel modules/eBPF probes for Flatcar use the value of the OS VERSION_ID field as KERNEL_RELEASE in the filename. A specific kernel release version does not uniquely identify a Flatcar configuration, because Flatcar is image-based instead of package-based. Here's a more specific example: the same kernel version can be part of various Flatcar releases (across channels alpha/beta/stable) with differences in configuration. This is why we use the VERSION_ID value during offline builds with driverkit. Flatcar version numbers are all higher than 1500.0.0, so there is no risk of collision with kernel version numbers. When locally building the kernel module on the system, we have access to the correct kernel build directory at /lib/modules/$(uname -r)/build with the right configuration and so for that branch, we need to reset KERNEL_RELEASE=$(uname -r). See also the driverkit PR that introduces a builder for Flatcar: falcosecurity/driverkit#131 Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

poiana · 2022-06-09T12:32:35Z

LGTM label has been added.

Git tree hash: 23987d0a536896f42d0ada75d1e9bc3b4fab122a

dwindsor · 2022-06-09T12:32:39Z

/lgtm

poiana · 2022-06-09T12:32:46Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dwindsor, jepio

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [dwindsor]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Pre-built kernel modules/eBPF probes for Flatcar use the value of the OS VERSION_ID field as KERNEL_RELEASE in the filename. A specific kernel release version does not uniquely identify a Flatcar configuration, because Flatcar is image-based instead of package-based. Here's a more specific example: the same kernel version can be part of various Flatcar releases (across channels alpha/beta/stable) with differences in configuration. This is why we use the VERSION_ID value during offline builds with driverkit. Flatcar version numbers are all higher than 1500.0.0, so there is no risk of collision with kernel version numbers. When locally building the kernel module on the system, we have access to the correct kernel build directory at /lib/modules/$(uname -r)/build with the right configuration and so for that branch, we need to reset KERNEL_RELEASE=$(uname -r). See also the driverkit PR that introduces a builder for Flatcar: falcosecurity/driverkit#131 Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

poiana added kind/feature New feature or request dco-signoff: yes area/build Further information is requested labels Apr 1, 2022

poiana requested review from fntlnz and leodido April 1, 2022 13:50

poiana added the size/L label Apr 1, 2022

dwindsor reviewed Apr 6, 2022

View reviewed changes

jepio force-pushed the flatcar-builder branch from 9a57978 to 614935a Compare April 20, 2022 11:38

jepio force-pushed the flatcar-builder branch 2 times, most recently from 92be8ca to 89dbfc1 Compare June 1, 2022 12:33

jepio added 7 commits June 7, 2022 14:05

Add 'flatcar' to completion targets

9968981

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

docs: add entry for flatcar builder

837b59f

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

builder/flatcar: override EXTRAVERSION in kernel Makefile

9227b42

This is required to build Flatcar kernel compatible modules, as otherwise the modules have the wrong version magic and can't be probed. Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

builder/flatcar: Add support for architecture argument

81d34f6

Signed-off-by: Jeremi Piotrowski <jpiotrowski@microsoft.com>

jepio force-pushed the flatcar-builder branch from 89dbfc1 to 81d34f6 Compare June 7, 2022 13:27

jepio mentioned this pull request Jun 7, 2022

Flatcar driver loader support falcosecurity/falco#2043

Merged

dwindsor approved these changes Jun 9, 2022

View reviewed changes

poiana assigned dwindsor Jun 9, 2022

poiana added the lgtm label Jun 9, 2022

poiana added the approved label Jun 9, 2022

dwindsor merged commit 0f87b2a into falcosecurity:master Jun 9, 2022

jepio deleted the flatcar-builder branch June 15, 2022 08:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add builder for Flatcar #131

Add builder for Flatcar #131

jepio commented Apr 1, 2022 •

edited

Loading

poiana commented Apr 1, 2022

dwindsor Apr 6, 2022 •

edited

Loading

jepio Apr 7, 2022

dwindsor Apr 7, 2022

jepio commented Apr 20, 2022

dwindsor commented May 31, 2022 •

edited

Loading

jepio commented Jun 1, 2022

dwindsor commented Jun 3, 2022

jepio commented Jun 3, 2022

dwindsor commented Jun 3, 2022

jepio commented Jun 7, 2022

poiana commented Jun 9, 2022

dwindsor commented Jun 9, 2022

poiana commented Jun 9, 2022

Add builder for Flatcar #131

Add builder for Flatcar #131

Conversation

jepio commented Apr 1, 2022 • edited Loading

poiana commented Apr 1, 2022

dwindsor Apr 6, 2022 • edited Loading

Choose a reason for hiding this comment

jepio Apr 7, 2022

Choose a reason for hiding this comment

dwindsor Apr 7, 2022

Choose a reason for hiding this comment

jepio commented Apr 20, 2022

dwindsor commented May 31, 2022 • edited Loading

jepio commented Jun 1, 2022

dwindsor commented Jun 3, 2022

jepio commented Jun 3, 2022

dwindsor commented Jun 3, 2022

jepio commented Jun 7, 2022

poiana commented Jun 9, 2022

dwindsor commented Jun 9, 2022

poiana commented Jun 9, 2022

jepio commented Apr 1, 2022 •

edited

Loading

dwindsor Apr 6, 2022 •

edited

Loading

dwindsor commented May 31, 2022 •

edited

Loading