Skip to content

Upstream kernel header packages with relative symlinks always fall back to Pixie prepackaged headers  #2037

@ddelnano

Description

@ddelnano

Describe the bug
Pixie requires linux kernel headers since BCC is used to compile our eBPF programs. These headers can either be supplied by installing the distro provided linux headers package or installed by pixie through our prepackaged linux headers.

When working with @feynmanliang, we realized that installing an openSUSE MicroOS linux headers package results in Pixie falling back to our prepackaged headers. While this should in theory result in equivalent behavior, the upstream distro's headers are preferred since they are more likely to be what the given kernel expects.

The openSUSE linux headers use a relative symlink and Pixie's logic to locate the headers fails for this case:

# File layout seen after installing openSUSE headers
# ls -alh  /lib/modules/6.11.0-1-default/
total 43M
drwxr-xr-x. 1 root root  674 Oct  2 18:10 .
drwxr-xr-x. 1 root root   32 Sep 28 00:36 ..
-rw-r--r--. 1 root root   65 Sep 25 05:56 .vmlinuz.hmac
-rw-r--r--. 1 root root 7.6M Sep 25 05:56 System.map
lrwxrwxrwx. 1 root root   46 Sep 25 05:56 build -> ../../../src/linux-6.11.0-1-obj/x86_64/default

# PEM logs that show installed headers are misdetected as missing
I20241002 18:32:19.083017 22431 kernel_version.cc:82] Obtained Linux version string from `uname`: 6.11.0-1-default
I20241002 18:32:19.083026 22431 linux_headers.cc:381] Detected kernel release (uname -r): 6.11.0-1-default
I20241002 18:32:19.083271 22431 linux_headers.cc:396] Not Found : Could not find 'source' or 'build' under /lib/modules/6.11.0-1-default.
I20241002 18:32:19.083294 22431 linux_headers.cc:244] Looking for host Linux headers at /host/lib/modules/6.11.0-1-default/build.
I20241002 18:32:19.083403 22431 linux_headers.cc:399] Not Found : Did not find host headers at resolved path: /host/../../../src/linux-6.11.0-1-obj/arm64/default.
I20241002 18:32:19.083415 22431 linux_headers.cc:342] Attempting to install packaged headers.

This is in contrast to Ubuntu's header package, which uses an absolute symlink:

$ ls -alh /lib/modules/$(uname -r) | grep build
lrwxrwxrwx  1 root root   37 Aug  8 21:15 build -> /usr/src/linux-headers-6.8.0-1013-gcp

To Reproduce
Steps to reproduce the behavior:

  1. Using a Ubuntu instance, install the linux-headers-$(uname -r) package
  2. Run stirling_wrapper and see that Pixie locates the headers properly
  3. Replace /lib/modules/$(uname -r)/build with an openSUSE style, relative symlink
$ cd /lib/modules/$(uname -r)
$ sudo ln -s ../../../../usr/src/linux-headers-$(uname -r) build
  1. See that stirling_wrapper falls back to prepackaged headers.

Expected behavior
Upstream installed linux headers should be correctly identified.

App information (please complete the following information):

  • Pixie version: v0.14.11
  • Node Kernel version: 6.11
  • Distro: openSUSE MicroOS
  • K8s cluster version: N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/datacollectorIssues related to Stirling (datacollector)

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions