Skip to content

RISC-V kCFI boot hang after LLVM commit 434e4e15f6a3 #2170

Open
Open
RISC-V kCFI boot hang after LLVM commit 434e4e15f6a3#2170
Labels
[ARCH] risc-vThis bug impacts ARCH=riscv[FEATURE] CFIRelated to building the kernel with Clang Control Flow Integrityboot failureThis issue results in a failure to boot

Description

@nathanchance
nathanchance
opened on Jul 8, 2026

After LLVM commit 434e4e15f6a3 ("[RISCV] Run MakeCompresible at Os as well (#207172)", 2026-07-06), I am seeing a boot hang with no output after OpenSBI when kCFI is enabled:

$ make -skj"$(nproc)" ARCH=riscv LLVM=1 mrproper defconfig

$ scripts/config -e CFI

$ make -skj"$(nproc)" ARCH=riscv LLVM=1 olddefconfig Image

$ boot-qemu.py -a riscv -k .
<hangs>

As this change should only impact files build with -Os, I built two sets of object files with compilers built from the problematic commit and its parent. I tried building with the -Os objects built using the parent commit compiler with the other objects built from the bad commit compiler to try and narrow things down but that did not fix the issue. CFI could be a red herring but it is the only configuration that appears to trigger this issue in my build tests.

Activity

  1. nathanchance
    added
    [ARCH] risc-vThis bug impacts ARCH=riscv
    boot failureThis issue results in a failure to boot
    on Jul 8, 2026
  2. nickdesaulniers
    added
    [FEATURE] CFIRelated to building the kernel with Clang Control Flow Integrity
    on Jul 9, 2026

  3. nickdesaulniers commented on Jul 9, 2026

    @nickdesaulniers
    nickdesaulniers
    on Jul 9, 2026
    Member

    cc @samitolvanen

  4. Gelbpunkt commented on Jul 15, 2026

    @Gelbpunkt
    Gelbpunkt
    on Jul 15, 2026

    I'm seeing what's probably a very similar (?) thing, but with the released LLVM 22.1.8 and our distribution configuration in postmarketOS. I'm booting with EDK2 and seeing nothing, no logs at all, even with earlycon, after the kernel is being booted if CFI is enabled, while it does boot normally with CFI disabled.

    Here's the broken config, disabling CFI makes it work: config.txt

  5. nathanchance commented on Jul 16, 2026

    @nathanchance
    nathanchance
    on Jul 16, 2026
    MemberAuthor

    I'm seeing what's probably a very similar (?) thing

    Hmmm, not sure they are the exact same issue but they could certainly be related.

    but with the released LLVM 22.1.8 and our distribution configuration in postmarketOS.

    However, due to this, I suspect it is some other issue that just manifests in an early boot crash, which is likely what happens in this case. I need to run this under gdb but I have been on vacation for a little bit so I have not had time.

    I'm booting with EDK2 and seeing nothing, no logs at all, even with earlycon, after the kernel is being booted if CFI is enabled, while it does boot normally with CFI disabled.

    Here's the broken config, disabling CFI makes it work: config.txt

    Your hang appears to be related to CONFIG_FUNCTION_TRACER=y + CONFIG_CFI=y, which could mean there is some bad interaction between -fpatchable-function-entry and -fsanitize=kcfi on RISC-V.

    $ printf 'CONFIG_%s=y\n' CFI FUNCTION_TRACER >arch/riscv/configs/repro.config

$ make -skj"$(nproc)" ARCH=riscv LLVM=1 mrproper defconfig repro.config Image

$ qemu-system-riscv64 -display none -nodefaults -bios default -M virt -append earlycon -kernel arch/riscv/boot/Image -initrd <boot_utils>/images/riscv/rootfs.cpio -m 1G -serial mon:stdio
<hangs after OpenSBI>

    Could you file a separate issue for this?

  6. nathanchance commented on Aug 1, 2026

    @nathanchance
    nathanchance
    on Aug 1, 2026
    MemberAuthor

    I attempted to use gdb to further inspect the boot process here but I am not sure we are even getting to the kernel/out of OpenSBI...

    $ boot-qemu.py -a riscv --append nokaslr -g --gdb-bin gdb -k .
...
Reading symbols from .../vmlinux...
Remote debugging using :1234
0x0000000000001000 in ?? ()
(gdb) break _start_kernel
Breakpoint 1 at 0xffffffff800010d4: file .../arch/riscv/kernel/head.S, line 212.
(gdb) c
Continuing.
^C
Program received signal SIGINT, Interrupt.
0x00000000802010bc in ?? ()
(gdb) lx-dmesg
Python Exception <class 'gdb.MemoryError'>: Cannot access memory at address 0xffffffff8168b7b0
❌️ Error occurred in Python: Cannot access memory at address 0xffffffff8168b7b0

    I will try to do some more side by side binary diffs next week.

  7. nathanchance commented on Aug 4, 2026

    @nathanchance
    nathanchance
    on Aug 4, 2026
    MemberAuthor

    Okay, I at least know why this LLVM change affects things and why my previous -Os objects from a good and a bad build trick did not work but I have not actually found out what is broken about it just yet (other than the fact that the pass is running on this code at all)...

    I tested changing all uses of -Os in this particular build configuration to -O2 (which meant changing arch/riscv/kernel/pi/Makefile and drivers/firmware/efi/libstub/Makefile) but things were still broken. In looking at llvm-readelf and llvm-nm, I realized that .init.text and its functions changed sizes between builds, which caused me to look at clang/lib/CodeGen/CodeGenModule.cpp and notice in CodeGenModule::SetLLVMFunctionAttributesForDefinition() that __cold implies optsize...

      // Track whether we need to add the optnone LLVM attribute,
  // starting with the default for this optimization level.
  bool ShouldAddOptNone =
      !CodeGenOpts.DisableO0ImplyOptNone && CodeGenOpts.OptimizationLevel == 0;
  // We can't add optnone in the following cases, it won't pass the verifier.
  ShouldAddOptNone &= !D->hasAttr<MinSizeAttr>();
  ShouldAddOptNone &= !D->hasAttr<AlwaysInlineAttr>();
...
    if (D->hasAttr<ColdAttr>()) {
      if (!ShouldAddOptNone)
        B.addAttribute(llvm::Attribute::OptimizeForSize);
      B.addAttribute(llvm::Attribute::Cold);
    }

    so __init (which implies __cold through its definition) functions will have this optimization applied to them. I could certainly see how this messes with CFI but I am not sure how yet.

  8. nathanchance commented on Aug 7, 2026

    @nathanchance
    nathanchance
    on Aug 7, 2026
    MemberAuthor

    If I take arch/riscv/mm/init.o from a build with the good compiler and put it in a build with the bad compiler, the kernel boots. If I remove __meminit from all functions in arch/riscv/mm/init.c and build with the bad compiler, the kernel boots. I am still trying to narrow down if there is a subset of functions that are problematic or if there is something else about removing __meminit that causes the problem to go away.

  9. nickdesaulniers commented on Aug 10, 2026

    @nickdesaulniers
    nickdesaulniers
    on Aug 10, 2026
    Member

    What's __meminit? Are we perhaps hitting a kCFI failure during early boot?

  10. nathanchance commented on Aug 10, 2026

    @nathanchance
    nathanchance
    on Aug 10, 2026
    MemberAuthor

    What's __meminit?

    __init when CONFIG_MEMORY_HOTPLUG is disabled.

    Are we perhaps hitting a kCFI failure during early boot?

    I am pretty sure I tested with CONFIG_CFI_PERMISSIVE=y to rule that out but I do not think it helped... I will double check though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    [ARCH] risc-vThis bug impacts ARCH=riscv[FEATURE] CFIRelated to building the kernel with Clang Control Flow Integrityboot failureThis issue results in a failure to boot

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions