Linux build: silence objtool warnings

[AttilaFueloep]

Motivation and Context

Follow up for #17401 (3084336).

Description

After #17401 the Linux build produces some stack related warnings. Silence them with the STACK_FRAME_NON_STANDARD macro.

How Has This Been Tested?

Built and loaded the module, verified that __warn_thunk isn't called.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Quality assurance (non-breaking change which makes the code more robust against bugs)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[AttilaFueloep]

Looks like RHEL 8 is missing the STACK_FRAME_NON_STANDARD assembler macro but has the C #define. Need to check the RHEL 8 Linux sources to find a solution.

[john-cabaj]

I've been running into this warning, though as a build error, while trying to get a zfs build for 6.15. After bisecting the kernel, it appears to be failing the build because of this commit:

36799069b48198e5ce92d99310060c4aecb4b3e3 is the first bad commit
commit 36799069b48198e5ce92d99310060c4aecb4b3e3
Author: Josh Poimboeuf <jpoimboe@kernel.org>
Date:   Fri Mar 14 12:29:11 2025 -0700

    objtool: Add CONFIG_OBJTOOL_WERROR
    
    Objtool warnings can be indicative of crashes, broken live patching, or
    even boot failures.  Ignoring them is not recommended.
    
    Add CONFIG_OBJTOOL_WERROR to upgrade objtool warnings to errors by
    enabling the objtool --Werror option.  Also set --backtrace to print the
    branches leading up to the warning, which can help considerably when
    debugging certain warnings.
    
    To avoid breaking bots too badly for now, make it the default for real
    world builds only (!COMPILE_TEST).
    
    Co-developed-by: Brendan Jackman <jackmanb@google.com>
    Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
    Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
    Link: https://lore.kernel.org/r/3e7c109313ff15da6c80788965cc7450115b0196.1741975349.git.jpoimboe@kernel.org

 lib/Kconfig.debug    | 11 +++++++++++
 scripts/Makefile.lib |  1 +
 2 files changed, 12 insertions(+)

[AttilaFueloep]

Thanks for noticing. I'd assume that this PR would fix the build for you. Did you try? (The failing check is due to an ancient kernel.)

[john-cabaj]

I can try shortly. I'm disabling the option in the kernel first, then will come back and try this patch (while re-enabling the config).

[behlendorf]

@AttilaFueloep for RHEL 8 adding a check to config/kernel-objtool.m4 which attempts to use STACK_FRAME_NON_STANDARD seems like the best thing to do. If the check fails we can define STACK_FRAME_NON_STANDARD and the assembler macro ourselves.

As you'd expect, I did verify the quick fix of adding only the missing assembler macro does work but that's going to be fragile long term.

[AttilaFueloep]

@behlendorf Well it's complicated. Newer kernels have an assembler macro for marking the stack non standard. for certain functions whereas the RHEL 8 kernel has not. To check for the macro inside config/kernel-objtool.m4 we'd need
tooling to assemble an .S file, which is currently missing. I'm currently trying to wrap my head around how to add this. The major problem is that you can't build a loadable kernel module out of assembly due to the MODULE_*() macros from module.h but all the tooling expects to build a module, so my added tooling fails in the modpost stage.

[john-cabaj]

@AttilaFueloep - this patch + CONFIG_OBJTOOL_WERROR=y works, as does CONFIG_OBJTOOL_WERROR=n

[AttilaFueloep]

@behlendorf

As you'd expect, I did verify the quick fix of adding only the missing assembler macro does work but that's going to be fragile long term.

Exactly, that's why my current plan is to check for the macro inside config/kernel-objtool.m4 and add an .section .discard... to the assembly file if it's missing, this would also also avoid copy pasting the GPL'ed macro.

[AttilaFueloep]

@john-cabaj Nice, thanks for confirming.

[behlendorf]

we'd need tooling to assemble an .S file, which is currently missing

Verifying we can build the kmod would absolutely be best, so don't let me deter you from getting that working. But adding a primitive check for the oddball case where the #define is there by the .macro is missing is perhaps good enough for this corner case. Something like this, d71dfa0, for the autotools bit.

[AttilaFueloep]

Yeah, nice. Looking at all the MODULE_* #defines I guess it would be quite challenging to make a module out of assembly. Given this blocks 2.2.8, I'll go with the grep approach, at least for now. (I'll file the changes I've made so far for a later retry.) I'll try to come up with something in a couple of hours. (Currently busy with other things.)

[AttilaFueloep]

If this passes the CI I'll squash and rebase.

[AttilaFueloep]

Squashed, rebased and added a refinement to the grep regexp I forgot to commit.

[AttilaFueloep]

Searching the build logs reveals that there still are some objtool warnings left. On newer kernels (Alma10 and Fedora builders, locally on Arch) objtool complains about {spl_panic(),luaD_throw()} is missing a __noreturn annotation. I'll try to address them in a follow up commit.

The Ubuntu22 builder has this strange warning which isn't seen on the other builders:

/home/zfs/zfs/module/zstd/lib/decompress/zstd_decompress_block.o: warning: objtool: ZSTD_decompressSequences_bmi2.constprop.0()+0x90c: call without frame pointer save/setup

Not sure what to make out of it.

[AttilaFueloep]

@john-cabaj It seems you are not seeing the remaining objtool warnings on your builds. Is something special with your kernel builds or does it fix itself on 6.15? Which distro are you using?

[AttilaFueloep]

The spl_panic () is tricky. ELF doesn't support the noreturn attribute so objtool needs to maintain a global list of extern noreturn functions. spl_panic(), if called from other compilation units and declared as noreturn, triggers falls through to next function warnings since the compiler doesn't add a return at the end and objtool doesn't know about it. So either way we end up with warnings.

The proper fix would be to add spl_panic() to the above mentioned list, which, for known reasons, won't happen. Not sure how to solve this, STACK_FRAME_NON_STANDARD doesn't help here.

[behlendorf]

@AttilaFueloep why don't we go ahead and merge this while figuring out the best way to handle the remaining objtool warnings. I can squash the commits when merging if you're okay with merging this.

{spl_panic(),luaD_throw()} is missing a __noreturn annotation

I don't yet see a good way to handle this while preserving the existing spl_panic() behavior. But I'll give it some thought.

[AttilaFueloep]

why don't we go ahead and merge this

Yes, please.

[AttilaFueloep]

@behlendorf I don't think that it's an issue with the spl_panic() design but rather an objtool limitation. Declaring spl_panic() noreturn works but creates a ton of fall through to next function warnings. This is documented, objtool needs a baked in list of noreturn functions to handle external calls to noreturn functions correctly. I'll open a separate issue later, and elaborate more on this there.

[AttilaFueloep]

@behlendorf Spelling out a problem always helps to understand it better.

Here's a rough outline for a solution:

1. Add a static function, say local_panic() via debug.h to each compilation unit. This one calls panic() from local scope.

2. Call spl_panic() with a pointer to local_panic() and use that instead.

Let me try it out.

[AttilaFueloep]

Nope, even worse.

[AttilaFueloep]

Found a way to silence the remaining objtool warnings, preparing a PR.

[AttilaFueloep]

I've run out of ideas how to fix the remaining warnings. Let's see if distros will adopt CONFIG_OBJTOOL_WERROR=y. I did a quick search, Arch and openSUSE do not. Ubuntu seems to work with it enabled, no idea why the noreturn warnings aren't a problem there (#17410 (comment), #17410 (comment)). EL sources I can't look at without paying them, so no dice.

That's pretty much it, I'll call it a day for now. To reiterate, the proper fix would be to add our external noreturn functions to objtools noretruns.h. I tried and it works fine, it would just need a bit more work on lua{,L}_error() and friends .

[john-cabaj]

@AttilaFueloep - I'm packaging ZFS for Ubuntu at present. CONFIG_OBJTOOL_WERROR=y was a problem for the 6.15 kernel, and the first zfs release for 6.15 disabled it. But as I said above, the latest patch fixed the issue and we've re-enabled CONFIG_OBJTOOL_WERROR.

[AttilaFueloep]

@john-cabaj Yes, I got that, just wondering why it works. If I build with this patch on Arch with a 6.14 kernel I still see some objtool warnings (".. misses __noreturn .."). Being valid warnings, I'd expect them to occur with the Ubuntu 6.15 kernel as well, failing the build if CONFIG_OBJTOOL_WERROR is set. Obviously it does not, maybe 6.15 related. Arch just released 6.15 so I'll give it a whirl once I've a minute.

[behlendorf]

@AttilaFueloep thanks for continuing to work on this. I'm still mulling it over myself. I'm going to work on wrapping up the 2.2.8 release with this change. We'll have to see in practice how much of a problem CONFIG_OBJTOOL_WERROR=y ends up being.

[AttilaFueloep]

@behlendorf Just tried a build on Arch + Linux 6.15 + --Werror and it fails. No idea why it succeeds on Ubuntu. Arch comes with CONFIG_OBJTOOL_WERROR disabled, I'd guess to many out-of-tree modules would break otherwise. I'd assume most distros will do the same. The major pitfall would be Red Hat based distros, but there's not much we can do about it.

We'll have to see in practice how much of a problem CONFIG_OBJTOOL_WERROR=y ends up being.

Agreed, let's just wait and hope the best, at least we're prepared.

[AttilaFueloep]

@behlendorf Thinking more about it, we can always do something like this. I don't like it but it's the best we can do I guess. What do you think?

AttilaFueloep@85c414e

[AttilaFueloep]

I can wrap this up in an auto check for --Werror.

[behlendorf]

That's clever. My main concern is that it has the potential to be fragile. We'd want to add auto tools checks for 1) CONFIG_OBJTOOL_WERROR=y / --Werror and that 2) objtool exists at the expected path, then put it behind a --enable-objtool-werror=yes|no|check flag so it can be controlled by packagers if needed. It doesn't solve the --enable-linux-builtin case, but at that point we could simply patch objtool ourselves. I think it's the best option so far, lets give it a try.

[AttilaFueloep]

Yeah, mostly reflects my thoughts, modulo the check for -Werror. We strip it from the flags anyhow, so no need to check if objtool supports it. I can add it if you prefer it this way though.

It doesn't solve the --enable-linux-builtin case

Good point, I'd say that's something for a follow-up.

So I'll do the auto tools dance and open a PR.

[AttilaFueloep]

By the way, what would be the best way to check for CONFIG_OBJTOOL_WERROR inside auto tools. I looked around but didn't found something obvious, I'm quite a noob regarding auto tools. Would an
AS_IF([test $CONFIG_OBJTOOL_WERROR == y].....) do the trick?

[behlendorf]

modulo the check for -Werror

Yeah, this probably isn't needed. My thinking was that if it's not supported we don't need to do any of this, but that's already covered by CONFIG_OBJTOOL_WERROR=y. So I'm fine with not adding it.

AS_IF([test $CONFIG_OBJTOOL_WERROR == y].....) do the trick?

config/kernel-config-defined.m4 contains a bunch CONFIG_* option checks, we can model this check after those. In this case since it's a boolean we should simply be able to check if it's defined or not.

[AttilaFueloep]

All right, I'll skip the -Werror check.

config/kernel-config-defined.m4 contains a bunch CONFIG_* option checks

Thanks, saved me some time!

[AttilaFueloep]

So I'll do the auto tools dance and open a PR.

#17456

AS_IF([test $CONFIG_OBJTOOL_WERROR == y].....) do the trick?

That was a silly question, I tend to forget that we're testing things by trying to compile things.