Release notes from dwarves

v1.30: CI testing:

2025-04-18T15:25:09Z

support for github CI tests to build pahole with gcc
and LLVM.
support for github CI tests to build pahole, a kernel
along with BTF using that pahole and run tests.
tests can also be run standalone; see toplevel README
for details.

DWARF loader:

better detection of abort during thread processing.

BTF encoder:

pahole now uses an improved scheme to detect presence of
newer libbpf functions for cases where pahole is built with
a non-embedded libbpf. A local weak declaration is added,
and if the function is non-NULL - indicating it is present -
the associated feature is avaialble. BTF feature detection
makes use of this now and BTF features declared in pahole
can provide a feature check function.
Type tags are now emitted for bpf_arena pointers if the
attributes btf_feature is specified.
kfunc tagging has been refactored into btf_encoder__collect_kfuncs
to simplify from the previous two-stage collect/tag process.
To support global variables other than per-CPU variables, code
was added to match a variable with the relevant section. However
variables in to-be-discarded sections have address value 0 and
appeared to be in the per-CPU section (since it starts at 0).
Add checks to ensure the variable really is in the relevant
ELF section.
To avoid expensive variable address checking in the above case,
filter out variables prefixed by _gendwarfksyms_ptr which are
present when CONFIG_GENDWARFKSYMS is set.
Memory access bugs reported by address sanitizer were also fixed.

Signed-off-by: Alan Maguire alan.maguire@oracle.com
Signed-off-by: Arnaldo Carvalho de Melo acme@redhat.com

v1.29: DWARF loader:

2025-01-21T15:01:32Z

Multithreading is now contained in the DWARF loader using a jobs queue and a
pool of worker threads.

BTF encoder:

The parallel reproducible BTF generation done using the new DWARF loader
multithreading model is as fast as the old non-reproducible one and thus is
now always performed, making the "reproducible_build" flag moot.

The memory consumption is now greatly reduced as well.

BTF loader:

Support for multiple BTF_DECL_TAGs pointing to same tag.

Example:

$ pfunct vmlinux -F btf -f bpf_rdonly_cast
bpf_kfunc bpf_fastcall void *bpf_rdonly_cast(const void *obj__ign, u32 btf_id__k);
$

Regression tests:

Verify that pfunct prints btf_decl_tags read from BTF.

pfunct:

Don't print functions twice when using -f.

Signed-off-by: Arnaldo Carvalho de Melo acme@redhat.com

v1.28: pahole:

2024-12-07T14:06:35Z

Various improvements to reduce the memory footprint of pahole, notably when
doing BTF encoding.
Show flexible arrays statistics, it detects them at the end of member types,
in the middle, etc. This should help with the efforts to spot problematic
usage of flexible arrays in the kernel sources, examples:

https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?id=6ab5318f536927cb
Introduce --with_embedded_flexible_array option.
Add '--padding N' to show only structs with N bytes of padding.
Add '--padding_ge N' to show only structs with at least N bytes of padding.
Introduce --running_kernel_vmlinux to find a vmlinux that matches the
build-id of the running kernel, e.g.:

$ pahole --running_kernel_vmlinux
/usr/lib/debug/lib/modules/6.11.7-200.fc40.x86_64/vmlinux
$ rpm -qf /usr/lib/debug/lib/modules/6.11.7-200.fc40.x86_64/vmlinux
kernel-debuginfo-6.11.7-200.fc40.x86_64
$

This is a shortcut to find the right vmlinux to use for the running kernel
and helps with regression tests.

pfunct:

Don't stop at the first function that matches a filter, show all of them.

BTF Encoder:

Allow encoding data about all global variables, not just per CPU ones.

There are several reasons why type information for all global variables to be
useful in the kernel, including drgn without DWARF, __ksym BPF programs return
type.

This is non-default, experiment with it using 'pahole --btf-features=+global_var'
Handle .BTF_ids section endianness, allowing for cross builds involving
machines with different endianness to work.

For instance, encoding BTF info on a s390 vmlinux file on a x86_64 workstation.
Generate decl tags for bpf_fastcall for eligible kfuncs.
Add "distilled_base" BTF feature to split BTF generation.
Use the ELF_C_READ_MMAP mode with libelf, reducing peak memory utilization.

BTF Loader:

Allow overiding /sys/kernel/btf/vmlinux with some other file, for testing,
via the PAHOLE_VMLINUX_BTF_FILENAME environment variable.

DWARF loader:

Allow setting the list of compile units produced from languages to skip via
the PAHOLE_LANG_EXCLUDE environment variable.
Serialize access to elfutils dwarf_getlocation() to avoid elfutils internal
data structure corruption when running multithreaded pahole.
Honour --lang_exclude when merging LTO built CUs.
Add the debuginfod client cache directory to the vmlinux search path.
Print the CU's language when a tag isn't supported.
Initial support for the DW_TAG_GNU_formal_parameter_pack,
DW_TAG_GNU_template_parameter_pack, DW_TAG_template_value_param and
DW_TAG_template_type_param DWARF tags.
Improve the parameter parsing by checking DW_OP_[GNU_]entry_value, this
makes some more functions to be made eligible by the BTF encoder, for instance
the perf_event_read() in the 6.11 kernel.

Core:

Use pahole to help in reorganizing its data structures to reduce its memory
footprint.

Regression tests:

Introduce a tests/ directory for adding regression tests, run it with:

$ tests/tests

Or run the individual tests directly.
Add a regression test for the reproducible build feature that establishes
as a baseline a detached BTF file without asking for a reproducible build and
then compares the output of 'bpftool btf dump file' for this file with the one
from BTF reproducible build encodings done with a growing number or threads.
Add a regression test for the flexible arrays features, checking if the various
comments about flexible arrays match the statistics at the final of the pahole
pretty print output.
Add a test that checks if pahole fails when running on a BTF system and BTF was
requested, previously it was falling back to DWARF silently.
Add test validating BTF encoding, reasons we skip functions: DWARF functions
that made it into BTF match signatures, functions we say we skipped, we did
indeed skip them in BTF encoding and that it was correct to skip these
functions.
Add regression test for 'pahole --prettify' that uses perf to record a simple
workload and then pretty print the resulting perf.data file to check that what
is produced are the expected records for such a file.

Link: https://lore.kernel.org/all/Z0jVLcpgyENlGg6E@x1/
Tested-by: Alan Maguire alan.maguire@oracle.com
Tested-by: Jiri Olsa jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo acme@redhat.com

v1.27: BTF encoder:

2024-06-11T19:33:03Z

Inject kfunc decl tags into BTF from the BTF IDs ELF section in the Linux
kernel vmlinux file.

This allows tools such as bpftools and pfunct to enumerate the available kfuncs
and to gets its function signature, the type of its return and of its
arguments. See the example in the BTF loader changes description, below.
Support parallel reproducible builds, where it doesn't matter how many
threads are used, the end BTF encoding result is the same.
Sanitize unsupported DWARF int type with greater-than-16 byte, as BTF doesn't
support it.

BTF loader:

Initial support for BTF_KIND_DECL_TAG:

$ pfunct --prototypes -F btf vmlinux.btf.decl_tag,decl_tag_kfuncs | grep ^bpf_kfunc | head
bpf_kfunc void cubictcp_init(struct sock * sk);
bpf_kfunc void cubictcp_cwnd_event(struct sock * sk, enum tcp_ca_event event);
bpf_kfunc void cubictcp_cong_avoid(struct sock * sk, u32 ack, u32 acked);
bpf_kfunc u32 cubictcp_recalc_ssthresh(struct sock * sk);
bpf_kfunc void cubictcp_state(struct sock * sk, u8 new_state);
bpf_kfunc void cubictcp_acked(struct sock * sk, const struct ack_sample * sample);
bpf_kfunc int bpf_iter_css_new(struct bpf_iter_css * it, struct cgroup_subsys_state * start, unsigned int flags);
bpf_kfunc struct cgroup_subsys_state * bpf_iter_css_next(struct bpf_iter_css * it);
bpf_kfunc void bpf_iter_css_destroy(struct bpf_iter_css * it);
bpf_kfunc s64 bpf_map_sum_elem_count(const struct bpf_map * map);
$ pfunct --prototypes -F btf vmlinux.btf.decl_tag,decl_tag_kfuncs | grep ^bpf_kfunc | wc -l
116
$

pretty printing:

Fix hole discovery with inheritance in C++.

Tested-by: Alan Maguire alan.maguire@oracle.com
Tested-by: Daniel Xu dxu@dxuuu.xyz
Tested-by: Jiri Olsa olsajiri@gmail.com
Link: https://lore.kernel.org/all/ZmIXxgbfIJGWmXer@x1/T/#u
Signed-off-by: Arnaldo Carvalho de Melo acme@redhat.com

v1.26: pahole:

2024-02-28T19:25:31Z

When expanding types using 'pahole -E' do it for union and struct typedefs and for enums too.

E.g: that 'state' field in 'struct module':

$ pahole module | head
struct module {
enum module_state state; /* 0 4 */

      /* XXX 4 bytes hole, try to pack */

      struct list_head           list;                 /*     8    16 */
      char                       name[56];             /*    24    56 */
      /* --- cacheline 1 boundary (64 bytes) was 16 bytes ago --- */
      struct module_kobject      mkobj;                /*    80    96 */
      /* --- cacheline 2 boundary (128 bytes) was 48 bytes ago --- */

now gets expanded:

$ pahole -E module | head
struct module {
enum module_state {
MODULE_STATE_LIVE = 0,
MODULE_STATE_COMING = 1,
MODULE_STATE_GOING = 2,
MODULE_STATE_UNFORMED = 3,
} state; /* 0 4 */

      /* XXX 4 bytes hole, try to pack */

Print number of holes, bit holes and bit paddings in class member types.

Doing this recursively to show how much waste a complex data structure has
is something that still needs to be done, there were the low hanging fruits
on the path to having that feature.

For instance, for 'struct task_struct' in the Linux kernel we get this
extra info:

--- task_struct.before.c 2024-02-09 11:38:39.249638750 -0300
+++ task_struct.after.c 2024-02-09 16:19:34.221134835 -0300
@@ -29,6 +29,12 @@

    /* --- cacheline 2 boundary (128 bytes) --- */
    struct sched_entity        se;                   /*   128   256 */

```
/* XXX last struct has 3 holes */
```

/* --- cacheline 6 boundary (384 bytes) --- */
struct sched_rt_entity     rt;                   /*   384    48 */
struct sched_dl_entity     dl;                   /*   432   224 */

```
  /* XXX last struct has 1 bit hole */
```

/* --- cacheline 10 boundary (640 bytes) was 16 bytes ago --- */
const struct sched_class  * sched_class;         /*   656     8 */
struct rb_node             core_node;            /*   664    24 */

@@ -100,6 +103,9 @@
/* --- cacheline 35 boundary (2240 bytes) was 16 bytes ago --- /
struct list_head tasks; / 2256 16 /
struct plist_node pushable_tasks; / 2272 40 */
+

```
/* XXX last struct has 1 hole */
```

/* --- cacheline 36 boundary (2304 bytes) was 8 bytes ago --- */
struct rb_node             pushable_dl_tasks;    /*  2312    24 */
struct mm_struct *         mm;                   /*  2336     8 */

@@ -172,6 +178,9 @@
/* XXX last struct has 4 bytes of padding */

    struct vtime               vtime;                /*  2744    48 */

```
/* XXX last struct has 1 hole */
```

/* --- cacheline 43 boundary (2752 bytes) was 40 bytes ago --- */
atomic_t                   tick_dep_mask;        /*  2792     4 */

@@ -396,9 +405,12 @@
/* --- cacheline 145 boundary (9280 bytes) --- /
struct thread_struct thread attribute((aligned(64))); / 9280 4416 */

  /* XXX last struct has 1 hole, 1 bit hole */

/* size: 13696, cachelines: 214, members: 262 */
/* sum members: 13518, holes: 21, sum holes: 162 */
/* sum bitfield members: 82 bits, bit holes: 2, sum bit holes: 46 bits */
/* member types with holes: 4, total: 6, bit holes: 2, total: 2 */
/* paddings: 6, sum paddings: 49 */
/* forced alignments: 2, forced holes: 2, sum forced holes: 88 */

};

Introduce --contains_enumerator=ENUMERATOR_NAME:

E.g.:

$ pahole --contains_enumerator S_VERSION
enum file_time_flags {
       S_ATIME   = 1,
       S_MTIME   = 2,
       S_CTIME   = 4,
       S_VERSION = 8,
}
$

The shorter form --contains_enum is also accepted.

Fix pretty printing when using DWARF, where sometimes the class (-C) and a specified "type_enum",
may not be present on the same CU, so wait till both are found.

Now this example that reads the 'struct perf_event_header' and 'enum perf_event_type'
from the DWARF info in ~/bin/perf to pretty print records in the perf.data file works
just like when using type info from BTF in ~/bin/perf:

$ pahole -F dwarf -V ~/bin/perf \
          --header=perf_file_header \
          --seek_bytes '$header.data.offset' \
          --size_bytes='$header.data.size' \
          -C 'perf_event_header(sizeof,type,type_enum=perf_event_type,filter=type==PERF_RECORD_MMAP2)' \
          --prettify perf.data --count 1
pahole: sizeof_operator for 'perf_event_header' is 'size'
pahole: type member for 'perf_event_header' is 'type'
pahole: type enum for 'perf_event_header' is 'perf_event_type'
pahole: filter for 'perf_event_header' is 'type==PERF_RECORD_MMAP2'
pahole: seek bytes evaluated from --seek_bytes=$header.data.offset is 0x3f0
pahole: size bytes evaluated from --size_bytes=$header.data.size is 0xd10
// type=perf_event_header, offset=0xc20, sizeof=8, real_sizeof=112
{
      .header = {
              .type = PERF_RECORD_MMAP2,
              .misc = 2,
              .size = 112,
      },
      .pid = 1533617,
      .tid = 1533617,
      .start = 94667542700032,
      .len = 90112,
      .pgoff = 16384,{
              .maj = 0,
              .min = 33,
              .ino = 35914923,
              .ino_generation = 26870,
      },{
              .build_id_size = 0,
              .__reserved_1 = 0,
              .__reserved_2 = 0,
              .build_id = { 33, 0, 0, 0, -85, 4, 36, 2, 0, 0, 0, 0, -10, 104, 0, 0, 0, 0, 0, 0 },
      },
      .prot = 5,
      .flags = 2,
      .filename = "/usr/bin/ls",
},
$

DWARF loader:

Add support for DW_TAG_constant, first seen in Go DWARF.
Fix loading DW_TAG_subroutine_type generated by the Go compiler, where it may
have a DW_AT_byte_size. Go DWARF. And pretty print it as if
it was from C, this helped in writing BPF programs to attach to Go binaries, using
uprobes.

BTF loader:

Fix loading of 32-bit signed enums.

BTF encoder:

Add 'pahole --btf_features' to allow consumers to specify an opt-in set of
features they want to use in BTF encoding.

Supported features are a comma-separated combination of
```
    encode_force    Ignore invalid symbols when encoding BTF.
    var             Encode variables using BTF_KIND_VAR in BTF.
    float           Encode floating-point types in BTF.
    decl_tag        Encode declaration tags using BTF_KIND_DECL_TAG.
    type_tag        Encode type tags using BTF_KIND_TYPE_TAG.
    enum64          Encode enum64 values with BTF_KIND_ENUM64.
    optimized_func  Encode representations of optimized functions
                    with suffixes like ".isra.0" etc
    consistent_func Avoid encoding inconsistent static functions.
                    These occur when a parameter is optimized out
                    in some CUs and not others, or when the same
                    function name has inconsistent BTF descriptions
                    in different CUs.
```
Specifying "--btf_features=all" is the equivalent to setting all of the
above. If pahole does not know about a feature specified in
--btf_features it silently ignores it.

The --btf_features can either be specified via a single comma-separated
list
--btf_features=enum64,float

...or via multiple --btf_features values
```
    --btf_features=enum64 --btf_features=float
```
These properties allow us to use the --btf_features option in the kernel
scripts/pahole_flags.sh script to specify the desired set of BTF
features.

If a feature named in --btf_features is not present in the version of
pahole used, BTF encoding will not complain. This is desired because it
means we no longer have to tie new features to a specific pahole
version.

Use --btf_features_strict to change that behaviour and bail out if one of
the requested features isn't present.

To see the supported features, use:

$ pahole --supported_btf_features
encode_force,var,float,decl_tag,type_tag,enum64,optimized_func,consistent_func
$

btfdiff:

Parallelize loading BTF and DWARF, speeding up a bit.
Do type expansion to cover "private" types and enumerations.

Signed-off-by: Arnaldo Carvalho de Melo acme@redhat.com

v1.24: BTF encoder:

2022-08-22T23:29:04Z

Add support to BTF_KIND_ENUM64 to represent enumeration entries
with more than 32 bits.
Support multithreaded encoding, in addition to DWARF
multithreaded loading, speeding up the process.

Selected just like DWARF multithreaded loading, using the
'pahole -j' option.
Encode 'char' type as signed.

BTF Loader:

Add support to BTF_KIND_ENUM64.

pahole:

Introduce --lang and --lang_exclude to specify the language the
DWARF compile units were originated from to use or filter.

Use case is to exclude Rust compile units while aspects of the
DWARF generated for it get sorted out in a way that the kernel
BPF verifier don't refuse loading the BTF generated from them.
Introduce --compile to generate compilable code in a similar fashion to:

bpftool btf dump file vmlinux format c > vmlinux.h

As with 'bpftool', this will notice type shadowing, i.e. multiple types
with the same name and will disambiguate by adding a suffix.
Don't segfault when processing bogus files.

Signed-off-by: Arnaldo Carvalho de Melo acme@redhat.com

v1.22: pahole:

2021-08-23T13:10:10Z

Allow encoding BTF to a separate BTF file (detached) instead of to a new
".BTF" ELF section in the file being encoded (vmlinux usually).
Introduce -j/--jobs option to specify the number of threads to use. Without
arguments means one thread per CPU. So far used for the DWARF loader, will
be used as well for the BTF encoder.
Show all different types with the same name, not just the first one found.
Introduce sorted type output (--sort), needed with multithreaded DWARF loading,
to use with things like 'btfdiff' that expects the output from DWARF and BTF
types to be comparable using 'diff'.
Stop assuming that reading from stdin means pretty printing as this broke
pre-existing scripts, introduce a explicit --prettify command line option.
Improve type resolution for the --header command line option.
Disable incomplete CTF encoder, this needs to be done using the external
libctf library.
Do not consider the ftrace filter when encoding BTF for kernel functions.
Add --kabi_prefix to avoid deduplication woes when using _RH_KABI_REPLACE(),
Add --with_flexible_array to show just types with flexible arrays.

DWARF Loader:

Multithreaded loading, requires elfutils >= 0.178.
Lock calls to non-thread safe elfutils' libdw functions (dwarf_decl_file()
and dwarf_decl_line())
Change hash table size to one that performs better with current typical
vmlinux files.
Allow tweaking the hash table size from the command line.
Stop allocating memory for strings obtained from libdw, just defer freeing
the Dwfl handler so that references to its strings can be safely kept.
Use a frontend cache for the latest lookup result.
Allow ignoring some DWARF tags when loading for encoding BTF, as BTF doesn't
have equivalents for things like DW_TAG_inline_expansion and DW_TAG_label.
Allow ignoring some DWARF tag attributes, such as DW_AT_alignment, not used
when encoding BTF.
Do not query for non-C attributes when loading a C language CU (compilation unit).

BTF encoder:

Preparatory work for multithreaded encoding, the focus for 1.23.

btfdiff:

Support diffing against a detached BTF file, e.g.: 'btfdiff vmlinux vmlinux.btf'
Support multithreaded DWARF loading, using the new pahole --sort option to have
the output from both BTF and DWARF sorted and thus comparable via 'diff'.

Build:

Support building with libc libraries lacking either obstacks or argp, such
as Alpine Linux's musl libc.
Support systems without getconf() to obtain the data cacheline size, such
as musl libc.
Add a buildcmd.sh for test builds, tested using the same set of containers
used for testing the Linux kernel perf tools.
Enable selecting building with a shared libdwarves library or statically.
Allow to use the libbpf package found in distributions instead of with the
accompanying libbpf git submodule.

Cleanups:

Address lots of compiler warnings accumulated by not using -Wextra, it'll
be added in the next release after allowing not to use it to build libbpf.
Address covscan report issues.

Documentation:

Improve the --nr_methods/-m pahole man page entry.
Clarify that currently --nr_methods doesn't work together witn -C.

Signed-off-by: Arnaldo Carvalho de Melo acme@redhat.com

v1.21: DWARF loader:

2021-04-12T14:53:29Z

Handle DWARF5 DW_OP_addrx properly

Part of the effort to support the subset of DWARF5 that is generated when building the kernel.
Handle subprogram ret type with abstract_origin properly

Adds a second pass to resolve abstract origin DWARF description of functions to aid
the BTF encoder in getting the right return type.
Check .notes section for LTO build info

When LTO is used, currently only with clang, we need to do extra steps to handle references
from one object (compile unit, aka CU) to another, a way for DWARF to avoid duplicating
information.
Check .debug_abbrev for cross-CU references

When the kernel build process doesn't add an ELF note in vmlinux indicating that LTO was
used and thus intra-CU references are present and thus we need to use a more expensive
way to resolve types and (again) thus to encode BTF, we need to look at DWARF's .debug_abbrev
ELF section to figure out if such intra-CU references are present.
Permit merging all DWARF CU's for clang LTO built binary

Allow not trowing away previously supposedly self contained compile units
(objects, aka CU, aka Compile Units) as they have type descriptions that will
be used in later CUs.
Permit a flexible HASHTAGS__BITS

So that we can use a more expensive algorithm when we need to keep previously processed
compile units that will then be referenced by later ones to resolve types.
Use a better hashing function, from libbpf

Enabling patch to combine compile units when using LTO.

BTF encoder:

Add --btf_gen_all flag

A new command line to allow asking for the generation of all BTF encodings, so that we
can stop adding new command line options to enable new encodings in the kernel Makefile.
Match ftrace addresses within ELF functions

To cope with differences in how DWARF and ftrace describes function boundaries.
Funnel ELF error reporting through a macro

To use libelf's elf_error() function, improving error messages.
Sanitize non-regular int base type

Cope with clang with dwarf5 non-regular int base types, tricky stuff, see yhs
full explanation in the relevant cset.
Add support for the floating-point types

S/390 has floats'n'doubles in its arch specific linux headers, cope with that.

Pretty printer:

Honour conf_fprintf.hex when printing enumerations

If the user specifies --hex in the command line, honour it when printing enumerations.

Signed-off-by: Arnaldo Carvalho de Melo acme@redhat.com

v1.20:

2021-02-04T21:34:13Z

BTF encoder:

Improve ELF error reporting using elf_errmsg(elf_errno()).
Improve objcopy error handling.
Fix handling of 'restrict' qualifier, that was being treated as a 'const'.
Support SHN_XINDEX in st_shndx symbol indexes, to handle ELF objects with
more than 65534 sections, for instance, which happens with kernels built
with 'KCFLAGS="-ffunction-sections -fdata-sections", Other cases may
include when using FG-ASLR, LTO.
Cope with functions without a name, as seen sometimes when building kernel
images with some versions of clang, when a SEGFAULT was taking place.
Fix BTF variable generation for kernel modules, not skipping variables at
offset zero.
Fix address size to match what is in the ELF file being processed, to fix using
a 64-bit pahole binary to generate BTF for a 32-bit vmlinux image.
Use kernel module ftrace addresses when finding which functions to encode,
which increases the number of functions encoded.

libbpf:

Allow use of packaged version, for distros wanting to dynamically link with
the system's libbpf package instead of using the libbpf git submodule shipped
in pahole's source code.

DWARF loader:

Support DW_AT_data_bit_offset

This appeared in DWARF4 but is supported only in gcc's -gdwarf-5,
support it in a way that makes the output be the same for both cases.

$ gcc -gdwarf-5 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
long int a:1; /* 0: 0 8 /
long int b:1; / 0: 1 8 /
long int c:1; / 0: 2 8 */
```
    /* XXX 29 bits hole, try to pack */
    /* Bitfield combined with next fields */

    int                        after_bitfield;       /*     4     4 */

    /* size: 8, cachelines: 1, members: 4 */
    /* sum members: 4 */
    /* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
    /* last cacheline: 8 bytes */
```
};
DW_FORM_implicit_const in attr_numeric() and attr_offset()
Support DW_TAG_GNU_call_site, its the standardized rename of the previously supported
DW_TAG_GNU_call_site.

build:

- Fix compilation on 32-bit architectures.

Signed-off-by: Arnaldo Carvalho de Melo acme@redhat.com

v1.19:

2020-11-23T19:21:26Z

Support split BTF, where a main BTF file, vmlinux, can be used to find types
and then a kernel module, for instance, can have just what is unique to it.

For instance, looking for a type in the main vmlinux BTF info:

$ pahole wmi_notify_handler
pahole: type 'wmi_notify_handler' not found
$

If we look at the 'wmi' module BTF info that is in:

$ ls -la /sys/kernel/btf/wmi
-r--r--r--. 1 root root 2866 Nov 18 13:35 /sys/kernel/btf/wmi
$

$ pahole /sys/kernel/btf/wmi -C wmi_notify_handler
typedef void (*wmi_notify_handler)(u32, void *);
$

'--btf_base=/sys/kernel/btf/vmlinux' was automatically added in this last
example, an option that was also introduced in this version where types used in
the wmi.ko module but present in vmlinux can be found so that there is no
duplicity of types.
Update libbpf to get the split BTF support and use some of its functions to
load BTF and speed up DWARF loading and BTF encoding.
Support cross-compiled ELF binaries with different endianness
Support showing typedefs for anonymous types, like structs, unions and enums,
see the "Align enumerators" entry below for an example, another:

$ pahole rwlock_t
typedef struct {
arch_rwlock_t raw_lock; /* 0 8 */
```
      /* size: 8, cachelines: 1, members: 1 */
      /* last cacheline: 8 bytes */
```
} rwlock_t;
$
Align enumerators:

$ pahole ZSTD_strategy
typedef enum {
ZSTD_fast = 0,
ZSTD_dfast = 1,
ZSTD_greedy = 2,
ZSTD_lazy = 3,
ZSTD_lazy2 = 4,
ZSTD_btlazy2 = 5,
ZSTD_btopt = 6,
ZSTD_btopt2 = 7,
} ZSTD_strategy;
$
Workaround bugs in the generation of DWARF records for functions in some gcc
versions that were causing breakage in the encoding of BTF:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060 "Missing DW_AT_declaration=1 in dwarf data"
Ignore zero-sized ELF symbols instead of erroring out.
Handle union forward declaration properly in the BTF loader.
Introduce --numeric_version for use in scripts and Makefiles:

$ pahole --version
v1.19
$ pahole --numeric_version
119
$

To avoid things like this in the kernel's scripts/link-vmlinux.sh:

pahole_ver=$(${PAHOLE} --version | sed -E 's/v([0-9]+).([0-9]+)/\1\2/')
Try sole pfunct argument as a function name, just like pahole with type names:

$ pfunct tcp_v4_rcv
int tcp_v4_rcv(struct sk_buff * skb);
$
Speed up pfunct using some of the load techniques used in pahole.
Discard CUs after BTF encoding as they're not used anymore, greatly reducing
memory usage and speeding up vmlinux BTF encoding.
Revamp how per-CPU variables are encoded in BTF.
Include BTF info for static functions.
Use BTF's string APIs for strings management, greatly improving performance
over the tsearch().
Increase size of DWARF lookup hash table, shaving off about 1 second out of
about 20 seconds total for Linux BTF dedup.
Stop BTF encoding when errors are found in some DWARF CU.
Implement --packed, to show just packed structures, for instance, here are
the top 5 packed data structures in the Linux kernel:

$ pahole --sizes --packed | sort -k2 -nr | head -5
e820_table 64004 0
boot_params 4096 0
efi_variable 2084 0
snd_soc_tplg_pcm 912 0
ntb_info_regs 800 0
$

And here is one of them:

$ pahole efi_variable
struct efi_variable {
efi_char16_t VariableName[512]; /* 0 1024 /
/ --- cacheline 16 boundary (1024 bytes) --- /
efi_guid_t VendorGuid; / 1024 16 /
long unsigned int DataSize; / 1040 8 /
__u8 Data[1024]; / 1048 1024 /
/ --- cacheline 32 boundary (2048 bytes) was 24 bytes ago --- /
efi_status_t Status; / 2072 8 /
__u32 Attributes; / 2080 4 */

/* size: 2084, cachelines: 33, members: 6 /
/ last cacheline: 36 bytes */
} attribute((packed));
$
Fix bug in distros such as OpenSUSE:15.2 where DW_AT_alignment isn't defined.

Signed-off-by: Arnaldo Carvalho de Melo acme@redhat.com