Facter not working on AIX with FFI resolvers

[SaltyAlan]

Describe the Bug

Facter not working on AIX with FFI resolvers, which prevents open source puppet from working

Expected Behavior

Facter and its dependent FFI to work on AIX, instead of failing with resolver errors. Puppet working properly.

Steps to Reproduce

Install the gems after prerequisite RPMs installed (as in the environment below):

# dnf list installed ruby ruby-devel libffi libffi-devel gcc10 make
Installed Packages
gcc10.ppc                                                                       10.3.0-6 
libffi.ppc                                                                      3.4.4-2  
libffi-devel.ppc                                                                3.4.4-2  
ruby.ppc                                                                        3.3.5-1  
ruby-devel.ppc                                                                  3.3.5-1  
make.ppc                                                                        1:4.4.1-1

# gem install facter-4.10.0.gem thor-1.2.2.gem ffi-1.17.1.gem
Fetching hocon-1.4.0.gem
Successfully installed thor-1.2.2
Successfully installed hocon-1.4.0
Successfully installed facter-4.10.0
Successfully installed thor-1.2.2
Building native extensions. This could take a while...
Successfully installed ffi-1.17.1
5 gems installed

Running facter then fails with FFI errors for networking facts:

# facter
[2025-01-02 11:16:04.310798 ] ERROR Facter::Resolvers::Aix::Networking - Resolving fact mtu, but got load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons:
        /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 at <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
[2025-01-02 11:16:04.328535 ] ERROR Facter::Resolvers::Aix::Networking - Resolving fact netmask, but got load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons:
        /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 at <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
[2025-01-02 11:16:04.344082 ] ERROR Facter::Resolvers::Aix::Networking - Resolving fact network, but got load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons:
        /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 at <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
[2025-01-02 11:16:04.359346 ] ERROR Facter::Resolvers::Aix::Networking - Resolving fact ip6, but got load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons:
        /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 at <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
[2025-01-02 11:16:04.374639 ] ERROR Facter::Resolvers::Aix::Networking - Resolving fact netmask6, but got load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons:
        /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 at <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
[2025-01-02 11:16:04.389665 ] ERROR Facter::Resolvers::Aix::Networking - Resolving fact primary_interface, but got load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons:
        /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 at <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
[2025-01-02 11:16:04.404717 ] ERROR Facter::Resolvers::Aix::Networking - Resolving fact network6, but got load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons:
        /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 at <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
[2025-01-02 11:16:04.419756 ] ERROR Facter::Resolvers::Aix::Networking - Resolving fact scope6, but got load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons:
        /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 at <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
[2025-01-02 11:16:04.437210 ] ERROR Facter::Resolvers::Aix::Networking - Resolving fact ip, but got load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons:
        /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 at <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
[2025-01-02 11:16:04.452798 ] ERROR Facter::Resolvers::Aix::Networking - Resolving fact mac, but got load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons:
        /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 at <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
[2025-01-02 11:16:04.455555 ] ERROR Facter::Resolvers::Aix::LoadAverages - Resolving fact load_averages, but got load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons:
        /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 at <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
 ...
 ...

And the execerror of the above errors:

#  /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
 exec(): 0509-036 Cannot load program ruby because of the following errors:
        0509-022 Cannot load module /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so.
        0509-114   The section sizes in the auxiliary header do not match the
                   section sizes in the corresponding section headers.
        0509-194 Examine file headers with the 'dump -ohv' command.

Environment

AIX version:
AIX 7.3 TL2 SP2

Prerequisite RPMs installed (installed from AIX toolbox for open source applications - https://www.ibm.com/support/pages/node/882892) :
ruby-3.3.5
ruby-devel-3.3.5
libffi-3.4.4
libffi-devel-3.4.4
gcc10-10.3.0-6

GEMs:
thor-1.2.2.gem
facter-4.10.0.gem
ffi-1.17.1.gem
hocon-1.4.0.gem

Additional Context

We are currently forced to run facter version 2 on AIX - as facter version 4 introduces the dependency on FFI. Facter version 2 is very old now and we need to upgrade it, but we are unable to get it to work.

I have tried several different versions of facter (all on version 4) and different versions of ffi, but all have the same behaviour. I have also tried on AIX version 7.2 and 7.3, but neither work. I have tried an earlier version of ruby (3.1.4) as well, which still failed. I also tried using gcc compilers of version 8 and 10, same behaviour with either.

I have also tried to install ffi with --disable-system-libffi option to bypass the system libffi and use its own- it makes no difference, same error occurs when running facter.

[joshcooper]

I think this is more of a "how to compile ffi with native extensions on AIX" than a facter issue. Using Ruby 3.3, if you do gem install ffi does the following work:

ruby -rffi -e "puts FFI::Library::LIBC"

[SaltyAlan]

Hello, thanks for the response.

I get that it is ffi related. Facter now depends on ffi, which does not work out of the box on AIX, thus causing facter to not work properly on AIX as well. Which in turn causes puppet to not work on AIX. Hard to know where to log this issue as they are separate components that are related.

Requested output:

# ruby -rffi -e "puts FFI::Library::LIBC"
<internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require': load failed - /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so. Please issue below command for detailed reasons: (LoadError)
     /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"

        from <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
        from /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi.rb:5:in `rescue in <top (required)>'
        from /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi.rb:2:in `<top (required)>'
        from <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:141:in `require'
        from <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:141:in `rescue in require'
        from <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:135:in `require'
<internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require': cannot load such file -- 3.3/ffi_c (LoadError)
        from <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'
        from /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi.rb:3:in `<top (required)>'
        from <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:141:in `require'
        from <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:141:in `rescue in require'
        from <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:135:in `require'
<internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require': cannot load such file -- ffi (LoadError)
        from <internal:/opt/freeware/share/ruby/3.3/rubygems/core_ext/kernel_require.rb>:136:in `require'

# /usr/sbin/execerror ruby "2 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so" "114 /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so"
exec(): 0509-036 Cannot load program ruby because of the following errors:
        0509-022 Cannot load module /opt/freeware/share/ruby/gems/3.3/gems/ffi-1.17.1/lib/ffi_c.so.
        0509-114   The section sizes in the auxiliary header do not match the
                   section sizes in the corresponding section headers.
        0509-194 Examine file headers with the 'dump -ohv' command.

The facter project has AIX as one of its supported platforms, the ffi project does not list such support. Is facter version 4 supposed to work on AIX?

Do I need to ask for ffi project for assistance instead?

Cheers,
Alan

[joshcooper]

To clarify, libffi works on AIX https://github.com/libffi/libffi/blob/d77b9fefa25f7f11dcc4e6380c661c4cf2960a85/.github/workflows/build.yml#L161-L165

but the ffi ruby bindings probably are not compiling the native extension ffi_c with the correct flags. I'd check with the ffi project.

Assuming those dependencies are resolved correctly, then facter should work on AIX.

[SaltyAlan]

Thank you for the feedback. I will log an issue with the ffi project.

[SaltyAlan]

This turned out to be an AIX bug - https://www.ibm.com/support/pages/apar/IJ49146 - ffi works once the fix for this bug is installed.

[joshcooper]

Nice work, thanks for letting us know!