Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Does not build on macOS 14 / aarch64 #10

Open
barracuda156 opened this issue Feb 27, 2024 · 6 comments
Open

Does not build on macOS 14 / aarch64 #10

barracuda156 opened this issue Feb 27, 2024 · 6 comments

Comments

@barracuda156
Copy link

Building xapian-ruby from rubygems fails on Sonoma. Moreover, something seems to hardcode C++ runtime, which cannot work in default macOS setups:

/bin/sh ../libtool  --tag=CXX   --mode=link g++ -std=gnu++11 -fno-strict-aliasing -Wall -Wno-unused -Wno-uninitialized -fvisibility=hidden -fvisibility-inlines-hidden -I/opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0/gems/xapian-ruby-1.4.22/xapian-core-1.4.22/include -g -O2 -avoid-version -module -shrext ".bundle"  -L/opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0/gems/xapian-ruby-1.4.22/lib -o _xapian.la -rpath /opt/local/lib/ruby3.3/site_ruby/3.3.0/arm64-darwin23 xapian_wrap.lo /opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0/gems/xapian-ruby-1.4.22/xapian-core-1.4.22/libxapian.la -lstdc++  
libtool: link: g++  -o .libs/_xapian.bundle -bundle  .libs/xapian_wrap.o   -L/opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0/gems/xapian-ruby-1.4.22/lib /opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0/gems/xapian-ruby-1.4.22/xapian-core-1.4.22/.libs/libxapian.dylib -lz -lstdc++  -g -O2  
ld: warning: ignoring duplicate libraries: '-lc++'
ld: Undefined symbols:
  _rb_ary_entry, referenced from:
      XapianSWIGQueryItor::operator*() const in xapian_wrap.o
  _rb_ary_new, referenced from:
      SwigDirector_Compactor::resolve_duplicate_metadata(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, unsigned long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const*) in xapian_wrap.o
  _rb_ary_push, referenced from:
      SwigDirector_Compactor::resolve_duplicate_metadata(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, unsigned long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const*) in xapian_wrap.o

By default most of macOS versions (but not all) use libc++, so trying to link against libstdc++ will break stuff. (Hardcoding libc++ will be wrong either, compiler should decide which runtime to use, it is done automatically.)

As of now, linking is broken:

. . .
  _rb_str_new_cstr, referenced from:
      SWIG_Ruby_NewPointerObj(void*, swig_type_info*, int) in xapian_wrap.o
      _wrap_PositionIterator_allocate(unsigned long) in xapian_wrap.o
      _wrap_PostingIterator_allocate(unsigned long) in xapian_wrap.o
      _wrap_TermIterator_allocate(unsigned long) in xapian_wrap.o
      _wrap_ValueIterator_allocate(unsigned long) in xapian_wrap.o
      _wrap_Document_allocate(unsigned long) in xapian_wrap.o
      _wrap_Registry_allocate(unsigned long) in xapian_wrap.o
      ...
  _rb_str_new_static, referenced from:
      Ruby_Format_TypeError(char const*, char const*, char const*, int, unsigned long) in xapian_wrap.o
  _rb_string_value_ptr, referenced from:
      SWIG_Ruby_ConvertPtrAndOwn(unsigned long, void**, swig_type_info*, int, swig_ruby_owntype*) in xapian_wrap.o
      SWIG_AsCharPtrAndSize(unsigned long, char**, unsigned long*, int*) in xapian_wrap.o
      Ruby_Format_TypeError(char const*, char const*, char const*, int, unsigned long) in xapian_wrap.o
      Ruby_Format_TypeError(char const*, char const*, char const*, int, unsigned long) in xapian_wrap.o
  _rb_uint2big, referenced from:
      _wrap_Query_get_num_subqueries(int, unsigned long*, unsigned long) in xapian_wrap.o
      _wrap_LatLongCoords_size(int, unsigned long*, unsigned long) in xapian_wrap.o
      _wrap_Database_size(int, unsigned long*, unsigned long) in xapian_wrap.o
      _wrap_Database_get_total_length(int, unsigned long*, unsigned long) in xapian_wrap.o
      _wrap_Database_get_revision(int, unsigned long*, unsigned long) in xapian_wrap.o
      _wrap_Database_check(int, unsigned long*, unsigned long) in xapian_wrap.o
      _wrap_Database_check(int, unsigned long*, unsigned long) in xapian_wrap.o
      ...
  _rb_undef_alloc_func, referenced from:
      _Init__xapian in xapian_wrap.o
      _Init__xapian in xapian_wrap.o
      _Init__xapian in xapian_wrap.o
      _Init__xapian in xapian_wrap.o
      _Init__xapian in xapian_wrap.o
  _rb_undef_method, referenced from:
      _Init__xapian in xapian_wrap.o
  _rb_unexpected_type, referenced from:
      _Init__xapian in xapian_wrap.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [_xapian.la] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
rake aborted!

/opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0/gems/xapian-ruby-1.4.22/Rakefile:9:in `system!'
/opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0/gems/xapian-ruby-1.4.22/Rakefile:40:in `block (2 levels) in <top (required)>'
/opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0/gems/xapian-ruby-1.4.22/Rakefile:35:in `chdir'
/opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0/gems/xapian-ruby-1.4.22/Rakefile:35:in `block in <top (required)>'
/opt/local/bin/rake3.3:25:in `load'
/opt/local/bin/rake3.3:25:in `<main>'
Tasks: TOP => default
(See full trace by running task with --trace)

rake failed, exit code 1

Gem files will remain installed in /opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0/gems/xapian-ruby-1.4.22 for inspection.
Results logged to /opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0/extensions/arm64-darwin-23/3.3.0/xapian-ruby-1.4.22/gem_make.out
Command failed:  cd "/opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/xapian-ruby-1.4.22" && /opt/local/bin/gem3.3 install --local --force --install-dir /opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/destroot/opt/local/lib/ruby3.3/gems/3.3.0 /opt/local/var/macports/build/_opt_svacchanda_SonomaPorts_ruby_rb-xapian-ruby/rb33-xapian-ruby/work/xapian-ruby-1.4.22/xapian-ruby.gem 
Exit code: 1

Xapian bindings for Ruby do build fine from the port, for example, so there is no inherent reason for them not to work. Just something goes wrong during the build here.
@barracuda156 barracuda156 mentioned this issue Feb 27, 2024
Closed
@barracuda156
Copy link
Author

Oh wait, why does it even use g++ for linking with clang? Or course this gonna fail. g++ should be used with the compiler is gcc (which normally, though not necessarily, implies libstdc++ runtime), but not with clang

@gessel
Copy link

gessel commented Sep 18, 2024

This also fails on FreeBSD, similar problem:

10 warnings generated.
/bin/sh ../libtool  --tag=CXX    --mode=link c++ -fno-strict-aliasing -Wall -Wno-unused -Wno-uninitialized -fvisibility=hidden -fvisibility-inlines-hidden
-I/usr/local/lib/ruby/gems/3.2/gems/xapian-ruby-1.4.22/xapian-core-1.4.22/include  -g -O2 -avoid-version -module -shrext ".so" 
-L/usr/local/lib/ruby/gems/3.2/gems/xapian-ruby-1.4.22/lib -o _xapian.la -rpath /usr/local/lib/ruby/site_ruby/3.2/amd64-freebsd14 xapian_wrap.lo
/usr/local/lib/ruby/gems/3.2/gems/xapian-ruby-1.4.22/xapian-core-1.4.22/libxapian.la -lstdc++
libtool: link: c++  -fPIC -DPIC -shared -nostdlib /usr/lib/crti.o /usr/lib/crtbeginS.o  .libs/xapian_wrap.o   -Wl,-rpath
-Wl,/usr/local/lib/ruby/gems/3.2/gems/xapian-ruby-1.4.22/xapian-core-1.4.22/.libs -Wl,-rpath -Wl,/usr/local/lib/ruby/gems/3.2/gems/xapian-ruby-1.4.22/lib
-L/usr/local/lib/ruby/gems/3.2/gems/xapian-ruby-1.4.22/lib /usr/local/lib/ruby/gems/3.2/gems/xapian-ruby-1.4.22/xapian-core-1.4.22/.libs/libxapian.so -lz -lrt -lstdc++ -L/usr/lib -lc++
-lm -lc -lgcc -lgcc_s /usr/lib/crtendS.o /usr/lib/crtn.o  -g -O2   -Wl,-soname -Wl,_xapian.so -o .libs/_xapian.so
ld: error: unable to find library -lstdc++
c++: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1

gcc13-13.3.0
FreeBSD 14.1-RELEASE-p3 GENERIC amd64

This seems to be a relatively common issue arising from a variation in library structure between BSD-based OSes and Linux based ones.
bazelbuild/rules_rust#978
Any chance of a patch?

@gessel
Copy link

gessel commented Sep 18, 2024

something like this:
brentleyjones/bazel@b922110

@barracuda156
Copy link
Author

something like this:

brentleyjones/bazel@b922110

@gessel This will break it for macOS when libstdc++ is used.
You just need to use C++ compiler to link the code. And do not pass any specific runtime flag. GCC or Clang will know what to link to.
This should never be hardcoded on Darwin.

Those -z flags, AFAIK, are not supported on MacOS btw, though I do not see from the patch if they are used there.

@gessel
Copy link

gessel commented Sep 19, 2024

The process in bundler is entirely automatic and the compilers available are GCC13 and LLVM/Clang 15. The compilers don't know what to do. The solution proposed is to detect the OS and it sounds like the conditional should be for *BSD leaving Dawin/MacOS to work as it does (assuming it does).

Strangely, FreeBSD has a port, https://www.freshports.org/databases/ruby-xapian/ which installs fine, but the gem install process seems to ignore the existing. The gem vs. ports management of Ruby gems seems to cause some issues here and there and perhaps that's a solution path. For example:

# gem list | grep simplecov
simplecov (0.22.0)
simplecov-html (0.13.1)
simplecov_json_formatter (0.1.4)
# pkg version  | grep simplecov
rubygem-simplecov-0.22.0           =
rubygem-simplecov-html-0.13.1      =
rubygem-simplecov_json_formatter-0.1.4 =

But, and this looks like it might be not the desired result:

# pkg version | grep xapian
php82-xapian-1.4.26                =
py311-xapian-1.4.26                =
ruby32-xapian-1.4.26               =
xapian-bindings-1.4.26             =
xapian-core-1.4.26,1               =
xapian-omega-1.4.26                =
# gem list | grep xapian
#

@gessel
Copy link

gessel commented Sep 26, 2024

I believe this has been fixed upstream.
https://trac.xapian.org/ticket/834#comment:8
I test built and I think we need to rev the xapian source to 1.4.27

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gessel @barracuda156