Different compiler needed on OpenBSD

[frenkel]

On OpenBSD you need to install a newer compiler to get Nokogiri 1.6.8 to compile. This newer compiler is called egcc (for external I think). It would be great if Nokogiri's install process could detect it, so a normal bundle install would run without problems. Currently we need to manually connect to the server and run CC=egcc bundle install. Would you be willing to add such an exception/workaround? If so, what would be the best place to put this? I can then make a pull request for it.

Thanks and keep up the good work!

[flavorjones]

I'm not sure I understand why this would be happening.

If you take a look at ext/nokogiri/extconf.rb you'll see we compile everything using RbConfig::MAKEFILE_CONFIG['CC'], which should be the same compiler used to compile Ruby itself.

Can you explain a bit about why "you need to install a newer compiler to get Nokogiri 1.6.8. to compile"? I'd like to understand a bit better the problem you're trying to solve.

[frenkel]

We install ruby from packages, and it was compiled using gcc 4.2 which is the default compile (i.e. CC=gcc). Apparently the newer libxml needs to be compiled with something newer, so we need to override the CC environment variable and use egcc which is gcc 4.9. It would be great when I can extend nokogiri with a check that it will use egcc when on OpenBSD, because otherwise you will always have to override CC manually.

[flavorjones]

Hi, i unfortunately still don't understand the problem you're trying to solve.

Can you show me what happens when you compile using the default settings? What breaks? Can you show the logs?

[frenkel]

The problem is simple: default gcc in OpenBSD is too old to compile libxml2. It gives this error during install of nokogiri:

Extracting libxml2-2.9.4.tar.gz into tmp/x86_64-unknown-openbsd6.0/ports/libxml2/2.9.4... OK
Running 'configure' for libxml2 2.9.4... OK                                     
Running 'compile' for libxml2 2.9.4... ERROR, review '/home/frank/.gem/ruby/2.2/gems/nokogiri-1.6.8/ext/nokogiri/tmp/x86_64-unknown-openbsd6.0/ports/libxml2/2.9.4/compile.log' to see what happened. Last lines are:
========================================================================        
/usr/include/unistd.h:404: warning: redundant redeclaration of 'ftruncate'      
/usr/include/sys/types.h:217: warning: previous declaration of 'ftruncate' was here
/usr/include/unistd.h:429: warning: redundant redeclaration of 'truncate'       
/usr/include/sys/types.h:218: warning: previous declaration of 'truncate' was here
In file included from xmlIO.c:33:                                               
/usr/include/stdlib.h:184: warning: redundant redeclaration of 'mktemp'         
/usr/include/unistd.h:423: warning: previous declaration of 'mktemp' was here   
/usr/include/stdlib.h:215: warning: redundant redeclaration of 'mkstemp'        
/usr/include/unistd.h:422: warning: previous declaration of 'mkstemp' was here  
/usr/include/stdlib.h:255: warning: redundant redeclaration of 'mkdtemp'        
/usr/include/unistd.h:504: warning: previous declaration of 'mkdtemp' was here  
/usr/include/stdlib.h:295: warning: redundant redeclaration of 'mkstemps'       
/usr/include/unistd.h:505: warning: previous declaration of 'mkstemps' was here 
xmlIO.c: In function 'xmlXzfileClose':                                          
xmlIO.c:1450: error: 'LZMA_OK' undeclared (first use in this function)          
xmlIO.c:1450: error: (Each undeclared identifier is reported only once          
xmlIO.c:1450: error: for each function it appears in.)                          
*** Error 1 in . (Makefile:1248 'xmlIO.lo': @echo "  CC      " xmlIO.lo;/bin/sh ./libtool --silent --tag=CC    --mode=compile cc -DHAVE_CONF...)
*** Error 1 in . (Makefile:1465 'all-recursive')                                
*** Error 1 in /home/frank/.gem/ruby/2.2/gems/nokogiri-1.6.8/ext/nokogiri/tmp/x86_64-unknown-openbsd6.0/ports/libxml2/2.9.4/libxml2-2.9.4 (Makefile:859 'all')
========================================================================        
*** extconf.rb failed ***                        

with CC=egcc it works fine. I want to fix this in the gem by detecting OpenBSD and using the newer compiler.

[flavorjones]

This seems like something that needs to be fixed in the documentation as well; introducing it into the build without noting that egcc is a requirement seems less than ideal.

If you'd like to send a PR to both the installation tutorial and the Nokogiri build script, those would be welcomed.

[frenkel]

So the answers to my original two questions: "Would you be willing to add such an exception/workaround? If so, what would be the best place to put this?" are: Yes and in the build script? Where can I find that script? I'm willing to create PR's for it yes :)

[flavorjones]

The answers to your two questions are:

1. yes, and
2. PRs to both the documentation and the build script.

The build script in this case is ext/nokogiri/extconf.rb, which I referenced in my first reply on this thread.

[jeremyevans]

If you install both libxml and nokogiri using OpenBSD packages, things work fine with the OpenBSD system compiler (gcc 4.2.1), even though OpenBSD's libxml port is for version 2.9.4. I'm working on update for the OpenBSD nokogiri port (from 1.6.6.2 -> 1.7.0.1) and noticed that nokogiri was now forcing egcc use even though it is not necessary or desirable.

Installing nokogiri with --use-system-libraries also works fine with the OpenBSD system compiler if libxml is installed via the OpenBSD package, which is recommended on OpenBSD. Defaulting to using egcc on OpenBSD and erroring if it is not installed should only be done if --use-system-libraries is not provided.

In the future, if you have questions about nokogiri usage on OpenBSD, you may want to ask the maintainer of the nokogiri port for OpenBSD (me).

[flavorjones]

@frenkel care to comment on the above?

[flavorjones]

@jeremyevans I had no idea there even was a port for OpenBSD, so please don't be offended by the fact that nobody asked. I'm happy to have you coordinate a fix for this. Do you have thoughts on what the right thing to do here is?

@knu, would you mind working with Jeremy on addressing any changes he feels need to be made?

[frenkel]

@flavorjones It seems right to only do this when not using system libraries. That way it will work in both cases on OpenBSD and won't unnecessarily require egcc.

[jeremyevans]

I've submitted a pull request that I think will fix the issue.

@flavorjones I apologize, I was in a bit of a hurry when I wrote my previous post, and I think I came off a bit harsh, which wasn't my intent. I definitely appreciate the work you are doing to maintain nokogiri. If you have questions about nokogiri usage on OpenBSD in the future, please do reach out to me and I'll do what I can to help.

[knu]

@flavorjones Sure, and I've merged the PR above. The term "port" in OpenBSD (and FreeBSD) refers to a recipe to build a package from source, and it looks like OpenBSD currently has no local patch in its port, which we, the developers and contributors, can all be proud of.

[knu]

@jeremyevans Thanks for the PR. I see you are listed as maintainer of the OpenBSD port, and I much appreciate what you have been doing for it and many other ports related to the ecosystem around Nokogiri! We'll do our best to reach out to you in the future whenever we know a change may affect OpenBSD, however we deal with a lot of platforms and sometimes we may make a mistake and accidentally break the build on a specific platform while trying to fix an issue for other platforms. So, as much as we take care, please watch this project and let us know if you find we are doing wrong for OpenBSD. We'll always be open to improvements. Thanks!

[frenkel]

@knu Good to hear such a positive attitude towards non-Linux systems. Keep up the good work!