library(sf) loads wrong version of GEOS

[dbaston]

I get the following error when building sf master against GEOS 3.7.0:

 unable to load shared object '/home/dan/R/x86_64-pc-linux-gnu-library/3.4/sf/libs/sf.so':
  /home/dan/R/x86_64-pc-linux-gnu-library/3.4/sf/libs/sf.so: undefined symbol: GEOSFrechetDistanceDensify_r

I verified that the symbol is present in libgeos_c.so:

nm -g /usr/local/lib/libgeos_c.so | grep Frechet
0000000000010210 T GEOSFrechetDistance
0000000000010230 T GEOSFrechetDistanceDensify
00000000000141e0 T GEOSFrechetDistanceDensify_r
0000000000014040 T GEOSFrechetDistance_r

The following command is used to link sf:

g++ -std=gnu++11 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o sf.so RcppExports.o bbox.o gdal.o gdal_geom.o gdal_read.o gdal_utils.o gdal_write.o geos.o hex.o proj.o raster2sf.o sfg.o signed_area.o stars.o wkb.o -lproj -L/usr/local/lib -lgdal -L/usr/local/lib -lgeos_c -L/usr/lib/R/lib -lR

which looks right to me.

[edzer]

I've seen the same thing when testing against 3.7.0rc2, after Brian Ripley warned @rsbivand (rgeos) and me about it a few days ago. Will try again with 3.7.0. Big mystery!

[edzer]

I reproduced it again (in a container) with GEOS 3.7.0, and got the same error. In earlier attempts I tried to remove GEOSFrechetDistanceDensify_r, but then it would break on the next symbol, and so on; as you showed, the symbols are there. Did they do something with GEOS?

[dbaston]

@edzer I'm able to build and run a dummy program using GEOSFrechetDistanceDensify_r:

#include <geos_c.h>
#include <iostream>

int main(int argc, char** argv) {
	auto handle = initGEOS_r(nullptr, nullptr);

	GEOSGeometry* geom1_ = GEOSGeomFromWKT_r(handle, "LINESTRING (0 0, 100 0)");
	GEOSGeometry* geom2_ = GEOSGeomFromWKT_r(handle, "LINESTRING (0 0, 50 50, 100 0)");

	double dist;
	int ret = GEOSFrechetDistanceDensify_r(handle, geom1_, geom2_, 0.5, &dist);

	std::cout << dist << std::endl;
}

with

g++ frechet.cpp -std=c++11 -o frechet -L /usr/local/lib -lgeos_c

[dbaston]

Running ldd sf.so also shows correct output:

libgeos_c.so.1 => /usr/local/lib/libgeos_c.so.1 (0x00007f4621c23000)

Yet, running the following:

LD_DEBUG=all LD_DEBUG_OUTPUT=/tmp/debug Rscript -e "dyn.load('/home/dan/dev/sf/src/sf.so')"
then

grep Frechet /tmp/debug*
shows that the wrong library is being loaded at runtime

/tmp/debug.28543: 28543: symbol=GEOSFrechetDistanceDensify_r; lookup in file=/usr/lib/x86_64-linux-gnu/libgeos_c.so.1

[edzer]

The link step of rgeos is

g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o rgeos.so dummy.o init.o local_stubs.o rgeos.o rgeos_R2geos.o rgeos_R2geosMP.o rgeos_bbox.o rgeos_buffer.o rgeos_coord.o rgeos_geos2R.o rgeos_linearref.o rgeos_misc.o rgeos_poly2nb.o rgeos_predicate_binary.o rgeos_predicate_unary.o rgeos_topology.o rgeos_topology_binary.o rgeos_validate.o rgeos_wkt.o -L/usr/local/lib -lgeos-3.7.0 -L/usr/local/lib -lgeos_c -L/usr/lib/R/lib -lR

which includes -lgeos-3.7.0, not in sf.

[edzer]

adding this didn't solve the problem.

[edzer]

After moving /usr/lib/x86_64-linux-gnu/libgeos_c.so.1.9.1 and /usr/lib/x86_64-linux-gnu/libgeos_c.so.1 out of the way, sf installed fine.

[rsbivand]

Yes, multiple libgeos will probably cause havoc. I haven't seen any rgeos issues on Fedora, but always delete earlier /usr/local/lib/libgeos* etc., and never let any packager install GDAL, PROJ or GEOS. The warning from Brian Ripley may relate to static builds (libgeos.a), but configure.ac presupposes dynamic (libgeos-3.7.0):

[rsb@localhost geos-3.7.0]$ geos-config --cclibs
-L/usr/local/lib -lgeos
[rsb@localhost geos-3.7.0]$ geos-config --clibs
-L/usr/local/lib -lgeos_c
[rsb@localhost geos-3.7.0]$ geos-config --libs
-L/usr/local/lib -lgeos-3.7.0
[rsb@localhost geos-3.7.0]$ geos-config --static-clibs
-L/usr/local/lib -lgeos_c -lgeos -lm
[rsb@localhost geos-3.7.0]$ geos-config --static-cclibs
-L/usr/local/lib -lgeos -lm

and configure.ac has:

GEOS_LIBS=`${GEOS_CONFIG} --libs`
GEOS_CLIBS=`${GEOS_CONFIG} --clibs`

Please advise on any need to change configure.ac.

[rsbivand]

rgeos configure.ac committed with:

GEOS_LIBS=`${GEOS_CONFIG} --cclibs`
GEOS_CLIBS=`${GEOS_CONFIG} --clibs`

[edzer]

configure controls compilation, but the error occurs at runtime. I can't think of a way in which configure could change this.

[rsbivand]

But it removes the version suffix from the rgeos link step. Beyond that, the loader will use the first libgeom it finds. I'm not seeing any issues on Fedora, feels like a platform issue. Time someone made a Fedora rocker for improved stability?

[edzer]

I think the issue Can't build against GEOS 3.7.0 is not true; the issue Can't build against GEOS 3.7.0 if there is also another GEOS installation on my system is, depending on where it is. Some diagnostics:

When hacking geos.cpp such that it doesn't use symbols provided only by GEOS 3.7.0 or 3.6.1, it compiles AND loads on R when GEOS 3.7.0 is in /usr/local/lib and GEOS 3.5.1 (or so) is in /usr/lib/something. Then:

root@313b7b08657b:/# ldd /usr/local/lib/R/site-library/sf/libs/sf.so  | grep geos
	libgeos_c.so.1 => /usr/local/lib/libgeos_c.so.1 (0x00007fb39962a000)
	libgeos-3.7.0.so => /usr/local/lib/libgeos-3.7.0.so (0x00007fb3964d2000)
root@313b7b08657b:/# echo $LD_LIBRARY_PATH
/usr/local/lib:

but after starting R:

> system("ldd /usr/local/lib/R/site-library/sf/libs/sf.so | grep geos")
	libgeos_c.so.1 => /usr/lib/x86_64-linux-gnu/libgeos_c.so.1 (0x00007fdad7954000)
	libgeos-3.5.1.so => /usr/lib/x86_64-linux-gnu/libgeos-3.5.1.so (0x00007fdad481d000)
> Sys.getenv("LD_LIBRARY_PATH")
[1] "/usr/lib/R/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/default-java/jre/lib/amd64/server:/usr/local/lib:"

A hacky solution to this might be

> Sys.setenv(LD_LIBRARY_PATH="/usr/local/lib:/usr/lib/R/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/jvm/default-java/jre/lib/amd64/server:")
> system("ldd /usr/local/lib/R/site-library/sf/libs/sf.so | grep geos")
	libgeos_c.so.1 => /usr/local/lib/libgeos_c.so.1 (0x00007f4909bc1000)
	libgeos-3.7.0.so => /usr/local/lib/libgeos-3.7.0.so (0x00007f4906a69000)

but I'm not sure if I would go as far as recommending this to anyone.

It would actually be good (sane, secure) to do a run-time check that the GEOS version linked to corresponds to the one found at compile time.

[rsbivand]

In rgdal this is done on attach (maybe should be done on load?) using GDALCheckVersion(); does GEOS provide a similar function?