allow linking with gmp instead of mpir

[DanGrayson]

[DanGrayson]

I've started a branch for this work at https://github.com/DanGrayson/M2/tree/use-native-gmp-mpfr-pari .

But it's complicated! Basically, we are forced to let libgmp use malloc instead of continuing to tell it to use GC_malloc, which we need to do so our top level objects get garbage collected.

[mikestillman]

I was wondering about this. For instance, frobby also uses gmp integers, inside of c++ data structure memory which, I think, gc is not aware of (having been called through malloc or new). For the part of frobby we use, I think it is not an issue, but if we try to use the rest, it is.

This gmp allocation issue has been thorny for a long time!

On Jun 4, 2015, at 1:22 PM, Daniel R. Grayson notifications@github.com wrote:

I've started a branch for this work at https://github.com/DanGrayson/M2/tree/use-native-gmp-mfpr-pari https://github.com/DanGrayson/M2/tree/use-native-gmp-mfpr-pari
But it's complicated! Basically, we are forced to let libgmp use malloc instead of continuing to tell it to use GC_malloc, which we need to do so our top level objects get garbage collected.

—
Reply to this email directly or view it on GitHub #285 (comment).

[DanGrayson]

I'm assuming that libgc cannot find thread local storage created with the __thread compiler keyword, but I could be wrong. Thread safe 3rd party libraries (such as pari) will use this. (Actually, pari comes in a non-gmp flavor, too.) So we cannot force them to use GC_malloc. Also, we can't set the gmp allocation functions on the fly, because that info is stored in global variables. (Pari sets them on the fly, which makes the gmp version of pari not thread safe, despite what pari documentation says.)

[DanGrayson]

In order to maximize the number of native or homebrew-supplied libraries we can use under Mac OS X, it seems we should compile with clang and clang++, to match what the libraries are compiled with. So I'm trying that now.

[mikestillman]

The clang provided with mac is way too old, I think. Perhaps with the latest Xcode it is OK, but this is a bit of a problem.

On Jun 4, 2015, at 4:53 PM, Daniel R. Grayson notifications@github.com wrote:

In order to maximize the number of native or homebrew-supplied libraries we can use under Mac OS X, it seems we should compile with clang and clang++, to match what the libraries are compiled with. So I'm trying that now.

—
Reply to this email directly or view it on GitHub #285 (comment).

[DanGrayson]

How do you compare ages of clang?

einsteinium$ ~/Downloads/clang+llvm-3.6.1-x86_64-apple-darwin/bin/clang --version
clang version 3.6.1 (tags/RELEASE_361/final)
Target: x86_64-apple-darwin14.1.0
Thread model: posix
einsteinium$ /usr/bin/clang --version
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin14.1.0
Thread model: posix

[DanGrayson]

I think the only way to fix these issue is to stop setting the GMP memory allocation functions (using mp_set_memory_functions) to versions that invoke LIBGC, because we can never be sure that third party libraries will store those pointers in a place where LIBGC can find them. So now I'm trying to finalize every top level object that contains a pointer to a GMP integer, a GMP rational, or an MPFR real.

[mikestillman]

Another option, possibly a bad one, is to use mpn routines at top level (for M2) and do memory allocation ourselves. mpn routines, I think, do no memory allocation.

How many gmp functions do you/we interface to?

On Jun 8, 2015, at 7:42 PM, Daniel R. Grayson notifications@github.com wrote:

I think the only way to fix these issue is to stop setting the GMP memory allocation functions (using mp_set_memory_functions) to versions that invoke LIBGC, because we can never be sure that third party libraries will store those pointers in a place where LIBGC can find them. So now I'm trying to finalize every top level object that contains a pointer to a GMP integer, a GMP rational, or an MPFR real.

—
Reply to this email directly or view it on GitHub #285 (comment).

[DanGrayson]

I think that won't work, because it's not just us using GMP routines.

On Mon, Jun 8, 2015 at 7:46 PM Mike Stillman notifications@github.com
wrote:

Another option, possibly a bad one, is to use mpn routines at top level
(for M2) and do memory allocation ourselves. mpn routines, I think, do no
memory allocation.

How many gmp functions do you/we interface to?

On Jun 8, 2015, at 7:42 PM, Daniel R. Grayson notifications@github.com
wrote:

I think the only way to fix these issue is to stop setting the GMP
memory allocation functions (using mp_set_memory_functions) to versions
that invoke LIBGC, because we can never be sure that third party libraries
will store those pointers in a place where LIBGC can find them. So now I'm
trying to finalize every top level object that contains a pointer to a GMP
integer, a GMP rational, or an MPFR real.

—
Reply to this email directly or view it on GitHub <
https://github.com/Macaulay2/M2/issues/285#issuecomment-110172694>.

—
Reply to this email directly or view it on GitHub
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Macaulay2_M2_issues_285-23issuecomment-2D110173282&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=PAZ_jTPLrSb1btCfAu4RkycWe57N0sGyzbjgeOn2wN0&m=vktisaeaQJrOfBNcbmyuvOiGEzGu9Ikpeuw5GngouSw&s=3yzeBBm_le0S5GcFOI44A6WHL5Np4RYXz_YRy10BuCU&e=
.

[DanGrayson]

Hmm, I take that back... let me think about it.

[mikestillman]

I was thinking that all engine objects should be finalized too. Then I can avoid (extra) garbage collection issues in the engine.

On Jun 8, 2015, at 7:49 PM, Daniel R. Grayson notifications@github.com wrote:

Hmm, I take that back... let me think about it.

—
Reply to this email directly or view it on GitHub #285 (comment).

[DanGrayson]

There are lots of engine objects that might contain pointers to GMP
integers.

Another option would be to copy the array of limbs in every GMP integer
returned by 3rd
party routines into memory obtained from GC and replace the limb pointer.

On Mon, Jun 8, 2015 at 8:10 PM Mike Stillman notifications@github.com
wrote:

I was thinking that all engine objects should be finalized too. Then I can
avoid (extra) garbage collection issues in the engine.

On Jun 8, 2015, at 7:49 PM, Daniel R. Grayson notifications@github.com
wrote:

Hmm, I take that back... let me think about it.