Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Making HPC-GAP work with Julia #3734

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

rbehrends
Copy link
Contributor

@rbehrends rbehrends commented Nov 11, 2019

This is still very raw and has a number of hacks (such as using calloc() for AllocateMemoryBlock()). The PR is currently an FYI/progress tracker, as a lot of the implementation is still subject to change. It also relies on previous HPC-GAP related PRs (namely #3502 and #2845).

Aside from making HPC-GAP work with Julia, it also aims at allowing HPC-GAP to be used as a library (which is necessary for GAP.jl, but also allows other code to link to it).

It currently runs only with gap --single-thread, as CreateThread() is disabled due to limitations in Julia. (It's also not a priority, as we are currently more concerned with calling GAP from Julia rather than having yet another GC for GAP.)

Configuration needs to happen with:

./configure --with-gc=julia --with-julia=/path/to/julia --enable-hpcgap --enable-native-tls

Remaining issues that I am working on are:

  • Proper initialization of thread-local state. This involves refactoring of RunThread() and DispatchThread().
  • Thread creation/initialization for the libgap version, building on the previous bullet point. This is necessary for GAP.jl.
  • Fixing errors in root scanning. Currently, we're seemingly missing out on some global variables, leading to eventual crashes.
  • Clean implementation of AllocateMemoryBlock(); this may require an internal API change for that function to permit a type parameter. AllocateMemoryBlock() presumes that a conservative GC is being used, which may not be the best decision, anyway.
  • Make CreateThread() work via the task system (Julia only recognizes its own preallocated threads, all other thread activity needs to be mapped to Julia tasks that are then being run by threads. This is how the Julia macro @threads works, for example.).
  • Insert GC safepoints around I/O (the Julia GC requires that for multi-threaded GC to work). This is necessary primarily for console I/O, which can block indefinitely while computations are running (I am currently not so much worried about a pause while reading a file, as GAP tends to read files only once, so a pause is more acceptable).

@rbehrends rbehrends added topic: HPC-GAP Issues and PRs related to HPC-GAP do not merge PRs which are not yet ready to be merged (e.g. submitted for discussion, or test results) do not comment yet PRs on which the author does not yet want any comment (e.g. only submitted for test results) do not review PRs which are not yet ready for a proper external review (e.g. only submitted for test results) labels Nov 11, 2019
@rbehrends rbehrends force-pushed the hpcgap-julia branch 2 times, most recently from 8fcadc1 to a8da44c Compare November 14, 2019 11:22
@fingolfin
Copy link
Member

The last commit message is "Make HPC-GAP guards work without Ward" -- but HPC-GAP already works without ward, and has done so for months. Perhaps "Change HPC-GAP to use the unward tool" or something like that could be used instead.

@rbehrends
Copy link
Contributor Author

I think a bit of subtlety got lost here. I had changed the message to "Make HPC-GAP guards work without Ward" (emphasis on guards); I had inserted "guards" as a result of your earlier suggestion to change the commit message. HPC-GAP did work without Ward, guards didn't.

If you think that this still doesn't describe the commit well, I think adding a longer explanation would be better than trying to squeeze an adequate description into a single line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do not comment yet PRs on which the author does not yet want any comment (e.g. only submitted for test results) do not merge PRs which are not yet ready to be merged (e.g. submitted for discussion, or test results) do not review PRs which are not yet ready for a proper external review (e.g. only submitted for test results) topic: HPC-GAP Issues and PRs related to HPC-GAP
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants