Skip to content

Split libjulia into runtime and codegen components #39129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

jpsamaroo
Copy link
Member

For the purposes of embedding Julia and generating smaller, mostly standalone binaries, we may want to be able to link against only Julia's runtime components to reduce the amount of memory linked into the process. This PR splits out the runtime components from libjulia-internal into libjulia-runtime-internal, and also provides a new libjulia-stub-internal library. The intention is that the user can set an environment variable that will cause libjulia to substitute our codegen implementation (still in libjulia-internal) for a set of codegen stubs in libjulia-stub-internal (from the pre-existing src/anticodegen.c). This implementation reuses the trampoline mechanism from libjulia to allow redirecting codegen-specific symbols to the right library.

This PR does not currently succeed in generating corecompiler.ji for reasons unknown to me; if someone can figure out what I'm doing wrong here, I would appreciate the help!

@JeffBezanson JeffBezanson self-assigned this Jan 22, 2021
@JeffBezanson
Copy link
Member

JeffBezanson commented Apr 27, 2021

Thanks for working on this. @vtjnash @Keno @staticfloat and I finally got around to brainstorming how it should work. Our vision is

  • Build a separate libjulia-codegen that contains codegen.cpp and llvm-*.cpp. Ideally only this library should depend on libLLVM.
  • libjulia-internal contains stub codegen entry points (anticodegen.c) as weak symbols.
  • If you're building your own executable via PackageCompiler, libjulia-codegen can be omitted just by not putting it on the link line.
  • Initially julia.exe can just link to libjulia-codegen at build time. But we might want to avoid loading it if --compile=no is passed.
  • The loader_lib.c trampoline tricks are not needed for this, since users of libjulia (embedders) don't need to call codegen entry points directly.

Let me know if you want to keep working on this, or if I should take over.

@vchuravy
Copy link
Member

Julian and I looked at this together last week, I think we are happy if you take this over, but we can also throw some cycles at this in May. Something odd was going on with jl_default_cgparams since it was being declared both in anticodegen and codegen.

@jpsamaroo
Copy link
Member Author

Do weak symbols work on non-Linux targets? Last time we discussed this, I was told that they didn't work everywhere.

@JeffBezanson
Copy link
Member

Ok, we don't need to use weak symbols. We can just use dlsym and plugin-style interface.

JeffBezanson added a commit that referenced this pull request Aug 19, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
JeffBezanson added a commit that referenced this pull request Aug 19, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
JeffBezanson added a commit that referenced this pull request Aug 19, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
JeffBezanson added a commit that referenced this pull request Aug 19, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
@JeffBezanson
Copy link
Member

Superseded by #41936

JeffBezanson added a commit that referenced this pull request Aug 25, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
JeffBezanson added a commit that referenced this pull request Sep 2, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
JeffBezanson added a commit that referenced this pull request Sep 2, 2021
follow-up to #39129

Co-authored-by: Julian Samaroo <jpsamaroo@jpsamaroo.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants