Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable JITLink on aarch64 linux. #45859

Closed
wants to merge 8 commits into from

Conversation

sunho
Copy link
Contributor

@sunho sunho commented Jun 29, 2022

Enable JITLink in order to turn on small code model on aarch64 linux. Fixes #42295.

JITLink is a new JIT linker in LLVM that solves various issues within RuntimeDyLD, notablely, lacking support of small code model, runtime features (i.e. TLS), and confusing error handling. The lack of small code model has caused random assertions and codegen bugs in aarch64 architecture which is tracked in #42295. Switching to JITLink can solve this issue, but aarch64 linux has not been supported in JITLink until recently. Now, most of the groundworks to support aarch64 linux were done in llvm upstream, and it's able to turn JITLink on.

This PR builds upon the works done in #43664. It simply turns on JITLink on aarch64 linux as well. Few modifications were done to properly register eh frame and debug info in linux.

On my local machine, it passes almost all testcases including challenging backtrace test cases. Few of them (mostly SSL TLS test cases) are failing which I think is unrelated to jit backend, but please tell me if I need to dig into them.

Needs JuliaLang/llvm-project#9.

cc @vchuravy @vtjnash @dnadlinger

@vchuravy
Copy link
Member

This is great! Thank you.

@giordano giordano added system:arm ARMv7 and AArch64 compiler:llvm For issues that relate to LLVM labels Jun 29, 2022
@dnadlinger
Copy link
Member

dnadlinger commented Jun 29, 2022

Nice work on the LLVM side of things – great to see that the required changes on the Julia side were minimal. (I hope the section name normalisation thing didn't cost you too much time; I was aware that this would likely not be needed for ELF, but unfortunately decided to obey YAGNI and not add the ifdef "clutter".)

@sunho
Copy link
Contributor Author

sunho commented Jun 30, 2022

Nice work on the LLVM side of things – great to see that the required changes on the Julia side were minimal. (I hope the section name normalisation thing didn't cost you too much time; I was aware that this would likely not be needed for ELF, but unfortunately decided to obey YAGNI and don't add the ifdef "clutter".)

I indeed took a bit of time to identify the section name issue, but it totally makes sense in darwin. I was also surprised such minimal changes in julia make everything work out of shelf; your JITLink work was very solid!

@maleadt
Copy link
Member

maleadt commented Jun 30, 2022

This also seems to fix / work around #41852!

@vchuravy
Copy link
Member

Is there a way we could detect at compile time if the backend supports JITlink? While discouraged people do build Julia against LLVM without our patches.

@sunho
Copy link
Contributor Author

sunho commented Jun 30, 2022

There is no such function yet, but unsupported platform/architecture can be easily detected by priting out JITLinkError. I can still create a llvm patch if such a function comes handy.

@vchuravy
Copy link
Member

Yeah the scenario I worry about is someone building against an unpatched version of LLVM 14. We normally don't backport big feature addition because of this.

@sunho
Copy link
Contributor Author

sunho commented Jun 30, 2022

It's indeed a bit complicated. I thought about making if JL_LLVM_VERSION < 140000 more specific, but then we have to tweak it every llvm14 version bump.

@vchuravy
Copy link
Member

Yeah, have all these landed for LLVM 15? We might delay until then, or make it something that can be set in Make.user and then tell people that are not using our fork to set a flag.

@sunho
Copy link
Contributor Author

sunho commented Jun 30, 2022

Yup, all of the commits should land in LLVM 15. We can delay until then.

@oschulz
Copy link
Contributor

oschulz commented Nov 3, 2022

Probably fixes #47399

@maleadt
Copy link
Member

maleadt commented Feb 17, 2023

LLVM 15 is happening, #48700, so we should be able to move this forwards :-) Time for a rebase?

@vchuravy vchuravy changed the base branch from master to vc/upgrade_llvm15 February 17, 2023 20:16
@giordano
Copy link
Contributor

This PR solves for me a segmentation fault during compilation of Julia on ThunderX2: #46431 (comment). Thanks @sunho!

@SimonDanisch
Copy link
Contributor

Thanks all!
This PR makes the new Julia versions finally work on my Pixel 3a, which has a really nice 8 core armv8 processor and can now profit from the much improved compile times and less ram usage! :)

@oschulz
Copy link
Contributor

oschulz commented Feb 28, 2023

This PR makes the new Julia versions finally work on my Pixel 3a, which has a really nice 8 core armv8 processor and can now profit from the much improved compile times and less ram usage! :)

Oh, nice! How do you run Julia on it?

@SimonDanisch
Copy link
Contributor

I installed ubuntu touch: https://devices.ubuntu-touch.io/device/sargo

But I think these instructions will also work and are less disruptive: https://discourse.julialang.org/t/using-julia-on-android/8086/33?u=sdanisch (got 1.6.4 running on it as well, before I went for the nuclear option^^)

@oschulz
Copy link
Contributor

oschulz commented Feb 28, 2023

Thanks @SimonDanisch , haven't played with Julia under Termux for a while, time to dust it off, then. :-)

@minecraft2048
Copy link

This commit makes Julia run on Rock Pi 4: https://discourse.julialang.org/t/julia-1-8-5-not-working-on-rock-pi-4/95425

@giordano
Copy link
Contributor

We lost this PR and it can't be reopened 😢

@vtjnash
Copy link
Member

vtjnash commented May 10, 2023

You can always get any PR branch back with github's APIs. It is available as a patch file:
https://patch-diff.githubusercontent.com/raw/JuliaLang/julia/pull/45859.patch
or a branch object to git pull
https://github.com/JuliaLang/julia/commits/refs/pull/45859/head
Only the /merge ref is gone (aka the PR).

@giordano
Copy link
Contributor

New attempt in #49745, but this got a bit mangled with #49530

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler:llvm For issues that relate to LLVM system:arm ARMv7 and AArch64
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[LLVM] Large code model on AArch64 broken with RTDyld
10 participants