Refactor Orc JIT to enable lazy compilation #974
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Refactor LLVM Orc JIT to actually enable the lazy compilation: https://llvm.org/docs/ORCv2.html#laziness Main modifications: - Create LLVM module for each wasm function, wrap it with thread safe module so that the modules can be compiled parallelly - Lookup function from aot module instance's func_ptrs but not directly call the function to decouple the module relationship - Compile the function when it is first called and hasn't been compiled - Create threads to pre-compile the WASM functions parallelly when loading - Set Lazy JIT as default, update document and build/test scripts
Merge main into dev/refactor_orc_jit
Merge main into dev/refactor_orc_jit
Merge main into dev/refactor_orc_jit
Merge main into dev/refactor_orc_jit
lum1n0us
reviewed
Jan 20, 2022
| if (orcjit_stop_compiling) { | ||
| break; | ||
| } | ||
| if (!module->func_ptrs[i]) { |
There was a problem hiding this comment.
L2819 和 L2792 会不会遇到多线程反复赋值的情况。比如：
- thread A 走到L2821的位置被挂起
- thread B 走到 L2819 发现没有赋值，又走一遍 L2821
- 结果是 thread A 和 B 都调用了
LVMOrcLLJITLookup，然后给func_ptrs[i]赋值两次？
There was a problem hiding this comment.
按照LLVM的说法，LLVMOrcLLJITLookup本身支持并发操作，并且同一个wasm函数只会被编译一次，编译了之后再次lookup会返回同一个指针:
Symbol lookup in ORC serves two other important functions, beyond providing addresses for symbols: (1) It triggers compilation of the symbol(s) searched for (if they have not been compiled already), and (2) it provides the synchronization mechanism for concurrent compilation.
这边是有可能多线程重复对一个指针赋值，但是即使重复赋值，赋的也是一样的值，另外这个应该不会有多线程读写冲突的问题，因为读写的指针类型是基本的数据类型，可以用一条机器指令完成，同一时刻只能总线上只能有一条指令在读或写一个指针。实际测试了很多case，没有发现有问题。参考：
https://www.cnblogs.com/zhouyazhou/p/7747926.html
lum1n0us
reviewed
Jan 20, 2022
| int32 group_stride; | ||
| } OrcJitThreadArg; | ||
|
|
||
| static bool orcjit_stop_compiling = false; |
There was a problem hiding this comment.
可能需要考虑用 volatile 强化线程间共享的变量
There was a problem hiding this comment.
应该不需要加volatile吧，编译的线程在调用orc lookup编译某个函数后，读这个变量，如果变成true，就不再编译其它函数，退出线程了。即使读到的这个变量还是fasle，最坏情况就是再编译一次下一个函数吧。
lum1n0us
reviewed
Jan 20, 2022
| "create orcjit compile thread failed"); | ||
| /* Terminate the threads created */ | ||
| orcjit_stop_compiling = true; | ||
| for (j = 0; j < i; j++) { |
There was a problem hiding this comment.
是否要考虑把 os_thread_join 的循环拿到外面？减少反复 join 的次数？
There was a problem hiding this comment.
这边是当创建线程失败时，去join其它已经创建的线程，不会反复join。移到外面的话，就要join所有线程，相当于等所有线程都编译好再执行，就不是真正的lazy编译了。这边创建了这些线程让它们进行编译，同时主线程还接着往前走调用jit函数，主线程一旦发现有函数没有编译，也会orc lookup编译一下。
|
LGTM |
wenyongh
referenced
this pull request
in wenyongh/wasm-micro-runtime
Jan 20, 2022
Refactor Orc JIT to enable lazy compilation (#974)
xujuntwt95329
pushed a commit
to xujuntwt95329/wasm-micro-runtime
that referenced
this pull request
Jan 24, 2022
Refactor LLVM Orc JIT to actually enable the lazy compilation and speedup the launching process: https://llvm.org/docs/ORCv2.html#laziness Main modifications: - Create LLVM module for each wasm function, wrap it with thread safe module so that the modules can be compiled parallelly - Lookup function from aot module instance's func_ptrs but not directly call the function to decouple the module relationship - Compile the function when it is first called and hasn't been compiled - Create threads to pre-compile the WASM functions parallelly when loading - Set Lazy JIT as default, update document and build/test scripts
vickiegpt
pushed a commit
to vickiegpt/wamr-aot-gc-checkpoint-restore
that referenced
this pull request
May 27, 2024
Refactor LLVM Orc JIT to actually enable the lazy compilation and speedup the launching process: https://llvm.org/docs/ORCv2.html#laziness Main modifications: - Create LLVM module for each wasm function, wrap it with thread safe module so that the modules can be compiled parallelly - Lookup function from aot module instance's func_ptrs but not directly call the function to decouple the module relationship - Compile the function when it is first called and hasn't been compiled - Create threads to pre-compile the WASM functions parallelly when loading - Set Lazy JIT as default, update document and build/test scripts
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refactor LLVM Orc JIT to actually enable the lazy compilation and speedup
the launching process:
https://llvm.org/docs/ORCv2.html#laziness
Main modifications:
so that the modules can be compiled parallelly
function to decouple the module relationship