Skip to content

[esm-integration] Add pthread support #24555

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 13, 2025

Conversation

sbc100
Copy link
Collaborator

@sbc100 sbc100 commented Jun 12, 2025

Under ESM integration all dependencies must be satisfied at import time, but pthreads requires that we have supply the memory via postMessage, so the memory is, by definition, not available at import time.

On order to work around this issue we create a smaller pthread stub/loader file that delays the import of the main program until the initial postMessage has been received. Once the memory is received we load main program using a dynamic import statement.

@sbc100 sbc100 requested review from brendandahl and dschuff June 12, 2025 21:54
@sbc100
Copy link
Collaborator Author

sbc100 commented Jun 12, 2025

I found a way to make it work! Not pretty, but it works!

sbc100 added a commit to sbc100/emscripten that referenced this pull request Jun 12, 2025
I think I was confused at some point and thought that only ASYNCIFY=1
was not supported.

Split out from emscripten-core#24555
@sbc100 sbc100 force-pushed the esm_integration_pthread branch 3 times, most recently from 1233179 to 967f7fa Compare June 12, 2025 22:39
sbc100 added a commit to sbc100/emscripten that referenced this pull request Jun 12, 2025
I think I was confused at some point and thought that only ASYNCIFY=1
was not supported.

Split out from emscripten-core#24555
@sbc100 sbc100 force-pushed the esm_integration_pthread branch from 967f7fa to cf8e793 Compare June 12, 2025 22:43
sbc100 added a commit that referenced this pull request Jun 13, 2025
I think I was confused at some point and thought that only ASYNCIFY=1
was not supported.

Split out from #24555
sbc100 added a commit to sbc100/emscripten that referenced this pull request Jun 13, 2025
This function doesn't need to take an argument or be declared in closure
externs anymore.

Split out from emscripten-core#24555
sbc100 added a commit that referenced this pull request Jun 13, 2025
This function doesn't need to take an argument or be declared in closure
externs anymore.

Split out from #24555
@sbc100 sbc100 force-pushed the esm_integration_pthread branch from cf8e793 to 18d5a3a Compare June 13, 2025 22:07
@sbc100 sbc100 requested a review from kripken June 13, 2025 22:08
@sbc100 sbc100 force-pushed the esm_integration_pthread branch 2 times, most recently from 9da3bec to b10d786 Compare June 13, 2025 22:09
@sbc100
Copy link
Collaborator Author

sbc100 commented Jun 13, 2025

This change has now been simplified a lot. I think its good to go now. PTAL

src/postamble.js Outdated
// When run as a worker thread run `init` immediately.
if ({{{ ENVIRONMENT_IS_WORKER_THREAD() }}}) await init()
if ({{{ ENVIRONMENT_IS_WORKER_THREAD() }}}) init()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if ({{{ ENVIRONMENT_IS_WORKER_THREAD() }}}) init()
if ({{{ ENVIRONMENT_IS_WORKER_THREAD() }}}) init();

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(pre-existing, but still)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted this line.

@sbc100 sbc100 force-pushed the esm_integration_pthread branch from b10d786 to 79d4b0b Compare June 13, 2025 22:50
Copy link
Member

@kripken kripken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I see, thanks. Tricky but nice how you got it to work.

Under ESM integration all dependencies must be satisfied at import time,
but pthreads requires that we have supply the memory via postMessage,
so the memory is, by definition, not available at import time.

On order to work around this issue we create a smaller pthread
stub/loader file that delays the import of the main program until the
initial `postMessage` has been received.  Once the memory is received we
load main program using a dynamic `import` statement.
@sbc100 sbc100 force-pushed the esm_integration_pthread branch from 79d4b0b to 968bbbc Compare June 13, 2025 23:52
@sbc100 sbc100 merged commit 6be9440 into emscripten-core:main Jun 13, 2025
10 of 14 checks passed
@sbc100 sbc100 deleted the esm_integration_pthread branch June 13, 2025 23:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants