Skip to content

When compiling, use a single clang command for all input sources. NFC #20713

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 15, 2023

Conversation

sbc100
Copy link
Collaborator

@sbc100 sbc100 commented Nov 14, 2023

When compiling, use a single clang command for all input sources. NFC

For folks that compiler a bunch of source files at once which should
help by reducing the number of clang subprocesses that we spawn. This
includes the new system library build process that we use internally.
See #20577.

Do the same for PCH compilation and pre-processing (-E).

@sbc100 sbc100 requested review from kripken and RReverser November 14, 2023 20:04
@sbc100
Copy link
Collaborator Author

sbc100 commented Nov 14, 2023

This change builds on #20712

shared.check_call(cmd)
return []

if state.mode == Mode.COMPILE_ONLY:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this limited to compile-only mode? Wouldn't it work for compile-and-link too?

Copy link
Collaborator Author

@sbc100 sbc100 Nov 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It compile-and-link mode it more complicated because we don't want to leave any object files in the current working directory. If we could find a way to extent this later to cover all cases that would be awesome but for now we want control over the output filenames in that case (just like the clang driver internally does).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we could just pass the args to Clang so e.g. emcc foo.c bar.c -o output.js becomes clang foo.c bar.c -o output.wasm but I guess it's not that simple?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the problem is that we don't really on clang as out linker-driver, we do it outselves. We run wam-ld and then a bunch of post-link tools from emcc.py.

If we could convert to using clang as the linker driver (i.e. have clang run wasm-ld for us) that would a really good step in the right direction.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other words, when linking we currently manage the names and locations of the temporary object files, and not clang. Maybe we could improve on this in the future.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. Leaving as a future improvement sgtm.

@sbc100 sbc100 force-pushed the single_clang_command branch from 81cd044 to e71b36f Compare November 14, 2023 22:26
@sbc100
Copy link
Collaborator Author

sbc100 commented Nov 14, 2023

This change should be much more readable now that #20712 has landed.

@sbc100 sbc100 force-pushed the single_clang_command branch 3 times, most recently from 6727618 to 69743ef Compare November 14, 2023 23:29
@sbc100 sbc100 force-pushed the single_clang_command branch 5 times, most recently from 4bcfad8 to 184908e Compare November 15, 2023 03:47
@sbc100 sbc100 enabled auto-merge (squash) November 15, 2023 04:00
@sbc100 sbc100 force-pushed the single_clang_command branch from 184908e to 1cd7599 Compare November 15, 2023 06:48
For folks that compiler a bunch of source files at once which should
help by reducing the number of clang subprocesses that we spawn.  This
includes the new system library build process that we use internally.
See emscripten-core#20577.

Do the same for PCH compilation and pre-processing (-E).
@sbc100 sbc100 force-pushed the single_clang_command branch from 1cd7599 to 7e4fc1a Compare November 15, 2023 06:59
@sbc100 sbc100 merged commit 94b36c0 into emscripten-core:main Nov 15, 2023
@sbc100 sbc100 deleted the single_clang_command branch November 15, 2023 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants