emscripten linking with many dependencies broken on Windows

[Moxinilian]

Hello.

I have been trying to compile a rather big project for wasm32-unknown-emscripten using cargo on Windows.
Unfortunately, the linking process fails when trying to run emcc.bat because the command line invoked is too long for Windows' limitations (about 8k characters).
error: linking with `emcc.bat` failed: exit code: 1
note: The command line is too long.

After manually counting, the command indeed is too big with in my case about 11k characters, as it includes the absolute path of every single .o and .rlib files. My project, while including piston as a dependency, does not have an incredibly large amount of them, which makes me believe this could be an important issue in the long run (this does not apply to Linux however as the maximum character limit usually is much higher).
After a little bit of digging, emscripten seems to provide a workaround for this problem by accepting parameters from files. However it has to be handled by the user.

Thanks in advance for your help.

[dataf3l]

why not use a folder like c:\a so that it works?

[Moxinilian]

Well yes, that would work in my specific case.
But do you really want that in a final product?

[ritiek]

Maybe #46113 is related.

[alexcrichton]

@Moxinilian can you provide the full error message? It looks like this may be a bug in emscripten, not in rustc.

[Moxinilian]

@alexcrichton I uploaded the project I tried to compile to this repo
https://github.com/Moxinilian/emscripten_bug
log.txt contains the entire output

edit: i'm adding the verbose output too
edit: done

[alexcrichton]

Ah ok thanks for the extra info @Moxinilian!

So for a bit of background, rustc does indeed have safeguards in place for reinvoking the linker if necessary when the command line is deemed too large. What's happening here, though, is that the error is happening at the wrong time.

The current detection relies on the spawning of the process itself to fail (aka the OS tells us about the failure) but it looks like cmd at least is succeeding. Some spawned process actually prints out The command line is too long but it's not clear to me whether that's cmd or emcc.bat.

[Moxinilian]

@alexcrichton Is there any additional test you'd want me to run to try to figure this out?

[alexcrichton]

If you'd like to debug further that'd be great. We'd first need to understand what's actually failing here (cmd or something in emscripten). Once that's known we could figure out if it's reasonable for us to catch the error and do something about it.

[Moxinilian]

@alexcrichton So I edited emcc.bat to print a dummy message when it gets called.
As the message did not show up, I came to the conclusion that cmd itself refuses to run the command.
I tried to run it with a shorter command and the message showed up.

This is surprising because that means rust would have failed to survive cmd's refusal.
I'm not an OS expert so I'm not sure what raw_os_error actually returns, but here cmd's return code is 1.

[alexcrichton]

Ok so sounds like we successfully spawn a cmd process and then cmd fails to spawn its own subprocess. That's going to be much trickier to handle an catch, but it at least sounds like what's happening here at least.

[Moxinilian]

A dirty way to catch it would be to simply check the error description. At least it is self explanatory so there won't be any false positive.