Compilation errors when built with O2 optimizations

[shaovoon]

If I remove the O2, I can built without errors. The emsdk is installed from sdk-incoming-64bit on 2016-08-21.

emsdk install --vs2015 sdk-incoming-64bit

I have plenty of free RAM(Total:16GB with no other programs running) and SSD space when the error happened.

F:\Emscripten\emscripten\incoming>emcc F:\App\SDL_App\TestEmscripten\TestEmscripten.cpp --bind -s FULL_ES2=1 -s USE_SDL=2 -s USE_SDL_IMAGE=2 -s ASSERTIONS=1 -D EMSCRIPTEN -std=c++11 -I F:\App\SDL_App\ext\glm-0.9.5.3\glm -O2 --llvm-opts 2 -o project.html
Traceback (most recent call last):
File "F:\Emscripten\emscripten\incoming\emcc", line 13, in
emcc.run()
File "F:\Emscripten\emscripten\incoming\emcc.py", line 1811, in run
JSOptimizer.flush()
File "F:\Emscripten\emscripten\incoming\emcc.py", line 1708, in flush
run_passes(chunks[0], title, just_split=False, just_concat=False)
File "F:\Emscripten\emscripten\incoming\emcc.py", line 1681, in run_passes
final = shared.Building.js_optimizer(final, passes, debug_level >= 4, JSOptimizer.extra_info, just_split=just_split, just_concat=just_concat)
File "F:\Emscripten\emscripten\incoming\tools\shared.py", line 1756, in js_optimizer
ret = js_optimizer.run(filename, passes, NODE_JS, debug, extra_info, just_split, just_concat)
File "F:\Emscripten\emscripten\incoming\tools\js_optimizer.py", line 555, in run
return temp_files.run_and_clean(lambda: run_on_js(filename, passes, js_engine, source_map, extra_info, just_split, just_concat))
File "F:\Emscripten\emscripten\incoming\tools\tempfiles.py", line 78, in run_and_clean
return func()
File "F:\Emscripten\emscripten\incoming\tools\js_optimizer.py", line 555, in
return temp_files.run_and_clean(lambda: run_on_js(filename, passes, js_engine, source_map, extra_info, just_split, just_concat))
File "F:\Emscripten\emscripten\incoming\tools\js_optimizer.py", line 450, in run_on_js
filenames = pool.map(run_on_chunk, commands, chunksize=1)
File "F:\Emscripten\python\2.7.5.3_64bit\lib\multiprocessing\pool.py", line 250, in map
return self.map_async(func, iterable, chunksize).get()
File "F:\Emscripten\python\2.7.5.3_64bit\lib\multiprocessing\pool.py", line 554, in get
raise self._value
WindowsError: [Error 2] The system cannot find the file specified

[kripken]

EMCC_DEBUG=1 might show more info that can help.

Also worth trying emcc --clear-cache to see if this was a disk error during system lib building.

[shaovoon]

I have clear cache and set EMCC_DEBUG=1. I got this in debug log.

DEBUG:root:testing for asm.js target, because if not present (i.e. this is plain vanilla llvm, not emscripten fastcomp), we will use the wasm target instead (set EMCC_WASM_BACKEND to skip this check)
DEBUG:root:check tells us to use asm.js backend

How to build fastcomp?

[kripken]

You can use the sdk or build manually from source.

[shaovoon]

One thing I do not understand, isn't sdk-incoming-64bit already Fastcomp enabled? Can you tell me which branch or tag is Fastcomp enabled by default?

[saschanaz]

I'm having the same issue on SDK version 1.36.9 which is downloaded and built by emsdk tool. Maybe something went wrong on SDK part?

[JMLX42]

@kripken same issue when building from sources with VS2015 yesterday with incoming and master.

js_optimizer.py creates a bunch of temporary js files, creates a bunch of commands to call the optimizer on each file and dispatches that to Python's Pool component (from pool.py).
But it looks like those temp files don't exist anymore when the pool runs or terminates.

[kripken]

This is likely windows-specific, so I don't think I can help much, maybe @juj has an idea. But I would check if you see that problem on small "hello world" programs too, might help narrow things down.

[saschanaz]

Yes I do. This simple code causes the problem.

#include <iostream>
using namespace std;
int main()
{
  cout << "Hello World!" << endl;
  cout << "Welcome to C++ Programming" << endl;
}

emcc helloworld.cpp -O2 -o helloworld.js

Full debug log:

PS C:\Users\sasch\Documents\GitHub> emcc helloworld.cpp -O2 -o helloworld.js
DEBUG:root:Cache: acquiring multiprocess file lock to Emscripten cache
DEBUG:root:Cache: done
DEBUG:root:Cache: released multiprocess file lock to Emscripten cache
DEBUG:root:failed to use vanilla file, will re-check: too many values to unpack
DEBUG:root:testing for asm.js target, because if not present (i.e. this is plain vanilla llvm, not emscripten fastcomp), we will use the wasm target instead (set EMCC_WASM_BACKEND to skip this check)
DEBUG:root:check tells us to use asm.js backend
WARNING:root:invocation: C:\Program Files\Emscripten\emscripten\tag-1.36.9\\emcc helloworld.cpp -O2 -o helloworld.js  (in C:\Users\sasch\Documents\GitHub)
INFO:root:(Emscripten: Running sanity checks)
DEBUG:root:compiling to bitcode
DEBUG:root:emcc step "parse arguments and setup" took 0.00 seconds
DEBUG:root:compiling source file: helloworld.cpp
DEBUG:root:running: C:/Program Files/Emscripten/clang/tag-e1.36.9/build_tag-e1.36.9_vs2015_64/RelWithDebInfo/bin\clang++ -target asmjs-unknown-emscripten -D__EMSCRIPTEN_major__=1 -D__EMSCRIPTEN_minor__=36 -D__EMSCRIPTEN_tiny__=9 -D_LIBCPP_ABI_VERSION=2 -Werror=implicit-function-declaration -nostdinc -Xclang -nobuiltininc -Xclang -nostdsysteminc -Xclang -isystemC:\Program Files\Emscripten\emscripten\tag-1.36.9\system\include\libcxx -Xclang -isystemC:\Program Files\Emscripten\emscripten\tag-1.36.9\system\lib\libcxxabi\include -Xclang -isystemC:\Program Files\Emscripten\emscripten\tag-1.36.9\system\include\compat -Xclang -isystemC:\Program Files\Emscripten\emscripten\tag-1.36.9\system\include -Xclang -isystemC:\Program Files\Emscripten\emscripten\tag-1.36.9\system\include\emscripten -Xclang -isystemC:\Program Files\Emscripten\emscripten\tag-1.36.9\system\include\libc -Xclang -isystemC:\Program Files\Emscripten\emscripten\tag-1.36.9\system\lib\libc\musl\arch\emscripten -Xclang -isystemC:\Program Files\Emscripten\emscripten\tag-1.36.9\system\local\include -O2 -std=c++03 -mllvm -disable-llvm-optzns helloworld.cpp -Xclang -isystemC:\Program Files\Emscripten\emscripten\tag-1.36.9\system\include\SDL -emit-llvm -c -o c:/users/sasch/appdata/local/temp\tmpvlvvfg\helloworld_0.o
DEBUG:root:emcc step "bitcodeize inputs" took 0.83 seconds
DEBUG:root:optimizing helloworld.cpp
DEBUG:root:emcc: LLVM opts: -O3 -disable-loop-vectorization -disable-slp-vectorization -vectorize-loops=false -vectorize-slp=false -vectorize-slp-aggressive=false  [num inputs: 1]
DEBUG:root:emcc step "process inputs" took 0.05 seconds
DEBUG:root:will generate JavaScript
DEBUG:root:including libcxx_noexcept.a
DEBUG:root:Cache: acquiring multiprocess file lock to Emscripten cache
DEBUG:root:Cache: done
DEBUG:root:Cache: released multiprocess file lock to Emscripten cache
DEBUG:root:including libcxxabi.bc
DEBUG:root:Cache: acquiring multiprocess file lock to Emscripten cache
DEBUG:root:Cache: done
DEBUG:root:Cache: released multiprocess file lock to Emscripten cache
DEBUG:root:including libc.bc
DEBUG:root:Cache: acquiring multiprocess file lock to Emscripten cache
DEBUG:root:Cache: done
DEBUG:root:Cache: released multiprocess file lock to Emscripten cache
DEBUG:root:including dlmalloc.bc
DEBUG:root:Cache: acquiring multiprocess file lock to Emscripten cache
DEBUG:root:Cache: done
DEBUG:root:Cache: released multiprocess file lock to Emscripten cache
DEBUG:root:emcc step "calculate system libraries" took 0.03 seconds
DEBUG:root:linking: ['c:/users/sasch/appdata/local/temp\\tmpvlvvfg\\helloworld_0_1.o', 'C:\\Users\\sasch\\.emscripten_cache\\asmjs\\libc.bc', 'C:\\Users\\sasch\\.emscripten_cache\\asmjs\\dlmalloc.bc', 'C:\\Users\\sasch\\.emscripten_cache\\asmjs\\libcxx_noexcept.a', 'C:\\Users\\sasch\\.emscripten_cache\\asmjs\\libcxxabi.bc']
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\tmpvlvvfg\helloworld_0_1.o to link
DEBUG:root:adding object C:\Users\sasch\.emscripten_cache\asmjs\libc.bc to link
DEBUG:root:adding object C:\Users\sasch\.emscripten_cache\asmjs\dlmalloc.bc to link
DEBUG:root:considering archive C:\Users\sasch\.emscripten_cache\asmjs\libcxx_noexcept.a
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\ios_2d8dc3c3.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\iostream_19b16a38.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\locale_7fdd6b67.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\memory_0ffda057.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\mutex_c029146d.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\new_27520e7b.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\string_d32c2a59.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\system_error_393f10af.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\condition_variable_aa94ca21.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\stdexcept_d35391e7.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\thread_799c1b94.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\future_31fc0cfa.cpp.o to link
DEBUG:root:adding object c:\users\sasch\appdata\local\temp\emscripten_temp\tmpx0rylr\exception_856bfa76.cpp.o to link
DEBUG:root:done running loop of archive C:\Users\sasch\.emscripten_cache\asmjs\libcxx_noexcept.a
DEBUG:root:adding object C:\Users\sasch\.emscripten_cache\asmjs\libcxxabi.bc to link
DEBUG:root:emcc step "link" took 0.49 seconds
DEBUG:root:saving intermediate processing steps to c:/users/sasch/appdata/local/temp\emscripten_temp
DEBUG:root:(not saving intermediate c:/users/sasch/appdata/local/temp\emscripten_temp\emcc-0-basebc.bc because deferring linking)
DEBUG:root:emcc: LLVM opts: -strip-debug -disable-verify -internalize -internalize-public-api-list=main,malloc,free,__errno_location,fflush,__cxa_can_catch,__cxa_is_pointer_type -globaldce -disable-loop-vectorization -disable-slp-vectorization -vectorize-loops=false -vectorize-slp=false -vectorize-slp-aggressive=false  [num inputs: 17]
DEBUG:root:emcc step "post-link" took 0.38 seconds
DEBUG:root:LLVM => JS
DEBUG:root:emscript: llvm backend: C:/Program Files/Emscripten/clang/tag-e1.36.9/build_tag-e1.36.9_vs2015_64/RelWithDebInfo/bin\llc c:\users\sasch\appdata\local\temp\tmpvlvvfg\helloworld.bc -march=js -filetype=asm -o c:\users\sasch\appdata\local\temp\emscripten_temp\tmpra1pkn.4.js -emscripten-global-base=8 -O2
DEBUG:root:  emscript: llvm backend took 0.485999822617 seconds
DEBUG:root:emscript: js compiler glue
DEBUG:root:  emscript: glue took 0.40700006485 seconds
DEBUG:root:emscript: python processing: function tables and exports
DEBUG:root:asm text sizes[[1388144, 16867], 3589, 243, 2690, 0, 3062, 23447, 1518, 268, 1908, 3027]
DEBUG:root:  emscript: python processing: function tables and exports took 0.00499987602234 seconds
DEBUG:root:emscript: python processing: finalize
DEBUG:root:  emscript: python processing: finalize took 0.00699996948242 seconds
DEBUG:root:emcc step "emscript (llvm => executable code)" took 1.26 seconds
DEBUG:root:wrote memory initialization to helloworld.js.mem
DEBUG:root:emcc step "source transforms" took 0.35 seconds
DEBUG:root:running js post-opts
DEBUG:root:applying js optimization passes: asm eliminate simplifyExpressions simplifyIfs registerize minifyNames asmLastOpts last minifyWhitespace cleanup
chunkification: num funcs: 728 actual num chunks: 3 chunk size range: 521475 - 408214
DEBUG:root:config forcing native optimizer at C:/Program Files/Emscripten/emscripten/tag-1.36.9_64bit_optimizer/RelWithDebInfo/optimizer.exe
DEBUG:root:js optimizer using native
DEBUG:root:config forcing native optimizer at C:/Program Files/Emscripten/emscripten/tag-1.36.9_64bit_optimizer/RelWithDebInfo/optimizer.exe
DEBUG:root:config forcing native optimizer at C:/Program Files/Emscripten/emscripten/tag-1.36.9_64bit_optimizer/RelWithDebInfo/optimizer.exe
DEBUG:root:config forcing native optimizer at C:/Program Files/Emscripten/emscripten/tag-1.36.9_64bit_optimizer/RelWithDebInfo/optimizer.exe
splitting up js optimization into 3 chunks, using 3 cores  (total: 1.34 MB)
DEBUG:root:Cache: acquiring multiprocess file lock to Emscripten cache
DEBUG:root:Cache: done
DEBUG:root:Cache: acquiring multiprocess file lock to Emscripten cache
DEBUG:root:Cache: released multiprocess file lock to Emscripten cache
DEBUG:root:failed to use vanilla file, will re-check: too many values to unpack
DEBUG:root:Cache: done
DEBUG:root:testing for asm.js target, because if not present (i.e. this is plain vanilla llvm, not emscripten fastcomp), we will use the wasm target instead (set EMCC_WASM_BACKEND to skip this check)
DEBUG:root:Cache: released multiprocess file lock to Emscripten cache
DEBUG:root:failed to use vanilla file, will re-check: too many values to unpack
DEBUG:root:testing for asm.js target, because if not present (i.e. this is plain vanilla llvm, not emscripten fastcomp), we will use the wasm target instead (set EMCC_WASM_BACKEND to skip this check)
DEBUG:root:Cache: acquiring multiprocess file lock to Emscripten cache
DEBUG:root:Cache: done
DEBUG:root:Cache: released multiprocess file lock to Emscripten cache
DEBUG:root:failed to use vanilla file, will re-check: too many values to unpack
DEBUG:root:testing for asm.js target, because if not present (i.e. this is plain vanilla llvm, not emscripten fastcomp), we will use the wasm target instead (set EMCC_WASM_BACKEND to skip this check)
DEBUG:root:check tells us to use asm.js backend
DEBUG:root:check tells us to use asm.js backend
DEBUG:root:check tells us to use asm.js backend
Traceback (most recent call last):
  File "C:\Program Files\Emscripten\emscripten\tag-1.36.9\\emcc", line 13, in <module>
    emcc.run()
  File "C:\Program Files\Emscripten\emscripten\tag-1.36.9\emcc.py", line 1811, in run
    JSOptimizer.flush()
  File "C:\Program Files\Emscripten\emscripten\tag-1.36.9\emcc.py", line 1708, in flush
    run_passes(chunks[0], title, just_split=False, just_concat=False)
  File "C:\Program Files\Emscripten\emscripten\tag-1.36.9\emcc.py", line 1681, in run_passes
    final = shared.Building.js_optimizer(final, passes, debug_level >= 4, JSOptimizer.extra_info, just_split=just_split, just_concat=just_concat)
  File "C:\Program Files\Emscripten\emscripten\tag-1.36.9\tools\shared.py", line 1769, in js_optimizer
    ret = js_optimizer.run(filename, passes, NODE_JS, debug, extra_info, just_split, just_concat)
  File "C:\Program Files\Emscripten\emscripten\tag-1.36.9\tools\js_optimizer.py", line 555, in run
    return temp_files.run_and_clean(lambda: run_on_js(filename, passes, js_engine, source_map, extra_info, just_split, just_concat))
  File "C:\Program Files\Emscripten\emscripten\tag-1.36.9\tools\tempfiles.py", line 78, in run_and_clean
    return func()
  File "C:\Program Files\Emscripten\emscripten\tag-1.36.9\tools\js_optimizer.py", line 555, in <lambda>
    return temp_files.run_and_clean(lambda: run_on_js(filename, passes, js_engine, source_map, extra_info, just_split, just_concat))
  File "C:\Program Files\Emscripten\emscripten\tag-1.36.9\tools\js_optimizer.py", line 450, in run_on_js
    filenames = pool.map(run_on_chunk, commands, chunksize=1)
  File "C:\Program Files\Emscripten\python\2.7.5.3_64bit\lib\multiprocessing\pool.py", line 250, in map
    return self.map_async(func, iterable, chunksize).get()
  File "C:\Program Files\Emscripten\python\2.7.5.3_64bit\lib\multiprocessing\pool.py", line 554, in get
    raise self._value
WindowsError: [Error 2] The system cannot find the file specified

[saschanaz]

I think this is not an unexistent fastcomp problem as my debug log says DEBUG:root:check tells us to use asm.js backend after the check.

[kripken]

Those DEBUG:root:failed to use vanilla file, will re-check: too many values to unpack look bad, does emcc --clear-cache help?

[shaovoon]

emcc --clear-cache does not help in my case.

[kripken]

What does ~/.emscripten_cache/is_vanilla.txt contain? (that's the file it reads before showing that error)

[shaovoon]

There is only 1 line in my is_vanilla.txt

0:F:/Emscripten/clang/fastcomp/build_incoming_vs2015_64/RelWithDebInfo/bin

This folder exists on my PC and is filled with clang tools.