Compile stuck at optimize chunk assets

[pherrymason]

I've started having problems to compile my build when --optimize-minimize is enabled.
I see that I've reached +/-205 build modules, and then the compilation gets stuck at 80% optimize chunk assets for a very long time (+40minutes).
If I wait enough, it ends up node failing with "fatal error: js allocation failed - process out of memory"

I guess it has to do with the uglifyjs plugin as I said this only happens when using --optimize-minime flag (When compiling through my grunt task, this also fails).

[sokra]

Could try to run uglifyjs on the output files? Maybe it's an uglifyjs bug...

[pherrymason]

I've done what you suggested, I first build with webpack with no optimization and then executed this script to compress using uglifyjs (v2) and got no issues:

#!/bin/bash
FILES=*.js
for f in $FILES
do
    echo "uglifyjs $f -o ./min/$f --source-map ./min/$f.map.js -c"
    uglifyjs $f -o ./min/$f --source-map ./min/$f.map.js -c 
done

Maybe I'm using different arguments options from those used in UglifyJsPlugin?

If I couldn't fix this, can I switch to this script to compress my build? Should I configure uglifyjs in a specific way?

[sokra]

hmm... Could you give me access to the repo or prepare a testcase so I can debug it?

[pherrymason]

I could pass you the bunch of files...

On Wed, Oct 15, 2014 at 11:25 AM, Tobias Koppers notifications@github.com
wrote:

hmm... Could you give me access to the repo or prepare a testcase so I can
debug it?

—
Reply to this email directly or view it on GitHub
#537 (comment).

[pherrymason]

How can I send you privately the repo/files?

[erkiesken]

I hit a similar problem. It seems that the UglifyJSPlugin can't handle a compressed return value from uglify.Compressor, specifically the ast.transform(compress) (see line 70 of UglifyJsPlugin.js) step hangs for me.

I added console logging like this around there:

if(options.compress !== false) {
    console.log("uglifying pre-compress", file);
    var compress = uglify.Compressor(options.compress);
    console.log("uglifying done-compress", file);
    ast = ast.transform(compress);
    console.log("uglifying done-transform", file);
    ast.figure_out_scope();
    if(options.mangle !== false) {
        ast.compute_char_frequency(options.mangle || {});
        ast.mangle_names(options.mangle || {});
    }
}

And in my output I can see it does indeed get stuck there:

3915ms build modules
13ms optimize
751ms build modules
21ms hashing
631ms create chunk assets
 78% additional chunk assets
uglifying pre-compress main.js
uglifying done-compress main.js
...hangs here...
^C%

Workaround for me for now was to init the uglify plugin with new webpack.optimize.UglifyJsPlugin({compress:false}) to skip that step.

The codebase in question is a quite large ReactJS/Reflux application.

[erkiesken]

If you have any ideas which Uglify compress options I should try to weed out what specific feature thet ast.transform chokes on, then let me know.

[jkimbo]

I've just come across the same problem with a large project with 26 entry points. If I disable source maps it works but hangs with source maps enabled. My current hypothesis is that because webpack keeps all the sourcemaps in memory as it is compiling it results in node hitting the 1.4Gb limit in v8. I'm not familiar with how the internals work but can the maps not be flushed to the files on disk instead of keeping them in memory?

[jkimbo]

I've managed to solve the issue by externalising some of the big libraries (React and jQuery). That seems to have reduced the memory usage considerably and I can now compress the bundles with source maps.

[jhnns]

Probably it's sufficient to flag libraries with noParse

[jkimbo]

@sokra so I've created a fairly contrived example repo that manages to reproduce the problem here: https://github.com/jkimbo/webpack-memory-issue

It has 20 entry points each of which requires fairly large libraries and then it runs it through the uglify plugin with sourcemaps. I realise it is in no way optimised but hopefully it can help debug this issue.

[pr0ton]

I am seeing a similar issue as well.

[pago]

I have the same problem without UglifyJS (continuous development build). The problem occurs after a couple of rebuilds and I believe I have an idea of where it leaks memory.

In my case, the count of ModuleNotFoundError objects created is growing at an extremely high pace. That error is thrown because of moment.js so I excluded it from parsing (noParse) and as a result the heap size stays stable (instead of growing further). If I read the dump in node-inspector correctly, it might be in the cache object of the Compilation module.

That's as much as I could find out so far.

[sokra]

You can try to disable the cache with cache: false.