Multicompiler with multiple output paths breaks WDS

[SpaceK33z]

Moved from webpack/webpack#3091 - by @drphelps:

I'm submitting a feature request

Webpack version:
2.x

Please tell us about your environment:
OSX 10.x

Current behavior:
Currently, a Multicompiler instance with a different output.path specified for each config will boil the outputPath property down to a "common" path. I'm referring to this block of code https://github.com/webpack/webpack/blob/547295ca0e91dd5bb2c446e2df19c887c6aa053c/lib/MultiCompiler.js#L51-L63. This breaks WDS filename resolution.

Expected/desired behavior:
WDS should serve resources from both paths or at least offer the ability to choose the first/preferred output.path when constructing the filename. I need to be able to output my files to separate (not necessarily nested) directories without breaking resolution.

• What is the motivation / use case for changing the behavior?
  In my setup, I am trying to build two separate apps in parallel and I need to be able to share resources between the two, including async modules. Consider the following config:

module.exports = [
  {
    context: __dirname,
    entry: "./app.js",
    output: {
      filename: "app.js",
      path: path.resolve(__dirname, './app/dist'),
      publicPath: '/static/javascript/dist/'
    },
    ...
  },
  {
    context: __dirname,
    entry: "./widget.js",
    output: {
      filename: "widget.js",
      path: path.resolve(__dirname, './widget/dist'),
      publicPath: '/widget/dist/'
    },
    ...
  }
]

[iMoses]

Just found that the hard way.

My use case is two configs, one for JavaScript and the other for static CSS, each requires a different output and resolves so I am using multi-compiler mode. I would like to be able to run webpack-dev-server simultaneously for both of them, using the same host & port configuration.

Is anybody working on it or should I take initiative? :)

[SpaceK33z]

Is anybody working on it or should I take initiative? :)

Nope, feel free.

[iMoses]

@SpaceK33z
I pushed a quick fix which does the trick for me.
Let me know what you think.

[rainercedric23]

Any updates on @iMoses fix? this is my issue also when I tried to run with multiple outputs. hopefully it can be pushed.

[SpaceK33z]

See webpack/webpack-dev-middleware#151

[ywmail]

This bug blocks me to use the WDS, but I found a solution from https://github.com/glenjamin/webpack-hot-middleware, the webpack-hot-middleware allows me to use multi compiler instances.

const firstConfig = require('./config/first');
const secondConfig = require('./config/second');

let express = require('express');
let middleware = require('webpack-dev-middleware');
let app = express();

// Dev Server
[firstConfig, secondConfig].forEach(function (config) {
    let compiler = webpack(config);
    app.use(middleware(compiler, {
        publicPath: config.output.publicPath
    }));

    // Enables HMR
    app.use(webpackHotMiddleware(compiler, {
        log: console.log, path: config.output.publicPath + '__webpack_hmr', heartbeat: 10 * 1000
    }));

});


let server = app.listen(18088);

[rainercedric23]

Hi @ywmail, can I see your firstConfig and second config file for reference. many thanks.

[ywmail]

part of the code:

server.js

const telcoConfig = require('./config/webpack.dev')({ env: 'development', app: 'telco' });
const marketplaceConfig = require('./config/webpack.dev')({ env: 'development', app: 'marketplace' });

webpack.dev.js

const hotMiddlewareScript = 'webpack-hot-middleware/client?path=__webpack_hmr&timeout=20000&reload=true&dynamicPublicPath=true';

    return webpackMerge(commonConfig({ env: ENV, app: app }), {

        entry: {

            'polyfills': ['./src/' + app + '/polyfills.browser.ts', hotMiddlewareScript],
            'vendor': ['./src/' + app + '/vendor.browser.ts', hotMiddlewareScript],
            'main': ['./src/' + app + '/main.browser.ts', hotMiddlewareScript]

        },

        devtool: 'cheap-module-source-map',

        output: {

            path: helpers.root('dist/' + app),

            publicPath: '/' + app + '/',

            filename: '[name].bundle.js',

            sourceMapFilename: '[name].map',

            chunkFilename: '[id].chunk.js',

            library: 'ac_[name]',
            libraryTarget: 'var',
        },

        plugins: [
        ],

    })

[akinnee]

Any update on this?

[elygre]

This would be very nice to have!

We are trying to fit webpack into an existing toolchain, requiring multiple configurations. The real killer would be to have a webpack-dev-server serving the assets from both the configs, at the paths specified by each.

[robgha01]

I just found out the hard way >.< any updates on this ? for now i guess i will have to not use this and stick to one embedded theme ...

[SpaceK33z]

Instead of constantly asking for updates, you can help. PR webpack/webpack-dev-middleware#151 is a good starting point, but was never finished.

[NogsMPLS]

Looks like this might be fixed with this merge into dev-middleware: webpack/webpack-dev-middleware#187

just needs to get published

[maciej-gurban]

I'm using webpack@2.6.1 and webpack-dev-server@2.4.4 versions, and having two webpack configs - each with different context and devServer.contentBase - still doesn't seem to work. Is it supposed to be supported?

I'm using this example as it is and it works. This still requires to manually include script tags into index.html like so:

<script src="./desktop.js"></script>
<script src="./mobile.js"></script>

Now if I change the configuration of mobile bundle so that the entry file path is different than in desktop file that previously compiled well (mobile.js) will now throw 404 in the browser.

Things work well to compile multiple apps with multiple webpack configs, but the moment file path to any of them is different, things break without explanation and webpack does not report any issues.

Edit: Okay, so apparently the only issue is that webpack won't understand the paths to other directories, but as long as all files are in the same directory it's enough to just do a single require('./my/other/file.js') inside those files to make it work.

[maciej-gurban]

@faceyspacey Since you've worked on this, is there any docs I could read to understand how to properly configure webpack to use the multicompiler settings? I believe my use case is exactly as what @SpaceK33z described. Or is that workflow supported at all?

[jonohayon]

Hi, was there any progress on this?
I see that webpack/webpack-dev-middleware#187 fixed this problem in the middleware part, so that's not a problem anymore, and the code actually looks like it's supposed to be able to run multi-compiler configurations -- anyone knows what may cause this not to work (as it looks like it should)?

[shellscape]

#187 resolved the root issue that @SpaceK33z moved over from webpack. If there are errors being thrown please do report it as a bug separately and we can triage. If there's only difficulty in getting it to work, please head to StackOverflow or the webpack gitter for more discussion and peer troubleshooting.

[jonaskello]

If you want to have the same output.publicPath for multiple configs, or have them nested so one is the root and the other is a sub-path, then you need to make sure the configs have the exact same setting for output.path otherwise only the first config will have effect. Not sure why this is, I spent several hours trial and error to find this out so I just wanted to mention it for future reference.

UPDATE: Seems like this only works if you have the exact same publicPath. Nesting them, for example having one config with output.publicPath = "/" and one with output.publicPath = "/sub" does not work.

UPDATE2: Seems like the trick is to set output.path to exactly follow output.publicPath. For example the following will work for nested paths:

config1.output.path = path.resolve(__dirname, "./dist");
config2.output.path = path.resolve(__dirname, "./dist/sub");
config1.output.publicPath = "/";
config2.output.publicPath = "/sub";

[SergeShkurko]

My solution to a similar problem with webpack multi compiller & react & HMR

I have 2 sub projects: app & widget

App located at localhost:8080/* (except localhost:8080/widget)
Widget located at localhost:8080/widget

webpack.config.js

module.exports = [{
  entry: {
    app: [
      'src/app/index.js',
      // path = output.publicPath + __webpack_hmr
      'webpack-hot-middleware/client?path=/__webpack_hmr&timeout=20000&reload=true',
    ],
  },
  output: {
    publicPath: '/',
  },
  plugins: [
    new HtmlWebpackPlugin({
      template: './src/app/index.html',
    }),
    new webpack.HotModuleReplacementPlugin(),
  ],
  // Other module 1 options
},{
  entry: {
    widget: [
      'src/app/index.js',
      // path = output.publicPath + __webpack_hmr !see the difference between entries!
      'webpack-hot-middleware/client?path=/widget__webpack_hmr&timeout=20000&reload=true',
    ],
  },
  output: {
    publicPath: '/widget',
  },
  plugins: [
    new HtmlWebpackPlugin({
      template: './src/widget/index.html',
    }),
    new webpack.HotModuleReplacementPlugin(),
  ],
  // Other module 2 options
}]

dev-server.js

const
  fs = require('fs'),
  http = require('http'),
  express = require('express'),
  webpack = require('webpack')

const app = express()

;(function() {
  const webpackConfig = require(process.env.WEBPACK_CONFIG || './webpack.config')

  webpackConfig.forEach((config, index) => {
    const compiler = webpack(config)

    app
      .use(require('webpack-dev-middleware')(compiler, {
        publicPath: config.output.publicPath,
      }))
      .use(require('webpack-hot-middleware')(compiler, {
        log: console.log,
        path: `${config.output.publicPath}__webpack_hmr`,
        heartbeat: 10 * 1000,
      }))
  })
})()

if (require.main === module) {
  const server = http.createServer(app)
  server.listen(process.env.PORT || 8080, () =>
    console.log('Listening on %j', server.address())
  )
}

And... add react-hot-loader in app & widget entrypoints

Works great 👍

[hytromo]

@jonaskello I had the same problem. It was created by me sharing data between the different configs returned.

My guess would be that webpack is not acting with an immutable way on this configs once you return them, so if you are sharing data between the config, when the first config is being parsed and mutated, part of the 2nd one is, too, and that in turn makes it invalid etc.

By not sharing data between the configs, it seems that it is working for any output or public path I set.