Feature: Jscodeshift Transformations for --migrate (#40)

* fix: Support for more complex query strings in loaders syntax

* style: Re-format source code for loaders transformation

* style: Don't run eslint on fixtures code

* WIP: Added tests for removeJsonLoader

* Do not pre-process test fixtures with eslint --fix since it makes our code less reliable.

Pre-processing fixtures with eslint makes our transform way less reliable
since we basically doing 2 transforms each time. In user-land it won't probably the case
and we need to test only our code.

Even if we want to provide re-formatting of the code with eslint for users, this should
probably be a sepaeate task.

* Use spaces instead of tabs for indentation in fixtures to get reliable results.

It looks like tabs are treated differently even by differrent tools. This was leading to some
false test fails. Replacing tabs with spaces makes it reliable for tests to run.

* Introduce removeJsonLoaderTransform that remove json-loader from loaders sections.

* style: Re-format code of resolve transformation

* Introduce uglifyJsPluginTransform

* Add .editorconfig to fixtures directories

* Add OccurrenceOrderPlugin transformation

* fixup! Introduce uglifyJsPluginTransform

* Made OccurrenceOrderPlugin in to a more generic removeDeprecatedPlugins transform

* Renamed safeTraverse to utils to allow multiple jscodeshift-helpers to be defined in it

* Share memberExpressionToPathString when working with plugins

* Introduce findPluginsByName function

* WIP on LoaderOptionsPlugin

* WIP on LoaderOptionsPlugin

* WIP Extract Plugin

* Gets basic case of extract text plugin working

* Fixed ExtractTextPlugin tests by properly formatting both input and output. Added .editorconfig

* Added tests for utils

* Fixed createOrUpdatePluginByName implementation

* Always use Literals for keys for now

* Fixed LoaderOptionsPlugin

* Return the full AST for compatibility

* Implemented BannerPlugin transformation.

See https://webpack.js.org/guides/migrating/#bannerplugin-breaking-change

* Use shorthand object syntax

* Added JSDoc for findPluginsByName util function

* Export all available transforms

* fixup! Implemented BannerPlugin transformation.

* Simplified directory structure and colocate tests with source files.

* Switch to snapshot testing and removed output fixtures

* All transformations return AST to allow chaining

* Split separate test cases into separate files

* Added tests and fixed some edge cases in BannerPlugin

* Introduce and export `transform` function

It takes source and array of transformations and returns the new source code.
Added "integration" tests.

* Adds new utils and tests

* Refactors loaders transforms

* Adds more refactoring

* Trims the input test

* Fixes eslint and parse bug

* Adds to the contributor docs about transforms

.eslintignore

@@ -1 +1 @@
-__testfixtures__
+**/__testfixtures__/*

CONTRIBUTING.md

@@ -128,6 +128,69 @@ to read on GitHub as well as in several git tools.
 For more information about what each part of the template mean, head up to the documentation in the
 [angular repo](https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#commit-message-format)
 
+## --migrate with the CLI
+
+This is a new feature in development for the CLI.
+
+```
+webpack --migrate <your-config-name>
+```
+
+The expected result of the above command is to take the mentioned `webpack` configuration and create a new configuration file which is compatible with webpack 2.
+It should be a valid new config and should keep intact all the features from the original config.
+The new config will be as readable as possible (may be add some comments).
+
+With [#40](https://github.com/webpack/webpack-cli/pull/40), we have been able to add basic scaffolding and do many of the conversions recommended in the [docs](https://webpack.js.org/guides/migrating/).
+
+### How it's being done
+
+We use [`jscodeshift`](https://github.com/facebook/jscodeshift) transforms called `codemods` to accomplish this.
+We have written a bunch of transformations under [/lib/transformations](https://github.com/webpack/webpack-cli/tree/master/lib/transformations),divided logically.
+We convert the existing webpack config to [AST](https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Parser_API). We then parse this tree for the specific features and modify it to conform to webpack v2...
+
+#### Structure of a transform
+
+The directory structure of a transform looks as follows -
+
+```sh
+|
+|--__snapshots__
+|--__testfixtures__
+|  |
+|  |--transform-name.input.js
+|
+|--transform-name.js
+|--transform-name.test.js
+```
+
+`transform-name.js`
+
+This file contains the actual transformation codemod. It applies specific transformation and parsing logic to accomplish its job
+There are utilities available under `/lib/utils.js` which can help you with this.
+
+`transform-name.test.js`
+
+This is where you declare a new test case for your transformation.
+Each test will refer to an input webpack config snippet.
+Conventionally we write them in `\_\_testfixtures\_\_`.
+
+```
+const defineTest = require('../defineTest');
+
+defineTest(__dirname, 'transform-name.input1.js');
+defineTest(__dirname, 'transform-name.input2.js');
+```
+
+`defineTest` is a helper test method which helps us to run tests on all the transforms uniformly.
+It takes the input file given as parameter and uses jest to create a snapshot of the output. This effectively tests the correctness of our transformation.
+
+### TODO
+
+This is still in a very raw form. We'd like to take this as close to a truly useful tool as possible.
+We will still need to
+  - Support all kinds of webpack configuration(made using merge tools)
+  - Test these transforms against real world configurations.
+
 ## Contributor License Agreement
 
 When submitting your contribution, a CLA (Contributor License Agreement) bot will come by to verify that you signed the CLA. If you are submitting a PR for the first time, it will link you to the right place to sign it. If you have committed your contributions using an email that is not the same as your email used on GitHub, the CLA bot can't accept your contribution.

lib/migrate.js

@@ -1,35 +1,36 @@
 const fs = require('fs');
-const jscodeshift = require('jscodeshift');
 const diff = require('diff');
 const chalk = require('chalk');
-const transformations = require('./transformations');
+const transform = require('./transformations').transform;
 const inquirer = require('inquirer');
 
 module.exports = (currentConfigLoc, outputConfigLoc) => {
 	let currentConfig = fs.readFileSync(currentConfigLoc, 'utf8');
-	let ast = jscodeshift(currentConfig);
-	let transformNames = Object.keys(transformations);
-	transformNames.forEach(key => transformations[key](jscodeshift, ast));
-	const outputConfig = ast.toSource();
+	const outputConfig = transform(currentConfig);
 	const diffOutput = diff.diffLines(currentConfig, outputConfig);
 	diffOutput.map(diffLine => {
-		if(diffLine.added) {
+		if (diffLine.added) {
 			process.stdout.write(chalk.green(`+ ${diffLine.value}`));
-		} else if(diffLine.removed){
+		} else if (diffLine.removed) {
 			process.stdout.write(chalk.red(`- ${diffLine.value}`));
 		}
 	});
-	inquirer.prompt([{
-		type: 'confirm',
-		name: 'confirmMigration',
-		message: 'Are you sure these changes are fine?',
-		default: 'Y'}]).then(answers => {
-			if(answers['confirmMigration']){
-      // TODO validate the config
+	inquirer
+		.prompt([
+			{
+				type: 'confirm',
+				name: 'confirmMigration',
+				message: 'Are you sure these changes are fine?',
+				default: 'Y'
+			}
+		])
+		.then(answers => {
+			if (answers['confirmMigration']) {
+				// TODO validate the config
 				fs.writeFileSync(outputConfigLoc, outputConfig, 'utf8');
 				process.stdout.write(chalk.green(`Congratulations! Your new webpack v2 config file is at ${outputConfigLoc}`));
 			} else {
-				process.stdout.write(chalk.yellow('You aborted the migration'));
+				process.stdout.write(chalk.yellow('Migration aborted'));
 			}
 		});
-};
+};

lib/transformations/__snapshots__/index.test.js.snap

@@ -0,0 +1,100 @@
+exports[`transform should respect recast options 1`] = `
+"
+module.exports = {
+	devtool: \'eval\',
+	entry: [
+		\'./src/index\'
+	],
+	output: {
+		path: path.join(__dirname, \'dist\'),
+		filename: \'index.js\'
+	},
+	module: {
+		rules: [{
+			test: /.js$/,
+			use: \"babel\",
+			include: path.join(__dirname, \'src\')
+		}]
+	},
+	resolve: {
+        modules: [\'node_modules\', path.resolve(\'/src\')],
+    },
+	plugins: [
+		new webpack.optimize.UglifyJsPlugin({
+            sourceMap: true,
+        }),
+		new webpack.optimize.LoaderOptionsPlugin({
+            \"debug\": true,
+            \"minimize\": true,
+        })
+	],
+	debug: true
+};
+"
+`;
+
+exports[`transform should transform only using specified transformations 1`] = `
+"
+module.exports = {
+	devtool: \'eval\',
+	entry: [
+		\'./src/index\'
+	],
+	output: {
+		path: path.join(__dirname, \'dist\'),
+		filename: \'index.js\'
+	},
+	module: {
+		rules: [{
+			test: /.js$/,
+			use: [\'babel\'],
+			include: path.join(__dirname, \'src\')
+		}]
+	},
+	resolve: {
+		root: path.resolve(\'/src\'),
+		modules: [\'node_modules\']
+	},
+	plugins: [
+		new webpack.optimize.UglifyJsPlugin(),
+		new webpack.optimize.OccurrenceOrderPlugin()
+	],
+	debug: true
+};
+"
+`;
+
+exports[`transform should transform using all transformations 1`] = `
+"
+module.exports = {
+	devtool: \'eval\',
+	entry: [
+		\'./src/index\'
+	],
+	output: {
+		path: path.join(__dirname, \'dist\'),
+		filename: \'index.js\'
+	},
+	module: {
+		rules: [{
+			test: /.js$/,
+			use: \'babel\',
+			include: path.join(__dirname, \'src\')
+		}]
+	},
+	resolve: {
+        modules: [\'node_modules\', path.resolve(\'/src\')]
+    },
+	plugins: [
+		new webpack.optimize.UglifyJsPlugin({
+            sourceMap: true
+        }),
+		new webpack.optimize.LoaderOptionsPlugin({
+            \'debug\': true,
+            \'minimize\': true
+        })
+	],
+	debug: true
+};
+"
+`;

lib/transformations/__snapshots__/utils.test.js.snap

@@ -0,0 +1,71 @@
+exports[`utils createLiteral should create basic literal 1`] = `
+Object {
+  "comments": null,
+  "loc": null,
+  "regex": null,
+  "type": "Literal",
+  "value": "strintLiteral",
+}
+`;
+
+exports[`utils createLiteral should create boolean 1`] = `
+Object {
+  "comments": null,
+  "loc": null,
+  "regex": null,
+  "type": "Literal",
+  "value": true,
+}
+`;
+
+exports[`utils createOrUpdatePluginByName should add an object as an argument 1`] = `
+"[new Plugin({
+  \"foo\": true
+})]"
+`;
+
+exports[`utils createOrUpdatePluginByName should create a new plugin with arguments 1`] = `
+"{ plugins: [new Plugin({
+  \"foo\": \"bar\"
+})] }"
+`;
+
+exports[`utils createOrUpdatePluginByName should create a new plugin without arguments 1`] = `"{ plugins: [new Plugin()] }"`;
+
+exports[`utils createOrUpdatePluginByName should merge options objects 1`] = `
+"[new Plugin({
+  \"foo\": false,
+  \"bar\": \"baz\",
+  \"baz-long\": true
+})]"
+`;
+
+exports[`utils createProperty should create properties for Boolean 1`] = `
+"{
+    \"foo\": true
+}"
+`;
+
+exports[`utils createProperty should create properties for Number 1`] = `
+"{
+    \"foo\": -1
+}"
+`;
+
+exports[`utils createProperty should create properties for String 1`] = `
+"{
+    \"foo\": \"bar\"
+}"
+`;
+
+exports[`utils createProperty should create properties for complex keys 1`] = `
+"{
+    \"foo-bar\": \"bar\"
+}"
+`;
+
+exports[`utils createProperty should create properties for non-literal keys 1`] = `
+"{
+    1: \"bar\"
+}"
+`;

lib/transformations/bannerPlugin/__snapshots__/bannerPlugin.test.js.snap

@@ -0,0 +1,30 @@
+exports[`bannerPlugin transforms correctly using "bannerPlugin-0" data 1`] = `
+"module.exports = {
+    plugins: [
+        new webpack.BannerPlugin({
+            raw: true,
+            entryOnly: true,
+            \'banner\': \'Banner\'
+        })
+    ]
+}
+"
+`;
+
+exports[`bannerPlugin transforms correctly using "bannerPlugin-1" data 1`] = `
+"// Should do nothing if there is no banner plugin
+module.exports = {
+    plugins: []
+}
+"
+`;
+
+exports[`bannerPlugin transforms correctly using "bannerPlugin-2" data 1`] = `
+"// Only transform if it uses the old format
+module.exports = {
+    plugins: [
+        new webpack.BannerPlugin({})
+    ]
+}
+"
+`;

lib/transformations/bannerPlugin/__testfixtures__/.editorconfig

@@ -0,0 +1,3 @@
+[*]
+indent_style = space
+indent_size = 4

lib/transformations/bannerPlugin/__testfixtures__/bannerPlugin-0.input.js

@@ -0,0 +1,5 @@
+module.exports = {
+    plugins: [
+        new webpack.BannerPlugin('Banner', { raw: true, entryOnly: true })
+    ]
+}

lib/transformations/bannerPlugin/__testfixtures__/bannerPlugin-1.input.js

@@ -0,0 +1,4 @@
+// Should do nothing if there is no banner plugin
+module.exports = {
+    plugins: []
+}

lib/transformations/bannerPlugin/__testfixtures__/bannerPlugin-2.input.js

@@ -0,0 +1,6 @@
+// Only transform if it uses the old format
+module.exports = {
+    plugins: [
+        new webpack.BannerPlugin({})
+    ]
+}

lib/transformations/bannerPlugin/bannerPlugin.js

@@ -0,0 +1,19 @@
+const utils = require('../utils');
+
+module.exports = function(j, ast) {
+	return utils.findPluginsByName(j, ast, ['webpack.BannerPlugin'])
+		.forEach(path => {
+			const args = path.value.arguments;
+			// If the first argument is a literal replace it with object notation
+			// See https://webpack.js.org/guides/migrating/#bannerplugin-breaking-change
+			if (args && args.length > 1 && args[0].type === j.Literal.name) {
+				// and remove the first argument
+				path.value.arguments = [path.value.arguments[1]];
+				utils.createOrUpdatePluginByName(j, path.parent, 'webpack.BannerPlugin', {
+					banner: args[0].value
+				});
+			}
+		});
+
+
+};

lib/transformations/bannerPlugin/bannerPlugin.test.js

@@ -0,0 +1,5 @@
+const defineTest = require('../defineTest');
+
+defineTest(__dirname, 'bannerPlugin', 'bannerPlugin-0');
+defineTest(__dirname, 'bannerPlugin', 'bannerPlugin-1');
+defineTest(__dirname, 'bannerPlugin', 'bannerPlugin-2');