Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Jscodeshift Transformations for --migrate #40

Merged
merged 43 commits into from
Feb 19, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
d8b57af
fix: Support for more complex query strings in loaders syntax
okonet Jan 26, 2017
465b915
style: Re-format source code for loaders transformation
okonet Jan 26, 2017
7da4804
style: Don't run eslint on fixtures code
okonet Jan 26, 2017
0278297
WIP: Added tests for removeJsonLoader
okonet Jan 26, 2017
cdd2cad
Do not pre-process test fixtures with eslint --fix since it makes our…
okonet Jan 27, 2017
97866e1
Use spaces instead of tabs for indentation in fixtures to get reliabl…
okonet Jan 27, 2017
dcdcd91
Introduce removeJsonLoaderTransform that remove json-loader from load…
okonet Jan 27, 2017
02c1425
style: Re-format code of resolve transformation
okonet Jan 27, 2017
eaab59a
Introduce uglifyJsPluginTransform
okonet Jan 27, 2017
c15acac
Add .editorconfig to fixtures directories
okonet Jan 27, 2017
4fa696d
Add OccurrenceOrderPlugin transformation
okonet Jan 27, 2017
1cbd96a
fixup! Introduce uglifyJsPluginTransform
okonet Feb 1, 2017
97e547a
Made OccurrenceOrderPlugin in to a more generic removeDeprecatedPlugi…
okonet Feb 1, 2017
6242628
Renamed safeTraverse to utils to allow multiple jscodeshift-helpers t…
okonet Feb 1, 2017
0f24d8d
Share memberExpressionToPathString when working with plugins
okonet Feb 1, 2017
c731d18
Introduce findPluginsByName function
okonet Feb 1, 2017
a8d8c31
WIP on LoaderOptionsPlugin
okonet Feb 1, 2017
d6a7364
WIP on LoaderOptionsPlugin
okonet Feb 3, 2017
e198fa1
WIP Extract Plugin
Feb 7, 2017
7ffb1e8
Gets basic case of extract text plugin working
Feb 11, 2017
8c2c9ad
Fixed ExtractTextPlugin tests by properly formatting both input and o…
okonet Feb 13, 2017
0dabfc3
Added tests for utils
okonet Feb 14, 2017
2bbfc4c
Fixed createOrUpdatePluginByName implementation
okonet Feb 15, 2017
333ff91
Always use Literals for keys for now
okonet Feb 15, 2017
734b3be
Fixed LoaderOptionsPlugin
okonet Feb 15, 2017
73ef8ce
Return the full AST for compatibility
okonet Feb 15, 2017
6d21726
Implemented BannerPlugin transformation.
okonet Feb 15, 2017
e8deaed
Use shorthand object syntax
okonet Feb 15, 2017
8f5a9d7
Added JSDoc for findPluginsByName util function
okonet Feb 15, 2017
5bcc5ec
Export all available transforms
okonet Feb 15, 2017
4601054
fixup! Implemented BannerPlugin transformation.
okonet Feb 15, 2017
83c65db
Simplified directory structure and colocate tests with source files.
okonet Feb 15, 2017
c12df12
Switch to snapshot testing and removed output fixtures
okonet Feb 15, 2017
45ccf97
All transformations return AST to allow chaining
okonet Feb 15, 2017
99624b5
Split separate test cases into separate files
okonet Feb 15, 2017
6f04f3d
Added tests and fixed some edge cases in BannerPlugin
okonet Feb 15, 2017
4223c3a
Introduce and export `transform` function
okonet Feb 15, 2017
067f71a
Adds new utils and tests
Feb 19, 2017
99e99cd
Refactors loaders transforms
Feb 19, 2017
3e064b4
Adds more refactoring
Feb 19, 2017
d758330
Trims the input test
Feb 19, 2017
d66f4e1
Fixes eslint and parse bug
Feb 19, 2017
e0539a1
Adds to the contributor docs about transforms
Feb 19, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .eslintignore
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__testfixtures__
**/__testfixtures__/*
63 changes: 63 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,69 @@ to read on GitHub as well as in several git tools.
For more information about what each part of the template mean, head up to the documentation in the
[angular repo](https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#commit-message-format)

## --migrate with the CLI

This is a new feature in development for the CLI.

```
webpack --migrate <your-config-name>
```

The expected result of the above command is to take the mentioned `webpack` configuration and create a new configuration file which is compatible with webpack 2.
It should be a valid new config and should keep intact all the features from the original config.
The new config will be as readable as possible (may be add some comments).

With [#40](https://github.com/webpack/webpack-cli/pull/40), we have been able to add basic scaffolding and do many of the conversions recommended in the [docs](https://webpack.js.org/guides/migrating/).

### How it's being done

We use [`jscodeshift`](https://github.com/facebook/jscodeshift) transforms called `codemods` to accomplish this.
We have written a bunch of transformations under [/lib/transformations](https://github.com/webpack/webpack-cli/tree/master/lib/transformations),divided logically.
We convert the existing webpack config to [AST](https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Parser_API). We then parse this tree for the specific features and modify it to conform to webpack v2...

#### Structure of a transform

The directory structure of a transform looks as follows -

```sh
|
|--__snapshots__
|--__testfixtures__
| |
| |--transform-name.input.js
|
|--transform-name.js
|--transform-name.test.js
```

`transform-name.js`

This file contains the actual transformation codemod. It applies specific transformation and parsing logic to accomplish its job
There are utilities available under `/lib/utils.js` which can help you with this.

`transform-name.test.js`

This is where you declare a new test case for your transformation.
Each test will refer to an input webpack config snippet.
Conventionally we write them in `\_\_testfixtures\_\_`.

```
const defineTest = require('../defineTest');

defineTest(__dirname, 'transform-name.input1.js');
defineTest(__dirname, 'transform-name.input2.js');
```

`defineTest` is a helper test method which helps us to run tests on all the transforms uniformly.
It takes the input file given as parameter and uses jest to create a snapshot of the output. This effectively tests the correctness of our transformation.

### TODO

This is still in a very raw form. We'd like to take this as close to a truly useful tool as possible.
We will still need to
- Support all kinds of webpack configuration(made using merge tools)
- Test these transforms against real world configurations.

## Contributor License Agreement

When submitting your contribution, a CLA (Contributor License Agreement) bot will come by to verify that you signed the CLA. If you are submitting a PR for the first time, it will link you to the right place to sign it. If you have committed your contributions using an email that is not the same as your email used on GitHub, the CLA bot can't accept your contribution.
Expand Down
35 changes: 18 additions & 17 deletions lib/migrate.js
Original file line number Diff line number Diff line change
@@ -1,35 +1,36 @@
const fs = require('fs');
const jscodeshift = require('jscodeshift');
const diff = require('diff');
const chalk = require('chalk');
const transformations = require('./transformations');
const transform = require('./transformations').transform;
const inquirer = require('inquirer');

module.exports = (currentConfigLoc, outputConfigLoc) => {
let currentConfig = fs.readFileSync(currentConfigLoc, 'utf8');
let ast = jscodeshift(currentConfig);
let transformNames = Object.keys(transformations);
transformNames.forEach(key => transformations[key](jscodeshift, ast));
const outputConfig = ast.toSource();
const outputConfig = transform(currentConfig);
const diffOutput = diff.diffLines(currentConfig, outputConfig);
diffOutput.map(diffLine => {
if(diffLine.added) {
if (diffLine.added) {
process.stdout.write(chalk.green(`+ ${diffLine.value}`));
} else if(diffLine.removed){
} else if (diffLine.removed) {
process.stdout.write(chalk.red(`- ${diffLine.value}`));
}
});
inquirer.prompt([{
type: 'confirm',
name: 'confirmMigration',
message: 'Are you sure these changes are fine?',
default: 'Y'}]).then(answers => {
if(answers['confirmMigration']){
// TODO validate the config
inquirer
.prompt([
{
type: 'confirm',
name: 'confirmMigration',
message: 'Are you sure these changes are fine?',
default: 'Y'
}
])
.then(answers => {
if (answers['confirmMigration']) {
// TODO validate the config
fs.writeFileSync(outputConfigLoc, outputConfig, 'utf8');
process.stdout.write(chalk.green(`Congratulations! Your new webpack v2 config file is at ${outputConfigLoc}`));
} else {
process.stdout.write(chalk.yellow('You aborted the migration'));
process.stdout.write(chalk.yellow('Migration aborted'));
}
});
};
};
100 changes: 100 additions & 0 deletions lib/transformations/__snapshots__/index.test.js.snap
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
exports[`transform should respect recast options 1`] = `
"
module.exports = {
devtool: \'eval\',
entry: [
\'./src/index\'
],
output: {
path: path.join(__dirname, \'dist\'),
filename: \'index.js\'
},
module: {
rules: [{
test: /.js$/,
use: \"babel\",
include: path.join(__dirname, \'src\')
}]
},
resolve: {
modules: [\'node_modules\', path.resolve(\'/src\')],
},
plugins: [
new webpack.optimize.UglifyJsPlugin({
sourceMap: true,
}),
new webpack.optimize.LoaderOptionsPlugin({
\"debug\": true,
\"minimize\": true,
})
],
debug: true
};
"
`;

exports[`transform should transform only using specified transformations 1`] = `
"
module.exports = {
devtool: \'eval\',
entry: [
\'./src/index\'
],
output: {
path: path.join(__dirname, \'dist\'),
filename: \'index.js\'
},
module: {
rules: [{
test: /.js$/,
use: [\'babel\'],
include: path.join(__dirname, \'src\')
}]
},
resolve: {
root: path.resolve(\'/src\'),
modules: [\'node_modules\']
},
plugins: [
new webpack.optimize.UglifyJsPlugin(),
new webpack.optimize.OccurrenceOrderPlugin()
],
debug: true
};
"
`;

exports[`transform should transform using all transformations 1`] = `
"
module.exports = {
devtool: \'eval\',
entry: [
\'./src/index\'
],
output: {
path: path.join(__dirname, \'dist\'),
filename: \'index.js\'
},
module: {
rules: [{
test: /.js$/,
use: \'babel\',
include: path.join(__dirname, \'src\')
}]
},
resolve: {
modules: [\'node_modules\', path.resolve(\'/src\')]
},
plugins: [
new webpack.optimize.UglifyJsPlugin({
sourceMap: true
}),
new webpack.optimize.LoaderOptionsPlugin({
\'debug\': true,
\'minimize\': true
})
],
debug: true
};
"
`;
71 changes: 71 additions & 0 deletions lib/transformations/__snapshots__/utils.test.js.snap
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
exports[`utils createLiteral should create basic literal 1`] = `
Object {
"comments": null,
"loc": null,
"regex": null,
"type": "Literal",
"value": "strintLiteral",
}
`;

exports[`utils createLiteral should create boolean 1`] = `
Object {
"comments": null,
"loc": null,
"regex": null,
"type": "Literal",
"value": true,
}
`;

exports[`utils createOrUpdatePluginByName should add an object as an argument 1`] = `
"[new Plugin({
\"foo\": true
})]"
`;

exports[`utils createOrUpdatePluginByName should create a new plugin with arguments 1`] = `
"{ plugins: [new Plugin({
\"foo\": \"bar\"
})] }"
`;

exports[`utils createOrUpdatePluginByName should create a new plugin without arguments 1`] = `"{ plugins: [new Plugin()] }"`;

exports[`utils createOrUpdatePluginByName should merge options objects 1`] = `
"[new Plugin({
\"foo\": false,
\"bar\": \"baz\",
\"baz-long\": true
})]"
`;

exports[`utils createProperty should create properties for Boolean 1`] = `
"{
\"foo\": true
}"
`;

exports[`utils createProperty should create properties for Number 1`] = `
"{
\"foo\": -1
}"
`;

exports[`utils createProperty should create properties for String 1`] = `
"{
\"foo\": \"bar\"
}"
`;

exports[`utils createProperty should create properties for complex keys 1`] = `
"{
\"foo-bar\": \"bar\"
}"
`;

exports[`utils createProperty should create properties for non-literal keys 1`] = `
"{
1: \"bar\"
}"
`;
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
exports[`bannerPlugin transforms correctly using "bannerPlugin-0" data 1`] = `
"module.exports = {
plugins: [
new webpack.BannerPlugin({
raw: true,
entryOnly: true,
\'banner\': \'Banner\'
})
]
}
"
`;

exports[`bannerPlugin transforms correctly using "bannerPlugin-1" data 1`] = `
"// Should do nothing if there is no banner plugin
module.exports = {
plugins: []
}
"
`;

exports[`bannerPlugin transforms correctly using "bannerPlugin-2" data 1`] = `
"// Only transform if it uses the old format
module.exports = {
plugins: [
new webpack.BannerPlugin({})
]
}
"
`;
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[*]
indent_style = space
indent_size = 4
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module.exports = {
plugins: [
new webpack.BannerPlugin('Banner', { raw: true, entryOnly: true })
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
// Should do nothing if there is no banner plugin
module.exports = {
plugins: []
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// Only transform if it uses the old format
module.exports = {
plugins: [
new webpack.BannerPlugin({})
]
}
19 changes: 19 additions & 0 deletions lib/transformations/bannerPlugin/bannerPlugin.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
const utils = require('../utils');

module.exports = function(j, ast) {
return utils.findPluginsByName(j, ast, ['webpack.BannerPlugin'])
.forEach(path => {
const args = path.value.arguments;
// If the first argument is a literal replace it with object notation
// See https://webpack.js.org/guides/migrating/#bannerplugin-breaking-change
if (args && args.length > 1 && args[0].type === j.Literal.name) {
// and remove the first argument
path.value.arguments = [path.value.arguments[1]];
utils.createOrUpdatePluginByName(j, path.parent, 'webpack.BannerPlugin', {
banner: args[0].value
});
}
});


};
5 changes: 5 additions & 0 deletions lib/transformations/bannerPlugin/bannerPlugin.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
const defineTest = require('../defineTest');

defineTest(__dirname, 'bannerPlugin', 'bannerPlugin-0');
defineTest(__dirname, 'bannerPlugin', 'bannerPlugin-1');
defineTest(__dirname, 'bannerPlugin', 'bannerPlugin-2');
Loading