Add phpcs baseline support

[frankdekker]

Added the ability to baseline violations.
Original PR: squizlabs/PHP_CodeSniffer#3387

Features

Create baseline

phpcs src --report-baseline=phpcs.baseline.xml --basepath=./

• Writes phpcs.baseline.xml in the current working directory. Pref be the root of the project
• basepath will be subtracted from the filepaths in the baseline xml.

Use default baseline
Will by default search in the working directory for phpcs.baseline.xml

phpcs src

Use custom location baseline

phpcs src --baseline-file=<path/to/baseline.xml>

Changes

Added classes to read the baseline file and setup data structure.
Added check in File::addMessage to check if the message is baselined.
Added --baseline-file to cli options
Added unit tests for the newly added files.

More precise baseline improvements

Improved the baseline calculation based on the approach I suggested in #2543

Now a code signature will be calculated and added to the baseline report file. The code signature is the hash of the code on the line of the violation, and 1 line before and after.

During standard inspection when comparing against the baseline, the same signature will be calculated for encountered violations and compared against the baseline signature. The violation will only be skipped if the code signature is identical to the one in the baseline.

As mentioned, this will prevent a sniff being disabled for the entire file. The sniff will now be specifically disabled for the calculated code signature. Modifications in the file won't break the baseline, unless you edit one of the 3 lines around the baseline. At this point I feel it's quite useful to resolve the violation anyway, as you're editing that close to it.

Example baseline:

<?xml version="1.0" encoding="UTF-8"?>
<phpcs-baseline version="3.6.1">
<violation file="src\Baseline\BaselineSet.php" sniff="PEAR.Commenting.FunctionComment.MissingParamName" signature="54accce13bb236dc361cf9d75895c274fce619d5"/>
<violation file="src\Baseline\BaselineSet.php" sniff="PEAR.Commenting.FunctionComment.MissingParamTag" signature="25a91da1dadc60a04a9a6615502ad4aa4fd397c2"/>
<violation file="src\Baseline\BaselineSet.php" sniff="PEAR.Functions.FunctionDeclaration.SpaceBeforeOpenParen" signature="edcfdf132eb46b842349404740ba7b8f224d82e1"/>
</phpcs-baseline>

[frankdekker]

Fixed a few issues with older phpunit versions. Was challenging to get working. As reminder for myself, and maybe helpful for others:

in composer.json:

"config": {
    "platform": {
        "php": "5.4.0"
    }
}

Then:

composer install
docker container run -it -v $(pwd):/app php:5.6  sh -c "cd /app; php vendor/bin/phpunit tests/AllTests.php"

[frankdekker]

@jrfnl Thanks for picking up this project :)

[jrfnl]

@frankdekker I appreciate that you ported this PR over. I'm not necessarily adverse to a feature like this, but I think this needs a lot more discussion on the principle of it and on how it should work before ever looking at an implementation of it.

There are lots of questions which need to be answered/discussed, to name a few:

• Should paths be relative to the ruleset location ? Or to a project root ? Or to some arbitrary user defined directory (as in this PR) ?
• How detailed should the baseline be ? This can range from nr of errors and warnings for a project to nr of errors/warnings per file in a project, to nr of errors/warnings per error code per file in a project, to exact errors/warnings per error code per line in a project etc.

I suggest closing the PR (for the time being) and opening an issue to discuss this further first.

I also suggest having a look at the handful of related issues open in the Squizlabs repo and to collect the various suggestions/options/things which need to be discussed from those so that the eventually proposed solution will have considered all the different opinions/suggestions.

[Potherca]

[...] and opening an issue to discuss this further first.

As the "Discussions" feature is (also) enabled for this repo, should we open an issue or a discussion to discuss this (and/or similar things).

[jrfnl]

[...] and opening an issue to discuss this further first.

As the "Discussions" feature is (also) enabled for this repo, should we open an issue or a discussion to discuss this (and/or similar things).

Good question. I'd say: open an issue for a feature request.

In my mind, the "Discussions" feature is more about users helping each other with questions on how to use PHPCS and possibly showcasing something they did with PHPCS.
The issues list is for tracking feature requests/bugs in PHPCS itself.

Does that make sense ?

[frankdekker]

[...] and opening an issue to discuss this further first.

As the "Discussions" feature is (also) enabled for this repo, should we open an issue or a discussion to discuss this (and/or similar things).

Good question. I'd say: open an issue for a feature request.

In my mind, the "Discussions" feature is more about users helping each other with questions on how to use PHPCS and possibly showcasing something they did with PHPCS. The issues list is for tracking feature requests/bugs in PHPCS itself.

Does that make sense ?

I've added a feature request. What would you propose for the choices in the issue? Can we add a poll?

[jrfnl]

[...] and opening an issue to discuss this further first.

As the "Discussions" feature is (also) enabled for this repo, should we open an issue or a discussion to discuss this (and/or similar things).

Good question. I'd say: open an issue for a feature request.
In my mind, the "Discussions" feature is more about users helping each other with questions on how to use PHPCS and possibly showcasing something they did with PHPCS. The issues list is for tracking feature requests/bugs in PHPCS itself.
Does that make sense ?

I've added a feature request. What would you propose for the choices in the issue? Can we add a poll?

If you can find a poll feature which GH markdown supports, go ahead and add a poll ;-)

I will look at the feature request and respond to it at a later point in time.

In the mean time, I propose we close this PR as until the way the feature should work has been decided upon, any implementation proposal is just noise.