add draft 7, but let's not make it default yet, as a lot of validators fail a LOT of tests. Is something wrong with the draft7 testrun

Author: ebdrup

README.md

@@ -8,20 +8,20 @@ for validators that cause side-effects on schema or data. The top 6 validators t
 
 # Performance
 
-![performance](https://chart.googleapis.com/chart?chxt=x,y&cht=bhs&chco=76A4FB&chls=2.0&chbh=62,4,1&chs=600x416&chxl=-1:|ajv|@exodus/schemasafe|is-my-json-valid|djv|@cfworker/json-schema|jsonschema&chd=t:100,69.8,49.8,12.6,4.2,1.1)
+![performance](https://chart.googleapis.com/chart?chxt=x,y&cht=bhs&chco=76A4FB&chls=2.0&chbh=62,4,1&chs=600x416&chxl=-1:|ajv|@exodus/schemasafe|is-my-json-valid|djv|@cfworker/json-schema|jsonschema&chd=t:100,69.2,51.5,13.1,5.1,1.2)
 
 |Validator|Relative speed|Number of test runs per second|
 |---------|:------------:|:----------------------------:|
-|[`ajv`](https://github.com/ajv-validator/ajv)|100%|17743 (± 0.42%)|
-|[`@exodus/schemasafe`](https://github.com/ExodusMovement/schemasafe)|69.8%|12383 (± 0.35%)|
-|[`is-my-json-valid`](https://github.com/mafintosh/is-my-json-valid#readme)|49.8%|8840 (± 0.27%)|
-|[`djv`](https://github.com/korzio/djv#readme)|12.6%|2239 (± 1.4%)|
-|[`@cfworker/json-schema`](https://github.com/cfworker/cfworker/tree/master/packages/json-schema/README.md)|4.2%|753 (± 1%)|
-|[`jsonschema`](https://github.com/tdegrunt/jsonschema#readme)|1.1%|197 (± 1.22%)|
+|[`ajv`](https://github.com/ajv-validator/ajv)|100%|17000 (± 0.34%)|
+|[`@exodus/schemasafe`](https://github.com/ExodusMovement/schemasafe)|69.2%|11760 (± 0.89%)|
+|[`is-my-json-valid`](https://github.com/mafintosh/is-my-json-valid#readme)|51.5%|8757 (± 0.37%)|
+|[`djv`](https://github.com/korzio/djv#readme)|13.1%|2223 (± 1.24%)|
+|[`@cfworker/json-schema`](https://github.com/cfworker/cfworker/tree/master/packages/json-schema/README.md)|5.1%|865 (± 1.15%)|
+|[`jsonschema`](https://github.com/tdegrunt/jsonschema#readme)|1.2%|198 (± 1.11%)|
 
 884 tests are run in each test run.
 
-Validators tested: [`@exodus/schemasafe (1.0.0-rc.3)`](https://github.com/ExodusMovement/schemasafe), [`@cfworker/json-schema (1.6.3)`](https://github.com/cfworker/cfworker/tree/master/packages/json-schema/README.md), [`ajv (6.12.6)`](https://github.com/ajv-validator/ajv), [`jsonschema (1.4.0)`](https://github.com/tdegrunt/jsonschema#readme), [`djv (2.1.3-alpha.0)`](https://github.com/korzio/djv#readme), [`is-my-json-valid (2.20.5)`](https://github.com/mafintosh/is-my-json-valid#readme), [`json-schema-library (4.1.4)`](https://github.com/sagold/json-schema-library), [`jsen (0.6.6)`](https://github.com/bugventure/jsen), [`tv4 (1.3.0)`](https://github.com/geraintluff/tv4), [`z-schema (5.0.0)`](https://github.com/zaggino/z-schema), [`jjv (1.0.2)`](https://github.com/acornejo/jjv), [`jassi (0.1.2)`](https://github.com/iclanzan/jassi), [`themis (1.1.6)`](https://github.com/playlyfe/themis), [`request-validator (0.3.3)`](https://github.com/bugventure/request-validator), [`json-schema-validator-generator (1.1.11)`](https://github.com/danwang/json-schema-validator-generator), [`jsck (0.3.2)`](https://github.com/pandastrike/jsck#readme), [`JSV (4.0.2)`](http://github.com/garycourt/JSV), [`json-model (0.2.24)`](https://github.com/geraintluff/json-model), [`skeemas (1.2.5)`](https://github.com/Prestaul/skeemas#readme), [`schemasaurus (0.7.8)`](https://github.com/AlexeyGrishin/schemasaurus), [`json-gate (0.8.23)`](https://github.com/oferei/json-gate#readme), [`revalidator (0.3.1)`](https://github.com/flatiron/revalidator), 
+Validators tested: [`@exodus/schemasafe (1.0.0-rc.3)`](https://github.com/ExodusMovement/schemasafe), [`@cfworker/json-schema (1.6.3)`](https://github.com/cfworker/cfworker/tree/master/packages/json-schema/README.md), [`ajv (6.12.6)`](https://github.com/ajv-validator/ajv), [`jsonschema (1.4.0)`](https://github.com/tdegrunt/jsonschema#readme), [`djv (2.1.3-alpha.0)`](https://github.com/korzio/djv#readme), [`is-my-json-valid (2.20.5)`](https://github.com/mafintosh/is-my-json-valid#readme), [`json-schema-library (4.1.4)`](https://github.com/sagold/json-schema-library), [`jsen (0.6.6)`](https://github.com/bugventure/jsen), [`tv4 (1.3.0)`](https://github.com/geraintluff/tv4), [`z-schema (5.0.0)`](https://github.com/zaggino/z-schema), [`jjv (1.0.2)`](https://github.com/acornejo/jjv), [`jassi (0.1.2)`](https://github.com/iclanzan/jassi), [`themis (1.1.6)`](https://github.com/playlyfe/themis), [`request-validator (0.3.3)`](https://github.com/bugventure/request-validator), [`json-schema-validator-generator (1.1.11)`](https://github.com/danwang/json-schema-validator-generator), [`json-model (0.2.24)`](https://github.com/geraintluff/json-model), [`jsck (0.3.2)`](https://github.com/pandastrike/jsck#readme), [`JSV (4.0.2)`](http://github.com/garycourt/JSV), [`skeemas (1.2.5)`](https://github.com/Prestaul/skeemas#readme), [`schemasaurus (0.7.8)`](https://github.com/AlexeyGrishin/schemasaurus), [`json-gate (0.8.23)`](https://github.com/oferei/json-gate#readme), [`revalidator (0.3.1)`](https://github.com/flatiron/revalidator), 
 
 (validators not in the results above where excluded because of failing tests - see below for details)
 
@@ -33,7 +33,7 @@ This test suite uses the [official JSON-schema test suite](https://github.com/js
 
 If a validator does not pass a test in the official test suite, it will show up in these results.
 
-![failing tests](https://chart.googleapis.com/chart?chxt=x,y&cht=bhs&chco=76A4FB&chls=2.0&chbh=14,4,1&chs=600x416&chxl=-1:|@exodus/schemasafe|@cfworker/json-schema|ajv|jsonschema|djv|is-my-json-valid|json-schema-library|jsen|tv4|z-schema|jjv|jassi|themis|request-validator|json-schema-validator-generator|jsck|JSV|json-model|skeemas|schemasaurus|json-gate|revalidator&chd=t:8,9,10,33,99,111,140,141,155,169,171,172,189,207,216,227,234,234,235,237,342,384&chxr=0,0,384&chds=0,384)
+![failing tests](https://chart.googleapis.com/chart?chxt=x,y&cht=bhs&chco=76A4FB&chls=2.0&chbh=14,4,1&chs=600x416&chxl=-1:|@exodus/schemasafe|@cfworker/json-schema|ajv|jsonschema|djv|is-my-json-valid|json-schema-library|jsen|tv4|z-schema|jjv|jassi|themis|request-validator|json-schema-validator-generator|json-model|jsck|JSV|skeemas|schemasaurus|json-gate|revalidator&chd=t:8,9,10,33,99,111,140,141,155,169,171,172,189,207,216,225,227,234,235,237,342,384&chxr=0,0,384&chds=0,384)
 
 |Validator|Number of failing tests (click for details)|
 |---------|-----------------------|
@@ -52,9 +52,9 @@ If a validator does not pass a test in the official test suite, it will show up
 |[`themis`](https://github.com/playlyfe/themis)|[189](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/themis.md)|
 |[`request-validator`](https://github.com/bugventure/request-validator)|[207](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/request-validator.md)|
 |[`json-schema-validator-generator`](https://github.com/danwang/json-schema-validator-generator)|[216](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/json-schema-validator-generator.md)|
+|[`json-model`](https://github.com/geraintluff/json-model)|[225](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/json-model.md)|
 |[`jsck`](https://github.com/pandastrike/jsck#readme)|[227](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/jsck.md)|
 |[`JSV`](http://github.com/garycourt/JSV)|[234](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/JSV.md)|
-|[`json-model`](https://github.com/geraintluff/json-model)|[234](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/json-model.md)|
 |[`skeemas`](https://github.com/Prestaul/skeemas#readme)|[235](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/skeemas.md)|
 |[`schemasaurus`](https://github.com/AlexeyGrishin/schemasaurus)|[237](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/schemasaurus.md)|
 |[`json-gate`](https://github.com/oferei/json-gate#readme)|[342](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/json-gate.md)|
@@ -69,13 +69,14 @@ Number of tests that caused side-effects. The schema or data was altered by the
 
 |Validator|Number of side-effects (BAD)|
 |---------|----------------------------|
-|[`json-model`](https://github.com/geraintluff/json-model)|[816](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/json-model-side-effects.md)|
 |[`revalidator`](https://github.com/flatiron/revalidator)|[818](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/revalidator-side-effects.md)|
+|[`json-model`](https://github.com/geraintluff/json-model)|[833](https://github.com/ebdrup/json-schema-benchmark/blob/master/reports/json-model-side-effects.md)|
 
 Validators not in the list have no side-effects on data or schemas.
 
 # Tests for other JSON-schema versions
 
+- [draft7](https://github.com/ebdrup/json-schema-benchmark/tree/master/draft7)
 - [draft6](https://github.com/ebdrup/json-schema-benchmark/)
 - [draft4](https://github.com/ebdrup/json-schema-benchmark/tree/master/draft4)
 

README.template

@@ -55,6 +55,7 @@ Validators not in the list have no side-effects on data or schemas.
 
 # Tests for other JSON-schema versions
 
+- [draft7](https://github.com/ebdrup/json-schema-benchmark/tree/master/draft7)
 - [draft6](https://github.com/ebdrup/json-schema-benchmark/)
 - [draft4](https://github.com/ebdrup/json-schema-benchmark/tree/master/draft4)
 

reports/SIDE-EFFECTS.template renamed to SIDE-EFFECTS.template

@@ -8,16 +8,16 @@ for validators that cause side-effects on schema or data. The top 6 validators t
 
 # Performance
 
-![performance](https://chart.googleapis.com/chart?chxt=x,y&cht=bhs&chco=76A4FB&chls=2.0&chbh=62,4,1&chs=600x416&chxl=-1:|ajv|@exodus/schemasafe|is-my-json-valid|z-schema|@cfworker/json-schema|jsonschema&chd=t:100,67.5,43,6.2,5,1.1)
+![performance](https://chart.googleapis.com/chart?chxt=x,y&cht=bhs&chco=76A4FB&chls=2.0&chbh=62,4,1&chs=600x416&chxl=-1:|ajv|@exodus/schemasafe|is-my-json-valid|z-schema|@cfworker/json-schema|jsonschema&chd=t:100,70.2,46.9,6.5,5.3,1.2)
 
 |Validator|Relative speed|Number of test runs per second|
 |---------|:------------:|:----------------------------:|
-|[`ajv`](https://github.com/ajv-validator/ajv)|100%|19250 (± 0.91%)|
-|[`@exodus/schemasafe`](https://github.com/ExodusMovement/schemasafe)|67.5%|12998 (± 1.04%)|
-|[`is-my-json-valid`](https://github.com/mafintosh/is-my-json-valid#readme)|43%|8268 (± 0.49%)|
-|[`z-schema`](https://github.com/zaggino/z-schema)|6.2%|1187 (± 1.53%)|
-|[`@cfworker/json-schema`](https://github.com/cfworker/cfworker/tree/master/packages/json-schema/README.md)|5%|962 (± 0.8%)|
-|[`jsonschema`](https://github.com/tdegrunt/jsonschema#readme)|1.1%|217 (± 1.22%)|
+|[`ajv`](https://github.com/ajv-validator/ajv)|100%|18434 (± 1.13%)|
+|[`@exodus/schemasafe`](https://github.com/ExodusMovement/schemasafe)|70.2%|12937 (± 3.11%)|
+|[`is-my-json-valid`](https://github.com/mafintosh/is-my-json-valid#readme)|46.9%|8643 (± 1.25%)|
+|[`z-schema`](https://github.com/zaggino/z-schema)|6.5%|1207 (± 0.55%)|
+|[`@cfworker/json-schema`](https://github.com/cfworker/cfworker/tree/master/packages/json-schema/README.md)|5.3%|982 (± 1.64%)|
+|[`jsonschema`](https://github.com/tdegrunt/jsonschema#readme)|1.2%|227 (± 0.35%)|
 
 692 tests are run in each test run.
 
@@ -77,6 +77,7 @@ Validators not in the list have no side-effects on data or schemas.
 
 # Tests for other JSON-schema versions
 
+- [draft7](https://github.com/ebdrup/json-schema-benchmark/tree/master/draft7)
 - [draft6](https://github.com/ebdrup/json-schema-benchmark/)
 - [draft4](https://github.com/ebdrup/json-schema-benchmark/tree/master/draft4)