Skip to content

Commit

Permalink
Add builder for SchemaValidatorsConfig (#1068)
Browse files Browse the repository at this point in the history
* Add builder for SchemaValidatorsConfig.

* Use an error message keyword instead of just a boolean

* Add tests

* Refactor

* Refactor

* Update docs

* Refactor

* Refactor openapi

* Update docs

* Update functional results

* Update version and benchmark

* Refactor

* Update docs

* Refactor

* Update docs

* Update docs
  • Loading branch information
justin-tay committed Jun 20, 2024
1 parent 1838371 commit 4f3345b
Show file tree
Hide file tree
Showing 103 changed files with 2,692 additions and 1,578 deletions.
98 changes: 48 additions & 50 deletions README.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions doc/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@ The implementation supports the use of custom keywords, formats, vocabularies an

There are currently no known issues with the required functionality from the specification.

The following are the tests results after running the [JSON Schema Test Suite](https://github.com/json-schema-org/JSON-Schema-Test-Suite) as at 29 Jan 2024 using version 1.3.1. As the test suite is continously updated, this can result in changes in the results subsequently.
The following are the tests results after running the [JSON Schema Test Suite](https://github.com/json-schema-org/JSON-Schema-Test-Suite) as at 18 Jun 2024 using version 1.4.1. As the test suite is continously updated, this can result in changes in the results subsequently.

| Implementations | Overall | DRAFT_03 | DRAFT_04 | DRAFT_06 | DRAFT_07 | DRAFT_2019_09 | DRAFT_2020_12 |
|-----------------|-------------------------------------------------------------------------|-------------------------------------------------------------------|---------------------------------------------------------------------|--------------------------------------------------------------------|------------------------------------------------------------------------|----------------------------------------------------------------------|------------------------------------------------------------------------|
| NetworkNt | pass: r:4703 (100.0%) o:2369 (100.0%)<br>fail: r:0 (0.0%) o:1 (0.0%) | | pass: r:600 (100.0%) o:251 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:796 (100.0%) o:318 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:880 (100.0%) o:541 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:1201 (100.0%) o:625 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:1226 (100.0%) o:634 (99.8%)<br>fail: r:0 (0.0%) o:1 (0.2%) |
| NetworkNt | pass: r:4803 (100.0%) o:2372 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | | pass: r:610 (100.0%) o:251 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:822 (100.0%) o:318 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:906 (100.0%) o:541 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:1220 (100.0%) o:625 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) | pass: r:1245 (100.0%) o:637 (100.0%)<br>fail: r:0 (0.0%) o:0 (0.0%) |

### Legend

Expand Down
102 changes: 40 additions & 62 deletions doc/config.md
Original file line number Diff line number Diff line change
@@ -1,69 +1,47 @@
### Configuration

To control the behavior of the library, we have introduced SchemaValidatorsConfig recently. It gives users great flexibility when using the library in different contexts.

For some users, it is just a JSON schema validator implemented mainly based on v4 with some additions from v5 to v7.

For others, it is used as a critical component in the REST API frameworks to validate the request or response. The library was developed as part of the [light-4j](https://github.com/networknt/light-4j) framework in the beginning.

Most of the configuration flags are used to control the difference between Swagger/OpenAPI specification and JSON schema specification as they are not the same. The future of the OpenAPI version might resolve this problem, but the release date is not set yet.
### Schema Validators Configuration

| Name | Description | Default Value
|---------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------
| `applyDefaultsStrategy` | The strategy for applying defaults when walking when missing or null nodes are encountered. | `ApplyDefaultsStrategy.EMPTY_APPLY_DEFAULTS_STRATEGY`
| `cacheRefs` | Whether the schemas loaded from refs will be cached and reused for subsequent runs. Setting this to `false` will affect performance but may be neccessary to prevent high memory usage for the cache if multiple nested applicators like `anyOf`, `oneOf` and `allOf` are used. | `true`
| `discriminatorKeywordEnabled` | Whether the `discriminator` keyword is handled according to OpenAPI 3. | `false`
| `errorMessageKeyword` | The keyword to use for custom error messages in the schema. If not set this features is disabled. This is typically set to `errorMessage` or `message`. | `null`
| `executionContextCustomizer` | This can be used to customize the `ExecutionContext` generated by the `JsonSchema` for each validation run. | `null`
| `failFast` | Whether to return failure immediately when an assertion is generated. | `false`
| `formatAssertionsEnabled` | The default is to generate format assertions from Draft 4 to Draft 7 and to only generate annotations from Draft 2019-09. Setting to `true` or `false` will override the default behavior. | `null`
| `javaSemantics` | Whether java semantics is used for the `type` keyword. | `false`
| `locale` | The locale to use for generating messages in the `ValidationMessage`. | `Locale.getDefault()`
| `losslessNarrowing` | Whether lossless narrowing is used for the `type` keyword. | `false`
| `messageSource` | This is used to retrieve the locale specific messages. | `DefaultMessageSource.getInstance()`
| `nullableKeywordEnabled` | Whether the `nullable` keyword is handled according to OpenAPI 3.0. This affects the `enum` and `type` keywords. | `false`
| `pathType` | The path type to use for reporting the instance location and evaluation path. Set to `PathType.JSON_PATH` to use JSON Path. | `PathType.JSON_POINTER`
| `preloadJsonSchema` | Whether the schema will be preloaded before processing any input. This will use memory but the execution of the validation will be faster. | `true`
| `preloadJsonSchemaRefMaxNestingDepth` | The max depth of the evaluation path to preload when preloading refs. | `40`
| `readOnly` | Whether schema is read only. This affects the `readOnly` keyword. | `null`
| `regularExpressionFactory` | The factory to use to create regular expressions for instance `JoniRegularExpressionFactory` or `GraalJSRegularExpressionFactory`. This requires the dependency to be manually added to the project or a `ClassNotFoundException` will be thrown. | `JDKRegularExpressionFactory.getInstance()`
| `schemaIdValidator` | This is used to customize how the `$id` values are validated. Note that the default implementation allows non-empty fragments where no base IRI is specified and also allows non-absolute IRI `$id` values in the root schema. | `JsonSchemaIdValidator.DEFAULT`
| `strict` | This is set whether keywords are strict in their validation. What this does depends on the individual validators. |
| `typeLoose` | Whether types are interpreted in a loose manner. If set to true, a single value can be interpreted as a size 1 array. Strings may also be interpreted as number, integer or boolean. | `false`
| `writeOnly` | Whether schema is write only. This affects the `writeOnly` keyword. | `null`

#### How to use config

When you create a `JsonSchema` instance from the `JsonSchemaFactory`, you can pass an object of SchemaValidatorsConfig as the second parameter.
When you create a `JsonSchema` instance from the `JsonSchemaFactory`, you can pass an object of `SchemaValidatorsConfig` as the second parameter.

```java
SchemaValidatorsConfig config = new SchemaValidatorsConfig();
config.setTypeLoose(false);
JsonSchema jsonSchema = JsonSchemaFactory.getInstance().getSchema(schema, config);
JsonSchemaFactory factory = JsonSchemaFactory.getInstance(VersionFlag.V202012);
SchemaValidatorsConfig config = SchemaValidatorsConfig.builder().typeLoose(false).build();
JsonSchema schema = factory.getSchema(schema, config);
```

#### Configurations

* typeLoose

When typeLoose is true, the validator will convert strings to different types to match the type defined in the schema. This is mostly used to validate the JSON request or response for headers, query parameters, path parameters, and cookies. For the HTTP protocol, these are all strings and might be defined as other types in the schema. For example, the page number might be an integer in the schema but passed as a query parameter in string. When it comes to validating arrays note that any item can also be interpreted as a size 1 array of that item so the item will be validated against the type defined for the array.

* strictness
This is a map of keywords to whether the keyword's validators should perform a strict or permissive analysis. When strict is true, validators will perform strict checking against the schema.
This is the default behavior. When set to false, validators are free to relax some constraints but not required. Each validator has its own understanding of what constitutes strict and
permissive.

* failFast

When set to true, the validation process stops immediately when the first error occurs. This mostly used on microservices that is designed to [fail-fast](https://www.networknt.com/architecture/fail-fast/), or users don't want to see hundreds of errors for a big payload. Please be aware that the validator throws an exception in the case the first error occurs. To learn how to use it, please follow the [test case](https://github.com/networknt/json-schema-validator/blob/master/src/test/java/com/networknt/schema/V4JsonSchemaTest.java#L352).

* handleNullableField

When a field is set as nullable in the OpenAPI specification, the schema validator validates that it is nullable; however, it continues with validation against the nullable field.

If handleNullableField is set to true && incoming field is nullable && value is field: null --> succeed

If handleNullableField is set to false && incoming field is nullable && value is field: null --> it is up to the type validator using the SchemaValidator to handle it.

The default value is true in the SchemaValidatorsConfig object.

For more details, please refer to this [issue](https://github.com/networknt/json-schema-validator/issues/183).

* javaSemantics

When set to true, use Java-specific semantics rather than native JavaScript semantics.

For example, if the node type is `number` per JS semantics where the value can be losslesly interpreted as `java.lang.Long`, the validator would use `integer` as the node type instead of `number`. This is useful when schema type is `integer`, since validation would fail otherwise.

For more details, please refer to this [issue](https://github.com/networknt/json-schema-validator/issues/334).

* losslessNarrowing

When set to true, can interpret round doubles as integers.

Note that setting `javaSemantics = true` will achieve the same functionality at this time.

For more details, please refer to this [issue](https://github.com/networknt/json-schema-validator/issues/344).

* pathType

This defines how path expressions are defined and returned once validation is performed through `ValidationMessage` instances. This can either be set to `PathType.JSON_POINTER` for [JSONPointer](https://www.rfc-editor.org/rfc/rfc6901.html) expressions,
or to `PathType.JSON_PATH` for [JSONPath](https://datatracker.ietf.org/doc/draft-ietf-jsonpath-base/) expressions. Doing so allows you to report the path for each finding and to potentially lookup nodes
(see [here](https://github.com/networknt/json-schema-validator/blob/c41df270a71f8423c63cfaa379d2e9b3f570b73e/doc/yaml-line-numbers.md#scenario-2---validationmessage-line-locations) for an example). By default, path expressions use a
`PathType.LEGACY` format which is close to JSONPath but does not escape reserved characters.
#### Details

| Name | Details
|---------------------------|------------------------------------------------
| `typeLoose` | When typeLoose is true, the validator will convert strings to different types to match the type defined in the schema. This is mostly used to validate the JSON request or response for headers, query parameters, path parameters, and cookies. For the HTTP protocol, these are all strings and might be defined as other types in the schema. For example, the page number might be an integer in the schema but passed as a query parameter in string. When it comes to validating arrays note that any item can also be interpreted as a size 1 array of that item so the item will be validated against the type defined for the array.
| `strict` | This is a map of keywords to whether the keyword's validators should perform a strict or permissive analysis. When strict is true, validators will perform strict checking against the schema. This is the default behavior. When set to false, validators are free to relax some constraints but not required. Each validator has its own understanding of what constitutes strict and permissive.
| `failFast` | When set to true, the validation process stops immediately when the first error occurs. This mostly used on microservices that is designed to [fail-fast](https://www.networknt.com/architecture/fail-fast/), or users don't want to see hundreds of errors for a big payload.
| `nullableKeywordEnabled` | When a field is set as `nullable` in the OpenAPI specification, the schema validator validates that it is `nullable`; however, it continues with validation against the `nullable` field. If `nullableKeywordEnabled` is set to true && incoming field is `nullable` && value is field: null --> succeed. If `nullableKeywordEnabled` is set to false && incoming field is `nullable` && value is field: null --> it is up to the type validator using the SchemaValidator to handle it.
| `javaSemantics` | When set to true, use Java-specific semantics rather than native JavaScript semantics. For example, if the node type is `number` per JS semantics where the value can be losslesly interpreted as `java.lang.Long`, the validator would use `integer` as the node type instead of `number`. This is useful when schema type is `integer`, since validation would fail otherwise.
| `losslessNarrowing` | When set to true, can interpret round doubles as integers. Note that setting `javaSemantics = true` will achieve the same functionality at this time.
| `pathType` | This defines how path expressions are defined and returned once validation is performed through `ValidationMessage` instances. This can either be set to `PathType.JSON_POINTER` for [JSONPointer](https://www.rfc-editor.org/rfc/rfc6901.html) expressions, or to `PathType.JSON_PATH` for [JSONPath](https://datatracker.ietf.org/doc/draft-ietf-jsonpath-base/) expressions. Doing so allows you to report the path for each finding and to potentially lookup nodes (see [here](https://github.com/networknt/json-schema-validator/blob/c41df270a71f8423c63cfaa379d2e9b3f570b73e/doc/yaml-line-numbers.md#scenario-2---validationmessage-line-locations) for an example).
Loading

0 comments on commit 4f3345b

Please sign in to comment.