Skip to content

Commit 9ff926f

Browse files
JulianJulian
committed
Merge branch 'additional-evaluated'
* additional-evaluated: Revert "Adapt test case generation from suite" Squashed 'json/' changes from ab3924a6..9ad349be Move is_valid function to module scope Revert "Add test for unevaluatedProperties" Adapt test case generation from suite Squashed 'json/' changes from 83e866b4..ab3924a6 Fix calculation of evaluated properties Add test for unevaluatedProperties Add helper function
2 parents 8290667 + 36623dd commit 9ff926f

17 files changed

+1735
-55
lines changed

‎json/.github/workflows/annotation-tests.yml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
name: Validate annotation tests
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- "annotations/**"
7+
8+
jobs:
9+
annotate:
10+
runs-on: ubuntu-latest
11+
12+
steps:
13+
- uses: actions/checkout@v4
14+
15+
- name: Set up Deno
16+
uses: denoland/setup-deno@v2
17+
with:
18+
deno-version: "2.x"
19+
20+
- name: Validate annotation tests
21+
run: deno --node-modules-dir=auto --allow-read --no-prompt bin/annotation-tests.ts

‎json/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -227,6 +227,7 @@ This suite is being used by:
227227

228228
### C++
229229

230+
* [Blaze](https://github.com/sourcemeta/blaze)
230231
* [Modern C++ JSON schema validator](https://github.com/pboettch/json-schema-validator)
231232
* [Valijson](https://github.com/tristanpenman/valijson)
232233

‎json/annotations/README.md

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# Annotations Tests Suite
2+
3+
The Annotations Test Suite tests which annotations should appear (or not appear)
4+
on which values of an instance. These tests are agnostic of any output format.
5+
6+
## Supported Dialects
7+
8+
Although the annotation terminology of didn't appear in the spec until 2019-09,
9+
the concept is compatible with every version of JSON Schema. Test Cases in this
10+
Test Suite are designed to be compatible with as many releases of JSON Schema as
11+
possible. They do not include `$schema` or `$id`/`id` keywords so
12+
implementations can run the same Test Suite for each dialect they support.
13+
14+
Since this Test Suite can be used for a variety of dialects, there are a couple
15+
of options that can be used by Test Runners to filter out Test Cases that don't
16+
apply to the dialect under test.
17+
18+
## Test Case Components
19+
20+
### description
21+
22+
A short description of what behavior the Test Case is covering.
23+
24+
### compatibility
25+
26+
The `compatibility` option allows you to set which dialects the Test Case is
27+
compatible with. Test Runners can use this value to filter out Test Cases that
28+
don't apply the to dialect currently under test. The terminology for annotations
29+
didn't appear in the spec until 2019-09, but the concept is compatible with
30+
older releases as well. When setting `compatibility`, test authors should take
31+
into account dialects before 2019-09 for implementations that chose to support
32+
annotations for older dialects.
33+
34+
Dialects are indicated by the number corresponding to their release. Date-based
35+
releases use just the year. If this option isn't present, it means the Test Case
36+
is compatible with any dialect.
37+
38+
If this option is present with a number, the number indicates the minimum
39+
release the Test Case is compatible with. This example indicates that the Test
40+
Case is compatible with draft-07 and up.
41+
42+
**Example**: `"compatibility": "7"`
43+
44+
You can use a `<=` operator to indicate that the Test Case is compatible with
45+
releases less then or equal to the given release. This example indicates that
46+
the Test Case is compatible with 2019-09 and under.
47+
48+
**Example**: `"compatibility": "<=2019"`
49+
50+
You can use comma-separated values to indicate multiple constraints if needed.
51+
This example indicates that the Test Case is compatible with releases between
52+
draft-06 and 2019-09.
53+
54+
**Example**: `"compatibility": "6,<=2019"`
55+
56+
For convenience, you can use the `=` operator to indicate a Test Case is only
57+
compatible with a single release. This example indicates that the Test Case is
58+
compatible only with 2020-12.
59+
60+
**Example**: `"compatibility": "=2020"`
61+
62+
### schema
63+
64+
The schema that will serve as the subject for the tests. Whenever possible, this
65+
schema shouldn't include `$schema` or `id`/`$id` because Test Cases should be
66+
designed to work with as many releases as possible.
67+
68+
### externalSchemas
69+
70+
This allows you to define additional schemas that `schema` makes references to.
71+
The value is an object where the keys are retrieval URIs and values are schemas.
72+
Most external schemas aren't self identifying (using `id`/`$id`) and rely on the
73+
retrieval URI for identification. This is done to increase the number of
74+
dialects that the test is compatible with. Because `id` changed to `$id` in
75+
draft-06, if you use `$id`, the test becomes incompatible with draft-03/4 and in
76+
most cases, that's not necessary.
77+
78+
### tests
79+
80+
A collection of Tests to run to verify the Test Case.
81+
82+
## Test Components
83+
84+
### instance
85+
86+
The JSON instance to be annotated.
87+
88+
### assertions
89+
90+
A collection of assertions that must be true for the test to pass.
91+
92+
## Assertions Components
93+
94+
### location
95+
96+
The instance location.
97+
98+
### keyword
99+
100+
The annotating keyword.
101+
102+
### expected
103+
104+
A collection of `keyword` annotations expected on the instance at `location`.
105+
`expected` is an object where the keys are schema locations and the values are
106+
the annotation that schema location contributed for the given `keyword`.
107+
108+
There can be more than one expected annotation because multiple schema locations
109+
could contribute annotations for a single keyword.
110+
111+
An empty object is an assertion that the annotation must not appear at the
112+
`location` for the `keyword`.
113+
114+
As a convention for this Test Suite, the `expected` array should be sorted such
115+
that the most recently encountered value for an annotation given top-down
116+
evaluation of the schema comes before previously encountered values.

‎json/annotations/assertion.schema.json

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
4+
"type": "object",
5+
"properties": {
6+
"location": {
7+
"markdownDescription": "The instance location.",
8+
"type": "string",
9+
"format": "json-pointer"
10+
},
11+
"keyword": {
12+
"markdownDescription": "The annotation keyword.",
13+
"type": "string"
14+
},
15+
"expected": {
16+
"markdownDescription": "An object of schemaLocation/annotations pairs for `keyword` annotations expected on the instance at `location`.",
17+
"type": "object",
18+
"propertyNames": {
19+
"format": "uri"
20+
}
21+
}
22+
},
23+
"required": ["location", "keyword", "expected"]
24+
}

‎json/annotations/test-case.schema.json

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
4+
"type": "object",
5+
"properties": {
6+
"description": {
7+
"markdownDescription": "A short description of what behavior the Test Case is covering.",
8+
"type": "string"
9+
},
10+
"compatibility": {
11+
"markdownDescription": "Set which dialects the Test Case is compatible with. Examples:\n- `\"7\"` -- draft-07 and above\n- `\"<=2019\"` -- 2019-09 and previous\n- `\"6,<=2019\"` -- Between draft-06 and 2019-09\n- `\"=2020\"` -- 2020-12 only",
12+
"type": "string",
13+
"pattern": "^(<=|=)?([123467]|2019|2020)(,(<=|=)?([123467]|2019|2020))*$"
14+
},
15+
"schema": {
16+
"markdownDescription": "This schema shouldn't include `$schema` or `id`/`$id` unless necesary for the test because Test Cases should be designed to work with as many releases as possible.",
17+
"type": ["boolean", "object"]
18+
},
19+
"externalSchemas": {
20+
"markdownDescription": "The keys are retrieval URIs and values are schemas.",
21+
"type": "object",
22+
"patternProperties": {
23+
"": {
24+
"type": ["boolean", "object"]
25+
}
26+
},
27+
"propertyNames": {
28+
"format": "uri"
29+
}
30+
},
31+
"tests": {
32+
"markdownDescription": "A collection of Tests to run to verify the Test Case.",
33+
"type": "array",
34+
"items": { "$ref": "./test.schema.json" }
35+
}
36+
},
37+
"required": ["description", "schema", "tests"]
38+
}

‎json/annotations/test-suite.schema.json

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
4+
"type": "object",
5+
"properties": {
6+
"description": {
7+
"type": "string"
8+
},
9+
"suite": {
10+
"type": "array",
11+
"items": { "$ref": "./test-case.schema.json" }
12+
}
13+
},
14+
"required": ["description", "suite"]
15+
}

‎json/annotations/test.schema.json

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"$schema": "http://json-schema.org/draft-07/schema#",
3+
4+
"type": "object",
5+
"properties": {
6+
"instance": {
7+
"markdownDescription": "The JSON instance to be annotated."
8+
},
9+
"assertions": {
10+
"markdownDescription": "A collection of assertions that must be true for the test to pass.",
11+
"type": "array",
12+
"items": { "$ref": "./assertion.schema.json" }
13+
}
14+
},
15+
"required": ["instance", "assertions"]
16+
}

0 commit comments

Comments
 (0)