Skip to content

Commit dad6e03

Browse files
rbassorbasso
authored
Merge pull request #602 from rbasso/canonical-schema
Add JSON Schema for 'canonical-data.json' and update the README.md
2 parents 52b9ffd + b750246 commit dad6e03

File tree

2 files changed

+207
-15
lines changed

2 files changed

+207
-15
lines changed

‎README.md

Lines changed: 70 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -36,21 +36,76 @@ There are three metadata files:
3636
## Test Data Format (canonical-data.json)
3737

3838
This data can be incorporated into test programs manually or extracted by a
39-
program. The file contains a single JSON object with a key for documentation
40-
and keys for various tests that may be meaningful for a problem.
41-
42-
The documentation uses the key "#" with a list of strings as the value.
43-
These strings document how the problem readme (`description.md`) is generally
44-
interpreted in test programs across different languages. In addition to a
45-
mainstream implementation path, this information can also document significant
46-
variations.
47-
48-
Each test case has the the following keys:
49-
- description: which will be used to name each generated test
50-
- The description should not simply explain **what** each case is (that is redundant information)
51-
- The description should explain **why** each case is there. For example, what kinds of implementation mistakes might this case help us find?
52-
- 'variable names': one or more variable names with values which will be passed to the solution method
53-
- expected: the expected result
39+
program. The file format is described in `canonical-schema.json`, but it
40+
is easier to understand with a example:
41+
42+
```json
43+
{ "exercise": "foobar"
44+
, "version" : "1.0.0"
45+
, "comments":
46+
[ " Comments are always optional and can be used almost anywhere. "
47+
, " "
48+
, " They usually document how the exercise's readme ('description.md') "
49+
, " is generally interpreted in test programs across different "
50+
, " languages. "
51+
, " "
52+
, " In addition to a mainstream implementation path, this information "
53+
, " can also document significant variations. "
54+
]
55+
, "cases":
56+
[ { "comments":
57+
[ " A test case must have a 'description' and a 'property'. "
58+
, " Anything else is optional. "
59+
, " "
60+
, " The 'property' is a string in lowerCamelCase identifying "
61+
, " the type of test, but most of the times it is just the "
62+
, " name of a function being tested. "
63+
, " "
64+
, " Test cases can have any number of additional keys, and "
65+
, " most of them also have an 'expected' one, defining the "
66+
, " value a test should return. "
67+
]
68+
, "description": "Foo'ing a word returns it reversed"
69+
, "property" : "foo"
70+
, "input" : "lion"
71+
, "expected" : "noil"
72+
}
73+
, { "description": "Bar'ing a name returns its parts combined"
74+
, "property" : "bar"
75+
, "firstName" : "Alan"
76+
, "lastName" : "Smithee"
77+
, "expected" : "ASlmainthee"
78+
}
79+
, { "comments":
80+
[ " Test cases can be arbitrarily grouped with a description "
81+
, " to make organization easier. "
82+
]
83+
, "description": "Abnormal inputs: numbers"
84+
, "cases":
85+
[ { "description": "Foo'ing a number returns nothing"
86+
, "property" : "foo"
87+
, "input" : "42"
88+
, "expected" : null
89+
}
90+
, { "description": "Bar'ing a name with numbers gives an error"
91+
, "property" : "bar"
92+
, "firstName" : "HAL"
93+
, "lastName" : "9000"
94+
, "expected" : { "error": "You should never bar a number" }
95+
}
96+
]
97+
}
98+
]
99+
}
100+
101+
```
102+
103+
Keep in mind that the description should not simply explain **what** each case
104+
is (that is redundant information) but also **why** each case is there. For
105+
example, what kinds of implementation mistakes might this case help us find?
106+
107+
There are also some convention about `expected` that you must follow:
108+
54109
- if the input is valid but there is no result for the input, the value at `"expected"` should be `null`.
55110
- if an error is expected (because the input is invalid, or any other reason), the value at `"expected"` should be an object containing exactly one property, `"error"`, whose value is a string.
56111
- The string should explain why the error would occur.

‎canonical-schema.json

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
{
2+
"comments":
3+
[ " This is a JSON Schema for 'canonical-data.json' files. "
4+
, " "
5+
, " It enforces just a general structure for all exercises, "
6+
, " without specifying how the test data should be organized "
7+
, " for each type of test. We do this to keep generality and "
8+
, " allow support for tests the do not fit well in the "
9+
, " 'function (input) == output' structure, like property "
10+
, " tests. "
11+
, " "
12+
, " The only thing enforced regarding how test data should be "
13+
, " structured is the error encoding, because it was agreed "
14+
, " and it doesn't restrict flexibility in a significant way. "
15+
, " "
16+
, " Standardized property names may help when automatically "
17+
, " deriving JSON parsers in some languages, so we followed "
18+
, " a few conventions from the 'Google JSON Style Guide'. "
19+
, " "
20+
, " Additionally, this schema strictly enforces letters, in "
21+
, " lowerCamelCase, for naming the 'property' being tested. We "
22+
, " expect this regularity will allow an easier automatic "
23+
, " generation of function's names in test generators, "
24+
, " slightly reducing the amount of manually generated code. "
25+
],
26+
27+
"$schema": "http://json-schema.org/draft-04/schema#",
28+
29+
"self": { "vendor" : "io.exercism"
30+
, "name" : "canonical-data"
31+
, "format" : "jsonschema"
32+
, "version": "1-0-0"
33+
},
34+
35+
"$ref": "#/definitions/canonicalData",
36+
37+
"definitions":{
38+
39+
"canonicalData":
40+
{ "description": "This is the top-level file structure"
41+
, "type" : "object"
42+
, "required" : ["exercise" , "version", "cases"]
43+
, "properties" :
44+
{ "exercise" : { "$ref": "#/definitions/exercise" }
45+
, "version" : { "$ref": "#/definitions/version" }
46+
, "comments" : { "$ref": "#/definitions/comments" }
47+
, "cases" : { "$ref": "#/definitions/testGroup" }
48+
}
49+
, "additionalProperties": false
50+
},
51+
52+
"exercise":
53+
{ "description": "Exercise's slug (kebab-case)"
54+
, "type" : "string"
55+
, "pattern" : "^[a-z]+(-[a-z]+)*$"
56+
},
57+
58+
"version" :
59+
{ "description" : "Semantic versioning: MAJOR.MINOR.PATCH"
60+
, "type" : "string"
61+
, "pattern" : "^(0|[1-9][0-9]*)(\\.(0|[1-9][0-9]*)){2}$"
62+
},
63+
64+
"comments":
65+
{ "description": "An array of strings to fake multi-line comments"
66+
, "type" : "array"
67+
, "items" : { "type": "string" }
68+
, "minItems" : 1
69+
},
70+
71+
"testGroup":
72+
{ "description": "An array of labeled test items"
73+
, "type" : "array"
74+
, "items" : { "$ref": "#/definitions/labeledTestItem" }
75+
, "minItems" : 1
76+
},
77+
78+
"labeledTestItem":
79+
{ "description": "A single test or group of tests with a description"
80+
, "oneOf": [ { "$ref": "#/definitions/labeledTest" }
81+
, { "$ref": "#/definitions/labeledTestGroup" }
82+
]
83+
},
84+
85+
"labeledTest":
86+
{ "description": "A single test with a description"
87+
, "type" : "object"
88+
, "required" : ["description", "property"]
89+
, "properties" :
90+
{ "description": { "$ref": "#/definitions/description" }
91+
, "comments" : { "$ref": "#/definitions/comments" }
92+
, "property" : { "$ref": "#/definitions/property" }
93+
, "expected" : { "$ref": "#/definitions/expected" }
94+
}
95+
},
96+
97+
"labeledTestGroup":
98+
{ "description": "A group of tests with a description"
99+
, "type" : "object"
100+
, "required" : ["description", "cases"]
101+
, "properties" :
102+
{ "description": { "$ref": "#/definitions/description" }
103+
, "comments" : { "$ref": "#/definitions/comments" }
104+
, "cases" : { "$ref": "#/definitions/testGroup" }
105+
}
106+
, "additionalProperties": false
107+
},
108+
109+
"description":
110+
{ "description": "A short, clear, one-line description"
111+
, "type" : "string"
112+
},
113+
114+
"property":
115+
{ "description": "A letters-only, lowerCamelCase property name"
116+
, "type" : "string"
117+
, "pattern" : "^[a-z]+([A-Z][a-z]+)*[A-Z]?$"
118+
},
119+
120+
"expected":
121+
{ "description": "The expected return value of a test case"
122+
, "properties":
123+
{ "error": { "$ref": "#/definitions/error" }
124+
}
125+
, "dependencies":
126+
{ "error": { "maxProperties": 1 }
127+
}
128+
},
129+
130+
"error":
131+
{ "description": "A message describing an error condition"
132+
, "type" : "string"
133+
}
134+
135+
}
136+
137+
}

0 commit comments

Comments
 (0)