v3.2.1: Unexpected token':'

[schw4rzlicht]

Hey there!

I just upgraded from 3.1.0 to 3.2.1 but verified this problem with 3.2.0 as well. I run

ajv validate -s resources/config.schema.json -d config.json.sample

with perfectly valid files but the tool just aborts with

error:  Unexpected token ':'

Like I said, 3.1.0 works fine.

[epoberezkin]

It must be related to supporting additional file types (cc @G-Rath). @schw4rzlicht - could you post the links to the files that cause the error?

[epoberezkin]

The issue would probably go away if you use config.sample.json file name - it is better anyway, so that the extension reflects the file type.

[schw4rzlicht]

could you post the links to the files that cause the error?

Sure. Just check out the current master of https://github.com/schw4rzlicht/twitch2ma (2ee8af36e31626264784365f62af30a130f65654), update ajv-cli manually (i locked it to 3.1.0) and run npm run validateConfig.

But:

The issue would probably go away if you use config.sample.json file name - it is better anyway, so that the extension reflects the file type.

That indeed fixed it. Thank you :)

[G-Rath]

This is because the file extension is technically .sample. which causes us to fallback to using require as it's not a supported file type.

Node (/require) then does a similar thing: it looks at the file extension, sees it's not .json and so assumes it's JS, and that file is not valid JS since you can't have "floating" objects or arrays in JS.

We can probably fix this by trying JSON.parse in the event of an unsupported file extension before falling back to require; that would preserve backwards compatibility but would only work for JSON file types (i.e not YAML).

I think a good long-term solution could be to add a --file-type flag that lets you explicitly provide the type for what's being provided for -d.

I'd argue it makes sense to not bother supporting this for the schema -s flag since it makes far less sense to have "sample" & "example" schema files that you actually use to valid something against (vs the high complexity of having to implement supporting --file-type based on relative position to -d & -s flags and ugh just describing it is already too messy 😂)

[epoberezkin]

Thanks. I agree that an additional option would be messy.

We can probably fix this by trying JSON.parse in the event of an unsupported file extension before falling back to require; that would preserve backwards compatibility but would only work for JSON file types (i.e not YAML).

What did it do before? try/catch on require with fallback to JSON.parse?

[G-Rath]

What did it do before?

As you said, but the other way around - because it only supported json, it just always try to parse as json and then fallback to require.

Now we check the extension to try and figure out what parser to use.

[epoberezkin]

I see. So what’s the plan for unknown extensions? Options are:

• keep as is (require)
• require -> JSON.parse
• JSON.parse -> require

Probably the last option is the most backwards compatible... On another hand, not too many complaints, so maybe best to keep as is not to complicate things.

[G-Rath]

I was originally going to say "let's wait and see", but tbh there's no real downside to just wrapping it all in a try/catch, doing JSON(5) -> YAML -> require.

We always install all the parsers, if the extension is known then we'll shortcut to the right parser, and it's only ~10 extra lines of code to implement. That would then also let you support cjson & such in .json since the JSON5 parser does support "normal" JSON too.

I'll make a draft PR to confirm the implementation complexity, and we can go from there.

[dcrossleyau]

We have some legacy JSON Schema files that use the filename suffix .schema so it would be useful for us if it operated with the -s and -r options too.

[epoberezkin]

What’s the file format?

[dcrossleyau]

Sorry for not being clear. They are JSON Schemas "draft-04", e.g. https://github.com/folio-org/raml/blob/raml1.0/schemas/uuid.schema

[epoberezkin]

So maybe you could work around it with .json symlinks? That is, if there is no way to rename the files.

[epoberezkin]

Probably JSON needs to be the default expected format anyway rather than JS...