Extend format keyword to support all data types

[jeremykentbgross]

The readme currently indicates that custom format validation is only applied to string properties. However my understanding from reading the draft 4 specifications is that any property could have a format attribute, it just so happens that all the predefined formats only apply to strings.

I have not (yet) checked if your implementation does indeed only support custom formats for strings and not other types, but that is what is currently stated in the readme. As noted in another issue, the readme currently broken/out of date in some other places, so perhaps this information is incorrect, however assuming it is correct, ideally when defining custom format validation functions you should be allowed to specifying the type they apply to.

There are many use cases for applying formats to other property types, some examples may be found even in software linked/recommend by the main json schema website.

I have several possible use cases for custom formats on other types in upcoming projects. For example on objects, the custom format matcher "isA:\w_" could trigger a custom format validation function to verify that the object represents data serialized from a class derived from the class matched by "\w_". While in theory this could be achieved with features such as "oneOf", that would assume the schema must know all possible descendant classes at the time it is authored, or be dynamically updated and recompiled whenever a new class factory is registered. This method on the other hand allows traversing inheritance tree information that may not be available to the schema, and even custom recursive validation by another schema found in said tree.

Many more examples abound, but the important thing is that the draft 4 specification implies custom formats can be applied to any type, unless i misunderstand their meaning. Ideally ajv stays in line with the standard on this point, and/or updates it's documentation to indicate it's existing compliance.

[jeremykentbgross]

The \w's was followed by a star, representing regex for matching a word, but it's not visible when posted.

[epoberezkin]

The draft 4 spec describes format keyword as optional, it only says that if it is implemented, some predefined attributes should be implemented too - see 7.2. It does say that it "generally" applies to a limited set of data types, so applying it only to strings is not against the standard, although one may imply that other types should be possible too:

A format attribute can generally only validate a given set of instance types. If the type of the instance to validate is not in this set, validation for this format attribute and instance SHOULD succeed.

At the same time it says:

The "format" keyword is defined to allow interoperable semantic validation for a fixed subset of values which are accurately described by authoritative resources, be they RFCs or other external specifications.

What would be the example of such external spec for non-strings? Also to me it seems semantically wrong using it for anything but strings, so Ajv indeed only supports formats for strings.

The validation scenario you describe is not really a "format" (neither in the way the spec defines it nor in plain language meaning: shape/size/pattern/plan/arrangement/presentation), it is some other data requirement.

Ajv allows to implement any requirements, including those that require asynchronous calls, using custom keywords - I think a custom keyword would better fit your use case (and you are not constrained with a string as a value, where you need to cram several things in one string as you are trying to do). See for example instanceof keyword - it is somewhat similar to what you are describing.

I will think again whether it's worth extending format to support other data types in addition to strings.

[nimerritt]

Could you please reconsider this. We use JSON-Schema for model validation and Swagger for documenting our APIs. Since Swagger extends the interger and numeric with a custom format we would like to have it validated when we use ajv. http://swagger.io/specification/#dataTypeFormat

Google also does this: https://developers.google.com/discovery/v1/type-format

I suppose this could be accomplished by overriding the types with a custom keyword but it seems more appropriate to have the format applied to a whitelisted subset of types.

[epoberezkin]

@nimerritt this seems like a more appropriate use case for format keyword than structural validation. So the proposal is to allow defining custom formats that would apply to some chosen type, not necessarily a string, correct? I'd still limit it to numbers and integers in this case, although it can be a usage recommendation rather than a limitation, it doesn't make much difference whether to allow all or some types.

[nimerritt]

Thanks for the quick response! :)

"So the proposal is to allow defining custom formats that would apply to some chosen type, not necessarily a string, correct?" -> Right on!

I would like to be able to specify that it is only applicable to the integer type, otherwise I will have to duplicate the type checking in the validator. Something along the lines of:

const int32_min = -1 * Math.pow(2, 31);
const int32_max = Math.pow(2, 31) - 1;

function validateInt32(n) {
  return n >= int32_min && n <= int32_max;
}

ajv.addFormat('int32', validateInt32, ['integer']);

would be great!

[epoberezkin]

Ok, this can be done. PR is welcome. API should be consistent with the existing format/custom keywords API (format definition can already be an object to support formatMinimum etc. and also async formats, so "type" will simply be a property in this definition defaulting to "string").

I don't want to support formats that apply to multiple types to be honest, although you can convince me otherwise if you know some valid use case.

[epoberezkin]

This should be added to 5.0.0 branch btw

[nimerritt]

I'll try to do a PR over the weekend :)

[pyramation]

Hi! I'm trying to shed light on this issue which was closed with 7361f83

I think formats for any object is truly important, and seems that it was resolved, however since I've checked out the 5.0.0 branch, I cannot seem to get formats to validate for all data types, and looks as if the tests only covered type number.

Before finding this issue to make current problem issue/idea clear (looks as though others also stumbled into this and there's been some decisions made), I've created a failing test with a hypotheical example, say, if an object is an immutable List class of some sort: https://github.com/pyramation/ajv/commit/cc19a8be22c4104e763c8734b7866952e90fb877

I think for simplicity sake, seems that every type having a format makes sense, does it not? String formats work well with RegExp and every other type just a naive validation function validate = f(x) ?

Other json schema implementations allow formats for all data types, so I'm hoping this is something ajv would also support. I'm also willing to help if I can get some pointers on editing the format code generation.

[epoberezkin]

@pyramation The reasons NOT to support formats for types other than string/number are above in comments.

Please share a specific use case where you think a format is a better fit (semantically) than a custom keyword.

[epoberezkin]

A better fit for your case where you validate that an object is an instance of List class is the custom keyword instanceof defined in ajv-keywords. Semantically, this validation has nothing to do with format.

If you use duck-typing and your object is not an instance of any class you can either change it and use class instead or define your own custom keyword e.g., x-list.

[pyramation]

yep, I believe instanceof and $ref can work. A case for format is that it is a nice shorthand way to keep your duck-typing scoped to only have to look at the type and format properties. But I think for now, my use case is solved :)