Option to merge defaults

[jonasfj]

In JSON schema it's often useful for declare defaults, with the default value for an optional property. This is powerful for documentation, and many validators supports inserting the default into the validated object, this way you don't hardcode defaults into your code, but declare the defaults in your input schema.

Example A)

var Ajv = require('ajv');
var ajv = Ajv({mergeDefaults: true}); // Specify that we want defaults merged

var validate = ajv.compile({
  type: "object",
  properties: {
    myProp: { // optional property
      type: "string",
      default: "Hello World" // default value
    }
  }
});

var data = {
  someOtherProp: "another value"
};

validate(data); // true
console.log(data)
// {someOtherProp: "another value", myProp: "Hello World"}

This would be extremely useful, and doing this outside of the schema validation is hard, as you want to do this on all sub-objects as well, some of which may exist under an anyOf, so you can't insert the defaults until you're sure which anyOf branch is satisfied.

It's quite possible that this is a post processing step, as nested anyOf branches means you sometimes can't do this until after everything is validated. A possible work around might be to insert the defaults into a clone of the existing object, that way multiple anyOf branches shouldn't be a problem.

Note, this only really relevant for objects, but could also be done for arrays which has items: [{default: ...}, {default: ...}] (though this is a corner case). It doesn't really make sense to do this when the top-level schema type is a value type like integer as the input is either valid (ie. an integer) or invalid (ie. another type or null).

Remark, after inserting a default value, validator should validate the default value too. This should ideally be done a schema compile-time. It is important to do this, because the default value of a property which has type: 'object' may be {} and the schema for that object may specify properties for this that has additional default values. It would also be nice to get errors about inconsistencies at compile-time.

[epoberezkin]

I agree that it's a useful feature. The reason it is absent is the number of complex and/or contradictory scenarios that should be covered. I am sure that there will be other complications that we don't immediately see. The additional problem with anyOf, for example, is that multiple branches can be satisfied. allOf is also not obvious. Post processing is not going to work, because I think that the expectation is that if default is provided and the value is missing, object with inserted default should be valid, not without. Pre-processing is difficult because it is not clear which branches are going to be valid. Some validators implement complex scenarios with changes and roll-backs...

I will probably implement it at some point, but it's not a simple task. PR is welcome of course :)

[jonasfj]

One could probably forbid default under allOf branches... (I suspect allOf is rarely used too).
But to determine of anyOf can have multiple branches we would have to build static analysis tools for JSON schemas ... Wow, that got complex really fast :)

Or you define that you choose the first anyOf branch...

[epoberezkin]

The first valid anyOf branch I assume. So I guess apply defaults first (if it wasn't applied in the previous branch), validate a branch, rollback if it wasn't valid...

[jonasfj]

Roll back seems hard.. Perhaps it's simplest (and possibly cheapest) to create a new object and
return along with the result. That way you don't mutate the input object too...

[atrniv]

@jonasfj you might be interested in taking a look at the defaults implementation in Themis. One of the main reasons I wrote it was because of the lack of support for defaults in most existing validators. We've been using it in production quite successfully for quite a while.

@epoberezkin, It is not a simple solution to implement and it does have a significant performance impact but ultimately it provides a level of convenience which makes up for the shortcomings.

[epoberezkin]

@atrniv, thanks. I was actually thinking about implementation in themis.

[epoberezkin]

The implementation uses default keywords only in properties and in items (when it's an array). It also ignores them if they are anywhere inside anyOf, oneOf and not as not only the required code is complex to implement but also because the meaning of these defaults is very ambiguous. I think the way it is done it covers the majority of use cases. The possible improvements to compilation are:

• warn when default keyword is ignored
• detect situations where minItems/maxItems (etc.) is guaranteed to pass (to simplify the compiled code) or to fail (to throw the exception during compilation).
• detect situation when property dependency in dependencies keyword is guaranteed to pass.
• validate defaults against the schema for the item/property (to throw compilation exception when it fails)

[paglias]

@epoberezkin does default support passing a function that returns a value?

For example:

  properties: {
    id: {
      type: 'uuid',
      default: function () { return uuid.v4() }
    },
}

If it doesn't then it would be very useful

[epoberezkin]

JSON schema is a JSON-document. It should be serialisable. You can do it with custom keyword on the parent object though, e.g.:

{
  uuid: {
    property: 'id',
    version: 4,
    generate: true
  }
}

[paglias]

@epoberezkin thanks!

[epoberezkin]

actually, with "inline" keyword you can do a more sane thing on the property itself as there you will have access to the parent object and the current property name (check the implementation of default for that):

{
  "properties": {
    "id": {
      "uuid": {
        "version": 4,
        "generate": true
      }
    }
  }
}

[paglias]

@epoberezkin would you mind pointing me to the implementation of default? I'm having some problem finding it,

I came up with this but it's probably wrong:

schemaValidator.addKeyword('uuid', {
  type: 'uuid',
  statements: true,
  inline (it, keyword, schema) {
    return `data || ${generateUUID()}`;
  },
});

[epoberezkin]

file defaults.def

[epoberezkin]

@paglias this doc about inline keywords can be useful

[ngryman]

@paglias I have the exact same need to generate a uuid if not specified. Would you mind sharing your solution? Thanks!

[paglias]

@ngryman sorry but it's been a long time and I don't remember if I found the solution. I'm not using ajv now

[rondonjon]

Hello @epoberezkin,

I just ran into this problem ...

not only the required code is complex to implement but also because the meaning of these defaults is very ambiguous.

I don't quite understand.

I am dealing with a property that can either be false or an object. If it is an object, AJV should check its child properties and use a default in case that a child property is missing. This is currently not working.

Since AJV is treating both cases well during validation, the code that selects the correct schema subbranch must already be there and properly working. Shouldn't it then also be possible to load the default value from this particular schema subbranch and use it if no actual value is present?

From your comment, I conclude that defaults are currently implemented elsewhere, but then my naive assumption would be that the problem could be solved by moving the "default picking" code to run right after the branch selection?

[epoberezkin]

@rondonjon that is not how Ajv works, it has no global awareness about branches, it just validates data against all of them and defaults have to be applied before validation. There is no way Ajv can decide which is the right branch without validating against it. But then if it is not valid, defaults have to be removed. So there is no "branch selection".

The suggestion to work around is exactly that - to explicitly select the branch using some kind of conditionals: "if/then/else" (will be in draft 7) or "select" (both are in ajv-keywords). It both solves defaults problem and excessive error reporting problem (in case of using oneOf/anyOf when all branches are invalid and Ajv has no idea what is the "right" branch).

[rondonjon]

Thanks for the clarification, @epoberezkin !

[mnpenner]

Would it not make more sense to implement this in a similar fashion to $ref? e.g.

"key": {
    "type": "string",
    "title": "Key",
    "default": {"$uuid": "v4"}
}

[hosseinalipour]

I recently have a problem with defaults value remaining in data even though the validation is failed,
I have multiple levels to check a schema and default values interfere with this pattern. I'm not sure what might be the obstacles to implement this in AJV?