Optimise Ajv cache usage

[0bex0]

Introducing a key which can be used to for caching in by Ajv library.
Currently the whole schema is used as a cache key in each ajv instance¹,
for very large schemas this leads to a lot of unnecessary memory usage since
a whole string representation of the schema is stored with each compile().

The ajv cache is never used, since there's already a cache implementation
in the code base, but by passing a much shorter key to ajv we optimise
memory usage.

Footnotes

1. https://ajv.js.org/guide/managing-schemas.html#cache-key-schema-vs-key-vs-id ↩

[cdimascio]

Thanks. Appreciate this update! Looks good 👍

[martinslota]

I'm noticing in our test suite that express-openapi-validator consumes more memory from version 5.5.0 onwards.

I have a feeling that this change might (ironically) be the cause of that: It turns out that despite the recommendation in ajv's docs, the addSchema() method stores the schema in this.schemas under the supplied key, but before it does that, it calls the _addSchema() method which sticks the schema in this._cache as well. So, at the end of the day, the schema gets cached twice when using addSchema() instead of compile().

[martinslota]

I'm noticing in our test suite that express-openapi-validator consumes more memory from version 5.5.0 onwards.

I have a feeling that this change might (ironically) be the cause of that: It turns out that despite the recommendation in ajv's docs, the addSchema() method stores the schema in this.schemas under the supplied key, but before it does that, it calls the _addSchema() method which sticks the schema in this._cache as well. So, at the end of the day, the schema gets cached twice when using addSchema() instead of compile().

A small follow-up on this: I have now taken version 5.5.0 and reverted the changes in this commit using patch-package and ran our tests again. The extra memory pressure disappeared.

[0bex0]

I'm noticing in our test suite that express-openapi-validator consumes more memory from version 5.5.0 onwards.

I have a feeling that this change might (ironically) be the cause of that: It turns out that despite the recommendation in ajv's docs, the addSchema() method stores the schema in this.schemas under the supplied key, but before it does that, it calls the _addSchema() method which sticks the schema in this._cache as well. So, at the end of the day, the schema gets cached twice when using addSchema() instead of compile().

I believe you're right, and _addSchema continues to cache the key using the schema object itself meaning the memory issue w.r.t the cache itself is not addressed either. I will create a PR to revert this change both the redundant caching and object cache key need to be addressed in the ajv library itself.