Implement Persistence Proposal

[ashleywxwx]

User story

As an API developer, I want in-memory persistence that can fetch an entity from another operation, so that I can validate scenarios that have dependent requests.

As an API developer, I want to define that different endpoints that all refer to the same collection of entities, so I can tie together different operations aren't a part of the same OAS path, such as a POST /widgets/widget and GET /widgets.

As an API developer, I want to specify as part of my OAS spec which response body property is associated with a collection's requestParam, so that I can later query for a specific persisted entity, such as GET /widget/{my-newly-persisted-id}

As an API developer, I want Prism to provide additional properties not a part of our initial POST request, so that I when I GET, derived properties like id or createdBy are returned.

As an API developer, I want to provide a file at runtime that defines initial data beyond OAS examples, so that I can consistently start with the same set of data without having to make POST requests before my tests run, while keeping this data out of the OAS spec.

Is your feature request related to a problem?

We've introduced Prism to some of the engineering teams we're working with. While they have previously worked against a QA environment, they are now looking to migrate some of their tests to use Prism instead. While ultimately we suggest leveraging Prism for non-functional/contract testing, we have a few scenarios where engineers still need data from a previous call in order to represent particular parts of their workflow. Having in-memory persistence could help bridge the gap without having to rely on more complex mocking behavior.

Describe the solution you'd like

Just some ideas, open to others:

In-memory - Having runtime persistence is sufficient for our needs, no need for durable data

Associating Operations to Collections - Similar to the custom x-faker property, provide a custom property such as x-mock-collection: widgets to the operation object, so that Prism can understand which operations refer to the same collection of data.

Associating Properties to Path Parameters - Similar to above, provide a custom property such as x-mock-identifier: userId to the parameters definition, and use this to know which property to use when handling a request with a path parameter used to fetch an object by it's id.

Derived Properties - Merge the persisted entity with a "dynamic" (generated) entity based on the schema definition/examples at response time, so that fields not present in the request can be returned in the response.

Data Loading - As a runtime flag, allow users to supply a file that will be loaded on startup and associated with named entity collections. We're open to any file type that's reasonable, JSON would probably be easiest though.

We hadn't considered the implications of distinguishing between a "persistence" call and a standard mocked call, but for our use case, it'll typically be an all-or nothing situation, so something like a runtime flag that enable persistence would be sufficient.

Additional context

We've chatted a bit about custom mocking as well, but I believe persistence would handle most of our use-cases without a need for custom mocking. Out of the above suggestions, data loading is a "nice to have" as we can find alternatives if not available.

[philsturgeon]

Great minds think alike! We've got some plans which look a lot like this, but we've got to fit them in with all the other evil plans we're working on. As always if anyone wants to take a swing at this one, please do, OSS loves contributions!

Otherwise, if folks would like to vote for this and help it get prioritized quicker, please vote on our roadmap: https://roadmap.stoplight.io/c/50-persisted-mock-data

[taras]

I just wanted to let you know that my team has been exploring this area quite a bit. We started working on a control API for simulators called Simulacrum. We currently have 3 simulators implemented - Auth0, LDAP, and GitHub GraphQL API. Auth0 & LDAP are open-source in https://github.com/thefrontside/simulacrum. We'll open source GitHub GraphQL API when it's ready enough to use.

I would like to create a plugin for Simulacrum that would make Prism one of the simulators that Simulacrum Control API can orchestrate. One of the steps on this path is creating a way to create a unified data graph that can be used to power multiple simulators. We recently created initial implementation and we wrote about it on our blog - https://frontside.com/blog/2022-02-21-graphgen-graph-factory/

There is a lot of work to do, but we're already using it to build big graphs that power multiple simulators at the same time. This could be used to generate data for Prism simulator. I'm happy to talk about this anytime.