A simple way to write advanced API Standards

[acunniffe]

The more a team's API standards and best practices can be checked with automation, the easier it is for developers to design APIs that meet the standards. This lets API reviewers focus their feedback on the actual product concerns, not the basic design or use of OpenAPI. After helping many teams like Snyk build more advanced API standards than they could with Spectral, we've learned a lot about how to improve the process of automating your API standards.

Here's a screenshot of some API Standards written with Optic -- and a live preview of the documentation the engine generates for you!

Design principles

• Rule authors are thinking in the API domain meet them there -- they don't want to think about JSON, the structure of an OpenAPI document or write JSONPath (which Spectral requires). Every time they do this it is a context change.
• Most API Standards are related heirarchilly. ie there are standards which apply for batch update operations, standards which apply to operations that return search results, etc.
• "Change" is first class -- the API standards with the most value to teams concern breaking changes and enforcing their versioning policies.
• Easy to read / easy to write -- API standards are often written once and updated a few times a year. The code has to look like what it does
• Self-documenting -- writing the code should explain what the rule does

Meeting Rule Authors in the API Domain

Optic lets you write rules about the entities in your OpenAPI specification Operation, Parameter, Property, Response...etc

Imagine we want to write a rule that says all Operations MUST have an operationId.

First we call Standard.Operation:

const allOperations = Standard.Operation({

})

If you're using a Typescript IDE, you'll notice argument you pass into Standard.Operation has the same keys as an Operation in OpenAPI. This makes it easy for developers who work with OpenAPI to understand where each rule runs:

const allOperations = Standard.Operation({
   summary: ...
   operationId: ...
   parameters: []
   ....
})

To write our rule about operationId we just add a requirement in our API standard. We did not even have to think about JSON Path or write $.paths[*][get,put,post,delete,options,head,patch,trace].operationId

Standard.Operation({
  operationId: requirement(
      "must have an operationId of format methodResource",
      (operationId, context) => {
        if (!operationId) throw new Error("does not have an operation id");
      }
  )
})

Imagine our team generates client SDKs and changing the operationId is a breaking change? We can write a changed rule right there, next to our requirement rule:

Standard.Operation({
  operationId: [
      requirement("must have an operationId of format methodResource",
        (operationId, context) => {
          if (!operationId) throw new Error("does not have an operation id");
      }),
      changed("operationId must remain consistent", (before, after) => {
        if (!before) return; // this is the case where it was not set, and now it is set. don't fail here
        if (before && after && before !== after) {
          throw new Error(
            `operationId was changed from '${before}' to '${after}'. This is not allowed`
          );
        }
    }
  ] 
})

Optic even self-documents the standard this rule tests for (alongside some of our other standards):

Advanced API Standards

Before Optic, most of the large teams we worked with struggled to automate their most important API standards. The Spectral rules they wrote could help them write valid OpenAPI and require certain metadata, enforce consistent naming, and a few other basic standards. They struggled to write rules for things like Pagination, long-running requests, and their versioning policies.

1. Advanced rules need to apply hierarchically

Let's say we want to describe our team's standard for Get Operations that lookup a resource by its ID. That standard is made up of many assertions, which are tested by multiple rules. We do not want these rules to run on our entire OpenAPI spec -- they should only run on subtrees that match the standard we are testing for. That's what applyWhen is for:

Standard.Operation({
  parameters: [...]
  responses: {...}
})
.applyWhen(
    "Get Resource by ID Operations",
    (op, context) =>
      context.method === "get" && apiPaths.matches(context.pathPattern, getByIdMatcher)
  )

Every parameter and response standard we write will run on operations that match the predicate we wrote. You can use applyWhen with all the entities we write standards for. For example one team wrote a standard that helped developers use x-extensible-enum extension properly. It runs on every schema (and property schema) in the specification, that uses the extension:

export default Standard.Schema({
  type: requirement("must use string type", (type) => {
    if (type !== "string") {
      throw new Error("expect string. actual " + type);
    }
  }),
  nullable: requirement("can never be null", (nullable) => {
    if (nullable) throw new Error("enums can not be null");
  }),
  "x-extensible-enum": [
    requirement("is an array of PascalCase strings", (value) => {
      if (
        Array.isArray(value) &&
        value.every((i) => typeof i === "string" && pascalCaseRegex.test(i))
      ) {
        // valid
      } else {
        throw new Error("x-extensible-enum must be an array of values");
      }
    }),
    changed("removing an option is a breaking change", (before, after) => {
      (before as Array<string>).forEach((item) => {
        if (!(after as Array<string>).includes(item)) {
          throw new Error(
            `${item} was removed from x-extensible-enum. This is a breaking change`
          );
        }
      });
    })
  ]
}).applyWhen("strings with `x-extensible-enum`", (schema) => {
  return Boolean(schema.hasOwnProperty("x-extensible-enum"));
});

Pretty powerful stuff.

2. The human-side

We've heard a common story from many of the team's who adopted Optic this year. At one time they were automating more advanced API standards, but rolled them back because they were too disruptive. Existing OpenAPI operations were failing the new checks, but there was no way to "fix" them because those "fixes" would be breaking changes. That's fundamentally why "linting" is the wrong way to think about OpenAPI validation -- you can fix lint errors in code and get to 0...with APIs it's often impossible to get to 0 without breaking the user.

Most teams have existing APIs, and because we live in the real world their API standards are going to change over time as their team learns-- that's a good thing. Optic makes it possible to apply your API standards "forward-only" -- that is to the new surface area, while letting the existing surface area be.

.applyWhen("to new operations", (schema, context) => {
  return context.lifecycle === 'added'
});

This has made it safe to write more advanced standards without worrying about upsetting teams who have existing APIs to maintain. In fact we've measured the before and after, and teams using forwards-only rules are automating 8x as many API standards as they had before.

---

Developer preview

Right now the new rule authoring is in a developer-preview. We're working out a design challenges and awaiting some feedback before we launch it. Please share your feedback, requirements, and feature requests

Check out a live example on CodeSandbox

Share feedback in this thread 👇

Book office hours if you have ideas to share