Keto check latency always over 200ms

[Luscha]

I already used Ory for AuthN in a project with a custom solution for AuthZ (gRBAC) and I wanted to opt for Ory Network stack for a new project.

I already check the current status of Keto and determined that its OPL suits our requirements.
What really concerns me is the latency Ory Network's Keto latency to perform a check.
In both a Developer-tier project and a Essentials-tier project i couldn't find a way to establish a connection with keto and get a response in less than 250-300 ms.

I'm located in italy, the poject is setup for Europe.
I tried with Ory CLI, golang ory.PermissionApi and keto v1alpha2 CheckServiceClient grpc client.

I also tried to enable edge cache in the essential-tier project.

I spent quite a lot of time reading documentation and blog posts to figure out if it was a misconfiguration on my side, or if I'm using keto in an improper way, but couldn't wrap my head around this unbelievably high latency to receive a response to the only question keto is designed to answer.

My plan was to use Ory network as AuthN AuthZ provider and setup an IAP on aws to authorize any request to my microservices following our authorization policies, but I really can't see how could a service with 200+ms latency be used to authorize every single incoming request.

Please help me because I tried 4 different providers and none could fit our requirements.

Repoducible environment

Namespace & rules:

import { Namespace, SubjectSet, Context } from "@ory/permission-namespace-types"

/*
Define your OPL rules here. Some examples:
*/
class User implements Namespace {
}

class Group implements Namespace {
  related: {
    members: (User | Group)[]
  }
}

class Shop implements Namespace {
  related: {
    owners: User[]
    collaborators: SubjectSet<Group, "members">[]
    viewers: SubjectSet<Group, "members">[]
    admins: SubjectSet<Group, "members">[]
    moderators: SubjectSet<Group, "members">[]
  }

  permits = {
    view: (ctx: Context): boolean =>
      this.related.owners.includes(ctx.subject) ||
      this.related.admins.includes(ctx.subject) ||
      this.related.collaborators.includes(ctx.subject) ||
      this.related.viewers.includes(ctx.subject) ||
      this.related.moderators.includes(ctx.subject),

    edit: (ctx: Context) =>
      this.related.owners.includes(ctx.subject) ||
      this.related.admins.includes(ctx.subject) ||
      this.related.collaborators.includes(ctx.subject)
  }
}

class Feature implements Namespace {
  related: {
    shops: Shop[]
    collaborators: SubjectSet<Group, "members">[]
    admins: SubjectSet<Group, "members">[]
  }

  permits = {
    allowed: (ctx: Context): boolean =>
      this.related.shops.includes(ctx.subject) ||
      (this.related.shops.traverse(s => s.permits.edit(ctx)) ||
        this.related.collaborators.includes(ctx.subject) ||
        this.related.admins.includes(ctx.subject)
      )
  }
}

class Resource implements Namespace {
  related: {
    shops: Shop[]
    collaborators: SubjectSet<Group, "members">[]
    admins: SubjectSet<Group, "members">[]
  }

  permits = {
    view: (ctx: Context): boolean =>
      this.related.collaborators.includes(ctx.subject) ||
      this.related.shops.traverse(s => s.permits.view(ctx)),

    edit: (ctx: Context) =>
      this.related.collaborators.includes(ctx.subject) ||
      this.related.admins.includes(ctx.subject) ||
      this.related.shops.traverse(s => s.permits.edit(ctx)),
  }
}

and set some test tuples:

[
    {
        "namespace": "Group",
        "object": "admin",
        "relation": "members",
        "subject_set": {
            "namespace": "User",
            "object": "matteo"
        }
    },
    {
        "namespace": "Group",
        "object": "moderator",
        "relation": "members",
        "subject_set": {
            "namespace": "User",
            "object": "alessandro"
        }
    },
    {
        "namespace": "Group",
        "object": "accountants-123",
        "relation": "members",
        "subject_set": {
            "namespace": "User",
            "object": "mario"
        }
    },
    {
        "namespace": "Group",
        "object": "accountants-456",
        "relation": "members",
        "subject_set": {
            "namespace": "User",
            "object": "rossi"
        }
    },
    {
        "namespace": "Shop",
        "object": "shops-123",
        "relation": "owners",
        "subject_set": {
            "namespace": "User",
            "object": "pacini"
        }
    },
    {
        "namespace": "Shop",
        "object": "shops-456",
        "relation": "owners",
        "subject_set": {
            "namespace": "User",
            "object": "ovolollo"
        }
    },
    {
        "namespace": "Shop",
        "object": "shops-123",
        "relation": "admins",
        "subject_set": {
            "namespace": "Group",
            "object": "admin",
            "relation": "members"
        }
    },
    {
        "namespace": "Shop",
        "object": "shops-456",
        "relation": "admins",
        "subject_set": {
            "namespace": "Group",
            "object": "admin",
            "relation": "members"
        }
    },
    {
        "namespace": "Shop",
        "object": "shops-123",
        "relation": "moderators",
        "subject_set": {
            "namespace": "Group",
            "object": "moderator",
            "relation": "members"
        }
    },
    {
        "namespace": "Shop",
        "object": "shops-456",
        "relation": "moderators",
        "subject_set": {
            "namespace": "Group",
            "object": "moderator",
            "relation": "members"
        }
    },
    {
        "namespace": "Resource",
        "object": "inbounds-123",
        "relation": "shops",
        "subject_set": {
            "namespace": "Shop",
            "object": "shops-123"
        }
    },
    {
        "namespace": "Resource",
        "object": "inbounds-456",
        "relation": "shops",
        "subject_set": {
            "namespace": "Shop",
            "object": "shops-456"
        }
    },
    {
        "namespace": "Resource",
        "object": "purchase-orders-123",
        "relation": "shops",
        "subject_set": {
            "namespace": "Shop",
            "object": "shops-123"
        }
    },
    {
        "namespace": "Resource",
        "object": "purchase-orders-456",
        "relation": "shops",
        "subject_set": {
            "namespace": "Shop",
            "object": "shops-456"
        }
    },
    {
        "namespace": "Shop",
        "object": "shops-123",
        "relation": "collaborators",
        "subject_set": {
            "namespace": "Group",
            "object": "accountants-123",
            "relation": "members"
        }
    },
    {
        "namespace": "Shop",
        "object": "shops-123",
        "relation": "viewers",
        "subject_set": {
            "namespace": "Group",
            "object": "accountants-456",
            "relation": "members"
        }
    },
    {
        "namespace": "Resource",
        "object": "purchase-orders-123",
        "relation": "collaborators",
        "subject_set": {
            "namespace": "Group",
            "object": "accountants-456",
            "relation": "members"
        }
    },
    {
        "namespace": "Shop",
        "object": "shops-456",
        "relation": "collaborators",
        "subject_set": {
            "namespace": "Group",
            "object": "accountants-456",
            "relation": "members"
        }
    },
    {
        "namespace": "Resource",
        "object": "inbounds-456",
        "relation": "collaborators",
        "subject_set": {
            "namespace": "Group",
            "object": "accountants-456",
            "relation": "members"
        }
    },
    {
        "namespace": "Resource",
        "object": "purchase-orders-456",
        "relation": "collaborators",
        "subject_set": {
            "namespace": "Group",
            "object": "accountants-456",
            "relation": "members"
        }
    },
    {
        "namespace": "Resource",
        "object": "purchase-orders-456",
        "relation": "admins",
        "subject_set": {
            "namespace": "Group",
            "object": "moderator",
            "relation": "members"
        }
    },
    {
        "namespace": "Feature",
        "object": "download-123",
        "relation": "shops",
        "subject_set": {
            "namespace": "Shop",
            "object": "shops-123"
        }
    },
    {
        "namespace": "Feature",
        "object": "reports-123",
        "relation": "shops",
        "subject_set": {
            "namespace": "Shop",
            "object": "shops-123"
        }
    },
    {
        "namespace": "Feature",
        "object": "reports-123",
        "relation": "collaborators",
        "subject_set": {
            "namespace": "Group",
            "object": "accountants-456",
            "relation": "members"
        }
    }
]

Ory CLI

 ory is allowed User:matteo view Resource inbounds-123

[alnr]

Hi @Luscha, thanks for your feedback. We're working hard on Keto performance. That includes a multi-region rollout (join the waitlist here), which could bring down latency depending on your geo location. Also, we have Keto latency improvements in our Q2 planning.

What kind of latency would you consider acceptable?

[Luscha]

Hi @alnr thank you for the answer.

Considering all requests should be authorized before even entering our perimeter, I would consider a sub-80ms latency acceptable.

In the case we adopt Keto as AuthZ provider we also need to "make a lot of checks" to make UI aware of the user's permissions regarding a particular resource (since keto still do not support any kind of bulk check or inverse tuple expansion); therefore and any latency greater than 100ms will make the website not usable