Support provisioning of access rules

[big-andy-coates]

Many orgs use ACLs/RBAC/Active Directory to protect cluster resources. This is to be encouraged! Therefore Creek should support provisioning the access control for public vs internal topics.

Provisioning

Aggregate topics

• Output topics (usual case), should be:
  • readable by anyone, though may need to extend this to allow the descriptor to define list of allowed entities that can read a topic, i.e. accessible to entities outside the aggregate, but only to defined set of entities, not everyone.
  • writable only by the aggregate service(s) that produces the data or, as an alternatively, it may be acceptable to be writable by any of the aggregate's service - but the former is better if possible.
• Input topics (usual case), should be the reverse, i.e.
  • writable by anyone, or defined list of external entities
  • readable by specific aggregate services, or may be all of the aggregate's services.

Service topics

Topics owned by services, but which are not part of the aggregate api, should be restricted so that those outside the aggregate can't access them. Ideally, only the exact aggregate service's the need to should be able to read/write them, as required.

Need to consider the repartition and state store changelog topics that streams creates... we could require these to be defined in service descriptors, to allow for exact access controls to be set, or could allow prefixed acls to support such internal topics, or both via config. The challenge here is that such internal topic names include the application id... which is-often configured at run time... tricky.

Topic naming patterns for aggregate and service topics would help here, e.g. aggregate topics are prefixed with aggregate name, (which itself can be hierarchal); service names are prefixed with aggregate names, and service topics prefixed with service name. With this, the internal topics would then follow the same naming strategy if application.id == service.name. Is this too restrictive though?

Updating

We need to consider more what to do when provisioning subsequent versions of a service. If we have access to the deployed version and the new version, then we can determine what resources and acls have been removed from the descriptor. Colud provide ability to 'tidy up' i.e. remove these from the cluster. Here be monsters though. Deleting user data is generally bad! Can make it optional. Removing acls may be acceptable?? But this is... a complicated area - not sure of effort vs value here.