Support CA Certs from other namespaces in BackendTLSPolicy

[arkodg]

What would you like to be added:

Move the type for CACertificateRefs in BackendTLSPolicy from LocalObjectReference to ObjectReference so a namespace field can be include to define a CA Cert Reference that lives in another namespace (not the backend service namespace)

Why this is needed:

• Allows a team to define a ConfigMap containing the CA Cert ( to validate the backends) thats common, once instead of 1 per backend per namespace
  • to reduce the number of resources that need to be created by the team
  • to reduce the amount of resources that need to be reconciled by the control plane (these are fairly large resources)
  • the workaround is to be creative with the WellKnownCACertificates option, which I'm hoping we can avoid

[youngnick]

If we do this, that will also require a ReferenceGrant, as per the usual cross-namespace reference requirements.

[arkodg]

agree @youngnick, it still results in a de duplication of the ConfigMap, which starts to impact at high scale (hundreds and thoursands of services)

[howardjohn]

what about using ClusterTrustBundle instead? it's now beta so may be viable?

not opposed to this as well possibly though

[robscott]

+1 on looking into ClusterTrustBundle as an option here. ConfigMap was a less than ideal solution as we waited for ClusterTrustBundle to stabilize. Now that it's reached beta, it's likely worth considering here.

cc @ahmedtd