Allow GRPCLB_NAME_PREFIX to be configured per service in DnsNameResolver

[ST-DDT]

What version of gRPC are you using?

1.18.0

Feature Request

Some third party servers that I try to connect to don't expose their grpc service addresses with the grpclb prefix, so it would be nice if I could configure the service name lookup by providing a custom service prefix.

Example

serverA: dns:/foobar.example.com -> _grpclb._tcp.foobar.example.com
serverB: dns:/barfoo.example.com?srvName=grpc -> _grpc._tcp.barfoo.example.com
serverC: dns:/lorep.example.com?srvName=ipsum -> _ipsum._tcp.lorep.example.com

[carl-mastrangelo]

cc @markdroth

[markdroth]

I don't think this is something we want to make configurable. Service owners that want to expose grpclb services should use the _grpclb prefix in DNS, as described in https://github.com/grpc/proposal/blob/master/A5-grpclb-in-dns.md.

Note that this prefix is only required when using grpclb for look-aside load balancing. Users using grpc without grpclb do not need any prefix in the first place.

Also, note that we are in the process of building a new look-aside load balancing architecture based on the Envoy xDS API, which will replace the existing grpclb functionality. This new system will be triggered via the gRPC service config, not via the DNS SRV records, so the entire question of what prefix to use here will be moot.

[ST-DDT]

I use the service names to resolve the server address from kubernetes. And AFAICT this works without the grpc-lb lib (unless it is included natively somewhere). I still have to enable the jndi service lookup via properties, but that isn't a problem.

I don't need the actual load-balancing, the current health checking round-robin is currently good enough, (sometimes there might only be one target available). I just need it to resolve the target IP:port, unfortunately I'm only a small consumer inside the cluster and thus don't have much influence on the cluster management.

Related: #5273 + #5218

[markdroth]

It sounds like what you want is a feature that allows the DNS resolver to use an arbitrary SRV record for non-grpclb lookups.

How specialized of a use-case is this? Does Kubernetes always create DNS SRV records that are useful with gRPC backends, or is this something specific to whatever cluster you're running on? If the latter, then it seems like a fairly specialized case that we wouldn't necessarily want to support, but if it turns out that this would be useful in all Kubernetes clusters, it may be something for us to consider.

It's not clear how this would work in the presence of grpclb. However, that may be okay, especially since we're trying to move away from the grpclb-style SRV records anyway. So we could just say that if people use this feature, then they won't be able to use grpclb-style SRV records.

If we do decide to support this, it's not something we should do as a Java-only feature; instead, it's something we should design such that it can be supported in all gRPC languages.

A better alternative for this would probably be to use a custom resolver that speaks the Kubernetes API directly instead of going through DNS. I believe that @saturnism has something like this already implemented for Java. (We've talked about supporting this in all gRPC languages, but it hasn't bubbled up to the top of our priority list yet.)

[ST-DDT]

How specialized of a use-case is this? Does Kubernetes always create DNS SRV records that are useful with gRPC backends, or is this something specific to whatever cluster you're running on? If the latter, then it seems like a fairly specialized case that we wouldn't necessarily want to support, but if it turns out that this would be useful in all Kubernetes clusters, it may be something for us to consider.

AFAIK Kubernetes automatically creates these svc entries for all Services created in Kubernetes. And I assume that you have one Service per Deployment (unless it does not expose ports for consumption). See also: https://kubernetes.io/docs/concepts/services-networking/service/#dns

A better alternative for this would probably be to use a custom resolver that speaks the Kubernetes API directly instead of going through DNS. I believe that @saturnism has something like this already implemented for Java. (We've talked about supporting this in all gRPC languages, but it hasn't bubbled up to the top of our priority list yet.)

If I use the Kubernetes API for the lookup then I have to provide credentials (to each client) in order to access it (unless RBAC is disabled). This would bring unnecessary complications for the setup. The current setup has the benefit that it does not require any additional libraries and is a well known concept for (Dev-)Ops.

Maybe we could add a very simply svc-dns only NameResolver, but I assume this would require a gRPC-global design decision.

[markdroth]

Maybe we could add a very simply svc-dns only NameResolver, but I assume this would require a gRPC-global design decision.

You could certainly implement such a resolver yourself without requiring any help from us. But if you want this to be something that gRPC supports out of the box, then yes, it would need to be a feature we'd have to decide to implement in all languages.