Querier throws ha-tracker error etcdserver: user name is empty

[paul-bormans]

Describe the bug
When enabling the ha-tracker (in the distributor; which is working fine btw), the queries instance is throwing errors:

level=error ts=2023-08-30T13:55:36.068147993Z caller=etcd.go:243 msg="watch error" key=ha-tracker/ err="rpc error: code = InvalidArgument desc = etcdserver: user name is empty"

To Reproduce
Configure distributor like so:

distributor:
  ha_tracker:
    enable_ha_tracker: true
    kvstore:
      etcd:
        endpoints:
        - cortex-etcd.monitoring.svc.cluster.local:2379
        password: ${ETCD_ROOT_PASSWORD}
        tls_enabled: false
        username: root
      store: etcd

Expected behavior
We expected ha-tracker to only work inside the distributor so to not impact querier at all; also ha-tracker is not configurable in the querier?

Environment:

• Infrastructure: Kubernetes
• Deployment tool: helm

Additional Context
Not sure what to attach, please ask for specific needs

[friedrichg]

I can see why is that not desired.
did you try disabling it for querier using an argument like -distributor.ha-tracker.enable=false ?

[paul-bormans]

Hello Friedrich, honestly i didn't thought of overriding that distributor conf for just querier, i can try for sure.
From Go code (ok i'm not the expert there) I was trying to discover the path where ha-tracker is initialized client side from the querier but i couldn't find that?
Paul