fix: support tls for cluster refresh op

[aballman]

Closes #780

[coveralls]

Pull Request Test Coverage Report for Build 95

• 1 of 1 (100.0%) changed or added relevant line in 1 file are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.004%) to 92.547%

---

Totals
Change from base Build 86:	0.004%
Covered Lines:	1900
Relevant Lines:	2053

---

💛 - Coveralls

[oliver006]

This is a great find, thanks for submitting this PR!
I assume you tried it with a TLS-enabled cluster and it works?

Is there any way to test this as part of the CI? I guess one would have to stand up a cluster with all nodes using TLS, correct? The docker image I use for cluster testing (https://github.com/Grokzen/docker-redis-cluster) unfortunately doesn't support TLS right now.

Not a blocker to merge this, was just wondering if you might have a suggestion.

[aballman]

@oliver006 This isn't quite as clean as the project you've referenced, but it does work for a local testing configuration that could be adapted to CI with a little work.

I put together a docker-compose for this

version: '3.5'
services:
  redis-cluster:
    image: 'redis:6.0-alpine'
    command: redis-cli --cluster create 172.20.0.31:6379 172.20.0.32:6380 172.20.0.33:6381 172.20.0.34:6382 172.20.0.35:6383 172.20.0.36:6374 --cluster-replicas 1 --cluster-yes --tls --cacert /usr/local/etc/redis/tls/CA-cert.pem
    networks:
      cluster_net:
        ipv4_address: 172.20.0.30
    depends_on:
      - redis-node-1
      - redis-node-2
      - redis-node-3
      - redis-node-4
      - redis-node-5
      - redis-node-6
    volumes:
      - ./redis-cluster/tls:/usr/local/etc/redis/tls
  redis-node-1:
    image: 'redis:6.0-alpine'
    command: redis-server /usr/local/etc/redis/redis.conf

    volumes:
      - redis-node-1-data:/var/lib/redis
      - ./redis-cluster/redis-node-1.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/tls:/usr/local/etc/redis/tls
    networks:
      cluster_net:
        ipv4_address: 172.20.0.31
    ports:
      - '6379:6379'
  redis-node-2:
    image: 'redis:6.0-alpine'
    command: redis-server /usr/local/etc/redis/redis.conf

    volumes:
      - redis-node-2-data:/var/lib/redis
      - ./redis-cluster/redis-node-2.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/tls:/usr/local/etc/redis/tls
    networks:
      cluster_net:
        ipv4_address: 172.20.0.32
    ports:
      - '6380:6380'
  redis-node-3:
    image: 'redis:6.0-alpine'
    command: redis-server /usr/local/etc/redis/redis.conf

    volumes:
      - redis-node-3-data:/var/lib/redis
      - ./redis-cluster/redis-node-3.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/tls:/usr/local/etc/redis/tls
    networks:
      cluster_net:
        ipv4_address: 172.20.0.33
    ports:
      - '6381:6381'
  redis-node-4:
    image: 'redis:6.0-alpine'
    command: redis-server /usr/local/etc/redis/redis.conf

    volumes:
      - redis-node-4-data:/var/lib/redis
      - ./redis-cluster/redis-node-4.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/tls:/usr/local/etc/redis/tls
    networks:
      cluster_net:
        ipv4_address: 172.20.0.34
    ports:
      - '6382:6382'
  redis-node-5:
    image: 'redis:6.0-alpine'
    command: redis-server /usr/local/etc/redis/redis.conf

    volumes:
      - redis-node-5-data:/var/lib/redis
      - ./redis-cluster/redis-node-5.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/tls:/usr/local/etc/redis/tls
    networks:
      cluster_net:
        ipv4_address: 172.20.0.35
    ports:
      - '6383:6383'
  redis-node-6:
    image: 'redis:6.0-alpine'
    command: redis-server /usr/local/etc/redis/redis.conf

    volumes:
      - redis-node-6-data:/var/lib/redis
      - ./redis-cluster/redis-node-6.conf:/usr/local/etc/redis/redis.conf
      - ./redis-cluster/tls:/usr/local/etc/redis/tls
    networks:
      cluster_net:
        ipv4_address: 172.20.0.36
    ports:
      - '6384:6384'

volumes:
  redis-node-1-data:
  redis-node-2-data:
  redis-node-3-data:
  redis-node-4-data:
  redis-node-5-data:
  redis-node-6-data:
networks:
  cluster_net:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/24

This is what my redis conf file looks, each one is unique but only to specify a unique port for each of them

# Custom config file to enable cluster mode
# on all Redis instances started via Docker
port 0
cluster-enabled yes
# The cluster file is created and managed by Redis
# We just need to declare it here
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

tls-port 6379
tls-cert-file /usr/local/etc/redis/tls/server.crt
tls-key-file /usr/local/etc/redis/tls/private.key
tls-ca-cert-file /usr/local/etc/redis/tls/CA-cert.pem
tls-auth-clients no
tls-cluster yes
tls-replication yes

and my config for generating the certificate

[req]
default_bits = 2048
distinguished_name = req_distinguished_name
req_extensions = req_ext
x509_extensions = v3_req
prompt = no

[req_distinguished_name]
OU = "Redis"

[req_ext]
subjectAltName = @alt_names

[v3_req]
keyUsage = keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[alt_names]
IP.1 = 172.20.0.31
IP.2 = 172.20.0.32
IP.3 = 172.20.0.33
IP.4 = 172.20.0.34
IP.5 = 172.20.0.35
IP.6 = 172.20.0.36
IP.7 = 127.0.0.1

My configuration here is heavily sourced from both of these:
https://pierreabreu.medium.com/building-redis-cluster-with-docker-compose-9569ddb6414a
https://www.dltlabs.com/blog/how-to-set-up-a-redis-cluster-with-tls-in-a-local-machine--679616

The latter has a guide on generating the TLS certificate.

It's important to use -extensions req_ext -extfile <tls-config-path> in the openssl x509 command, otherwise you won't have SANs for the IPs.

With the following set

REDIS_ADDR=rediss://127.0.0.1:6379
REDIS_EXPORTER_TLS_CA_CERT_FILE=<path to ca>

Running with these settings against the master branch yields this log from the exporter

┃ ❯ curl -s localhost:9121/metrics | grep -E "redis_cluster_known_nodes \d+"
ERRO[0001] Couldn't connect to redis instance (rediss://127.0.0.1:6379)

When using the above changes, it should report the cluster node count

┃ ❯ curl -s localhost:9121/metrics | grep -E "redis_cluster_known_nodes \d+"
redis_cluster_known_nodes 6

I also verified that this works in a plaintext cluster as well.

[oliver006]

This is probably the most complete and beautiful response I've ever seen in this repo - thanks for being so thorough and explaining everything. I shall link this from the README to preserve it for future generations of engineers that want to set up a Redis test cluster with TLS enabled.

[oliver006]

Released as v1.49.0