Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clustered Redis Slots Refresh Ignores TLS Usage #780

Closed
aballman opened this issue Mar 28, 2023 · 3 comments · Fixed by #781
Closed

Clustered Redis Slots Refresh Ignores TLS Usage #780

aballman opened this issue Mar 28, 2023 · 3 comments · Fixed by #781
Assignees
Labels

Comments

@aballman
Copy link
Contributor

aballman commented Mar 28, 2023

Describe the problem
When connecting to a TLS enabled cluster (specifically AWS Elasticache, but doesn't seem relevant), there is a timeout when refreshing cluster slots due to TLS not being enabled.

ERRO[0017] Cluster refresh failed: redisc: all nodes failed
read tcp xxx.xxx.xxx.xxx:50221->xxx.xxx.xxx.xxx:6379: i/o timeout

In the gomodule/redigo library, which is a dependency of mna/redisc there are two pathways to connect that are being utilized by redis_exporter Dial and DialURL

DialURL is used when connecting not in cluster mode, and Dial is used when clustering is enabled.

Clustering Enabled:
https://github.com/oliver006/redis_exporter/blob/de4f5eb0bfc3e15e3aa882db262572ed3bb45dcb/exporter/redis.go#LL98C10-L98C10

Clustering Disabled:
https://github.com/oliver006/redis_exporter/blob/de4f5eb0bfc3e15e3aa882db262572ed3bb45dcb/exporter/redis.go#LL53

The problem here is that in the redigo lib when using DialURL it automatically appends the DialUseTLS DialOption when the scheme is set to rediss. This does not happen in Dial
https://github.com/gomodule/redigo/blob/d6854479365f0307560fa28e18e2bd0634b05229/redis/conn.go#L389

What version of redis_exporter are you running?
Please run redis_exporter --version if you're not sure what version you're running.
1.48.0

Running the exporter
What's the full command you're using to run the exporter? (please remove passwords and other sensitive data)

REDIS_PASSWORD=<redacted>
REDIS_EXPORTER_IS_CLUSTER=true
REDIS_ADDR=<redacted>
REDIS_EXPORTER_SKIP_TLS_VERIFICATION=true
./redis_exporter

Expected behavior
Exporter does not time out when attempting to refresh cache slots on a TLS enabled cluster.

Screenshots
N/A

Additional context
N/A

@aballman
Copy link
Contributor Author

aballman commented Mar 28, 2023

Related to #723

@oliver006
Copy link
Owner

This is a great find!

@aballman
Copy link
Contributor Author

Also related to #752

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants