Add a debug level logging message for mismatched DNS Records and IPv4/v6 Addresses #21552
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
I've recently been chasing down a mysterious log in our Consul clusters which sounds quite scary:
The root cause of this error is that a client was making an
AAAA
query to our consul cluster, which received an answer in the form of IPv4 addresses, which cannot satisfy the request. During message serialization, this caused the answer to be silently dropped, and only manifest as a failure at the top level, which ends up being an error that is too generic to be useful.There's a discussion and other users who have had this issue and been confused here in the Hashicorp Forum
This PR adds a
debug
level log message to the code path in serialization. I chose debug so that this shouldn't be too chatty in existing consul deployments, thoughwarn
feels slighly more appropriate to me.It's slightly unfortunate that we have to plumb a
logger
into themessageSerializer
in order to do this. An alternative is that we could create a more descriptive error and bubble it up through the message serializer, which would then be logged out logged here. A new error type felt more invasive and more likely to break something, but I'm happy to take this approach if folks at Hashicorp prefer.Testing & Reproduction steps
Repro:
consul agent -dev
AAAA
record:dig @127.0.0.1 -p 8600 AAAA consul.service.consul
[ERROR] agent.dns: error serializing DNS results: error="no data"
Test this PR:
[DEBUG] agent.dns: unable to return DNS AAAA record for for ipv4 address: question=consul.service.consul. query-type=28 answer=127.0.0.1
Links
Discussion on Hashicorp forum with other confused users: https://discuss.hashicorp.com/t/after-updating-to-latest-consul-im-getting-error-serializing-dns-results-errors-in-my-logs/68319
PR Checklist