-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
service did not deregistered properly #3122
Comments
Hi @waterdudu thanks for opening an issue. Are there any messages in the logs on your Consul servers when this is happening on the agent (the dereg/sync loop and when you try to remove it from the catalog manually)? |
Hi @slackpad Since log level is INFO, there are only a few synced node lines except for the
|
Ran into the same issue, only way for me to get rid of the zombies was to stop the consul agent, nuke its data directory and then start it again. |
Looks like this is still an issue - at least on the 1.4.3 version. Also, deregistering a service makes a rendom number of other services turn critical for some seconds until they have been run through the next health check. Deleting the data directory fixes the problem. |
Same issue on the version 1.8.3 |
Hello everyone, My apologies that this is being revisited after so long. Without more information than was available in the provided logs at the If any of you are still actively facing issues related to this, please reply. We'd like to understand whether the different commenters were experiencing the same or different issues. On getting the deregistration to work One problem I noticed in the original post is that the call to the deregister service endpoint uses the Deregister service unfortunately only works with a service ID, and I can see how that error message would be quite confusing (because a service with that name does exist, just not one with that id). So...
... should be ...
It looks like service ID was correctly used in the call to the I've marked this issue with the On the origins of the sync/deregister loop We are not aware of a mechanism purely within Consul that would cause the sync/deregister loop. It is our understanding that nomad must be the cause of re-registering of the service. |
I have both consul cluster and nomad cluster running in the same 3 nodes(node-0, node-1, node-2) and a service call
goddess-of-harvest
running with count of 2, deployed by nomad.After releasing several versions of that service, I noticed a zombie service occured.
After checking the consul log file, I noticed the service was synced and deregistered infinitely. Below is the consul log.
I tried to call consul deregister api
curl -v --request PUT "http://127.0.0.1:8500/v1/agent/service/deregister/goddess-of-harvest-fe"
. Nothing happened and the service can still be seen inv1/catalog/service
api response.What was strange, however, the consul log said
service does not exist
.I also tried to call deregister api
curl --request PUT --data @payload.json http://127.0.0.1:8500/v1/catalog/deregister
with payload file to deregister the service.Althought that PUT request returned true, consul catalog api told me the service was still available.
curl -s localhost:8500/v1/catalog/service/goddess-of-harvest-fe | jq .
consul info
for both Client and ServerClient:
Server:
My consul version is
Consul v0.7.1
and noamd version isNomad v0.5.4
.Thanks
The text was updated successfully, but these errors were encountered: