-
Notifications
You must be signed in to change notification settings - Fork 112
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provider produced inconsistent result after apply #160
Comments
Thanks @kpettijohn, this is indeed an issue. I made some tests and I find Consul's behavior weird with policies and multiple datacenters. I'll keep you updated and hopefully post a fix shortly. |
Hi @kpettijohn, I think I found multiple ways to trigger this bug. Can you give more information your Consul cluster? Do you have only one datacenter named |
In my case I only have one datacenter ( |
Since the provider did not correctly checked the error response from Consul, we lack some info. Can you try running:
? You will need to replace |
Here is the output from the command above.
I added the following flags
Running the curl again after the first 200 success returns the following error.
|
Can you remove the policy |
Removing the policy allows me to create it again.
|
Does
let you read the policy properly? |
It does seem to let me read the policy.
|
This is weird, those two calls, the Can you send your Consul configuration file with the secrets removed? Maybe my configuration differs from yours. |
Maybe this depends on the Consul version we are running. Which one are you using? |
I am running Server configuration: datacenter = "mydc"
data_dir = "/var/lib/consul"
server = true
bootstrap_expect = 3
ui = true
ports {
grpc = 8502
http = -1 //disabled
https = 8501
}
retry_join = ["10.10.10.101", "10.10.10.102", "10.10.10.103"]
encrypt = "encrypt-key"
verify_incoming = false
verify_incoming_rpc = true
verify_outgoing = true
verify_server_hostname = true
auto_encrypt {
allow_tls = true
}
ca_file = "/etc/pki/tls/certs/consul-agent-ca.pem"
cert_file = "/etc/pki/tls/certs/mydc-server-consul-0.pem"
key_file = "/etc/pki/tls/private/mydc-server-consul-0-key.pem"
acl = {
enabled = true
default_policy = "deny"
enable_token_persistence = true
tokens = {
agent = "agent-token"
}
}
addresses = {
https = "0.0.0.0"
} |
After looking into things I bit more I added a log statement to the resource consul_acl_policy and I now see the following error from Consul.
|
Did you added that in I can reproduce the bug with multiple datacenters and this is the error message I get too. I'm running tests based on your configuration but still can't reproduce the bug. Are you sure all three masters are in |
OK I think I found the issue and it seems that it might just be a bad configuration on my end. Overall I had a typo in the datacenter name on the consul provider configuration which allowed it to create the policy but when reading it back it would error as the DC didn't exist. Thanks for your help tracking things down @remilapeyre! |
Failing to read a policy from the server does not necessarly mean that the policy has been removed, the network can be down, the correct datacenter may not be reachable etc. We must be conservative when removing resources from the state and only create a new one if it's actualy needed. See hashicorp#160
Failing to read a policy from the server does not necessarly mean that the policy has been removed, the network can be down, the correct datacenter may not be reachable etc. We must be conservative when removing resources from the state and only create a new one if it's actualy needed. See #160
Terraform Version
Run
terraform -v
to show the version. If you are not running the latest version of Terraform, please upgrade because your issue may have already been fixed.Affected Resource(s)
Please list the resources as a list, for example:
consul_acl_policy
If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.
Terraform Configuration Files
Debug Output
Please provider a link to a GitHub Gist containing the complete debug output: https://www.terraform.io/docs/internals/debugging.html. Please do NOT paste the debug output in the issue; just paste a link to the Gist.
https://gist.github.com/kpettijohn/81cdd2588f7526b35f74c25d3a127c3d
Panic Output
If Terraform produced a panic, please provide a link to a GitHub Gist containing the output of the
crash.log
.Expected Behavior
Successful Terraform apply
Actual Behavior
Terraform throws the following error after creating the new policy in Consul.
After the first error if another Terraform apply is attempted it will fail again but with another error saying that a policy with that name already exists.
Steps to Reproduce
Please list the steps required to reproduce the issue, for example:
terraform apply
Important Factoids
Are there anything atypical about your accounts that we should know? For example: Running in EC2 Classic? Custom version of OpenStack? Tight ACLs?
I have ACLs enabled and am currently using the
Bootstrap Token (Global Management)
token.References
The text was updated successfully, but these errors were encountered: