Failed to initalize provider error

[tw-carexpress]

Hello, I am following the instructions on the README.md

I have on my cluster the following:

kubectl v1.20.0
cert-manager v1.2.0
step-certificates-1.15.6 0.15.6 helm charts
step-issuer cloned from https://github.com/smallstep/step-issuer

Everything seems to be working fine, but when I modify the stepissuer.yaml inside the config/samples/ directory with the base 64 root cert, plus child etc etc ( following step by the step the guide) ...
At the moment of checking the status of the Issuer I get the following:
`apiVersion: certmanager.step.sm/v1beta1
kind: StepIssuer
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"certmanager.step.sm/v1beta1","kind":"StepIssuer","metadata":{"annotations":{},"name":"step-issuer","namespace":"default"},"spec":{"caBundle":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJpekNDQVRHZ0F3SUJBZ0lRTE1aTlFGdVpVbXRyTHNEUUp3dEREekFLQmdncWhrak9QUVFEQWpBa01TSXcKSUFZRFZRUURFeGxUZEdWd0lFTmxjblJwWm1sallYUmxjeUJTYjI5MElFTkJNQjRYRFRJeE1ESXhOekV6TURjegpPVm9YRFRNeE1ESXhOVEV6TURjek9Wb3dKREVpTUNBR0ExVUVBeE1aVTNSbGNDQkRaWEowYVdacFkyRjBaWE1nClVtOXZkQ0JEUVRCWk1CTUdCeXFHU000OUFnRUdDQ3FHU000OUF3RUhBMElBQkxDSjZjNktMckFiNHlWNTNzVVYKMThETHdVQ25WSUFwYlloR0p6TEE1TVVHNWhNVEJjcDgyT3R0dWNSenEydWMvWUJBa1lvdXZ4UmxwaXZlQ1V4aApqNGlqUlRCRE1BNEdBMVVkRHdFQi93UUVBd0lCQmpBU0JnTlZIUk1CQWY4RUNEQUdBUUgvQWdFQk1CMEdBMVVkCkRnUVdCQlFZcGxtL2tpRFljNjVRd3RGZHJsVW9rSG9rZERBS0JnZ3Foa2pPUFFRREFnTklBREJGQWlFQWpOcVAKYVcvNjJiaHRYdmQ4Q3Nta0dGajJSVkRXd1ZJZjBYTk1KV041NHZBQ0lDZUJ2SzhBUkZ1QTRwaUVzOHhUQktDTAo0dm1tRHBEMUNKL0JhVU9jMjNnSQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==","provisioner":{"kid":"w75BC1ZFGGpBP579V_JXsAKT9JK-89ZRkAb6mdGjLI8","name":"admin","passwordRef":{"key":"password","name":"step-certificates-provisioner-password"}},"url":"https://step-certificates.default.svc.cluster.local"}}
creationTimestamp: "2021-02-17T13:21:53Z"
generation: 1
managedFields:

• apiVersion: certmanager.step.sm/v1beta1
  fieldsType: FieldsV1
  fieldsV1:
  f:metadata:
  f:annotations:
  .: {}
  f:kubectl.kubernetes.io/last-applied-configuration: {}
  f:spec:
  .: {}
  f:caBundle: {}
  f:provisioner:
  .: {}
  f:kid: {}
  f:name: {}
  f:passwordRef:
  .: {}
  f:key: {}
  f:name: {}
  f:url: {}
  manager: kubectl-client-side-apply
  operation: Update
  time: "2021-02-17T13:21:53Z"
• apiVersion: certmanager.step.sm/v1beta1
  fieldsType: FieldsV1
  fieldsV1:
  f:status:
  .: {}
  f:conditions: {}
  manager: manager
  operation: Update
  time: "2021-02-17T13:22:23Z"
  name: step-issuer
  namespace: default
  resourceVersion: "7416479"
  uid: 85ca1a6b-8eda-4aa3-9d2e-4325e7e33ac5
  spec:
  caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJpekNDQVRHZ0F3SUJBZ0lRTE1aTlFGdVpVbXRyTHNEUUp3dEREekFLQmdncWhrak9QUVFEQWpBa01TSXcKSUFZRFZRUURFeGxUZEdWd0lFTmxjblJwWm1sallYUmxjeUJTYjI5MElFTkJNQjRYRFRJeE1ESXhOekV6TURjegpPVm9YRFRNeE1ESXhOVEV6TURjek9Wb3dKREVpTUNBR0ExVUVBeE1aVTNSbGNDQkRaWEowYVdacFkyRjBaWE1nClVtOXZkQ0JEUVRCWk1CTUdCeXFHU000OUFnRUdDQ3FHU000OUF3RUhBMElBQkxDSjZjNktMckFiNHlWNTNzVVYKMThETHdVQ25WSUFwYlloR0p6TEE1TVVHNWhNVEJjcDgyT3R0dWNSenEydWMvWUJBa1lvdXZ4UmxwaXZlQ1V4aApqNGlqUlRCRE1BNEdBMVVkRHdFQi93UUVBd0lCQmpBU0JnTlZIUk1CQWY4RUNEQUdBUUgvQWdFQk1CMEdBMVVkCkRnUVdCQlFZcGxtL2tpRFljNjVRd3RGZHJsVW9rSG9rZERBS0JnZ3Foa2pPUFFRREFnTklBREJGQWlFQWpOcVAKYVcvNjJiaHRYdmQ4Q3Nta0dGajJSVkRXd1ZJZjBYTk1KV041NHZBQ0lDZUJ2SzhBUkZ1QTRwaUVzOHhUQktDTAo0dm1tRHBEMUNKL0JhVU9jMjNnSQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
  provisioner:
  kid: w75BC1ZFGGpBP579V_JXsAKT9JK-89ZRkAb6mdGjLI8
  name: admin
  passwordRef:
  key: password
  name: step-certificates-provisioner-password
  url: https://step-certificates.default.svc.cluster.local
  status:
  conditions:
• lastTransitionTime: "2021-02-17T13:22:23Z"
  message: failed initialize provisioner
  reason: Error
  status: "False"
  type: Ready
  `

As you can see it says failed to initialize provisioner but Im not sure why this is happening and dunno how I can debug further.

[maraino]

@creamteam-de Can you see more errors in the logs for step-issuer pod? I think there should be a more clarifying error.

But in any case, this error is generally displayed on these cases:

• step-issuer fails to connect with step-ca
• step-issuer cannot connect with step-ca with the given ca bundle
• step-issuer cannot find a JWK provisioner in step-ca with the given kid
• step-issuer cannot decode the JWK encrypted key with the given password

[xlejo]

You check that the password is encode without new lines in the end, like \n?

Encode your password like this: printf 'password' | base64 -w 0.

If you try with: echo 'password' | base64 -w 0 the password will not work.

[wranders]

If anyone else encounters this, check the logs of step-certificates.

kubectl logs pod/step-certificates-0 | grep error

I encountered this on two occasions.

1. My CA was signed by an intermediate and I mistakenly added only the Root to the caBundle. Adding both certificates fixed that issue.
2. I created a new provisioner for the service and added it to ca.json (in Helm values.yaml), then updated via Helm. The error showed that the kid could not be found. Appearantly step-certificates only loads ca.json on start, and updating via Helm does not automatically trigger a restart. Fixed by restarting the StatefulSet.
   • kubectl rollout restart statefulset/step-certificates

Error logs led me right to the solution in both cases.