Skip to content

Cluster install fails due to Vault failure #3345

Closed
@tdcox

Description

@tdcox

Summary

Cluster install terminates prematurely due to a health check error from Vault:

error creating cluster configuring the git auth: creating the git auth config service: wait for vault to be initialized and unsealed: reading vault health: Error making API request.

Steps to reproduce the behavior

jx create cluster gke \
--cluster-name='d21' \
--default-admin-password='xxxxx' \
--environment-git-owner='tdcox' \
--enhanced-apis=true \
--enhanced-scopes=true \
--git-username='tdcox' \
--git-private=false \
--labels='demo=true' \
--machine-type='n1-standard-4' \
--max-num-nodes='3' \
--min-num-nodes='2' \
--no-tiller=true \
--preemptible=true \
--project-id='jx-mar19' \
--prow=true \
--skip-login=true \
--vault=true \
--zone='europe-west1-d'

Expected behavior

Completion of the install script.

Actual behavior

Ingress rules recreated
Vault jx-vault-d21 exposed
System vault created named jx-vault-d21 in namespace jx.
Lets set up a Git user name and API token to be able to perform CI/CD

error creating cluster configuring the git auth: creating the git auth config service: wait for vault to be initialized and unsealed: reading vault health: Error making API request.

URL: GET http://jx-vault-d21.jx.35.195.246.238.nip.io/v1/sys/health?drsecondarycode=299&performancestandbycode=299&sealedcode=299&standbycode=299&uninitcode=299
Code: 503. Raw Message:

<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.15.8</center>
</body>
</html>
error: configuring the git auth: creating the git auth config service: wait for vault to be initialized and unsealed: reading vault health: Error making API request.

URL: GET http://jx-vault-d21.jx.35.195.246.238.nip.io/v1/sys/health?drsecondarycode=299&performancestandbycode=299&sealedcode=299&standbycode=299&uninitcode=299
Code: 503. Raw Message:

<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx/1.15.8</center>
</body>
</html>

D21_logs_2019-03-14.txt

Looks like the vault pod has a repeating error failed to set up mount table

Jx version

The output of jx version is:

NAME               VERSION
jx                 1.3.974
Kubernetes cluster v1.11.7-gke.4
kubectl            v1.13.4
helm client        Client: v2.13.0+g79d0794
git                git version 2.21.0
Operating System   Mac OS X 10.13.6 build 17G4015

Jenkins type

  • Classic Jenkins
  • Serverless Jenkins

Kubernetes cluster

Metadata

Metadata

Assignees

Labels

area/vaultkind/bugIssue is a bugpriority/important-soonMust be staffed and worked on either currently, or very soon, ideally in time for the next release.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions