gcve: Add tags for failure domain testing, reprovision for vsphere 8 and add inline comments #8148

chrischdi · 2025-05-28T11:57:17Z

No description provided.

k8s-ci-robot · 2025-05-28T11:57:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chrischdi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~infra/gcp/terraform/k8s-infra-gcp-gcve/OWNERS~~ [chrischdi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

chrischdi · 2025-05-28T17:12:05Z

/assign @sbueringer @fabriziopandini

fabriziopandini · 2025-05-29T09:31:21Z

Q: are you planning to adding setup GCP Essential Contacts for VMware Engine notifications to this PR?

fabriziopandini · 2025-05-29T10:15:31Z

infra/gcp/terraform/k8s-infra-gcp-gcve/vsphere/projects.tf

@@ -18,13 +18,13 @@ locals {
  gcp_gcve_project_name = "k8s-infra-e2e-gcp-gcve-project"
 }

-## create the projects (resource pool, folder, assign permissions per resource pool and folder)
+# Creates the projects (resource pool, folder, assign permissions per resource pool and folder)


What about adding a little bit more context, e.g.

Suggested change

# Creates the projects (resource pool, folder, assign permissions per resource pool and folder)

# Creates the projects; each project will map to a resources of type `gcve-vsphere-project` in Boskos

# and, when assigned to a prow container, it will provide access to a subset of vSphere resources:

# - A vSphere folder, e.g. `/Datacenter/vm/prow/k8s-infra-e2e-gcp-gcve-project-001`

# - A vSphere resource pool, e.g. `/Datacenter/host/k8s-gcve-cluster/Resources/prow/k8s-infra-e2e-gcp-gcve-project-001`

# - A range of IP assigned by boskos-userdata.sh

chrischdi · 2025-06-02T09:00:14Z

atlantis/plan: infra/gcp/terraform/k8s-infra-gcp-gcve

chrischdi · 2025-06-02T09:01:04Z

atlantis help

k8s-infra-ci-robot · 2025-06-02T09:01:07Z

atlantis
Terraform Pull Request Automation

Usage:
  atlantis <command> [options] -- [terraform options]

Examples:
  # show atlantis help
  atlantis help

  # run plan in the root directory passing the -target flag to terraform
  atlantis plan -d . -- -target=resource

  # apply all unapplied plans from this pull request
  atlantis apply

  # apply the plan for the root directory and staging workspace
  atlantis apply -d . -w staging

Commands:
  plan     Runs 'terraform plan' for the changes in this pull request.
           To plan a specific project, use the -d, -w and -p flags.
  apply    Runs 'terraform apply' on all unapplied plans from this pull request.
           To only apply a specific plan, use the -d, -w and -p flags.
  unlock   Removes all atlantis locks and discards all plans for this PR.
           To unlock a specific plan you can use the Atlantis UI.
  approve_policies
           Approves all current policy checking failures for the PR.
  version  Print the output of 'terraform version'
  help     View help.

Flags:
  -h, --help   help for atlantis

Use "atlantis [command] --help" for more information about a command.

chrischdi · 2025-06-02T09:02:33Z

atlantis plan -d infra/gcp/terraform/k8s-infra-gcp-gcve

k8s-infra-ci-robot · 2025-06-02T09:02:35Z

Error: User @chrischdi does not have permissions to execute 'plan' command.

upodroid · 2025-06-02T11:36:18Z

atlantis plan

k8s-infra-ci-robot · 2025-06-02T11:36:46Z

Ran Plan for 3 projects:

dir: infra/gcp/terraform/k8s-infra-gcp-gcve workspace: default
dir: infra/gcp/terraform/k8s-infra-gcp-gcve/maintenance-jumphost workspace: default
dir: infra/gcp/terraform/k8s-infra-gcp-gcve/vsphere workspace: default

2. dir: `infra/gcp/terraform/k8s-infra-gcp-gcve/maintenance-jumphost` workspace: `default`

Show Output

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
~ update in-place

Terraform will perform the following actions:

  # google_compute_instance.jumphost will be updated in-place
~ resource "google_compute_instance" "jumphost" {
        id                         = "projects/broadcom-451918/zones/us-central1-f/instances/maintenance-jumphost"
      ~ metadata                   = {
          ~ "user-data" = (sensitive value)
        }
        name                       = "maintenance-jumphost"
        tags                       = []
        # (22 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

▶️ To apply this plan, comment:

atlantis apply -d infra/gcp/terraform/k8s-infra-gcp-gcve/maintenance-jumphost

🚮 To delete this plan and lock, click here

🔁 To plan this project again, comment:

atlantis plan -d infra/gcp/terraform/k8s-infra-gcp-gcve/maintenance-jumphost

Plan: 0 to add, 1 to change, 0 to destroy.

3. dir: `infra/gcp/terraform/k8s-infra-gcp-gcve/vsphere` workspace: `default`

Plan Error

Show Output

running 'sh -c' '/usr/local/bin/terraform1.10.5 plan -input=false -refresh -out "/atlantis/repos/kubernetes/k8s.io/8148/default/infra/gcp/terraform/k8s-infra-gcp-gcve/vsphere/default.tfplan"' in '/atlantis/repos/kubernetes/k8s.io/8148/default/infra/gcp/terraform/k8s-infra-gcp-gcve/vsphere': exit status 1
╷
│ Error: No value for required variable
│ 
│   on variables.tf line 19:
│   19: variable "vsphere_user" {
│ 
│ The root module input variable "vsphere_user" is not set, and has no
│ default value. Use a -var or -var-file command line argument to provide a
│ value for this variable.
╵
╷
│ Error: No value for required variable
│ 
│   on variables.tf line 23:
│   23: variable "vsphere_password" {
│ 
│ The root module input variable "vsphere_password" is not set, and has no
│ default value. Use a -var or -var-file command line argument to provide a
│ value for this variable.
╵
╷
│ Error: No value for required variable
│ 
│   on variables.tf line 27:
│   27: variable "vsphere_server" {
│ 
│ The root module input variable "vsphere_server" is not set, and has no
│ default value. Use a -var or -var-file command line argument to provide a
│ value for this variable.
╵
╷
│ Error: No value for required variable
│ 
│   on variables.tf line 32:
│   32: variable "nsxt_user" {
│ 
│ The root module input variable "nsxt_user" is not set, and has no default
│ value. Use a -var or -var-file command line argument to provide a value for
│ this variable.
╵
╷
│ Error: No value for required variable
│ 
│   on variables.tf line 36:
│   36: variable "nsxt_password" {
│ 
│ The root module input variable "nsxt_password" is not set, and has no
│ default value. Use a -var or -var-file command line argument to provide a
│ value for this variable.
╵
╷
│ Error: No value for required variable
│ 
│   on variables.tf line 41:
│   41: variable "nsxt_server" {
│ 
│ The root module input variable "nsxt_server" is not set, and has no default
│ value. Use a -var or -var-file command line argument to provide a value for
│ this variable.
╵

Plan Summary

3 projects, 1 with changes, 1 with no changes, 1 failed

⏩ To apply all unapplied plans from this Pull Request, comment:
```
atlantis apply
```
🚮 To delete all plans and locks from this Pull Request, comment:
```
atlantis unlock
```

fabriziopandini

/lgtm

infra/gcp/terraform/k8s-infra-gcp-gcve/main.tf

upodroid · 2025-06-09T18:47:00Z

atlantis unlock

k8s-infra-ci-robot · 2025-06-09T18:47:03Z

All Atlantis locks for this PR have been unlocked and plans discarded

sbueringer

Thx! Really great documentation, appreciate it

infra/gcp/terraform/k8s-infra-gcp-gcve/README.md

sbueringer · 2025-06-10T18:02:03Z

infra/gcp/terraform/k8s-infra-gcp-gcve/README.md

@@ -17,9 +17,10 @@ More specifically, to allow prow jobs to create VM on vSphere, a few resources a
 Also, the network of the prow container is going to be paired to the VMware engine network, thus


Suggested change

Also, the network of the prow container is going to be paired to the VMware engine network, thus

Also, the network of the prow container is going to be peered to the VMware engine network, thus

infra/gcp/terraform/k8s-infra-gcp-gcve/README.md

infra/gcp/terraform/k8s-infra-gcp-gcve/docs/terraform.md

sbueringer · 2025-06-10T18:40:15Z

infra/gcp/terraform/k8s-infra-gcp-gcve/docs/terraform.md

@@ -30,15 +30,21 @@ gcloud auth application-default login
 Ensure all the env variables expected by the terraform manifest you are planning to run are set:
 - [vsphere](../vsphere/)


Suggested change

- [vsphere](../vsphere/)

- [vsphere](../vsphere/README.md)

infra/gcp/terraform/k8s-infra-gcp-gcve/README.md

fabriziopandini

Thanks for working on this
/lgtm

sbueringer · 2025-06-25T16:57:21Z

/lgtm

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 28, 2025

k8s-ci-robot requested review from BenTheElder and dims May 28, 2025 11:57

k8s-ci-robot added sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 28, 2025

chrischdi force-pushed the pr-gcve-add-tags branch from 1581fd8 to e622c49 Compare May 28, 2025 16:00

BenTheElder requested a review from upodroid May 28, 2025 16:47

k8s-ci-robot assigned fabriziopandini and sbueringer May 28, 2025

fabriziopandini reviewed May 29, 2025

View reviewed changes

chrischdi mentioned this pull request Jun 3, 2025

Add more GCVE documentation #8146

Merged

chrischdi changed the title ~~gcve: Add tags for failure domain testing and add more comments~~ [WIP] gcve: Add tags for failure domain testing and add more comments Jun 5, 2025

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 5, 2025

chrischdi force-pushed the pr-gcve-add-tags branch from e622c49 to b9ed008 Compare June 5, 2025 11:32

k8s-ci-robot removed the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 5, 2025

gcve: fixup overview image

2acf0a8

k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 5, 2025

chrischdi changed the title ~~[WIP] gcve: Add tags for failure domain testing and add more comments~~ [WIP] gcve: Add tags for failure domain testing, reprovision for vsphere 8 and add inline comments Jun 5, 2025

gcve: fixup vsphere readme and add dependency to nsx for datasource

a10c77f

fabriziopandini reviewed Jun 6, 2025

View reviewed changes

infra/gcp/terraform/k8s-infra-gcp-gcve/main.tf Show resolved Hide resolved

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 6, 2025

k8s-infra-ci-robot mentioned this pull request Jun 9, 2025

deploy datadog to GCP #8175

Merged

sbueringer reviewed Jun 10, 2025

View reviewed changes

infra/gcp/terraform/k8s-infra-gcp-gcve/README.md Outdated Show resolved Hide resolved

sbueringer reviewed Jun 10, 2025

View reviewed changes

Review fixes

093a1eb

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 11, 2025

chrischdi commented Jun 11, 2025

View reviewed changes

infra/gcp/terraform/k8s-infra-gcp-gcve/README.md Show resolved Hide resolved

chrischdi added 2 commits June 16, 2025 08:28

fixup dns

fd57733

Fixup DNS and docs and document nsx gateway

03f2c92

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jun 16, 2025

chrischdi added 2 commits June 18, 2025 10:23

increase mtu for nsx-t segment

b1372b3

gcve: document OVA upload and environment recreation

c057bfd

chrischdi changed the title ~~[WIP] gcve: Add tags for failure domain testing, reprovision for vsphere 8 and add inline comments~~ gcve: Add tags for failure domain testing, reprovision for vsphere 8 and add inline comments Jun 18, 2025

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 18, 2025

chrischdi mentioned this pull request Jun 18, 2025

[WIP] GCVE: scale-up to Standard deployment #8205

Open

fabriziopandini reviewed Jun 20, 2025

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 20, 2025

k8s-ci-robot merged commit 21dcaad into kubernetes:main Jun 20, 2025
3 checks passed

k8s-ci-robot added this to the v1.34 milestone Jun 20, 2025

-# Creates the projects (resource pool, folder, assign permissions per resource pool and folder)
+# Creates the projects; each project will map to a resources of type `gcve-vsphere-project` in Boskos
+# and, when assigned to a prow container, it will provide  access to a subset of vSphere resources:
+# - A vSphere folder, e.g. `/Datacenter/vm/prow/k8s-infra-e2e-gcp-gcve-project-001`
+# - A vSphere resource pool, e.g. `/Datacenter/host/k8s-gcve-cluster/Resources/prow/k8s-infra-e2e-gcp-gcve-project-001`
+# - A range of IP assigned by boskos-userdata.sh

		@@ -17,9 +17,10 @@ More specifically, to allow prow jobs to create VM on vSphere, a few resources a
		Also, the network of the prow container is going to be paired to the VMware engine network, thus

		@@ -30,15 +30,21 @@ gcloud auth application-default login
		Ensure all the env variables expected by the terraform manifest you are planning to run are set:
		- [vsphere](../vsphere/)

gcve: Add tags for failure domain testing, reprovision for vsphere 8 and add inline comments #8148

gcve: Add tags for failure domain testing, reprovision for vsphere 8 and add inline comments #8148

Conversation

chrischdi commented May 28, 2025

Uh oh!

k8s-ci-robot commented May 28, 2025

Uh oh!

chrischdi commented May 28, 2025

Uh oh!

fabriziopandini commented May 29, 2025

Uh oh!

fabriziopandini May 29, 2025

Choose a reason for hiding this comment

Uh oh!

chrischdi commented Jun 2, 2025

Uh oh!

chrischdi commented Jun 2, 2025

Uh oh!

k8s-infra-ci-robot commented Jun 2, 2025

Uh oh!

chrischdi commented Jun 2, 2025

Uh oh!

k8s-infra-ci-robot commented Jun 2, 2025

Uh oh!

upodroid commented Jun 2, 2025

Uh oh!

k8s-infra-ci-robot commented Jun 2, 2025

2. dir: infra/gcp/terraform/k8s-infra-gcp-gcve/maintenance-jumphost workspace: default

3. dir: infra/gcp/terraform/k8s-infra-gcp-gcve/vsphere workspace: default

Plan Summary

Uh oh!

fabriziopandini left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

upodroid commented Jun 9, 2025

Uh oh!

k8s-infra-ci-robot commented Jun 9, 2025

Uh oh!

sbueringer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sbueringer Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sbueringer Jun 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fabriziopandini left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sbueringer commented Jun 25, 2025

Uh oh!

Uh oh!

2. dir: `infra/gcp/terraform/k8s-infra-gcp-gcve/maintenance-jumphost` workspace: `default`

3. dir: `infra/gcp/terraform/k8s-infra-gcp-gcve/vsphere` workspace: `default`