Skip to content

Commit

Permalink
Use E2E_KUBECONFIG instead of KUBECONFIG
Browse files Browse the repository at this point in the history
  • Loading branch information
@uwe-mayer
uwe-mayer committed Jul 31, 2024
1 parent 693d868 commit 4a8f594
Show file tree
Hide file tree
Showing 3 changed files with 23 additions and 13 deletions.
6 changes: 3 additions & 3 deletions Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,16 +75,16 @@ e2e-local: generate-manifests generate envtest ## Run e2e tests against mock api
unset USE_EXISTING_CLUSTER && KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) -p path)" go test ./test/e2e/... -coverprofile cover.out -v

.PHONY: e2e-remote
e2e-remote: ## Run e2e tests against a remote Greenhouse cluster.
e2e-remote: ## Run e2e tests against a remote Greenhouse cluster. E2E_KUBECONFIG must be set.
USE_EXISTING_CLUSTER=true go test ./test/e2e/... -coverprofile cover.out -v

.PHONY: e2e-local-cluster
e2e-local-cluster: e2e-local-cluster-create ## Run e2e tests on a local KIND cluster.
USE_EXISTING_CLUSTER=true KUBECONFIG=$(shell pwd)/test/e2e/local-cluster/e2e.kubeconfig INTERNAL_KUBECONFIG=$(shell pwd)/test/e2e/local-cluster/e2e.internal.kubeconfig go test ./test/e2e/... -coverprofile cover.out -v
USE_EXISTING_CLUSTER=true E2E_KUBECONFIG=$(shell pwd)/test/e2e/local-cluster/e2e.kubeconfig INTERNAL_KUBECONFIG=$(shell pwd)/test/e2e/local-cluster/e2e.internal.kubeconfig go test ./test/e2e/... -coverprofile cover.out -v

.PHONY: e2e-local-cluster-create
e2e-local-cluster-create:
cd test/e2e/local-cluster && go run .
cd test/e2e/local-cluster && go run . -v


.PHONY: fmt
Expand Down
15 changes: 12 additions & 3 deletions pkg/test/env.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,8 @@ var (
pollInterval = 1 * time.Second
updateTimeout = 30 * time.Second

persisted_kubeconfig = os.Getenv("KUBECONFIG")

// TestBeforeSuite configures the test suite.
TestBeforeSuite = func() {
logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true)))
Expand All @@ -116,9 +118,11 @@ var (
installWebhooks := len(allRegisterWebhookFuncs) > 0 && os.Getenv("TEST_INSTALL_WEBHOOKS") != "false"
if useExistingGreenhouseCluster {
// we are making use of https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/envtest#pkg-constants to prevent starting a new control plane
kubeconfig := os.Getenv("KUBECONFIG")
Expect(kubeconfig).NotTo(BeEmpty(), "the environment variable KUBECONFIG must be set to run the tests against a remote cluster")
fmt.Printf("using existing cluster with kubeconfig: %s\n", kubeconfig)
e2e_kubeconfig := os.Getenv("E2E_KUBECONFIG")
Expect(e2e_kubeconfig).NotTo(BeEmpty(), "the environment variable E2E_KUBECONFIG must be set to run the tests against a remote cluster")
// we overwrite the KUBECONFIG env var expected by envtest to make sure tests are not accidentally running against existing k8s context
os.Setenv("KUBECONFIG", e2e_kubeconfig)
fmt.Printf("Running tests against existing cluster with kubeconfig: %s\n", e2e_kubeconfig)
installCRDs = false
installWebhooks = false
} else {
Expand Down Expand Up @@ -205,6 +209,11 @@ var (
Eventually(func() error {
return testEnv.Stop()
}).Should(Succeed(), "there should be no error stopping the test environment")

if useExistingGreenhouseCluster {
// we reset the KUBECONFIG env var to its original value
os.Setenv("KUBECONFIG", persisted_kubeconfig)
}
}
)

Expand Down
15 changes: 8 additions & 7 deletions test/e2e/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,11 @@ will run the e2e test suite without making assumptions on the infrastructure to

Leveraging envtest, we will basically have three different test scenarios. The following env vars steer these:

| Env Var | Meaning |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `USE_EXISTING_CLUSTER` | If set to `true`, the e2e test suite will not spin up a local apiserver and etcd. Instead, it will expect an existing greenhouse installation on the cluster inferred from the `KUBECONFIG` environment variable. |
| `INTERNAL_KUBECONFIG` | The path to the kubeconfig file for accessing the Greenhouse cluster itself from the running instance. This is used when `USE_EXISTING_CLUSTER` is set to `true`. KIND makes it necessary to set this separately to the `KUBECONFIG` as the internal api server adress differs to the external. Other setups may not use this. If unset `KUBECONFIG` is used. |
| Env Var | Meaning |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --- |
| `USE_EXISTING_CLUSTER` | If set to `true`, the e2e test suite will not spin up a local apiserver and etcd. Instead, it will expect an existing greenhouse installation on the cluster inferred from the `E2E_KUBECONFIG` environment variable. |
| `E2E_KUBECONFIG` | Required when `USE_EXISTING_CLUSTER` is `true`. Points to the remote cluster the e2e test suite is running against. | |
| `INTERNAL_KUBECONFIG` | The path to the kubeconfig file for accessing the Greenhouse cluster itself from the running instance. This is used when `USE_EXISTING_CLUSTER` is set to `true`. KIND makes it necessary to set this separately to the `E2E_KUBECONFIG` as the internal api server adress differs to the external. Other setups may not use this. If unset `E2E_KUBECONFIG` is used. |

## Run everything local a.k.a. `USE_EXISTING_CLUSTER = false` or unset

Expand All @@ -37,10 +38,10 @@ We can run our e2e test suite against a running greenhouse installation by expos
export USE_EXISTING_CLUSTER=true
```

This will stop envtest from spinning up a local apiserver and etcd and expect an existing greenhouse installation on the cluster infered from the set `KUBECONFIG` environment variable:
This will stop envtest from spinning up a local apiserver and etcd and expect an existing greenhouse installation on the cluster infered from the set `E2E_KUBECONFIG` environment variable:

```bash
export KUBECONFIG=/path/to/greenhouse.kubeconfig
export E2E_KUBECONFIG=/path/to/greenhouse.kubeconfig
```

To run the e2e test suite against a remote installation:
Expand All @@ -49,7 +50,7 @@ To run the e2e test suite against a remote installation:
make e2e-remote
```

Test setup asserts `KUBECONFIG` is set and working and will fail otherwise.
Test setup asserts `E2E_KUBECONFIG` is set and working and will fail otherwise.

### Run against a local Greenhouse installation in KIND cluster a.k.a. `USE_EXISTING_CLUSTER = true` and `INTERNAL_KUBECONFIG` set

Expand Down

0 comments on commit 4a8f594

Please sign in to comment.