[meta] CI tests jobs are successful when goss tests are failing

Example:
https://devops-ci.elastic.co/job/elastic+helm-charts+pull-request+integration-apm-server/842/APM_SERVER_SUITE=oss,KUBERNETES_VERSION=1.18,label=docker&&virtual/console

```
10:14:10 Testing with pod: helm-apm-server-oss-apm-server-cc68b684c-4cd4h
10:14:11 2021/10/07 08:14:11 template: test:10:22: executing "test" at <.Env.ELASTIC_USERNAM...>: map has no entry for key "ELASTIC_USERNAME"
10:14:11 command terminated with exit code 1
...
10:14:19 Finished: SUCCESS
```

Related to:

https://github.com/elastic/helm-charts/blob/5cfd9f74ae0b829148b21f31a2c0deecc9938c3b/helpers/examples.mk#L12-L24

	goss: ## Run goss tests
	for i in $$(seq 1 5); do \
	if [ -z "$$GOSS_CONTAINER" ]; then \
	sleep 5; \
	echo "Retrieving pod ($$i/5)"; \
	GOSS_CONTAINER=$$(kubectl get --no-headers=true pods -l "$(GOSS_SELECTOR)" -o custom-columns=:metadata.name --field-selector=status.phase=Running --sort-by=.metadata.creationTimestamp \| tail -1 ); \
	else \
	echo "Testing with pod: $$GOSS_CONTAINER" && \
	kubectl cp "test/$(GOSS_FILE)" "$$GOSS_CONTAINER:/tmp/$(GOSS_FILE)" && \
	kubectl exec "$$GOSS_CONTAINER" -- sh -c "cd /tmp/ && curl -s -L \"https://github.com/aelsabbahy/goss/releases/download/$(GOSS_VERSION)/goss-linux-amd64\" -o goss && chmod +rx ./goss && ./goss --gossfile \"$(GOSS_FILE)\" validate --retry-timeout 300s --sleep 5s --color --format documentation"; \
	break; \
	fi; \
	done

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[meta] CI tests jobs are successful when goss tests are failing #1385

jmlrt
openedon Oct 7, 2021

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[meta] CI tests jobs are successful when goss tests are failing #1385

Description

jmlrtopenedon Oct 7, 2021

Metadata