feat: pass tags and global chart variables when deploying an application #6427

aure-olli · 2020-01-07T04:05:42Z

Submitter checklist

Change is code complete and matches issue description.
Change is covered by existing or new tests.

Description

This PR passes useful tags and global chart variables to helm when deploying an application.

Prior to this PR, the only way to check in which environment a chart was deployed was to read .Release.Namespace. It was impossible to use such conditions in chart requirements, and preview environment were hard to detect.

Tags can be used to condition chart requirements.
Global variables can be used in chart templates.

jx promote and jx preview now pass the following variables to helm:

tags.jx-ns-<namespace>=true (ex: tags.jx-ns-jx-production=true)
tags.jx-<env-type>=true (ex: tags.jx-preview=true)
tags.jx-env-<env>=true (ex: tags.jx-env-production=true)
global.jx-ns-<namespace>=true (ex: tags.jx-ns-jx-production=true)
global.jx-<env-type>=true (ex: tags.jx-preview=true)
global.jx-env-<env>=true (ex: tags.jx-env-production=true)
global.jx-ns=<namespace> (ex: tags.jx-ns=jx-production)
global.jx-type-env=<env-type> (ex: tags.jx-type-env=preview)
global.jx-env=<env> (ex: tags.jx-env=production)
global.jx-preview-app=<preview-app> preview only (ex: global.jx-preview-app=my-app)
global.jx-preview-pr=<preview-pr> preview only (ex: global.jx-preview-pr=6)

jx step helm install and jx step helm apply now pass the following variables to helm:

tags.jx-ns-<namespace>=true (ex: tags.jx-ns-jx-production=true)
global.jx-ns-<namespace>=true (ex: tags.jx-ns-jx-production=true)
global.jx-ns=<namespace> (ex: tags.jx-ns=jx-production)

Additionally, jx step helm install now accepts the --set-string parameter, similarly to helm install.

Special notes for the reviewer(s)

I don't know where such feature can be documented.

Regular environments are currently deployed using jx step helm apply --namespace <namespace>, without any mention of which environment is actually being deployed. This unfortunately limits the variables I can pass when deploying a regular environment. Maybe another command or option would be useful.

Which issue this PR fixes

fixes #6312

houndci-bot · 2020-01-07T04:05:46Z

pkg/cmd/promote/promote.go

@@ -1171,3 +1176,27 @@ func (o *PromoteOptions) SearchForChart(filter string) (string, error) {
 	o.HelmRepositoryURL = repoUrl
 	return appName, nil
 }
+
+func (o *PromoteOptions) GetEnvChartValues(targetNS string, env *v1.Environment) ([]string, []string) {


exported method PromoteOptions.GetEnvChartValues should have comment or be unexported

houndci-bot · 2020-01-07T04:05:46Z

pkg/cmd/promote/promote.go

@@ -1171,3 +1176,27 @@ func (o *PromoteOptions) SearchForChart(filter string) (string, error) {
 	o.HelmRepositoryURL = repoUrl
 	return appName, nil
 }
+
+func (o *PromoteOptions) GetEnvChartValues(targetNS string, env *v1.Environment) ([]string, []string) {


exported method PromoteOptions.GetEnvChartValues should have comment or be unexported

houndci-bot · 2020-01-07T04:05:47Z

pkg/cmd/promote/promote.go

@@ -1171,3 +1176,27 @@ func (o *PromoteOptions) SearchForChart(filter string) (string, error) {
 	o.HelmRepositoryURL = repoUrl
 	return appName, nil
 }
+
+func (o *PromoteOptions) GetEnvChartValues(targetNS string, env *v1.Environment) ([]string, []string) {


exported method PromoteOptions.GetEnvChartValues should have comment or be unexported

jenkins-x-bot · 2020-01-07T04:05:59Z

Hi @aure-olli. Thanks for your PR.

I'm waiting for a jenkins-x member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

abayer · 2020-01-07T11:22:38Z

/ok-to-test

abayer · 2020-01-07T11:42:00Z

lint failure is two things - a case of import ordering, and some formatting. As a general rule, I’d recommend running make fmt before pushing every time (I personally slip up on this sometimes due to having an old and slow laptop as my primary work computer, and make fmt takes a few minutes there thanks to the import formatting), and if you’ve got time, make lint as well to catch any other potential lint errors.

We’re gradually working on getting the logs scrubbed well enough in the UI to be 99% sure of not exposing unmasked secrets there. Once we’ve (finally!) got that, we’ll be able to give contributors like yourself access to the UI. Sorry. =(

abayer · 2020-01-07T12:31:11Z

And the integration failure is due to a bunch of compilation errors in pkg/cmd/add/add_app_test.go - if you don’t have the time for a full make test, I’d suggest running make build-all, which will compile all the test code but not run any tests.

Tags can be used to condition chart requirements. Global variables can be used in chart templates. `jx promote` and `jx preview` now pass the following variables to helm: - `tags.jx-ns-<namespace>=true` (ex: `tags.jx-ns-jx-production=true`) - `tags.jx-<env-type>=true` (ex: `tags.jx-preview=true`) - `tags.jx-env-<env>=true` (ex: `tags.jx-env-production=true`) - `global.jx-ns-<namespace>=true` (ex: `tags.jx-ns-jx-production=true`) - `global.jx-<env-type>=true` (ex: `tags.jx-preview=true`) - `global.jx-env-<env>=true` (ex: `tags.jx-env-production=true`) - `global.jx-ns=<namespace>` (ex: `tags.jx-ns=jx-production`) - `global.jx-type-env=<env-type>` (ex: `tags.jx-type-env=preview`) - `global.jx-env=<env>` (ex: `tags.jx-env=production`) - `global.jx-preview-app=<preview-app>` (ex: `global.jx-preview-app=my-app`) - `global.jx-preview-pr=<preview-pr>` (ex: `global.jx-preview-pr=6`) `jx step helm install` and `jx step helm apply` now pass the following variables to helm: - `tags.jx-ns-<namespace>=true` (ex: `tags.jx-ns-jx-production=true`) - `global.jx-ns-<namespace>=true` (ex: `tags.jx-ns-jx-production=true`) - `global.jx-ns=<namespace>` (ex: `tags.jx-ns=jx-production`) Additionally, `jx step helm install` now accepts the `--set-string` parameter, similarly to `helm install`. fixes jenkins-x#6312 Signed-off-by: Aurélien Lambert <aure@olli-ai.com>

aure-olli · 2020-01-08T03:08:55Z

Hi !

Thank you for you advice, it's much helpful. I also face problem with my modest computer: I had to reboot several times because I forgot to remove -p 4, make test takes forever even when I only want one test, and I had to let make generate run during the night ^^ Many tests interact with git and kubectl, which provokes many errors so it's sometimes hard to see the true errors.

Anyway, I hope everything is fixed.

abayer · 2020-01-08T13:07:30Z

Yeah, I always set my k8s context to an existing cluster and set my namespace to jx before running the full tests. That said, I should really fix the lingering non-integration tests that barf if you don’t have a cluster.

abayer · 2020-01-08T13:09:18Z

/lgtm

jenkins-x-bot · 2020-01-08T13:09:32Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abayer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [abayer]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

In jenkins-x#6427 was added template variables to help templating. Unfortunately, template variables names must be valid go variable names to be used. Tags are still snake-cased. `jx promote` and `jx preview` now pass the following variables to helm: - `tags.jx-ns-<namespace>=true` (ex: `tags.jx-ns-jx-production=true`) - `tags.jx-<env-type>=true` (ex: `tags.jx-preview=true`) - `tags.jx-env-<env>=true` (ex: `tags.jx-env-production=true`) - `global.jxNs<namespace>=true` (ex: `global.jxNsJxProduction=true`) - `global.jx<env-type>=true` (ex: `global.jxPreview=true`) - `global.jxEnv<env>=true` (ex: `global.jxEnvProduction=true`) - `global.jxNs=<namespace>` (ex: `global.jxNs=jx-production`) - `global.jxTypeEnv=<env-type>` (ex: `global.jxTypeEnv=preview`) - `global.jxEnv=<env>` (ex: `global.jxEnv=production`) - `global.jxPreviewApp=<preview-app>` (ex: `global.jxPreviewApp=my-app`) - `global.jxPreviewPr=<preview-pr>` (ex: `global.jxPreviewPr=6`) `jx step helm install` and `jx step helm apply` now pass the following variables to helm: - `tags.jx-ns-<namespace>=true` (ex: `tags.jx-ns-jx-production=true`) - `global.jxNs<namespace>=true` (ex: `global.jxNsJxProduction=true`) - `global.jxNs=<namespace>` (ex: `global.jxNs=jx-production`) Signed-off-by: Aurélien Lambert <aure@olli-ai.com>

In #6427 was added template variables to help templating. Unfortunately, template variables names must be valid go variable names to be used. Tags are still snake-cased. `jx promote` and `jx preview` now pass the following variables to helm: - `tags.jx-ns-<namespace>=true` (ex: `tags.jx-ns-jx-production=true`) - `tags.jx-<env-type>=true` (ex: `tags.jx-preview=true`) - `tags.jx-env-<env>=true` (ex: `tags.jx-env-production=true`) - `global.jxNs<namespace>=true` (ex: `global.jxNsJxProduction=true`) - `global.jx<env-type>=true` (ex: `global.jxPreview=true`) - `global.jxEnv<env>=true` (ex: `global.jxEnvProduction=true`) - `global.jxNs=<namespace>` (ex: `global.jxNs=jx-production`) - `global.jxTypeEnv=<env-type>` (ex: `global.jxTypeEnv=preview`) - `global.jxEnv=<env>` (ex: `global.jxEnv=production`) - `global.jxPreviewApp=<preview-app>` (ex: `global.jxPreviewApp=my-app`) - `global.jxPreviewPr=<preview-pr>` (ex: `global.jxPreviewPr=6`) `jx step helm install` and `jx step helm apply` now pass the following variables to helm: - `tags.jx-ns-<namespace>=true` (ex: `tags.jx-ns-jx-production=true`) - `global.jxNs<namespace>=true` (ex: `global.jxNsJxProduction=true`) - `global.jxNs=<namespace>` (ex: `global.jxNs=jx-production`) Signed-off-by: Aurélien Lambert <aure@olli-ai.com>

Complementary to jenkins-x#6427 and jenkins-x#6682 all calls to `helm lint` (including `jx step helm release`) also pass some variables - `tags.jx-lint=true` - `global.jxLint=true` - `global.jxTypeEnv=lint` This avoids having `helm lint` errors when the templates are looking for variables like `.Values.global.jxNs` due to `global` being `nil` Signed-off-by: Aurélien Lambert <aure@olli-ai.com>

Complementary to #6427 and #6682 all calls to `helm lint` (including `jx step helm release`) also pass some variables - `tags.jx-lint=true` - `global.jxLint=true` - `global.jxTypeEnv=lint` This avoids having `helm lint` errors when the templates are looking for variables like `.Values.global.jxNs` due to `global` being `nil` Signed-off-by: Aurélien Lambert <aure@olli-ai.com>

In jenkins-x#6427 was added template variables to help templating. Unfortunately, template variables names must be valid go variable names to be used. Tags are still snake-cased. `jx promote` and `jx preview` now pass the following variables to helm: - `tags.jx-ns-<namespace>=true` (ex: `tags.jx-ns-jx-production=true`) - `tags.jx-<env-type>=true` (ex: `tags.jx-preview=true`) - `tags.jx-env-<env>=true` (ex: `tags.jx-env-production=true`) - `global.jxNs<namespace>=true` (ex: `global.jxNsJxProduction=true`) - `global.jx<env-type>=true` (ex: `global.jxPreview=true`) - `global.jxEnv<env>=true` (ex: `global.jxEnvProduction=true`) - `global.jxNs=<namespace>` (ex: `global.jxNs=jx-production`) - `global.jxTypeEnv=<env-type>` (ex: `global.jxTypeEnv=preview`) - `global.jxEnv=<env>` (ex: `global.jxEnv=production`) - `global.jxPreviewApp=<preview-app>` (ex: `global.jxPreviewApp=my-app`) - `global.jxPreviewPr=<preview-pr>` (ex: `global.jxPreviewPr=6`) `jx step helm install` and `jx step helm apply` now pass the following variables to helm: - `tags.jx-ns-<namespace>=true` (ex: `tags.jx-ns-jx-production=true`) - `global.jxNs<namespace>=true` (ex: `global.jxNsJxProduction=true`) - `global.jxNs=<namespace>` (ex: `global.jxNs=jx-production`) Signed-off-by: Aurélien Lambert <aure@olli-ai.com>

Complementary to jenkins-x#6427 and jenkins-x#6682 all calls to `helm lint` (including `jx step helm release`) also pass some variables - `tags.jx-lint=true` - `global.jxLint=true` - `global.jxTypeEnv=lint` This avoids having `helm lint` errors when the templates are looking for variables like `.Values.global.jxNs` due to `global` being `nil` Signed-off-by: Aurélien Lambert <aure@olli-ai.com>

houndci-bot reviewed Jan 7, 2020

View reviewed changes

jenkins-x-bot requested review from abayer and garethjevans January 7, 2020 04:05

jenkins-x-bot added the needs-ok-to-test label Jan 7, 2020

jenkins-x-bot added the size/L label Jan 7, 2020

aure-olli force-pushed the envs branch from 9ba0e91 to 6aeb353 Compare January 7, 2020 04:19

jenkins-x-bot added ok-to-test and removed needs-ok-to-test labels Jan 7, 2020

aure-olli force-pushed the envs branch from 6aeb353 to a93d931 Compare January 8, 2020 03:05

jenkins-x-bot assigned abayer Jan 8, 2020

jenkins-x-bot added the lgtm label Jan 8, 2020

jenkins-x-bot added the approved label Jan 8, 2020

jenkins-x-bot merged commit 322302a into jenkins-x:master Jan 8, 2020

aure-olli mentioned this pull request Feb 3, 2020

fix: Turns globals.jx-* template variables into valid variable names #6682

Merged

2 tasks

aure-olli mentioned this pull request Feb 6, 2020

feat: helm lint calls pass variables too #6699

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: pass tags and global chart variables when deploying an application #6427

feat: pass tags and global chart variables when deploying an application #6427

aure-olli commented Jan 7, 2020

houndci-bot Jan 7, 2020

houndci-bot Jan 7, 2020

houndci-bot Jan 7, 2020

jenkins-x-bot commented Jan 7, 2020

abayer commented Jan 7, 2020

abayer commented Jan 7, 2020

abayer commented Jan 7, 2020

aure-olli commented Jan 8, 2020

abayer commented Jan 8, 2020

abayer commented Jan 8, 2020

jenkins-x-bot commented Jan 8, 2020

feat: pass tags and global chart variables when deploying an application #6427

feat: pass tags and global chart variables when deploying an application #6427

Conversation

aure-olli commented Jan 7, 2020

Submitter checklist

Description

Special notes for the reviewer(s)

Which issue this PR fixes

houndci-bot Jan 7, 2020

Choose a reason for hiding this comment

houndci-bot Jan 7, 2020

Choose a reason for hiding this comment

houndci-bot Jan 7, 2020

Choose a reason for hiding this comment

jenkins-x-bot commented Jan 7, 2020

abayer commented Jan 7, 2020

abayer commented Jan 7, 2020

abayer commented Jan 7, 2020

aure-olli commented Jan 8, 2020

abayer commented Jan 8, 2020

abayer commented Jan 8, 2020

jenkins-x-bot commented Jan 8, 2020