Selecting multiple profiles doesn't appear to work

[mattmelgard]

Expected behavior

When executing skaffold run -p servicedeps,airflow I would expect both profiles to be built and deployed.

Actual behavior

I might be missing something about how this would be expected to work, but I would expect the releases of both profiles to be deployed, where instead I'm only seeing the last item in the list being deployed. I also tried using a patch instead, but that doesn't seem to have worked either.

Information

• Skaffold version: skaffold version doesn't seem to have any output for whatever reason, but I'm using the hombrew install manifest located at https://ghcr.io/v2/homebrew/core/skaffold/manifests/2.1.0
• Operating system: macOS 13.1 (22C65)
• Installed via: Homebrew
• Contents of skaffold.yaml:

apiVersion: skaffold/v3
kind: Config
metadata:
  name: backend

# Release configuration that is common to our helm managed services.
.common-service-helm-config: &common-service-helm-config
  upgradeOnChange: true

# Release configuration that is common to all of our services.
.common-service-release-config: &common-service-release-config
  <<: *common-service-helm-config
  remoteChart: syndio/syndio-service
  version: 1.13.0

# Service dependencies that are deployed in the initial setup phase and are typically required
# for local development to run.
.release-servicedeps: &release-servicedeps
  releases:
    - &servicedeps
      name: servicedeps
      remoteChart: syndio/syndio-service-deps
      version: 0.6.0
      wait: true
      upgradeOnChange: false
      valuesFiles:
        - helm/shared-values/development/service-deps.values.yaml

# The list of all releases aside from service dependencies.
.all-releases: &all-releases
  releases:
    - *servicedeps
    - &accountsapi
      name: accountsapi
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - accounts/deploy/accountsapi/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &permissionsapi
      name: permissionsapi
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - permissions/deploy/permissionsapi/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &payeq-jobs
      name: payeq-jobs
      remoteChart: syndio/payeq-jobs
      version: 0.12.0
      <<: *common-service-helm-config
      valuesFiles:
        - apiv2/deploy/ce/values/config/development.values.yaml
        - apiv2/deploy/ce/values/jobs/development.values.yaml
      setValueTemplates:
        image.tag: "{{.IMAGE_TAG_backend_python}}"
        <<: *env-value-overrides
    - &apiv2
      name: apiv2
      <<: *common-service-release-config
      valuesFiles:
        - apiv2/deploy/ce/values/config/development.values.yaml
        - apiv2/deploy/ce/values/api/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_python}}"
        <<: *env-value-overrides
    - &admin
      name: admin
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - admin/deploy/admin/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &apiv3
      name: apiv3
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - apiv3/deploy/apiv3/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &gateway
      name: gateway
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - gateway/deploy/gateway/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &oppeqapi
      name: oppeqapi
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - oppeq/deploy/oppeqapi/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &oppeqliteapi:
      name: oppeqliteapi
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - oppeqlite/deploy/oppeqliteapi/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &payeqapi
      name: payeqapi
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - payeq/deploy/payeqapi/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &payeqqr
      name: payeqqr
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - payeq/deploy/payeqqr/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &orchestrationapi
      name: orchestrationapi
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - orchestrationapi/deploy/orchestrationapi/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &statsapi
      name: statsapi
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - stats/deploy/statsapi/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &pubsubrouter
      name: pubsubrouter
      <<: *common-service-release-config
      valuesFiles:
        - apiv2/deploy/ce/values/config/development.values.yaml
        - apiv2/deploy/ce/values/pubsubrouter/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_python}}"
        <<: *env-value-overrides
    - &pubsubuploader
      name: pubsubuploader
      <<: *common-service-release-config
      valuesFiles:
        - apiv2/deploy/ce/values/config/development.values.yaml
        - apiv2/deploy/ce/values/pubsubuploader/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_python}}"
        <<: *env-value-overrides
    - &pubsubworker
      name: pubsubworker
      <<: *common-service-release-config
      valuesFiles:
        - apiv2/deploy/ce/values/config/development.values.yaml
        - apiv2/deploy/ce/values/pubsubworker/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_python}}"
        <<: *env-value-overrides
    - &stats_cloudfunctions
      name: statscloudfunctions
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - stats/deploy/cloud_functions/values/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_python}}"
        <<: *env-value-overrides
    - &logsapi
      name: logsapi
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - logging/deploy/logsapi/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &oppeqliteqr
      name: oppeqliteqr
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - oppeqlite/deploy/oppeqliteqr/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides

# The set of default releases to include when initializing the local environment
# for the first time.
.release-defaults: &release-defaults
  releases:
    - *servicedeps
    - *accountsapi
    - *permissionsapi
    - *payeq-jobs
    - *apiv2
    - *admin
    - *apiv3
    - *gateway
    - *oppeqapi
    - *oppeqliteapi
    - *payeqapi
    - *payeqqr
    - *orchestrationapi
    - *statsapi

# The set of optional releases that can be enabled or disabled as needed and are not required
# for core functionality.
.release-optional: &release-optional
  releases:
    - *pubsubrouter
    - *pubsubuploader
    - *pubsubworker
    - *logsapi
    - *oppeqliteqr

# Services that are optional, but that depend on Google Cloud infrastructure in some way
# and require GCP authentication to a live GCP project in order to function.
# To use these services, assign the contents of a GCP service account key to the 
# variable DEV_GCLOUD_SA_KEY (documented above) and re-deploy with the needs-gcp profile
# enabled
.releases-needing-gcp-auth: &releases-needing-gcp-auth
  releases:
    - &ingestqr
      name: ingestqr
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - ingest/deploy/ingestqr/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides
    - &encryptionapi
      name: encryptionapi
      <<: *common-service-release-config
      valuesFiles:
        - helm/shared-values/development/values.yaml
        - encryption/deploy/encryptionapi/development.values.yaml
      setValueTemplates:
        app.image.tag: "{{.IMAGE_TAG_backend_go}}"
        <<: *env-value-overrides

build:
  local:
    push: false
    useBuildkit: true
    concurrency: 0 # A value of 0 means no limit
  tagPolicy:
    gitCommit:
      variant: AbbrevCommitSha
  artifacts:
    # We use a custom build step for building the go services which involves compiling
    # the binaries on the host to take advantage of the go build cache.
    - &backend_go
      image: backend-go
      custom:
        buildCommand: ./development/scripts/go_build.sh
        dependencies:
          paths:
            - "*/**/*.go"
            - "*/**/config.default.yaml"
            - "*/**/scripts/*"
          ignore:
            - "apiv2/"
            - "development"
    - &backend_python
      image: backend-python
      context: ./apiv2
      docker:
        dockerfile: ./deploy/ce/Dockerfile
    - &backend_cloudfunctions
      image: backend-cloudfunctions
      context: .
      docker:
        dockerfile: ./stats/deploy/cloud_functions/dev/Dockerfile

deploy:
  kubeContext: docker-desktop
  helm:
    <<: *release-defaults

profiles:
  - name: all
    deploy:
      helm:
        <<: *all-releases
  - name: airflow
    deploy:
      helm:
        releases:
          - name: airflow
            remoteChart: airflow-helm/airflow
            version: 8.6.1
            upgradeOnChange: false
            valuesFiles:
              - helm/shared-values/development/airflow.yaml
  - name: needs-gcp
    deploy:
      helm:
        <<: *releases-needing-gcp-auth
  - name: servicedeps
    patches:
      - op: replace
        path: /build/artifacts
        value: []
    deploy:
      helm:
        releases:
          - *servicedeps
  - name: payeq-core
    deploy:
      helm:
        releases:
          - *accountsapi
          - *permissionsapi
          - *payeq-jobs
          - *apiv2
          - *admin
          - *apiv3
          - *gateway
          - *payeqapi
  - name: accounts
    build:
      artifacts:
        - *backend_go
    deploy:
      helm:
        releases:
          - *accountsapi
          - *permissionsapi
  - name: oppeq
    build:
      artifacts:
        - *backend_go
    deploy:
      helm:
        releases:
          - *oppeqapi
          - *oppeqliteapi
          - *oppeqliteqr
          - *stats_cloudfunctions
  - name: python-workers
    build:
      artifacts:
        - *backend_python
    deploy:
      helm:
        releases:
          - *pubsubrouter
          - *pubsubuploader
          - *pubsubworker
  - name: python-services
    build:
      artifacts:
        - *backend_python
    deploy:
      helm:
        releases:
          - *payeq-jobs
          - *apiv2

portForward:
  - resourceType: service
    resourceName: syndio-service-deps-postgres
    port: 5432
  - resourceType: service
    resourceName: syndio-service-deps-redis
    port: 6379
  - resourceType: service
    resourceName: syndio-service-deps-fake-gcs
    port: 4443
  - resourceType: service
    resourceName: syndio-service-deps-pubsub
    port: 8085
  - resourceType: service
    resourceName: payeq-gateway
    port: 80
    localPort: 3000

Steps to reproduce the behavior

1. Run Docker Desktop local k8s cluster on macOS
2. Create a skaffold.yaml with multiple profiles similar to the one I'm using
3. Run skaffold run with two of those profiles enabled
4. See only the last profile listed being released to the cluster

[renzodavid9]

Hi @mattmelgard, according to the given description, I just want to check, are you trying to extend the list of existing deploy.helm.releases (the one that is not under any profile) adding more releases with every profile? Assuming that's the case, using:

...
deploy:
  helm:
    releases:
    - #releaseP0...

profiles:
  - name: p1
    deploy:
      helm:
        releases:
          - #releaseP1

With a command like skaffold run -p p1 will make the original deploy.helm.releases list to be replaced with the one from p1 profile, will not extend it, this is the intended behavior; the same will happen with two profiles defining the deploy.helm.releases list, the last profile will override the list. However, I think with patch you should be able to extend the releases list, could you share more details about how you were trying to accomplish this with patch? Here is a short example that tries to mimic your case:

apiVersion: skaffold/v3
kind: Config


.release-defaults: &release-defaults
  releases:
    - name: chart1
      remoteChart: SOME-REMOTE-CHART-1
      setValues:
        image: image-1


metadata:
  name: test

deploy:
  helm:
    <<: *release-defaults

profiles:
  - name: p1
    patches:
      - op: add
        path: "/deploy/helm/releases/-"
        value:
          name: chart2
          remoteChart: SOME-REMOTE-CHART-2
          setValues:
            image: image-2

Running skaffold run -p p1 with the previous configuration should result in the releases list extended by the p1 profile. Thanks!

[mattmelgard]

@renzodavid9 ah, let me try that, I think I may have left out the /- last time when using the patches approach. I know that's supposed to append to the end of the list so that might have been it. Will report back tomorrow once I've had a chance to test that out.

[mattmelgard]

Okay, that does seem to work for me so I'm going to go ahead and close this. Thanks for your help!