Skip to content

Migration from loki-distributed to loki fails when using umbrella chart #15878

Open
@MatusJenca2

Description

Describe the bug
I tried to migrate from loki-distributed to loki in our monitoring-stack using this guide

However i got this error.

helm upgrade --install monitoring ~/kubernetes-monitoring-stack/chart -f loki/values-loki-migration.yaml --reuse-values --dependency-update 
Error: template: dnation-kubernetes-monitoring-stack/charts/loki/templates/write/statefulset-write.yaml:89:20: executing "dnation-kubernetes-monitoring-stack/charts/loki/templates/write/statefulset-write.yaml" at <include "loki.image" .>: error calling include: template: dnation-kubernetes-monitoring-stack/charts/loki-distributed/templates/_helpers.tpl:78:33: executing "loki.image" at <.global.registry>: nil pointer evaluating interface {}.registry
helm.go:86: 2025-01-22 12:38:47.333622711 +0100 CET m=+0.542132745 [debug] template: dnation-kubernetes-monitoring-stack/charts/loki/templates/write/statefulset-write.yaml:89:20: executing "dnation-kubernetes-monitoring-stack/charts/loki/templates/write/statefulset-write.yaml" at <include "loki.image" .>: error calling include: template: dnation-kubernetes-monitoring-stack/charts/loki-distributed/templates/_helpers.tpl:78:33: executing "loki.image" at <.global.registry>: nil pointer evaluating interface {}.registry

I suspect that this is related to this issue in helm as both loki and loki-distributed charts use loki.image variable in templates and to quote Helm docs:

Defined templates (templates created inside a {{ define }} directive) are globally accessible. That means that a chart and all of its subcharts will have access to all of the templates created with {{ define }}.
For that reason, all defined template names should be namespaced.

Because we use an ubrella chart, there's a conflict. I don't see any way around this other than code change, that's why I am creating this isssue.
To Reproduce
Steps to reproduce the behavior:

  1. Create a subchart with both loki and loki-distributed as dependencies i.e. our monitoring-stack.
  2. Install with loki-distributed enabled and loki disabled - This is current state
  3. Follow migration guide
  4. Deploy with both subcharts enabled as per the guide (step 1)

Expected behavior
Both loki and loki distributed are deployed, we may continue to step 2 of the guide

Environment:

  • Infrastructure: Kubernetes
  • Deployment tool: helm

Screenshots, Promtail config, or terminal output
values before migration

kube-prometheus-stack:
  kubeScheduler:
    serviceMonitor:
      insecureSkipVerify: true

  kubeControllerManager:
    serviceMonitor:
      insecureSkipVerify: true
# I tried to manually enter global.registry wherever needed, still got the same error
global:
  registry: https://hub.docker.com/
loki:
  enabled: false
loki-distributed:
  global:
    registry: https://hub.docker.com/
  registry: https://hub.docker.com/
  enabled: true
# loki config omitted, may contain sensitive information

values-loki-migration.yaml

# Drop logs from loki 
promtail:
  config:
    snippets:
      common:
      - source_labels:
        - "__meta_kubernetes_pod_label_app_kubernetes_io_component"
      regex: "(canary|read|write)"
      action: "drop"
loki2:
  registry: https://hub.docker.com/
  global:
    registry: https://hub.docker.com/
  enabled: true
  migrate:
    fromDistributed:
      enabled: true
      memberlistService: monitoring-loki-memberlist
  auth_enabled: true
  minio:
    enabled: false
  loki:
    storage:
      bucketNames:
        chunks: monitoring-loki-chunks  # an example bucket name for chunks
        ruler: monitoring-loki-ruler  # an example bucket name for rules
      s3:
        endpoint: https://minio:minio123@minio.minio-tennant.svc.cluster.local/loki
        s3ForcePathStyle: true
    limits_config:
      retention_period: 0  # infinite retention
    compactor:
    retention_enabled: false # infinite retention
global:
  registry: https://hub.docker.com/

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions