Skip to content

chore: remove query worker deployment #93

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 18, 2025

Conversation

almeidabbm
Copy link
Contributor

@almeidabbm almeidabbm commented Aug 7, 2025

TL;DR

Removed the query worker deployment from the Lightdash Helm chart and bumped chart version.

What changed?

  • Incremented chart version from 1.7.0 to 1.7.2
  • Removed queryWorkerDeployment.yaml template file
  • Removed the queryWorker configuration section from values.yaml
  • Updated the scheduler tasks configuration by removing the exclude: "runAsyncWarehouseQuery" line

How to test?

  1. Install the updated chart and verify that no query worker pods are created
  2. Confirm that the scheduler properly handles all tasks without exclusions
  3. Verify that the application continues to function correctly without the query worker component

Why make this change?

The query worker functionality has likely been consolidated into the main application or scheduler component, eliminating the need for a separate deployment. This simplifies the architecture and reduces resource usage while maintaining the same functionality.

---
# Source: lightdash/templates/workerDeployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-name-lightdash-worker
  labels:
    helm.sh/chart: lightdash-1.7.1
    app.kubernetes.io/name: lightdash
    app.kubernetes.io/instance: release-name
    app.kubernetes.io/version: "0.1121.0"
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: worker
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: lightdash
      app.kubernetes.io/instance: release-name
      app.kubernetes.io/component: worker
  template:
    metadata:
      annotations:
        checksum/config: e34ea7f0e34062e9df2c72775b566b546167408ce7d9e6f41fc087065652d808
        checksum/secrets: 76610acacf72560b5e3057d7c4efad837eb0d46795fbeb27e30f3a6411e1edd7
      labels:
        app.kubernetes.io/name: lightdash
        app.kubernetes.io/instance: release-name
        app.kubernetes.io/component: worker
    spec:
      securityContext:
        {}
      serviceAccountName: release-name-lightdash
      containers:
        - name: lightdash
          securityContext:
            {}
          image: "lightdash/lightdash:0.1121.0"
          imagePullPolicy: IfNotPresent
          command: ["node", "dist/scheduler.js"]
          args: 
          env:
            - name: PGPASSWORD
              valueFrom:
                secretKeyRef:
                  name: release-name-postgresql
                  key: password
            - name: PORT
              value: "8080"
            - name: SCHEDULER_CONCURRENCY
              value: "3"
          envFrom:
            - configMapRef:
                name: release-name-lightdash
            
            - secretRef:
                name: release-name-lightdash
            
          livenessProbe:
            initialDelaySeconds: 10
            timeoutSeconds: 5
            periodSeconds: 10
            httpGet:
              path: /api/v1/health
              port: 8080
          readinessProbe:
            initialDelaySeconds: 35
            periodSeconds: 35
            timeoutSeconds: 30
            httpGet:
              path: /api/v1/health
              port: 8080
          resources:
            requests:
              cpu: 475m
              ephemeral-storage: 1Gi
              memory: 725Mi
      terminationGracePeriodSeconds: 90

Copy link
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@almeidabbm almeidabbm self-assigned this Aug 7, 2025
@almeidabbm almeidabbm requested a review from owlas August 7, 2025 09:20
@almeidabbm almeidabbm marked this pull request as ready for review August 7, 2025 09:20
@almeidabbm almeidabbm force-pushed the 08-07-chore_remove_query_worker_deployment branch from 0be9998 to 9e905ea Compare August 18, 2025 16:51
@almeidabbm almeidabbm merged commit d8ca9a0 into main Aug 18, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant