Worker deployments have no concurrency limit and will use all CPU cores by default

[Gardelll]

Worker deployments have no concurrency limit and will use all CPU cores by default, which can lead to memory exhaustion on bare metal clusters.

The worker process is started using the following command:

celery -A plane worker -l info

According to the documentation, -c needs to be set to limit the number of concurrent processes.

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

✅ Successfully generated implementation plan.

💡 Use @coderabbitai <feedback> to replan

Examples

• Example 1
• Example 2

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[coderabbitai]

Coding Plan

Approach

Add a configurable Celery worker concurrency limit to prevent memory exhaustion on bare metal clusters where the default behavior (using all CPU cores) can overwhelm system resources.

Observations

The Plane Helm charts consist of two distributions (plane-ce and plane-enterprise) with worker deployments that execute Celery via ./bin/docker-entrypoint-worker.sh. Environment variables are managed through ConfigMaps (app-vars) and Secrets, which are injected into worker pods via envFrom. Currently, there are no Celery-specific concurrency settings defined. The GUNICORN_WORKERS variable serves as a precedent for hardcoded worker configuration in the ConfigMap. Only the worker deployment runs actual Celery worker processes—beat-worker runs the scheduler, while automation-consumer and outbox-poller use custom mechanisms unrelated to Celery workers.

Assumptions

Assumption 1: Will the Celery worker process respect the environment variable without backend code changes?

Options Considered:

1. Use CELERY_WORKER_CONCURRENCY (modern Celery 4.x+ pattern)
2. Use CELERYD_CONCURRENCY (older/legacy pattern)

Chosen Option: 1

Rationale: CELERY_WORKER_CONCURRENCY follows modern Celery conventions and is more likely to be supported in current versions. Celery supports configuration via environment variables following the pattern CELERY_ + uppercase config key. The standard CELERY_WORKER_CONCURRENCY variable should be recognized by Celery automatically when the worker starts.

Assumption 2: What should the default concurrency value be?

Options Considered:

1. Default to "1" (most conservative, matches GUNICORN_WORKERS pattern)
2. Default to "2" or "4" (balanced performance vs. safety)
3. No default, leave empty/optional (maintains current behavior unless explicitly set)

Chosen Option: 1

Rationale: Default to "1" provides the safest baseline preventing memory exhaustion while allowing users to increase as needed for their infrastructure. The ticket indicates memory exhaustion issues from defaulting to all CPU cores.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Add Celery Worker Concurrency Configuration

This phase introduces a configurable concurrency limit for Celery workers across both chart distributions. The configuration follows existing patterns for worker-related settings in values.yaml and environment variable injection via ConfigMaps.

Task 1: Add Concurrency Setting to Plane-CE Values

Add a new configuration option for Celery worker concurrency in the Community Edition chart.

• Add celeryWorkerConcurrency field to the worker section in charts/plane-ce/values.yaml
• Set default value to 1 following the conservative pattern established by GUNICORN_WORKERS
• Position the new field near other worker-related configuration options (replicas, resource limits)

Task 2: Add Concurrency Setting to Plane-Enterprise Values

Add the equivalent configuration option for the Enterprise Edition chart.

• Add celeryWorkerConcurrency field to the services.worker section in charts/plane-enterprise/values.yaml
• Use the same default value of 1 for consistency across editions
• Maintain the nested structure pattern used by enterprise worker configuration

Task 3: Update Plane-CE ConfigMap with Environment Variable

Expose the concurrency setting as an environment variable that Celery workers will consume.

• Update charts/plane-ce/templates/config-secrets/app-env.yaml to add CELERY_WORKER_CONCURRENCY to the ConfigMap data section
• Reference the value from .Values.worker.celeryWorkerConcurrency with appropriate default handling
• Place the new variable near GUNICORN_WORKERS for logical grouping of worker-related settings

Task 4: Update Plane-Enterprise ConfigMap with Environment Variable

Apply the same ConfigMap update for the Enterprise Edition.

• Update charts/plane-enterprise/templates/config-secrets/app-env.yaml to add CELERY_WORKER_CONCURRENCY to the ConfigMap data section
• Reference the value from .Values.services.worker.celeryWorkerConcurrency following the enterprise nested value path pattern
• Ensure consistent variable naming and placement with the CE chart

🤖 Prompt for AI agents

Add configurable Celery worker concurrency limits to both Plane-CE and
Plane-Enterprise Helm charts to prevent memory exhaustion.

**For Plane-CE:**
- Add `celeryWorkerConcurrency: 1` field in the `worker` section of
`charts/plane-ce/values.yaml`
- Add `CELERY_WORKER_CONCURRENCY` environment variable to
`charts/plane-ce/templates/config-secrets/app-env.yaml` ConfigMap, referencing
`.Values.worker.celeryWorkerConcurrency`

**For Plane-Enterprise:**
- Add `celeryWorkerConcurrency: 1` field in the `services.worker` section of
`charts/plane-enterprise/values.yaml`
- Add `CELERY_WORKER_CONCURRENCY` environment variable to
`charts/plane-enterprise/templates/config-secrets/app-env.yaml` ConfigMap,
referencing `.Values.services.worker.celeryWorkerConcurrency`

**Consistency:**
- Use default value of `1` for both editions
- Position fields near existing worker configuration options
- Place environment variable near `GUNICORN_WORKERS` in ConfigMaps

---

🚀 Next Steps

🤖 All AI agent prompts combined

Add configurable Celery worker concurrency limits to both Plane-CE and
Plane-Enterprise Helm charts to prevent memory exhaustion.

**For Plane-CE:**
- Add `celeryWorkerConcurrency: 1` field in the `worker` section of
`charts/plane-ce/values.yaml`
- Add `CELERY_WORKER_CONCURRENCY` environment variable to
`charts/plane-ce/templates/config-secrets/app-env.yaml` ConfigMap, referencing
`.Values.worker.celeryWorkerConcurrency`

**For Plane-Enterprise:**
- Add `celeryWorkerConcurrency: 1` field in the `services.worker` section of
`charts/plane-enterprise/values.yaml`
- Add `CELERY_WORKER_CONCURRENCY` environment variable to
`charts/plane-enterprise/templates/config-secrets/app-env.yaml` ConfigMap,
referencing `.Values.services.worker.celeryWorkerConcurrency`

**Consistency:**
- Use default value of `1` for both editions
- Position fields near existing worker configuration options
- Place environment variable near `GUNICORN_WORKERS` in ConfigMaps

💡 Iterate on the plan with: @coderabbitai <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For assumption 1 go ahead with option 3 and replan.

---

💬 Have feedback or questions? Drop into our discord!

[mguptahub]

@akshat5302 : Please work with @pablohashescobar and get it resolved