You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bug: five settings are accepted and persisted but have no runtime effect #10840
Five settings can be set, are persisted, and read back correctly via the API — but have no effect at runtime. They fail silently, which makes them worse than a missing feature: an operator configures them, verifies they are stored, and reasonably concludes the system is protected.
Setting
Level
Observed effect
Error handler
Workspace
none — never fires
on_failure
Schedule
none — never fires
concurrent_limit
Script and Flow
none — no limiting observed
cache_ttl
Flow step
none (on a script it works correctly)
delete_after_secs
Flow
none — nothing is deleted
Filing these together because the pattern is the actual finding — there is no obvious rule that predicts which fields are wired up. Happy to split into separate issues if that is easier to triage.
To reproduce
Each was tested with a control run — once with the setting, once without — since testing only the configured case measures the environment rather than the setting.
delete_after_secs is the clearest to reproduce: set it to 30 on a flow, deploy, confirm via API that delete_after_secs = 30 is returned. Run the flow. After 160 seconds: flow job present, result present, step job present, flow status present. Nothing deleted, nothing redacted.
concurrent_limit: set on both a script and a flow, then start more runs than the limit allows. All run concurrently.
cache_ttl: has no effect when set on a flow step; the same value in a script's script.yaml correctly freezes results.
Error handler / on_failure: neither the workspace error handler nor a schedule's on_failure runs when a job fails. Note that the error_handler_path on an HTTP trigger does fire correctly with the same bare path format (f/foo/handler), so path formatting is not the cause here.
Expected behavior
Either the settings take effect, or they are rejected at write time.
Screenshots
No response
Browser information
Not applicable — reproduced via API.
Application version
CE v1.792.2 (self-hosted, Docker Compose)
Additional Context
For contrast, these were verified to work correctly in the same test pass: timeout, retry, suspend, mock, continue_on_error and skip_failures on a step, same_worker on a flow, and cache_ttl on a script.
There is no discernible pattern — it is not "all flow-level fields" or "all workspace settings". same_worker is flow-level and works; concurrent_limit is script-level and does not.
Possibly related to #7311 and #5453 for the error-handler part.
Re-verified on CE v1.796.0-12-gffdf17ef8d (image ghcr.io/windmill-labs/windmill:main, digest sha256:f8513becee64ac47b0099a84adde4bb90962f396768070671b0ab9790e343f87), on a fresh stack from the repo's own docker-compose.yml. Two of the five confirmed still present, this time with a control run.
concurrent_limit — still no effect, and not because of a missing time window
The obvious objection to the original report was that concurrency_time_window_s wasn't set. It was tested both ways, plus a control with no limit at all. Script sleeps 5s; three runs fired together; three default workers available, so real parallelism is possible.
All three durations were ~5040 ms and the start timestamps land within 100 ms of each other in every case. A and B are indistinguishable from C.
delete_after_secs — still no effect
Set to 30 on a flow, confirmed via GET /flows/get/{path} that delete_after_secs=30 came back. Ran the flow, waited 170 s, then GET /jobs_u/get/{id}:
HTTP 200 — type=CompletedJob, result={'ok': True}
Nothing deleted, nothing redacted.
Method note
Worth flagging for anyone reproducing this: a newly created script is not immediately runnable. Windmill starts an automatic dependencies job on create, and runs fired before it finishes never execute. In a first attempt the only "completed" job was the lock job (Successful lock file generation, 48 ms), which is easy to mistake for the script having run. Waiting for a successful trial run before measuring avoids this.
Happy to split this issue per setting if that's easier to triage, and to test the remaining three (cache_ttl on a flow step, workspace error handler, schedule on_failure) the same way.
Thanks for the careful writeup, and in particular for the control runs.
Four of the five are behaving as designed: they are Enterprise Edition features, and CE builds compile the enforcement out entirely. The fifth, cache_ttl on a flow step, does not reproduce for me.
I checked against a CE build of main (no enterprise cargo feature, same shape as the published CE image).
both halves are EE: the flow-completion block that schedules the deletion (backend/windmill-worker/src/worker_flow.rs:1833) and the sweeper cleanup_scheduled_job_deletions (backend/src/monitor.rs:1863)
Details from the repro:
concurrent_limit: on CE the server does log this, at ERROR level, once per pulled job:
Concurrent limits are an EE feature only, ignoring constraints
delete_after_secs: on CE nothing is even queued for deletion. After a flow run with delete_after_secs: 15, job_delete_schedule stayed empty and the result was intact well past the deadline. Worth knowing: delete_after_secs: 0 (immediate deletion, the old delete_after_use behaviour) is inline in the flow-completion path and does work on CE.
Workspace error handler: same, no handler job pushed after a failing run. Workspace settings does show an "Workspace error/success handler is an EE feature" banner in the UI, but POST /workspaces/edit_error_handler returns 200 on CE and persists the value.
Your note that error_handler_path on an HTTP trigger fires correctly is right and consistent: trigger-level error handlers live in windmill-trigger and have no EE gate. Different code path from the workspace and schedule handlers.
cache_ttl on a flow step: could not reproduce
On the same CE build, a step with cache_ttl: 300 returned an identical Math.random() value across three runs, in all three shapes:
inline (rawscript) step with cache_ttl set on the module
step referencing a workspace script that has cache_ttl set on the script
step referencing a workspace script with cache_ttl overridden on the module
The cache key is (job kind, script hash or path+code, hashed args). So a step whose resolved inputs differ between runs (anything derived from flow_input, a previous step's result, or a timestamp) misses every time even though the setting is live, which is the most likely explanation. If you still see it with args that are identical run to run, please open a separate issue with the flow value and I will dig in, that would be a real bug.
On the actual finding
You are right that there is no rule you could have inferred, and that is the part worth fixing. The UI gates all four (eeOnly toggles, disabled inputs, the EE banner in workspace settings), but the API accepts and persists them with a 200 on CE, so configuring through the API or the CLI gives no signal whatsoever. I will look at surfacing this at write time on CE rather than leaving it to be discovered by measurement.
Filing it as one pattern rather than five reports was the right call, and the control runs are what made it quick to triage. Thank you.
Activity
S-a-r-i-u-s commentedon Aug 26, 2026
Re-verified on
CE v1.796.0-12-gffdf17ef8d(imageghcr.io/windmill-labs/windmill:main, digestsha256:f8513becee64ac47b0099a84adde4bb90962f396768070671b0ab9790e343f87), on a fresh stack from the repo's owndocker-compose.yml. Two of the five confirmed still present, this time with a control run.concurrent_limit— still no effect, and not because of a missing time windowThe obvious objection to the original report was that
concurrency_time_window_swasn't set. It was tested both ways, plus a control with no limit at all. Script sleeps 5s; three runs fired together; threedefaultworkers available, so real parallelism is possible.concurrent_limit=1, no windowconcurrent_limit=1,window=Noneconcurrent_limit=1,concurrency_time_window_s=60concurrent_limit=1,window=60All three durations were ~5040 ms and the start timestamps land within 100 ms of each other in every case. A and B are indistinguishable from C.
delete_after_secs— still no effectSet to
30on a flow, confirmed viaGET /flows/get/{path}thatdelete_after_secs=30came back. Ran the flow, waited 170 s, thenGET /jobs_u/get/{id}:Nothing deleted, nothing redacted.
Method note
Worth flagging for anyone reproducing this: a newly created script is not immediately runnable. Windmill starts an automatic
dependenciesjob on create, and runs fired before it finishes never execute. In a first attempt the only "completed" job was the lock job (Successful lock file generation, 48 ms), which is easy to mistake for the script having run. Waiting for a successful trial run before measuring avoids this.Happy to split this issue per setting if that's easier to triage, and to test the remaining three (
cache_ttlon a flow step, workspace error handler, scheduleon_failure) the same way.rubenfiszel commentedon Aug 26, 2026
Thanks for the careful writeup, and in particular for the control runs.
Four of the five are behaving as designed: they are Enterprise Edition features, and CE builds compile the enforcement out entirely. The fifth,
cache_ttlon a flow step, does not reproduce for me.I checked against a CE build of
main(noenterprisecargo feature, same shape as the published CE image).The four EE ones
apply_completed_job_error_handlers, behind#[cfg(all(feature = "enterprise", feature = "private"))]—backend/windmill-queue/src/jobs.rs:1089on_failure(alsoon_success/on_recovery)apply_schedule_handlers, same gate —backend/windmill-queue/src/jobs.rs:1424concurrent_limit/concurrency_time_window_s#[cfg(not(feature = "enterprise"))]branch —backend/windmill-queue/src/jobs.rs:4247delete_after_secs> 0backend/windmill-worker/src/worker_flow.rs:1833) and the sweepercleanup_scheduled_job_deletions(backend/src/monitor.rs:1863)Details from the repro:
concurrent_limit: on CE the server does log this, at ERROR level, once per pulled job:delete_after_secs: on CE nothing is even queued for deletion. After a flow run withdelete_after_secs: 15,job_delete_schedulestayed empty and the result was intact well past the deadline. Worth knowing:delete_after_secs: 0(immediate deletion, the olddelete_after_usebehaviour) is inline in the flow-completion path and does work on CE.on_failure: confirmed with a0 * * * * *schedule on a throwing script. The scheduled job failed, no handler job was pushed. Documented as EE: https://www.windmill.dev/docs/core_concepts/error_handlingPOST /workspaces/edit_error_handlerreturns 200 on CE and persists the value.error_handler_pathon an HTTP trigger fires correctly is right and consistent: trigger-level error handlers live inwindmill-triggerand have no EE gate. Different code path from the workspace and schedule handlers.cache_ttlon a flow step: could not reproduceOn the same CE build, a step with
cache_ttl: 300returned an identicalMath.random()value across three runs, in all three shapes:rawscript) step withcache_ttlset on the modulecache_ttlset on the scriptcache_ttloverridden on the moduleThe cache key is
(job kind, script hash or path+code, hashed args). So a step whose resolved inputs differ between runs (anything derived fromflow_input, a previous step's result, or a timestamp) misses every time even though the setting is live, which is the most likely explanation. If you still see it with args that are identical run to run, please open a separate issue with the flow value and I will dig in, that would be a real bug.On the actual finding
You are right that there is no rule you could have inferred, and that is the part worth fixing. The UI gates all four (
eeOnlytoggles, disabled inputs, the EE banner in workspace settings), but the API accepts and persists them with a 200 on CE, so configuring through the API or the CLI gives no signal whatsoever. I will look at surfacing this at write time on CE rather than leaving it to be discovered by measurement.Filing it as one pattern rather than five reports was the right call, and the control runs are what made it quick to triage. Thank you.