Repository navigation
@cluster(cores=N) on the local backend does not parallelize anything #152
Description
Summary
@cluster(cores=8) with cluster_type="local" runs the function in the caller's own process, sequentially. cores has no effect. A user who reaches for the local backend expecting a quick way to get multiprocessing gets one core and no warning.
Evidence
from clustrix import cluster, configure
configure(cluster_type="local", cluster_host=None)
@cluster(cores=8)
def work(n):
import os, math
return (os.getpid(), sum(math.sqrt(i) for i in range(n)))
pid, _ = work(4_000_000)this interpreter pid : 23668
pid the work ran in : 23668
same process? : True
The path: _choose_execution_mode (clustrix/decorator.py:395) returns "local" when cluster_host is unset, and the local branch at decorator.py:143 is
# Execute locally without parallelization
return func(*args, **func_kwargs)LocalJobManager.submit_job is no different — local_executor.py:504 builds a pool and never uses it:
executor = LocalExecutor(max_workers=job_config.get("cores"), use_threads=True)
record["result"] = executor.execute_single(func, args, kwargs)and execute_single (local_executor.py:56-69) is a bare return func(*args, **kwargs). Both constructor arguments are dead.
Where the parallelism actually lives
LocalExecutor (local_executor.py:19) is real and works. _create_executor (:43) picks ThreadPoolExecutor or ProcessPoolExecutor from use_threads, and create_local_executor (:434) fills that in via choose_executor_type (:339) when use_threads is None:
pickle.dumpson the function and every argument — any failure selects threads- a lowercased
inspect.getsourcesubstring scan foropen(,requests.,urllib.,http.,ftp.,sql,database,time.sleep,threading.— a hit selects threads - otherwise processes
Measured on 12 cores with 8 workers: CPU-bound work gets 5.59x through processes and 1.00x through threads; I/O-bound work gets 6.03x through threads and 5.92x through processes with warm pools. So the machinery is sound. Nothing in the @cluster local path reaches it.
The one route that does parallelize, and its two problems
parallel=True can reach _execute_local_parallel, but only if find_parallelizable_loops selects a loop AND the function accepts a _parallel_<var> keyword (decorator.py:540, _accepts_chunk_kwargs). Two consequences:
- It almost never fires.
loop_analysis.py:552only accepts a loop whose body reads no name other than the loop variable, which excludes essentially every useful loop. - When it does fire, the caller must cooperate or lose. Each chunk runs the whole function with
_parallel_i=[...]and the function is expected to restrict its own loop to that chunk. A function that ignores the keyword does the full work once per chunk._combine_local_resultsthen concatenates, so the return shape changes.
Why this matters
This is the same shape as the fabricated-result family fixed in #139: the user states an intent, the library accepts it, and silently does something else. Here the cost is performance rather than a wrong answer — but cores=8 returning one core with no warning is not defensible, and the local backend is the first thing anyone tries.
Related: @cluster(cores=8, cluster_type="local") does not do what it looks like either — cluster_type is not a decorator keyword. It lands in **kwargs and warns:
@cluster received unrecognised option(s) cluster_type; they have no effect.
The backend must be set with configure(cluster_type="local"). That warning exists and works; the confusion is worth a documentation fix regardless of what happens here.
Options
- Make
coresmean something locally. A single call tof(x)is one unit of work, so there is nothing to split without either a parallelizable loop or a mapped input. Honest version:cores>1locally is only meaningful withparallel=Trueand a cooperating function. - Wire
LocalJobManagerto the pool it already builds — at minimum stop constructing aLocalExecutorwhose arguments are dead. - Warn when
cores>1is passed to a path that ignores it, in the same spirit as the unrecognised-kwarg warning. - Fix the loop detector so the parallel path is reachable for realistic code (that is
#132's territory).
Option 3 is the minimum: silence is what makes this a defect rather than a limitation.
Definition of done
-
coreseither changes local behaviour or produces a warning saying it does not -
LocalJobManager.submit_jobhas no dead arguments - A test asserts the observed degree of parallelism for the local backend, rather than asserting the call returned
- The documentation states what the local backend does, with a measured comparison rather than a claim
Activity
jeremymanning commentedon Aug 20, 2026
Tracked as a sub-issue of #159 (defect in current functionality). Consolidated after a three-agent audit that verified every open issue against the code; see #159 for the plan and the ordering.
jeremymanning commentedon Aug 20, 2026
Fixed — five rounds, four adversarial reviews
The reported defect, and the one underneath it
@cluster(cores=N)on the local backend did not parallelize. True — and the bound was unreachable even where parallelization did happen:_create_local_work_chunkscut work intoos.cpu_count() * 2pieces regardless ofcores.With
os.cpu_countpinned to 2:coresnow bounds the local pool, the chunk count follows the pool rather than the machine, and wherecorescannot apply the decorator says so instead of staying quiet.Invalid values are refused at decoration rather than absorbed:
cores=0,cores=-2,cores=1.5,cores="8"and — less obviously —cores=True, becauseboolsubclassesintand would otherwise have been read as one worker.LocalExecutoragrees at its own door.What adversarial review found that the tests did not
The first fix was correct and its headline change could be reverted with the entire suite green. Deleting the
max_workersargument at the call site restored the original bug; 1717 tests still passed. The proving test was parametrized[2, 4], and both values are ≤cpu_counton any normal machine, so the old broken chunking still supplied enough chunks to fill the pool.That set the standard for the rest: every subsequent round re-ran the specific mutants that had previously survived, rather than accepting a green suite as evidence.
max_workersat the call site// (workers * 2)→// workerscores or SHIPPED_DEFAULT_CORES(theconfigure(default_cores=N)route)chunks[::-1]— result order<= WARNING→<= ERRORpropagatebreaklastResort→return TrueThe warning had to be delivered, not merely emitted
A first attempt throttled the warning per function per reason — and recorded the throttle at emit-attempt. If the first call happened before logging was configured, the only warning was discarded and 50 later calls were silent. Measured: 0 warnings delivered.
Gating on delivery then required predicting what
loggingwill do. Compared against realloggingacross 20 configurations, the implementation initially disagreed on three. Filters are now handled by failing safe rather than by prediction — running a filter to decide would execute it twice per delivered message and corrupt any filter that counts or rate-limits.The structure matters and is easy to get backwards:
logger.warningsits outside the gate, which guards only the throttle bookkeeping. So a filter never silences the warning — it un-throttles it. A filter on the root logger cannot silence clustrix at all, becausecallHandlersignores ancestor logger filters.Correctness, verified rather than assumed
63 fresh combinations of loop length ×
cores— including lengths shorter than the pool, not divisible by it, andcoresfar exceeding the range — all returned the same answer as the undecorated call.The returned shape does depend on
cores, and that is now pinned with exact values rather than types:Documented in
limitations.rst, which also had a now-false claim corrected: chunk count followscores, notos.cpu_count(). What that shape should be is a separate design question, filed as #170; it is deliberately unchanged here because altering it is a breaking change.Two claims corrected rather than left standing
4126a03's message says the width test "passes at cores of 2, 4 and 8". 8 was not in the shipped parametrize. It is now, and the discrepancy is recorded.limitations.rstclaimed a run-time bound such asrange(n)is declined. False —SafeRangeEvaluatorresolves names from local variables. Onlyrange(len(data)), an expression bound, a closure and a module global are genuinely declined; each was verified individually.Also fixed in passing
psutilandtraitletswere undeclared test dependencies, so a cleanpip install -e ".[dev]"could not pass the suite.jeremymanning commentedon Aug 22, 2026
Verified on the merged v0.2.0 tree — the fix survived all six branch merges:
(within the full non-billable run: 2760 passed / 0 failed).
@cluster(cores=N)on the local backend now bounds the worker pool, and a core count that cannot do anything is reported instead of dropped.Issue #152: cores either sizes the local pool or says it is being ign…
Issue #152: cores bounds the pool, and the bound is now reachable
Issue #166: the documentation checker now opens the notebooks
Issue #152: the cores fix now fails when it is broken
Issue #152: the config route into the pool, and a warning nobody heard
Issue #152: a warning a NullHandler swallowed, and a shape nobody pinned
Issue #152: the three gates nobody pinned, and a filter that ate the …
Issue #152: the mutant that turned repetition into silence, and the m…
Issue #127: re-execute the local-parallel notebook so its stored outp…