Skip to content

Commit

Permalink
feat: demo setting n_workers
Browse files Browse the repository at this point in the history
  • Loading branch information
tekumara committed May 8, 2024
1 parent f9cbcaa commit 984d53b
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion flows/dask_flow.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@ def say_goodbye(name: str) -> None:

# TODO: can the task runner be parameterised so we don't duplicate the flow with dask_kubes_flow?
# see https://github.com/PrefectHQ/prefect/issues/5560
@flow(task_runner=DaskTaskRunner())


# creates a LocalCluster https://docs.dask.org/en/stable/deploying-python.html#localcluster
@flow(task_runner=DaskTaskRunner(cluster_kwargs={"n_workers": 2}))
def dask(names: list[str]) -> None:
for name in names:
# tasks must be submitted to run on dask
Expand Down

0 comments on commit 984d53b

Please sign in to comment.