Skip to content

prewarm reusable env#1099

Open
dansola wants to merge 4 commits into
mainfrom
prewarm
Open

prewarm reusable env#1099
dansola wants to merge 4 commits into
mainfrom
prewarm

Conversation

@dansola
Copy link
Copy Markdown
Contributor

@dansola dansola commented May 21, 2026

Motivation

When a reusable TaskEnvironment is first invoked, the backend lazily spawns its actor pool — pulling images, running init containers, starting workers. Subsequent tasks land on the warm pool, but the first one pays the full cold start. Users who know they'll hit a reusable env later in a workflow have no way to ask the SDK to start that pool earlier; today they have to author a hand-rolled dummy task and fire it from the driver. SE-760 adds a first-class API for it.

Summary

Adds TaskEnvironment.prewarm() (async) and TaskEnvironment.prewarm_sync() (matching download_sync / load_sync convention). Calling either submits a hidden no-op sub-action on the reusable env, which triggers GetOrCreateEnvironment on the backend and spins up min_replica_count workers. The heavy task — invoked later in the driver — lands on the warm pool instead of paying the cold start.

  • await env.prewarm() blocks until the noo HEALTHY on return) — standard Python async semantics.
  • asyncio.create_task(env.prewarm()) is the pool warms concurrently with whatever async work the driver does next.
  • env.prewarm_sync() blocks until completiions.

The synthesized no-op is a real task registen time (__post_init__ calls _register_prewarm_task when reusable is not None). It shares the env's image, ReusePolicy, env_ve so its actor version hash matches the env's real tasks — without this match, the warm-up would land on a different pool a PrewarmTaskResolver materializes the task on the worker without touching user code, so the no-op is loadable even when thee bundle that doesn't define it explicitly.

Per-env naming (<env>.prewarm_<env>, prefix form) makes prewarm sub-actions sort together in the UI. prewarm() logs thfired so users can see the warm window they're operating in.

No-op cases (all warn and return):

  • prewarm() on a non-reusable env
  • Running in local execution mode
  • Called outside a task / run context

Example Executions

Without Prewarm: https://union-internal.hosted.unionai.cloud/v2/domain/development/project/flytesnacks/runs/ux2bz5jljgkhlrxqt9st

With Prewarm: https://union-internal.hosted.unionai.cloud/v2/domain/development/project/flytesnacks/runs/uff6qnnx59wjzvx5f6kz

Comment thread examples/reuse/prewarm.py
# reference (see Python's background-tasks idiom) so the task can't be
# GC'd mid-execution; in this short driver the local frame is alive for
# the whole duration so it's safe to discard.
asyncio.create_task(heavy_env.prewarm()) # noqa: RUF006
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks cool, but you need to have one await before it really triggers sometime

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants