Skip to content

Historical resource scheduler: hints by digest keys, size classes with a cold-start policy, and where each reservation came from - #2832

Merged
amankrx merged 4 commits into
TraceMachina:mainfrom
amankrx:pr/historical-resource-scheduler
Sep 30, 2026
Merged

amankrx merged 4 commits into
TraceMachina:mainfrom
amankrx:pr/historical-resource-scheduler

Conversation

@amankrx

@amankrx amankrx commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

What and why

The historical_resource scheduler sized only actions with a hint keyed by target, so anything else ran with whatever the client sent, and a producer fed by origin events had no key it could write. Hints are now read by a ladder of keys: target and mnemonic, target, the action digest, the command digest, then the mnemonic alone, so a hint reaches every client, with or without Bazel request metadata. classes names the fleet's size ladder, a hint may name a class instead of numbers, and an action no hint matches takes the class picked by class_by_mnemonic, then class_by_timeout (Bazel test sizes arrive as timeouts), then default_class; without classes the raw cold_start numbers fill only the properties the client left out. A scheduler.hints.resolutions counter says where each action's reservation came from, so a hint feed can be judged by how far it drives the cold_start share down. A record is indexed under every key family it names, so one carrying a target and a digest is found by either; each ladder step is probed only when the action carries that key, so a mnemonic alone never outranks a digest hint; a hint that reserves nothing falls to the cold start and is counted there; the cold-start rules take the first named class that exists, and a rule naming a class the ladder lacks is refused at load, as is a reserved property the nested scheduler does not declare as a minimum; an unknown class in a hint is dropped once at load, not logged per action; and cold_start reserves disk too.

How was this verified?

historical_resource_scheduler_test: a command digest hint matches an action without metadata, an action digest hint matches only its action, classes resolve hints and the cold start, the cold start fills only absent properties, a record with a target and a digest is found by either, a mnemonic alone does not outrank a digest hint, an all-zero hint falls to the cold start, and a rule naming no class is refused while the default class stands in. Older hint files still load, since every new field is optional. The config reference is regenerated and the snippet lint passes.

Risk

Nothing changes for a scheduler without classes or cold_start: the target keys resolve as before and the digest keys only add matches. It reads ladder_kb from #2829, which is merged.

AI assistance

An agent drafted the change and I reviewed every line.


This change is Reviewable

…h a cold-start policy, and where each reservation came from
@vercel

vercel Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
nativelink Ready Ready Preview Sep 30, 2026 3:55am UTC
nativelink-aidm Ready Ready Preview Sep 30, 2026 3:55am UTC

Request Review

…rce-scheduler

# Conflicts:
#	web/apps/docs/content/docs/reference/nativelink-config/main.mdx
Comment thread nativelink-scheduler/src/historical_resource_scheduler.rs Outdated
Comment thread nativelink-scheduler/src/historical_resource_scheduler.rs Outdated
Comment thread nativelink-scheduler/src/historical_resource_scheduler.rs Outdated
Comment thread nativelink-scheduler/src/historical_resource_scheduler.rs Outdated
Comment thread nativelink-scheduler/src/historical_resource_scheduler.rs
Comment thread nativelink-config/src/schedulers.rs
Comment thread nativelink-scheduler/src/historical_resource_scheduler.rs Outdated
…e action carries, zero hints fall to cold start, rules and property types checked at load, disk on the cold start
@amankrx
amankrx merged commit 59cbad2 into TraceMachina:main Sep 30, 2026
47 of 49 checks passed

This branch was successfully deployed

2 active deployments
Preview – nativelink — 17a4d0d6 Deployed Sep 30, 2026 by vercel[bot]
Preview – nativelink-aidm — 17a4d0d6 Deployed Sep 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants