Repository navigation
Proposal: Introduce Categorical Sampling Modes for the Prompt API #203
Description
Activity
Thanks for introducing this topic, @isaacahouma. We'd appreciate developer and WebML CG feedback on this proposal.
Chrome folks would like to experiment along these lines as part of the Sampling Params feature, currently in Origin Trial.
From a DevRel perspective, this is a very welcome simplification of the API that's in line with what I saw developers intend to do with the previously existing raw parameters: either make the model as predictable as possible, or make the model as creative as possible.
With this is mind, and speaking as a non-native learner of English, I was wondering about the exact nuance of the following enum pairs, which sound very similar to me:
Pair 1
- deterministic: For tasks requiring strict consistency and reproducibility (e.g., code generation or factual extraction).
- precise: For highly focused outputs with minimal variation.
Code generation is something I'd like to be precise as well (like when I say write a loop that counts from 1 to 10, it should not end at 8). But code generation isn't necessarily deterministic (as per the Cambridge Dictionary definition: "believing that everything that happens must happen as it does and could not have happened any other way, or relating to this belief"), since you could create the same precise result with a
forloop or awhileloop.Pair 2
- creative: For tasks where variety is preferred over strict factual accuracy.
- imaginative: For maximum token diversity and brainstorming.
These terms are listed as synonyms of each other in the Cambridge Dictionary.
Proposal
Could we maybe simplify the terms, and instead add more steps?
enum AILanguageModelSamplingMode { "most-predictable", "predictable", "balanced", "creative", "most-creative" };@tomayac Your proposal sounds great to me and I will rename the categories as suggested if there are no objections.
Reacted by Thomas Steiner@sushraja-msft, do you think that Edge can implement these modes for the model it is using?
Another bikeshed option:
enum AILanguageModelCreativity { "very-low", "low", "medium", "high", "very-high"};Another bikeshed option:
enum AILanguageModelCreativity { "very-low", "low", "medium", "high", "very-high"};This makes it sound like you're only manipulating one angle, that of creativity, but, based on many conversations with developers, in most folks' minds you're manipulating two angles, predictability vs. creativity.
- added a commit that references this issue
on May 1, 2026 - added 2 commits that reference this issue
on May 1, 2026 @sushraja-msft, do you think that Edge can implement these modes for the model it is using?
Yes shouldn’t be a problem.
- Web developer may need predictable with model A but more predictable with model B.
- developers may set most-predictable and start parsing responses and that can break between browsers.
UA may need to tune the temperature, topk to match Gemini like results. Having fewer classes would make that easier. Is there a reason to have 5, do 3 buckets (Predictable, Balanced, Creative) work for web devs ?
- added a commit that references this issue
on May 8, 2026 1 remaining item
- added 3 commits that reference this issue
on May 12, 2026 Hi Sushanth,
Sorry for the delay!
We are definitely open to reducing the number of categories in the future. For this initial experimentation phase, we wanted to actually see what developers would be using first to inform our decision. By offering a slightly wider spectrum of presets right now (from most-predictable to most-creative), we want collect real-world usage data during the Origin Trial. If the metrics show that developers only really gravitate toward two or three of these modes, we can definitely consolidate and reduce the number of categories before finalizing the API shape. What do you think?
RESOLUTION: Experiment with categorical sampling modes to inform the Prompt API design. (issue #203)
To provide the working group with visibility into how developers are interacting with sampling parameters (such as
temperatureandtop-K), we want to share some aggregated data collected from members of our Early Preview Program (EPP).When surveyed about their current implementations:
- 45% of respondents stated they are actively tuning sampling parameters in their workflows.
For the segment actively tuning parameters, their feedback regarding a potential transition to semantic presets breaks down as follows:
Developer Feedback Cohort Percentage Indicated that predefined presets would likely be acceptable 50% Expressed a definitive need for raw parameter access 28% No strong opinion / Undecided yet 14% Our Approach to Addressing the 28% Cohort
We recognize that moving away from raw numerical scalars represents a significant shift for the 28% of tuning developers who currently rely on fine-grained control. Rather than treating this as a simple feature preference, our goal is to evaluate this constraint against verified production requirements.
In our communications to the preview community, we are actively soliciting objective, historical data from this cohort to map out where semantic presets may fall short. Specifically, we are asking developers who have tested the new presets to provide:
- The specific prompt and required output variance.
- The exact numerical settings (
temperature/top-K) their production implementation relied on. - The specific presets tested and the exact point of failure where the output failed to achieve the verified production result.
Gathering this historical data will allow us to evaluate the viability of the 5-tier preset spectrum against real-world boundaries and help us explore workable alternatives before final standardization.
How to Chime In on This Thread
We welcome broader feedback from the community on this issue. To help us move beyond hypothetical preferences and focus on core architectural needs, we ask that contributions to this thread focus on concrete product requirements and evidence of shortcomings of the presets vs raw access. Thanks!
- added a commit that references this issue
on Jun 16, 2026 When we receive further data from developers (via @michaelwasserman likely) we will revisit this issue again.
Update: Origin Trial Extended through M159 & Expanded Preset Spectrum
Hello,
Following up on the earlier discussions regarding preset granularity and developer feedback, we wanted to share an update on our ongoing experimentation.
To address feedback from developers who felt the initial 5-tier presets had granularity gaps, we have expanded the preset spectrum and extended the Prompt API Sampling Parameters Origin Trial through Chrome M159.
Expanded Preset Spectrum
The expanded 7-tier spectrum of
samplingModepresets aims to better cover granularity in both the predictable and creative spaces while keeping"balanced"aligned with the API default:enum AILanguageModelSamplingMode { "most-predictable", // Strict consistency / factual extraction "predictable", // Highly focused outputs "slightly-predictable",// Moderately focused, consistent outputs "balanced", // Default state for standard prompting "slightly-creative", // Moderately varied, expressive outputs "creative", // Tasks favoring variety over strict facts "most-creative" // Maximum token diversity and brainstorming };
Chromium Issue Tracker
https://issues.chromium.org/issues/502214118
Observations
Currently, the Prompt API does not expose sampling parameters to standard web pages. Previously, raw sampling parameters like
temperatureandtopKwere available, but they were recently deprecated in web page contexts and restricted only to Chrome Extensions (see webmachinelearning/prompt-api#170).This deprecation was a direct result of W3C TAG feedback regarding interoperability (see w3ctag/design-reviews#1093). Raw sampling parameters are model-internal scalars whose behavior drifts silently across different models and versions. Consequently, the same numerical input could produce vastly different behaviors depending on the user's browser and the specific model version they had downloaded, leading to silent breaks in API expectations.
Impact
Completely removing sampling controls limits the usefulness of the Prompt API, as developers lose the ability to fine-tune the model's output variety for specific use cases. Without these controls, developers cannot easily force a model to produce highly deterministic output for factual extraction, nor can they increase variance for creative brainstorming tasks.
Request for Specification
To provide developers with an easy lever for output variety while maintaining cross-browser interoperability, the Prompt API specification should introduce Categorical Sampling Modes.
Instead of passing raw numerical scalars, developers would pass a semantic enum (
samplingMode) during session creation. Each browser vendor would then be responsible for mapping these semantic presets to the optimal raw parameters (e.g.,temperature,topK,topP,minP) for their specific underlying model.We propose the following 5 behavioral categories:
deterministic: For tasks requiring strict consistency and reproducibility (e.g., code generation or factual extraction).precise: For highly focused outputs with minimal variation.balanced: The default state, serving as the "sweet spot" for standard conversational interactions.creative: For tasks where variety is preferred over strict factual accuracy.imaginative: For maximum token diversity and brainstorming.To prevent ambiguity, this parameter should be mutually exclusive with any legacy raw parameters where they still exist. Providing both a
samplingModeand a raw parameter liketemperatureshould result in aTypeError.Example
The proposed Web IDL additions would look like this: