June SWML updates

[hey-august]

This ticket collects updates to SWML to be handed off to Muriel at the start of June. The assigned action item is the creation of the handoff ticket in the appropriate Product repository.

Voice selection

Want an update to the voice selection flow. The current UI is:

• Language (long dropdown list)
• Voice (long dropdown list)
• Fillers (text box)

We should more closely match the SWML interface:

• Engine - dropdown menu, since this is a well-defined and short list
• Model (optional text box)
• Voice (I could see either a dropdown or a text box - frankly, a text box is probably the better option.
• Remove Fillers, this parameter is deprecated in favor of speech_fillers and function_fillers

I also recommend changing the tab from "Languages" to "Voices and Languages"

[Devon-White]

Params

Parameter	Description	Value Type	Valid Values/Constraints	Default	Required
ai_model	The AI model that the AI Agent will use during the conversation.	string	gpt-4o-mini, gpt-4.1-mini, gpt-4.1-nano	gpt-4o-mini	False
enable_thinking	Enables thinking output for the AI Agent. When set to true, the AI Agent will be able to utilize thinking capabilities.	boolean	true, false	false	False
enable_vision	Enables visual input processing for the AI Agent. The image that will be used for visual processing will be gathered from the users camera if video is available on the call. When set to true, the AI Agent will be able to utilize visual processing capabilities, while leveraging the get_visual_input function.	boolean	true, false	false	False
initial_sleep_ms	Amount of time, in ms, to wait before the AI Agent starts processing.	integer	Min: 0, Max: 300,000	0	False
thinking_model	The AI model that the AI Agent will use when utilizing thinking capabilities.	string	Same as ai_model	Value of ai_model	False
video_listening_file	URL of a video file to play when AI is listening to the user speak. Only works for calls that support video.	string	Valid URL	None	False
video_idle_file	URL of a video file to play when AI is idle. Only works for calls that support video.	string	Valid URL	None	False
video_talking_file	URL of a video file to play when AI is talking. Only works for calls that support video.	string	Valid URL	None	False
vision_model	The AI model that the AI Agent will use for vision capabilities.	string	Same as ai_model	Value of ai_model	False

[Devon-White]

Languages

Parameter	Description	Value Type	Valid Values/Constraints	Default	Required
emotion	Enables emotion for the set TTS engine. This allows the AI to express emotions when speaking. A global emotion or specific emotions for certain topics can be set within the prompt of the AI. IMPORTANT: Only works with Cartesia TTS engine.	string	auto	auto	False
speed	The speed to use for the specified TTS engine. This allows the AI to speak at a different speed at different points in the conversation. The speed behavior can be defined in the prompt of the AI. IMPORTANT: Only works with Cartesia TTS engine.	string	auto	auto	False
name	User defined name for the voice	string	Any string	None	True
model	The model to use for the specified TTS engine (e.g. arcana).	string	See link	None	False
engine DEPRECATED	The engine to use for the language. For example, "elevenlabs". Deprecated. Set the engine with the voice parameter.	string	See link	gcloud	False
function_fillers	An array of strings to be used as fillers in the conversation when the agent is calling a SWAIG function. The filler is played asynchronously during the function call.	string[]	Any String	None	False
speech_fillers	An array of strings to be used as fillers in the conversation. This helps the AI break silence between responses.	string[]	Any string	None	False
fillers DEPRECATED	An array of strings to be used as fillers in the conversation and when the agent is calling a SWAIG function. Deprecated: Use speech_fillers and function_fillers instead.	string[]	Any string	None	False

[Devon-White]

Hints

Hints can now also be objects (can be string or object)

New Hint object:

Parameter	Description	Value Type	Valid Values/Constraints	Default	Required
hint	The hint to match. This will match the string exactly as provided.	string	Any string	None	True
pattern	A regular expression to match the hint against. This will ensure that the hint has a valid matching pattern before being replaced.	string	Any string	None	True
replace	The text to replace the hint with. This will replace the portion of the hint that matches the pattern.	string	Any string	None	True
ignore_case	If true, the hint will be matched in a case-insensitive manner.	boolean	false | true	false	False

[Devon-White]

Configuration

Missing all of POM & context.steps

POM

Parameter	Description	Value Type	Valid Values/Constraints	Default	Required
pom	An array of objects that defines the prompt object model (POM) for the AI. Each object represents a section in the POM.	object[]	POM parameters	None	False
pom.title	The title of the section. Will be a heading in the rendered prompt.	string	Any string	None	False
pom.body	The body of the section. This will be a paragraph in the rendered prompt. This parameter is required if bullets is not present.	string	Any string	None	Conditionally Required
pom.bullets	An array of strings that represent the bullets of the section. This will be a list of short statements/rules in the rendered prompt. This parameter is optional if body is present.	string[]	Any string	None	Conditionally Required
pom.subsections	An array of objects that defines the prompt object model (POM) for the AI. Each object represents a section in the POM allowing the users to nest sections within sections.	object[]	Any valid POM object/section	None	False
pom.numbered	If true, the section will be numbered in the rendered prompt.	boolean	true, false	false	False
pom.numberedBullets	If true, the bullets will be numbered in the rendered prompt.	boolean	true, false	false	False

Contexts

Parameter	Description	Value Type	Valid Values/Constraints	Default	Required
contexts.default	The default context to use at the beginning of the conversation. This object accepts the steps object.	object	Steps object	None	True
contexts.[key: string]	An additional context steps to define the flow of the conversation. This is a key that is user-defined and can be any string. Additional context can be swapped to from the default context or any other user-defined context. The value is an object that accepts the steps object.	object	Steps object	None	False
contexts.steps	An array of objects that define the steps in the context. These steps are used to define the flow of the conversation.	object[]	Steps parameters	None	True
contexts.steps.name	The name of the step. The name must be unique within the context. The name is used for referencing the step in the context.	string	Any string	None	True
contexts.steps.text	The prompt or instructions given to the AI at this step.	string	Any string	None	True
contexts.steps.end	A boolean value that determines if the step is the last in the context. If true, the context ends after this step. Important: This cannot be used along with the valid_steps parameter.	boolean	true, false	false	False
contexts.steps.functions	An array of strings, where each string is the name of a SWAIG.function that can be executed from this step.	string[]	Any string	None	False
contexts.steps.step_criteria	The criteria that must be met for the AI to proceed to the next step. The criteria is a instruction given to the AI. It's highly recommended you create a custom criteria for the step to get the intended behavior.	string	Any string	None	False
contexts.steps.skip_user_turn	A boolean value, if set to true, will skip the user's turn to respond in the conversation and proceed to the next step.	boolean	true, false	false	False
contexts.steps.valid_contexts	An array of context names that the AI can transition to from this step. This must be a valid contexts.name that is present in your contexts object.	string[]	Valid context names	None	False
contexts.steps.valid_steps	An array of valid step names that the conversation can proceed to from this step. If the array is empty, or the valid_steps key is not present, the conversation will proceed to the next step in the context. Important: This cannot be used along with the end parameter.	string[]	Valid step names	Proceeds to the next step	False

[Devon-White]

Functions

Parameter	Description	Value Type	Valid Values/Constraints	Default	Required
native_functions	Prebuilt functions the AI agent is able to call (from this list of available native functions)	string[]	check_time | wait_seconds	None	False

Functions.Functions (naming is a bit weird here - also technically incorrect. Should be functions.function)

Parameter	Description	Value Type	Valid Values/Constraints	Default	Required
data_map	An object containing properties to process or validate the input, perform actions based on the input, or connect to external APIs or services in a serverless fashion. See the docs for more info	object	Valid data Map object	None	False
description	A description of the context and purpose of the function, to explain to the agent when to use it. Replaces the deprecated purpose parameter.	string	Any string	None	True
active	Whether the function starts in an enabled state or not.	boolean	true | false	true	False
parameters	A JSON object that defines the expected user input parameters and their validation rules for the function. Replaces the deprecated argument parameter.	object	Any valid JSON Schema definition for params	None	False
fillers	A JSON object defining the fillers that should be played when calling a swaig function. This helps the AI break silence between responses. The filler is played asynchronously during the function call. Learn more at fillers docs	object	Valid fillers object	None	False
meta_data	A powerful and flexible environmental variable which can accept arbitrary data that is set initially in the SWML script or from the SWML set_meta_data action. This data can be referenced locally to the function. All contained information can be accessed and expanded within the prompt - for example, by using a template string.	object	Any object	None	False
meta_data_token	Scoping token for meta_data. If not supplied, metadata will be scoped to function's web_hook_url.	string	Any string	Set by SignalWire	False
wait_file	A file to play while the function is running. wait_file_loops can specify the amount of times that files should continuously play.	string	Any valid URL that returns a media file	None	False
wait_file_loops	The amount of times that wait_file should continuously play/loop.	string | integer	Any string or integer	None	False
wait_for_fillers	Whether to wait for fillers to finish playing before continuing with the function.	boolean	true | false	false	False

Reserved Functions

Reference

These are Pre-defined functions on the SignalWire platform that will execute at certain points in the call. They are defined EXACTLY like a regular SWAIG function, but the name acts as a reserved key. When a user defined a function with one of the below names, the function will act like a reserved function. Perhaps we should display these and what they do?

Function Name	Purpose
start_hook	Triggered when the call is answered. Sends the set properties of the function to the defined web_hook_url.
stop_hook	Triggered when the call is ended. Sends the set properties of the function to the defined web_hook_url.
summarize_conversation	Triggered when the call is ended. The post_prompt must be defined for this function to be triggered. Provides a summary of the conversation and any set properties to the defined web_hook_url.

[Devon-White]

Any additional changes can be found here: https://github.com/signalwire/swml-schema-ts/releases

I think the above covers most, but there might be some already existent params that were updated some time ago, so might be worth to double check the releases for this.