[FEATURE] Bidirectional Streaming

[pgrayy]

Overview

Bidirectional streaming enables real-time, continuous communication between clients and AI models in both directions simultaneously. Unlike traditional request-response patterns, this approach allows for simultaneous data exchange where both client and model can send and receive data incrementally. This creates a more natural interaction flow where content is processed as it arrives, without waiting for complete messages, and conversations can adapt dynamically based on ongoing inputs and outputs.

Model Support

Several providers are building models with bidirectional streaming capabilities. Some examples include:

• Amazon: Amazon's Nova Sonic model offers real-time speech processing, interruption handling, context-aware responses, and low latency interactions, making it particularly effective for voice assistants (docs).
• OpenAI: OpenAI also provides models with bidirectional streaming capabilities, allowing for dynamic conversation flows where the model can receive new information while generating a response (announcement).

Request

Support a bidirectional streaming interface in Strands.

Prototype

To help facilitate discussion, we have implemented a prototype for bidirectional streaming under https://github.com/pgrayy/strands-sdk-python-async (see README for instructions on testing). The prototype implements a flexible architecture designed to handle real-time, two-way communication between clients and AI models. The implementation focuses on supporting audio-based interactions with Nova Sonic while establishing patterns that could extend to other models and modalities. The key components are:

• Bidirectional Agent: The Agent class in the bidirectional module provides an async context manager for sending data (send), an async generator for receiving data (receive), and a method to initialize bidirectional streaming (bistream). For example usage, please see https://github.com/pgrayy/sdk-python-async/blob/main/scripts/agents/bidirectional.py.
• Model Sender/Receiver: The abstract Sender and Receiver interfaces define the contract for model providers. The Sender handles outgoing events to the model with context managers for different content types (text, audio, tools), while the Receiver processes incoming events from the model and constructs message history.
• Event System: Events are structured as typed objects representing different kinds of streaming content, including session events (start/end), prompt events (start/end), and content events (text, audio, system, tool).
• Nova Implementation: The Nova implementation demonstrates how to adapt a specific model to the bidirectional interface, providing a concrete example of the architecture in action.

[jsamuel1]

I'd be keen to see how we can use a bidirectional streaming agent in front of a multi-agent flow, and have it receive back async data from multi-agent without "barging in" to the conversation.

[jsamuel1]

#189 related.

[mkmeral]

Fantastic work on the prototype implementation! The Sender/Receiver pattern with async generators is elegant and the Nova integration proves bidirectional streaming works beautifully in practice. I've also built a working demo with Nova Sonic at Strands Live that validates this approach.

Key Insight: Your Patterns Scale Beyond Audio

What's exciting is that your core architectural patterns—bidirectional streaming, concurrent tool execution, event-driven processing—naturally extend beyond voice assistants. I'm seeing architectural convergence across AI domains:

Pattern	Voice AI (Your Nova)	Physical AI (VLA/Robotics)	Unified Primitive
Perception	Audio → Speech	Vision → Processing	Generic InputStream
Reasoning	Nova Sonic	VLA Models	BidirectionalModel
Action	Tool + Speech	Motor Commands	Generic OutputStream
Feedback	Tool Results	Sensor Data	Event loop integration

Proposed Enhancement: Generic Stream Primitives

Building on your excellent foundation, we could extend the architecture to support:

# Your current Nova-specific approach (proven to work!)
async with agent.send() as (prompt_name, sender):
    async with sender.context_audio(prompt_name) as audio_content_name:
        # Audio streaming logic

# Generic extension for multi-modal applications  
StreamingAgent(
    model=BidirectionalModel(),  # Nova, VLA, or custom
    input_streams={"audio": AudioConfig(), "vision": SensorConfig()},
    output_streams={"speech": AudioConfig(), "actions": ActuatorConfig()},
    tools=existing_strands_tools  # Your tool integration works unchanged!
)

Additional Use Cases This Unlocks:

• Real-time robotics with VLA models (Vision-Language-Action)
• Multimodal AI systems combining voice, vision, and physical control
• Smart environments with concurrent audio/sensor processing
• Hybrid workflows (voice-controlled automation, AR assistants)

The architectural foundation you've built is exactly what's needed—we'd just be making the stream primitives more generic to unlock broader applications!