README.md

Getting Started   •   Create Extensions   •   TEN Framework Repository

🎉 TEN Agent with OpenAI Realtime API and RTC

Try OpenAI Realtime API and RTC at agent.theten.ai.

Combining OpenAI Realtime API for ultra-low latency with RTC’s AI noise suppression ensures smooth, high-quality interactions. On top of that, the seamless integration of weather and news tools makes TEN Agent even more versatile.

TEN Agent Features

• OpenAI Realtime API and RTC integration: TEN Agent is the world-class multimodal AI agent to integrate the OpenAI Realtime API and RTC.

• High-Performance Real-Time Multimodal Interactions: Offers high-performance, low-latency solutions for complex audio-visual AI applications.

• Multi-Language and Multi-Platform Support : Supports extension development in C++, Go, Python, etc. Runs on Windows, Mac, Linux, and mobile devices.

• Edge-Cloud Integration: Flexibly combines edge and cloud-deployed extensions, balancing privacy, cost, and performance.

• Flexibility Beyond Model Limitations: Easily build complex AI applications through simple drag-and-drop programming, integrating audio-visual tools, databases, RAG, and more.

• Real-Time Agent State Management: Manages and adjusts agent behavior in real-time for dynamic responsiveness.

Ready-to-use Extensions

Stay Tuned

Before we get started, be sure to star our repository and get instant notifications for all new releases!

How to build TEN Agent locally

Prerequisites

Keys

• Agora App ID and App Certificate (certificate only required if enabled in the Agora Console)
• Azure SST(ASR) and TTS API keys
• OpenAI API key

Installation

• Docker / Docker Compose
• Node.js(LTS) v18

Minimum system requirements

• CPU >= 2 Core
• RAM >= 4 GB

Docker setting on Apple Silicon

You will need to uncheck "Use Rosetta for x86_64/amd64 emulation on Apple Silicon" option for Docker if you are on Apple Silicon, otherwise the server is not going to work.

Next step

1. Modify config files

In the root of the project, use cp command to create .env from the .env.example .

It will be used to store information for docker compose later.

cp ./.env.example ./.env

2. Setup API keys

Open the .env file and fill in the keys and regions.

# Agora App ID 
# Agora App Certificate(only required if enabled in the Agora Console)
AGORA_APP_ID=
AGORA_APP_CERTIFICATE=

# Azure STT key and region
AZURE_STT_KEY=
AZURE_STT_REGION=

# Azure TTS key and region
AZURE_TTS_KEY=
AZURE_TTS_REGION=

# OpenAI API key
OPENAI_API_KEY=

3. Start agent development containers

In the same directory, run the docker compose up command to compose containers:

docker compose up

Or using the docker compose up -d command, start the container in detached mode.(more here)

docker compose up -d

4. Enter container and build agent

Open up a separate terminal window, enter the container and build the agent:

docker exec -it ten_agent_dev bash

make build

5. Start the server

Once the build is done, make run-server on port 8080:

make run-server

Finish and verify

TEN Agent

Open up localhost:3000 in browser to play the TEN Agent.

TEN Graph Designer

Open up another tab go to localhost:3001, and use Graph Designer to create, connect and edit extensions on canvas.

Once you save the graph, you can return to localhost:3000 and select the corresponding graph to view the changes.

Join Community

• Discord: Ideal for sharing your applications and engaging with the community.
• GitHub Discussion: Perfect for providing feedback and asking questions.
• GitHub Issues: Best for reporting bugs and proposing new features. Refer to our contribution guidelines for more details.
• X (formerly Twitter): Great for sharing your agents and interacting with the community.

Code Contributors

Contribution Guidelines

Contributions are welcome! Please read the contribution guidelines first.

License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.