TEN Agent, powered by the world’s first real-time multimodal framework. It is open-source, with the ability to speak, see, and access a knowledge base. By taking advantage of TEN Framework, TEN Agent has the following features:
-
High-Performance Real-Time Multimodal Interactions: Offers high-performance, low-latency solutions for complex audio-visual AI applications.
-
Multi-Language and Multi-Platform Support : Supports extension development in C++, Go, Python, etc. Runs on Windows, Mac, Linux, and mobile devices.
-
Edge-Cloud Integration: Flexibly combines edge and cloud-deployed extensions, balancing privacy, cost, and performance.
-
Flexibility Beyond Model Limitations: Easily build complex AI applications through simple drag-and-drop programming, integrating audio-visual tools, databases, RAG, and more.
-
Real-Time Agent State Management: Manages and adjusts agent behavior in real-time for dynamic responsiveness.
Before we get started, be sure to star our repository and get instant notifications for all new releases!
TEN Agent is a multimodal agent powered by TEN , demonstrating its capabilities in speech, vision, and reasoning through RAG from local documentation.
- Agora App ID and App Certificate (certificate is not required)
- Azure SST and TTS API keys (feel free to use another provider)
- OpenAI API key
- CPU >= 2 Core
- RAM >= 4 GB
You will need to uncheck "Use Rosetta for x86_64/amd64 emulation on Apple Silicon" option for Docker if you are on Apple Silicon, otherwise the server is not going to work.
In the root of the project, use cp command to create .env from the example.
It will be used to store information for docker compose later.
cp ./.env.example ./.envOpen the .env file and fill in the keys and regions. This is also where you can choose to use any different extensions:
# Agora App ID and Agora App Certificate
AGORA_APP_ID=
# Leave empty unless you have enabled the certificate within the Agora account.
AGORA_APP_CERTIFICATE=
# Azure STT key and region
AZURE_STT_KEY=
AZURE_STT_REGION=
# Azure TTS key and region
AZURE_TTS_KEY=
AZURE_TTS_REGION=
# OpenAI API key
OPENAI_API_KEY=In the same directory, run the docker compose up command to compose containers:
docker compose upOpen up a separate terminal window, enter the container and build the agent:
docker exec -it ten_agent_dev bash
make buildOnce the build is done, make run-server on port 8080:
make run-serverOpen up http://localhost:3000 in browser to play and test the TEN Agent.
Open up another tab go to http://localhost:3001, and use Graph Designer to create, connect and edit extensions on canvas.
| Features | TEN Agent | Pipecat | LiveKit:KITT | Vapi.ai | DailyBots | Play.ai |
|---|---|---|---|---|---|---|
| Vision | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Rich TTS Support for different languages | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Go support for extension | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| C++ support for extension | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| RAG support | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
| Workflow builder for extension | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ |
| Rich LLM Support | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Python support for extension | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Open source | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
- Discord: Ideal for sharing your applications and engaging with the community.
- GitHub Discussion: Perfect for providing feedback and asking questions.
- GitHub Issues: Best for reporting bugs and proposing new features. Refer to our contribution guidelines for more details.
- X (formerly Twitter): Great for sharing your agents and interacting with the community.
Contributions are welcome! Please read the contribution guidelines first.
This project is licensed under the Apache 2.0 License - see the LICENSE file for details.