简体中文 | English
py-xiaozhi is a Xiaozhi voice client implemented in Python, designed for code learning and experiencing AI Xiaozhi's voice features without hardware requirements. This repository is ported from xiaozhi-esp32
- AI Voice Interaction: Supports voice input and recognition, enabling intelligent human-computer interaction with natural and smooth conversation experience
- Visual Multimodal: Supports image recognition and processing, providing multimodal interaction capabilities to understand image content
- Smart Wake-up: Supports multiple wake words for activation, eliminating manual operations (configurable)
- Auto Conversation Mode: Implements continuous conversation experience, improving user interaction fluency
- System Control Tools: System status monitoring, application management, volume control, device management, etc.
- Schedule Management Tools: Full-featured schedule management supporting create, query, update, delete events, intelligent categorization and reminders
- Timer Task Tools: Countdown timer function, supporting delayed execution of MCP tools, parallel multi-task management
- Music Player Tools: Online music search and playback, supporting playback control, lyrics display, local cache management
- 12306 Query Tools: 12306 railway ticket query, supporting ticket query, transfer query, train route query
- Search Tools: Web search and webpage content retrieval, supporting Bing search and intelligent content parsing
- Recipe Tools: Rich recipe library, supporting recipe search, category query, intelligent recommendations
- Map Tools: Amap services, supporting geocoding, route planning, nearby search, weather query
- Bazi Fortune Tools: Traditional Bazi fortune analysis, supporting Bazi calculation, marriage analysis, lunar calendar query
- Camera Tools: Image capture and AI analysis, supporting photo recognition and intelligent Q&A
- Device Management Architecture: Unified device management based on Thing pattern, supporting asynchronous invocation of properties and methods
- Smart Home Control: Supports light, volume, temperature sensor and other device control
- State Sync Mechanism: Real-time state monitoring, supporting incremental updates and concurrent state retrieval
- Extensible Design: Modular device drivers, easy to add new device types
- Multi-level Audio Processing: Supports Opus codec, real-time resampling
- Voice Activity Detection: VAD detector implements intelligent interruption, supporting real-time voice activity monitoring
- Wake Word Detection: Offline speech recognition based on Vosk, supporting multiple wake words and pinyin matching
- Audio Stream Management: Independent input/output streams, supporting stream rebuilding and error recovery
- Graphical Interface: Modern GUI based on PyQt5, supporting Xiaozhi expressions and text display, enhancing visual experience
- Command Line Mode: Supports CLI operation, suitable for embedded devices or non-GUI environments
- System Tray: Background running support, integrated system tray functionality
- Global Shortcuts: Supports global shortcut operations, improving usability
- Settings Interface: Complete settings management interface, supporting configuration customization
- Encrypted Audio Transmission: Supports WSS protocol, ensuring audio data security and preventing information leakage
- Device Activation System: Supports v1/v2 dual-protocol activation, automatically handles verification codes and device fingerprints
- Error Recovery: Complete error handling and recovery mechanism, supporting reconnection
- System Compatibility: Compatible with Windows 10+, macOS 10.15+, and Linux systems
- Protocol Support: Supports WebSocket and MQTT dual-protocol communication
- Multi-environment Deployment: Supports GUI and CLI dual modes, adapting to different deployment environments
- Platform Optimization: Audio and system control optimization for different platforms
- Modular Architecture: Clear code structure and responsibility separation, convenient for secondary development
- Async-first: Event-driven architecture based on asyncio, high-performance concurrent processing
- Configuration Management: Hierarchical configuration system, supporting dot notation access and dynamic updates
- Logging System: Complete logging and debugging support
- API Documentation: Detailed code documentation and usage guides
- Python Version: 3.9 - 3.12
- Operating System: Windows 10+, macOS 10.15+, Linux
- Audio Devices: Microphone and speaker devices
- Network Connection: Stable internet connection (for AI services and online functions)
- Memory: At least 4GB RAM (8GB+ recommended)
- Processor: Modern CPU supporting AVX instruction set
- Storage: At least 2GB available disk space (for model files and cache)
- Audio: Audio devices supporting 16kHz sampling rate
- Voice Wake-up: Requires downloading Vosk speech recognition model
- Camera Function: Requires camera device and OpenCV support
- Carefully read the Project Documentation - startup tutorials and file descriptions are all there
- main is the latest code, you need to manually reinstall pip dependencies after each update to prevent missing new dependencies
Getting Started with Xiaozhi Client from Zero (Video Tutorial)
- Event-driven Architecture: Asynchronous event loop based on asyncio, supporting high-concurrency processing
- Layered Design: Clear separation of application layer, protocol layer, device layer, and UI layer
- Singleton Pattern: Core components adopt singleton pattern, ensuring unified resource management
- Plugin-based: MCP tool system and IoT devices support plugin-based extension
- Audio Processing: Opus codec, real-time resampling
- Speech Recognition: Vosk offline model, voice activity detection, wake word recognition
- Protocol Communication: WebSocket/MQTT dual-protocol support, encrypted transmission
- Configuration System: Hierarchical configuration, dot notation access, dynamic updates
- Async-first: Full system async architecture, avoiding blocking operations
- Memory Management: Intelligent caching, garbage collection
- Audio Optimization: 5ms low-latency processing, queue management, streaming transmission
- Concurrency Control: Task pool management, semaphore control, thread safety
- Encrypted Communication: WSS/TLS encryption, certificate verification
- Device Authentication: Dual-protocol activation, device fingerprint recognition
- Permission Control: Tool permission management, API access control
- Error Isolation: Exception isolation, fault recovery, graceful degradation
py-xiaozhi/
├── src/
│ ├── application.py # Application main entry
│ ├── audio_codecs/ # Audio codecs
│ ├── audio_processing/ # Audio processing module
│ ├── core/ # Core components
│ ├── display/ # Display interface
│ ├── iot/ # IoT device management
│ ├── mcp/ # MCP tool system
│ ├── protocols/ # Communication protocols
│ ├── utils/ # Utility functions
│ └── views/ # View components
├── config/ # Configuration files
├── models/ # Speech models
├── assets/ # Resource files
└── libs/ # Third-party libraries
# Clone project
git clone https://github.com/huangjunsen0406/py-xiaozhi.git
cd py-xiaozhi
# Install dependencies
pip install -r requirements.txt
# Code formatting
./format_code.sh
# Run program
python main.py- Async-first: Use
async/awaitsyntax, avoid blocking operations - Error Handling: Complete exception handling and logging
- Configuration Management: Use
ConfigManagerfor unified configuration access - Test-driven: Write unit tests to ensure code quality
- Add MCP Tools: Create new tool modules in
src/mcp/tools/directory - Add IoT Devices: Inherit
Thingbase class to implement new devices - Add Protocols: Implement
Protocolabstract base class - Add Interfaces: Extend
BaseDisplayto implement new UI components
+----------------+
| |
v |
+------+ Wake/Button +------------+ | +------------+
| IDLE | -----------> | CONNECTING | --+-> | LISTENING |
+------+ +------------+ +------------+
^ |
| | Speech recognition complete
| +------------+ v
+--------- | SPEAKING | <-----------------+
Complete +------------+
Welcome to submit issue reports and code contributions. Please ensure you follow these specifications:
- Code style complies with PEP8 specifications
- Submitted PRs include appropriate tests
- Update relevant documentation
No particular order
Xiaoxia zhh827 四博智联-李洪刚 HonestQiao vonweller 孙卫公 isamu2025 Rain120 kejily 电波bilibili君
Whether it's interface resources, device compatibility testing, or financial support, every bit of help makes the project better
