An interactive, realtime web application featuring an AI boyfriend companion with natural voice conversations. The application is built on WebRTC for seamless bidirectional audio streaming with OpenAI's Realtime API, enabling low-latency voice interactions. The interactive avatar is powered by HeyGen Streaming Avatar SDK, providing realtime facial animations and lip-sync capabilities. The architecture supports dynamic tool calling with real-time function execution, creating a truly immersive and responsive companion experience.
received_1277596923499329.mp4
- Real-time Voice Conversations: Have natural, seamless bidirectional conversations with your AI boyfriend using OpenAI's Realtime API and WebRTC technology
- Interactive Avatar: See your AI boyfriend come to life with an animated avatar powered by HeyGen Streaming Avatar SDK that responds and reacts in real-time
- Personalized Interactions: Dynamic function execution that allows your AI boyfriend to:
- Check the current time and timezone for you
- Adjust the theme (dark/light mode) to match your preferences
- Create fun moments with party mode and confetti animations
- Help with tasks like copying to clipboard
- Launch websites and assist with browsing
- Modern UI Components: Beautiful, comprehensive Shadcn/ui component library built on Radix UI foundations for a polished experience
- Type-Safe: Full TypeScript support with strict ESLint configuration ensuring reliability
- Responsive Design: Mobile-friendly interface with adaptive layouts - chat with your AI boyfriend anywhere, anytime
- Theme Support: Built-in light/dark mode switching to match your mood and preferences
- Code Editor: Integrated CodeMirror for live code editing and visualization when needed
- Analytics: Vercel Analytics integration for usage tracking
- Token Usage Display: Real-time monitoring of API token consumption to keep track of usage
- Framework: Next.js 15 with App Router
- Styling: Tailwind CSS + Shadcn/ui
- Language: TypeScript
- Avatar: HeyGen Streaming Avatar
- Forms: React Hook Form with Zod validation
- State Management: React hooks (custom
use-webrtchook for WebRTC abstraction) - UI Library: 30+ Radix UI components
- Code Editor: CodeMirror with language support
- Notifications: Sonner toast notifications
- Icons: Phosphor Icons
- Node.js 18+ or Deno
- OpenAI API Key (or Azure OpenAI API Key)
- HeyGen API Key and Avatar ID
- Modern browser with WebRTC support
git clone https://github.com/cameronking4/shadcn-openai-realtime-webrtc.git
cd shadcn-openai-realtime-webrtcCreate a .env file in the root directory:
# OpenAI API Configuration
OPENAI_API_KEY=your-openai-api-key
# HeyGen API Configuration
NEXT_PUBLIC_HEYGEN_API_KEY=your-heygen-api-key
NEXT_PUBLIC_HEYGEN_AVATAR_ID=your-heygen-avatar-id- OPENAI_API_KEY: Your OpenAI API key for accessing the Realtime API. Get it from OpenAI Platform
- NEXT_PUBLIC_HEYGEN_API_KEY: Your HeyGen API key for avatar streaming. Get it from HeyGen Dashboard
- NEXT_PUBLIC_HEYGEN_AVATAR_ID: The avatar ID from your HeyGen account to use for your AI boyfriend avatar
If using Node.js:
npm installIf using Deno:
deno installnpm run devdeno task startThe application will be available at http://localhost:3000.
- Open the app in your browser:
http://localhost:3000. - Grant camera and microphone permissions when prompted.
- Select your preferred voice for your AI boyfriend.
- Click the call button to start your conversation session.
- Begin talking naturally - your AI boyfriend will respond in real-time with voice and animated expressions.
This project is licensed under the MIT License. See the LICENSE file for details.
- OpenAI for their API and models.
- Next.js for the framework.
- Tailwind CSS for styling.
- Simon Willison's Weblog for inspiration
- Originator: skrivov for the WebRTC and Nextjs implementation