🚫 Blocking ads in real-time with AI-powered augmented reality!
This is an augmented reality experience built for Snapchat Spectacles, where you can:
- Detect advertisements in your real-world environment using AI
- Block ads visually by overlaying them with AR content
- Use pinch gestures to trigger ad detection and blocking
- Experience real-time AI-powered advertisement identification
This project is based on the Snapchat Spectacles Depth Cache Sample, which provides the foundational depth sensing and world space conversion capabilities.
https://www.linkedin.com/feed/update/urn:li:activity:7341484830682923008/
- Lens Studio (Snapchat's AR development environment)
- Google Gemini AI (for advertisement detection)
- Depth Sensing (for accurate 3D placement)
- Gesture Recognition (pinch to activate)
- Remote Service Gateway (for AI API integration)
-
Load the Lens
Import the project into Lens Studio and load it on your Snapchat Spectacles. -
Point at Advertisements
Look at any advertisements in your environment (billboards, signs, digital displays, etc.). -
Pinch to Activate
Use the pinch gesture to trigger the AI-powered ad detection system. -
Watch Ads Get Blocked
The AI will identify advertisements and overlay them with blocking content in real-time.
- Real-time Detection: Uses Google Gemini AI to identify advertisements in camera feed
- Bounding Box Analysis: Precisely locates ad boundaries for accurate blocking
- Depth-Aware Placement: Uses depth sensing to place blocking content in 3D space
- Smart Labeling: AI provides descriptive labels for each detected advertisement
- Multiple Ad Support: Can detect and block up to 25 advertisements simultaneously
- SceneController.ts: Main orchestration and gesture handling
- GeminiAPI.ts: Integration with Google Gemini AI for ad detection
- DepthCache.ts: Depth frame management and world space conversion
- SpeechUI.ts: User interface for speech input (future feature)
- ResponseUI.ts: Display of AI responses and blocking status
- Google Gemini 2.5 Pro: Advanced AI model for advertisement detection
- JSON Response Schema: Structured data format for bounding box coordinates
- Image Processing: Real-time camera frame analysis and encoding
- 3D Plane Generation: Creates blocking overlays in world space
- Depth-Aware Positioning: Uses depth data for accurate 3D placement
- Gesture Recognition: Pinch-to-activate functionality
- Visual Feedback: Loading states and response bubbles
This is a proof of concept (PoC) that demonstrates the potential of AI-powered AR ad blocking. Key considerations:
- Privacy: Camera data is processed through Google's Gemini API
- Accuracy: AI detection may have false positives/negatives
- Performance: Real-time processing requires stable internet connection
- Ethical Use: Intended for personal use and research purposes
- Install Lens Studio (version 5.10.1 or later)
- Import Project: Open
Ad Blocker.esprojin Lens Studio - Configure API: Set up Google Gemini API credentials
- Test on Device: Deploy to Snapchat Spectacles for testing
- Snapchat Spectacles (2024)
- Front and Back Camera support
- Gesture Recognition via pinch detection
- Depth Sensing for accurate 3D placement
- Offline Detection: Local AI models for privacy
- Custom Blocking: User-defined blocking patterns
- Speech Commands: Voice-activated ad blocking
- Analytics: Track blocked ads and effectiveness
- Multi-language: Support for international advertisements
Enjoy blocking ads in the real world! 🚫✨