glsl-pipeline-SpookySkull 💀 🎃

A mesmerizing Halloween experience brought to life with 3D graphics, voice recognition, and facial landmarks.

Watch the video

🚀 Run and play with interactive demo on github pages (LLM endpoint will be active 72h):

• github.io/glsl-pipe-SpookySkull
• tested on Google Chrome browser

Features ⭐️

Striking 3D Skull

• Utilizes Three.js for 3D graphics rendering.
• Applies custom GLSL shaders to bring life-like appearance and animations to the skull.
• Incorporates a surreal animation effect inspired by examples from glsl-pipeline.

Voice Recognition

• Utilizes browser's native Speech Recognition API.
• Ask the spooky skull questions with just a click.
• Backend integration with Local LLMv2 Endpoint for speech processing and analytics.

MediaPipe Facial Landmarking

• Real-time facial landmark detection with head tracking using MediaPipe.
• The skull in the scene mimics the user's head orientation, bringing a personalized interactive experience.
• GPU accelerated for a seamless experience.

Spooky Soundtracks

• Background music to keep the spooky vibes.
• ♪ Music used: Spook by PeriTune | Licensed under Creative Commons Attribution 3.0

Local LLMv2 Endpoint

• A locally running endpoint to handle speech-to-text analytics and provide real-time outputs.
• Custom prompt enginnering for SpookySkull character.
• powerd by Llama-2 & text-generation-webui

GLSL Pipeline

• Advanced GLSL pipelining for efficient graphical rendering.
• Custom shader logic incorporated via Lygia library.

Technologies Used ⚙️

• Three.js
• MediaPipe
• glsl-pipeline
• Web Speech API
• lygia.xyz
• Node.js
• text-generation-webui

Acknowledgements

Thanks to:

• Patricio Gonzalez Vivo for the glsl-pipeline & lygia.xyz libraries.
• oobabooga for text-generation-webui.
• open source community for quantified language models.
• META for open sourcing Llama-2.
• Node.js
• Three.js
• Web Speech API

Happy Halloween 🎃 👻