Skip to content

SignSync is a sign-language translation system I built to support both Indian Sign Language (ISL) and American Sign Language (ASL). I used MediaPipe for gesture tracking and integrated NLP and deep learning models to convert signs into speech and text. The system achieved around 82% accuracy in translation.

Notifications You must be signed in to change notification settings

Vigneshgbe/SignSync-Speech-to-Sign

Repository files navigation

✨ SignSync – Speech ↔ Sign Language Converter

🚀 SignSync is an AI-powered communication platform that bridges the gap between spoken language and sign language.
It converts speech/text into sign language animations (ISL & ASL) and vice versa, enabling inclusive communication for the deaf and hard-of-hearing community.


🌍 Why SignSync?

  • 🧑‍🦽 Accessibility First: Helps people with hearing impairments communicate seamlessly.
  • 🌐 Two-Way Communication: Converts speech to signs and signs to speech in real-time.
  • 🇮🇳 Supports ISL & ASL: Covers both Indian Sign Language and American Sign Language.
  • 🤖 AI & ML Integration: Uses NLP + Computer Vision to ensure accurate translation.

✨ Key Features

  • 🎙 Speech → Sign Language (real-time conversion)
  • ✋ Sign Language → Text/Speech (gesture recognition)
  • 🌐 Multi-language support (future roadmap)
  • 📊 Dashboard for usage analytics
  • 🔉 Text-to-Speech & Voice output
  • 🎥 Animated Sign Avatars for ISL & ASL

🛡️ Tech Stack

  • Frontend: React.js + TailwindCSS
  • Backend: Node.js + Express / Python Flask (for ML APIs)
  • AI/ML: TensorFlow / PyTorch (gesture recognition, NLP models)
  • Database: MongoDB / PostgreSQL
  • Other Tools: OpenCV, MediaPipe, Google Speech-to-Text, TTS APIs

📂 Repository Structure

signsync/
├── frontend/        # React-based UI
├── backend/         # APIs for speech ↔ sign conversion
├── models/          # Trained ML models (gesture recognition, NLP)
├── dataset/         # Sign language datasets
├── docs/            # Documentation, research papers
└── README.md        # You're here

##⚡ Installation
# Clone repo
git clone https://github.com/Sweety-Vigneshg/speech-to-sign-project.git

# Backend setup
cd backend
pip install -r requirements.txt

# Frontend setup
cd frontend
npm install

# Run servers
npm start   # for frontend
python app.py   # for backend ML service

About

SignSync is a sign-language translation system I built to support both Indian Sign Language (ISL) and American Sign Language (ASL). I used MediaPipe for gesture tracking and integrated NLP and deep learning models to convert signs into speech and text. The system achieved around 82% accuracy in translation.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published