TACTO App

Never Miss a Beat - Situational Awareness for the Deaf and Hard of Hearing

Live Demo | Documentation | Report Bug | Request Feature

---

Overview

TACTO is an innovative Progressive Web App (PWA) designed to enhance situational awareness for deaf and hard-of-hearing individuals. Using real-time audio analysis, the app detects critical environmental sounds (fire alarms, sirens, doorbells, crying babies) and provides strong haptic feedback and visual alerts.

Key Problem Solved

Deaf and hard-of-hearing individuals often miss critical audio cues that could impact their safety, security, and daily life. TACTO bridges this gap by transforming your smartphone into a vigilant auditory assistant.

---

Key Features

Feature	Description
Real-Time Sound Detection	Advanced audio pattern recognition for 5+ critical sound types
Intelligent Haptic Patterns	Unique vibration patterns for each alert type
Visual Alerts	High-contrast flashing notifications
Bilingual Support	Full English/Spanish internationalization (i18n)
PWA Architecture	Installable, offline-capable, native-like experience
Background Monitoring	Continuous detection even when phone is locked
Customizable Sensitivity	Adjust microphone sensitivity to your environment
Dark Mode	Comfortable viewing in any lighting condition
Alert History	Track and review past notifications

---

Demo

** Coming Soon:** Live demo link and video walkthrough

Screenshots

| Splash Screen | Onboarding | Dashboard | |: ---:|:---:|:---:| | | | | | Clean, accessible design | Interactive user guide | Real-time monitoring |

---

Architecture & Tech Stack

Frontend

• Core: HTML5, CSS3, Vanilla JavaScript (ES Modules)
• Build Tool: Vite 5.0
• Mobile Framework: Capacitor 6.0 (for native features)
• UI/UX:
  • FontAwesome 6 (Icons)
  • Driver. js (User tours)
  • Custom CSS Variables (theming)
  • Fonts: Poppins, Coolvetica, Supreme

Backend/Data Processing

• Python 3.9+ for dataset generation
• Freesound API for training data acquisition
• Web Audio API for real-time sound analysis

Key Technical Decisions

Decision	Rationale
Vanilla JS (ESM)	Zero framework overhead, maximum performance
Capacitor over Cordova	Modern API, better TypeScript support, active maintenance
Client-side Processing	Privacy-first: no audio data sent to servers
localStorage over IndexedDB	Simpler API for small data sets, sufficient for MVP

---

Project Structure

Tacto-app/
├── . github/                      # GitHub configuration
│   ├── ISSUE_TEMPLATE/          # Issue templates
│   └── PULL_REQUEST_TEMPLATE.md # PR template
├── docs/                        # Documentation
│   ├── ARCHITECTURE.md          # Technical architecture
│   └── screenshots/
|   |---- notebooks/
|         |---- Audio_sampling_TACTO_new.ipnyb            
├── Tacto/                       # PWA Application
│   └── public/
│       ├── src/
│       │   ├── components/      # Reusable UI (Alert, Modal, Nav)
│       │   ├── pages/           # Main Views (History, Settings)
│       │   ├── sections/        # App Sections (Onboarding, Splash)
│       │   ├── utils/           # Utilities (i18n, Storage)
│       │   └── main.mjs         # App entry point
│       ├── libs/                # Third-party libraries
│       ├── font/                # Local fonts
│       ├── service-worker.js    # PWA offline support
│       ├── capacitor.config.json
│       └── package.json
├── TactoAPP/                    # Mobile app builds
├── main.py                      # Dataset generation script
├── creditos_obligatorios.csv    # Audio licensing credits
├── . env. example                 # Environment variables template
├── .gitignore
├── requirements.txt             # Python dependencies
├── LICENSE
├── CHANGELOG.md
├── CONTRIBUTING.md
├── SECURITY.md
└── README.md

---

Notebooks & Analysis

This project includes Jupyter notebooks for data analysis and model development:

• Sound Detection Analysis - Audio pattern analysis and classification experiments

See the notebooks directory for more details.

Getting Started

Prerequisites

• Node.js 16+ and npm
• Python 3.9+ (for dataset generation only)
• Modern web browser with Web Audio API support
• (Optional) Android Studio / Xcode for mobile builds

Installation

1. Clone the repository

   git clone https://github.com/Chillipeeper1/Tacto-app.git
   cd Tacto-app

2. Install PWA dependencies

   cd Tacto/public
   npm install

3. Run development server

   npm run dev

   Open http://localhost:5173 in your browser

4. Build for production

   npm run build
   # Output will be in /dist folder

Dataset Generation (Optional)

If you want to train your own sound detection model:

1. Install Python dependencies

   pip install -r requirements.txt

2. Configure Freesound API

   cp .env.example .env
   # Edit .env and add your FREESOUND_API_KEY

3. Download audio samples

   python main.py

---

Technical Highlights

Challenge 1: Real-Time Audio Processing

Problem: Web Audio API raw data is noisy and requires complex pattern recognition.
Solution: Implemented frequency analysis with adjustable sensitivity thresholds and cooldown periods to prevent false positives.

// Simplified example from main.mjs
analyser.getByteFrequencyData(dataArray);
const average = dataArray.reduce((a, b) => a + b) / dataArray.length;
if (average > sensitivity && ! cooldown) {
  triggerAlert(detectSoundType(dataArray));
}

Challenge 2: Cross-Platform Vibration

Problem: iOS and Android have different vibration API implementations.
Solution: Created abstraction layer with pattern fallbacks.

Challenge 3: Offline Functionality

Problem: App must work without internet connection.
Solution: Service Worker with intelligent caching strategy for all assets.

---

Performance

• Lighthouse Score: 95+ (Performance, Accessibility, Best Practices)
• First Contentful Paint: < 1.2s
• Time to Interactive: < 2.5s
• Bundle Size: < 500KB (gzipped)

---

Roadmap

• MVP: Basic sound detection (5 types)
• Haptic feedback system
• Bilingual support (EN/ES)
• Machine Learning model integration
• Wearable device support (smartwatches)
• Cloud sync for multi-device
• Custom sound training
• iOS App Store release
• Google Play Store release

See the open issues for a full list of proposed features and known issues.

---

Contributing

Contributions are what make the open-source community amazing! Any contributions you make are greatly appreciated.

Please read our Contributing Guidelines before submitting PRs.

Quick Start for Contributors

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

---

Team & Credits

Core Development Team

• JuanMg - Full Stack Development (Front & Back)
• Zabdiel Rios Cervantes (Chillipeeper1) - Illustrations, Machine Learning, project lead

Acknowledgments

• Audio samples from Freesound. org (Creative Commons)
• Icon library by FontAwesome
• Inspiration from accessibility-first design principles

---

License

This project is licensed under the MIT License - see the LICENSE file for details.

---

Security

Found a security vulnerability? Please read our Security Policy and report it responsibly.

---

Contact & Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Email: Contact via GitHub profile

---

💖 Support the Project

If TACTO helped you or someone you know, consider:

• Starring this repository
• Reporting bugs
• Sharing with your network
• Contributing ideas

---