🎵 Mujica - Music Cluster Visualization

AI-Powered Music Genre Classification & Interactive Visualization

---

📖 Overview

Mujica is an interactive web application that visualizes music clustering based on AI-powered genre classification. Upload MP3 files or YouTube URLs to see songs positioned in a beautiful decagon visualization based on their genre probabilities.

🎯 Core Concept

Songs are positioned inside a 10-sided polygon (decagon) where each vertex represents one of 10 music genres. The position of each song is determined by weighted genre probabilities from a trained BiLSTM neural network with attention mechanism.

✨ Features

🤖 AI-Powered Classification

• BiLSTM with Attention - Deep learning model with 90%+ accuracy
• Multi-Segment Analysis - Analyzes 10 segments across the entire song for better accuracy
• 58 Audio Features - MFCCs, spectral features, tempo, harmony, and more
• 10 Genres - Blues, Classical, Country, Disco, Hip-Hop, Jazz, Metal, Pop, Reggae, Rock

🎨 Interactive Visualization

• D3.js Decagon - Beautiful 10-sided polygon visualization
• Real-Time Updates - Songs appear instantly after processing
• Interactive Exploration - Hover, click, zoom, and pan
• Genre Color Coding - Each genre has its unique color

📤 Dual Upload Methods

• MP3 File Upload - Drag & drop interface with validation (up to 50MB)
• YouTube Integration - Paste any YouTube URL to analyze songs

📊 Detailed Analysis

• Primary Genre - Top prediction with confidence percentage
• Probability Breakdown - All 10 genre probabilities with bar charts
• Song Metadata - Duration, upload date, source information
• Easy Management - Delete songs, view history

Song Positioning

Songs are positioned using a weighted calculation:

• Each genre probability contributes to the X and Y coordinates
• Higher probability = closer to that genre's vertex
• Mixed genres appear in intermediate positions
• Node size reflects confidence level

🔌 API Endpoints

Upload

• POST /api/upload/mp3 - Upload MP3 file
• POST /api/upload/youtube - Process YouTube URL

Songs

• GET /api/songs - List songs (paginated, filterable)
• GET /api/songs/{id} - Get song details
• DELETE /api/songs/{id} - Delete song

Visualization

• GET /api/cluster-data - Get all data for visualization
• GET /api/health - Health check

📚 Full API documentation: http://localhost:8000/docs

🧠 Model Architecture

BiLSTM with Attention Mechanism

Input (58 features)
    ↓
Batch Normalization
    ↓
LSTM Layer 1 (256 units) + Attention
    ↓
LSTM Layer 2 (128 units) + Attention
    ↓
LSTM Layer 3 (64 units)
    ↓
Dense Layer (128 units)
    ↓
Dense Layer (64 units)
    ↓
Output (10 genres)

Training Details:

• Dataset: GTZAN Music Genre
• Samples: ~10,000 audio clips
• Accuracy: 90%+ on test set
• Framework: PyTorch

Multi-Segment Analysis

For improved accuracy, each song is analyzed using:

• 10 evenly-spaced segments across the full song duration
• 3 seconds per segment (standard training length)
• Averaged probabilities from all segments
• Fallback to single segment for clips < 3 seconds

This captures the full structure of the song rather than just a single clip!

📋 Step-by-Step Setup

Step 1: Backend Setup (15-20 minutes)

1.1 Navigate to Backend Directory

cd c:/Users/muzam/mujica/soundscape/backend

1.2 Create Virtual Environment

# Create venv
python -m venv venv

# Activate (Windows)
venv\Scripts\activate

# Activate (Linux/Mac)
source venv/bin/activate

1.3 Install Python Dependencies

pip install -r requirements.txt

Note: This may take 10-15 minutes as it installs PyTorch and other large packages.

1.4 Install FFmpeg (Required for YouTube Downloads)

FFmpeg is required to convert YouTube audio to MP3 format.

Windows:

# Option 1: Using Chocolatey (recommended)
choco install ffmpeg

# Option 2: Manual Installation
# 1. Download from: https://www.gyan.dev/ffmpeg/builds/
# 2. Extract to C:\ffmpeg
# 3. Add C:\ffmpeg\bin to your System PATH
# 4. Restart terminal/IDE

Mac:

brew install ffmpeg

Linux (Ubuntu/Debian):

sudo apt update
sudo apt install ffmpeg

Linux (Fedora/RHEL):

sudo dnf install ffmpeg

Verify Installation:

ffmpeg -version

You should see version information. If not, restart your terminal and try again.

1.5 Verify Model File

Check that the model file exists:

# Should see: pytorch_genre_classifier_best.pkl
dir ml_models\

If missing, copy from your training directory:

copy ..\..\..\models\classification_model\pytorch_genre_classifier_best.pkl ml_models\

1.6 Test Backend

# Start the server
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

You should see:

🚀 Starting SoundScape Backend...
📊 Creating database tables...
🤖 Loading ML model...
✓ Model loaded successfully on cpu
  Test Accuracy: XX.XX%
✓ Backend ready!
INFO:     Uvicorn running on http://0.0.0.0:8000

Test it: Open http://localhost:8000/docs in your browser to see the API documentation.

Press Ctrl+C to stop the server when done testing.

Step 2: Frontend Setup (10-15 minutes)

2.1 Navigate to Frontend Directory

# Open a NEW terminal (keep backend running in the first one)
cd c:/Users/muzam/mujica/soundscape/frontend

2.2 Install Node Dependencies

npm install

Note: This may take 5-10 minutes.

2.3 Verify Environment File

Check that .env file exists with:

VITE_API_URL=http://localhost:8000/api

2.4 Start Development Server

npm run dev