[Creative Apps] CodeSonify — Transform Source Code Into Music

[jpablortiz96]

Track

Creative Apps (GitHub Copilot)

Project Name

CodeSonify

GitHub Username

@jpablortiz96

Repository URL

https://github.com/jpablortiz96/codesonify

Project Description

CodeSonify transforms source code into real music. It analyzes the structural elements of any codebase — functions, loops, conditionals, classes, variables, comments — and maps each construct to distinct musical elements: functions become ascending melodies, loops create rhythmic patterns, conditionals trigger chord changes, and bugs produce dissonant sounds.

The tool supports 7 programming languages (JavaScript, TypeScript, Python, Java, C#, Go, Rust) with automatic language detection, and offers 5 musical styles: Classical, Electronic, Ambient, Jazz, and Rock. Each language receives its own musical key signature, and code complexity directly controls the tempo — complex code literally sounds more intense.

CodeSonify delivers three integrated experiences:

1. A polished web application with real-time canvas visualization, live audio playback via Tone.js, interactive stats dashboard, and one-click MIDI file export.

2. An MCP Server exposing 5 tools to GitHub Copilot in VS Code: sonify code, analyze code structure, compare code musically, sonify git diffs, and export MIDI — enabling developers to hear their code without leaving the editor.

3. A Diff Sonification engine that transforms code changes into music: added lines sound bright and ascending (major key), removed lines sound dark and descending (minor key) — letting developers literally hear a code review or pull request.

The MIDI export feature generates standard .mid files compatible with any DAW (GarageBand, FL Studio, Ableton), turning code into real, editable, shareable music. Built entirely with GitHub Copilot assistance, CodeSonify bridges the gap between programming and music in a way that's creative, functional, and immediately engaging.

Demo Video or Screenshots

🎬 Demo Video

📹 Watch the full demo on YouTube →

The video demonstrates:

• Code-to-music transformation with real-time visualization
• Multiple musical styles (Classical, Electronic, Jazz)
• Diff Mode: hearing code changes as ascending/descending melodies
• MIDI file export (real .mid download)
• MCP Server integration with GitHub Copilot in VS Code Agent Mode

Primary Programming Language

TypeScript/JavaScript

Key Technologies Used

• TypeScript — Type-safe codebase with strict mode throughout
• MCP SDK (@modelcontextprotocol/sdk) — Model Context Protocol server for VS Code/GitHub Copilot integration
• Express 5 — Web server and REST API
• Tone.js — Web Audio synthesis and real-time music playback
• Canvas API — Animated note particle visualization
• Zod — Runtime input validation for MCP tool schemas
• GitHub Copilot — AI-assisted development (code suggestions, Chat for architecture, Agent Mode for MCP testing)
• Node.js — Server runtime
• MIDI format — Pure TypeScript MIDI file generator (zero external dependencies)

Submission Type

Individual

Team Members

No response

Submission Requirements

• My project meets the track-specific challenge requirements
• My repository includes a comprehensive README.md with setup instructions
• My code does not contain hardcoded API keys or secrets
• I have included demo materials (video or screenshots)
• My project is my own work with proper attribution for any third-party code
• I agree to the Code of Conduct
• I have read and agree to the Disclaimer
• My submission does NOT contain any confidential, proprietary, or sensitive information
• I confirm I have the rights to submit this content and grant the necessary licenses

Quick Setup Summary

# Clone and run in 3 commands:
git clone https://github.com/TU_USUARIO/codesonify.git
cd codesonify
npm install
npm start

Open http://localhost:3000 in your browser. Paste any code, select a musical style, and click "Sonify Code" to hear your code as music.

For MCP Server (GitHub Copilot integration):

1. Open the project in VS Code
2. Open .vscode/mcp.json and click "Start"
3. Open Copilot Chat → switch to Agent Mode
4. Try: Sonify this code: function hello() { console.log("world"); }

Requirements: Node.js v18+, VS Code with GitHub Copilot (for MCP features)

Technical Highlights

What I'm most proud of:

1. Pure TypeScript MIDI Generator (Zero Dependencies)
I built a complete MIDI file generator from scratch — no libraries. It handles multi-track MIDI format 1, variable-length quantity encoding, tempo meta events, program changes, and note on/off events. The result is a real .mid file that opens in any DAW. Understanding the binary MIDI spec and implementing it in TypeScript was the most rewarding challenge.

2. MCP Server with 5 Specialized Tools
The MCP integration goes beyond a basic demo. Five distinct tools — sonify_code, analyze_code_structure, compare_code_musically, sonify_git_diff, and export_midi — each with proper Zod validation schemas and rich markdown-formatted responses. Developers can hear their code directly from GitHub Copilot Chat in Agent Mode without leaving VS Code.

3. Diff Sonification Engine
This is the feature I'm most creatively proud of. The diff parser analyzes unified diffs (or compares two code versions) and maps additions to ascending major-key melodies and deletions to descending minor-key bass lines. The result is that you can literally hear a code review — a clean refactoring sounds harmonious, while a messy change sounds chaotic.

4. Musical Design Decisions
Each programming language maps to a unique musical key (JavaScript = C Mixolydian, Python = F Pentatonic, Rust = B Blues). Code complexity directly controls tempo. Nesting depth controls octave/pitch. These aren't arbitrary — they create meaningful, reproducible musical fingerprints for any codebase.

5. Real-time Visualization
The Canvas-based particle system renders each note as a colored dot moving across a timeline, with glowing effects for active notes. Different instruments get different colors. The result is a visual representation of code structure that's both beautiful and informative.

Challenges & Learnings

Challenges:

Audio Timing Precision: Web Audio (Tone.js) requires careful scheduling to avoid clicks and timing drift. I learned to pre-schedule all notes relative to Tone.now() rather than using setTimeout, which solved the synchronization issues between tracks.

Express 5 Breaking Changes: Express 5 changed the wildcard route syntax from '*' to '/{*path}', which caused runtime errors. This taught me to always check for breaking changes when using latest package versions.

MIDI Binary Format: Implementing the MIDI spec from scratch required understanding variable-length quantity encoding, the difference between Format 0 and Format 1, and how channel 10 is reserved for percussion. The binary math was tricky but educational.

Musical Mapping Balance: Finding the right balance between musical correctness and code representation took iteration. Early versions produced chaotic noise because every token generated a note. The solution was assigning different instruments, volumes, and durations so each code element has its own sonic space — melody for functions, bass for variables, percussion for loops.

Key Learnings:

• MCP servers are a powerful way to extend GitHub Copilot — the integration feels natural and useful
• The Model Context Protocol SDK is well-designed and straightforward to implement
• GitHub Copilot Chat was invaluable for exploring creative ideas and debugging audio issues
• TypeScript strict mode catches many bugs at compile time that would be painful at runtime
• Building something at the intersection of two domains (music + code) forces deep understanding of both

Contact Information

jpablortiz96@hotmail.com

Country/Region

Colombia