Project Info

Winner

Treehacks Grand Prize (3rd); [Suno] Best Musical Hack

ChromaChord

Inspiration

The inspiration behind this idea is to have the most extensive overview of music theory, especially in Jazz. Using a physics based approach, we could correctly identify all ~4000 chords in an octave and rank them by dissonance. With the power of LLMs in today's world, it is easier than ever to offload the burden of improvisation to LLMs. However, it was important to us in the beginning of this process to stray away from the LLM models that have little interpretability. This way of constructing the chords and ranking their dissonance from the ground up builds concrete understanding of which chord works well with what. Now, introducing ChromaChord, a solution at the intersection of acoustics, graph theory, and a strong desire to have better tools for learning how to play jazz!

What it does

ChromaChord is a real-time AI jazz assistant that listens to your improvisation and helps you navigate harmony. As you play, it analyzes incoming notes using chroma vector representations, maps harmonic relationships onto a graph of possible chord transitions, and suggests musically coherent next chords. Of course, humans have limitations, and there are only so many paths we can physically traverse. So after the extensive sifting with the chroma vectors, we can use AI to generate harmony, melodies, matching the progression that was played.

How we built it

Part 1: Chroma Vectors MIDI input is captured live from guitar/keyboard. Notes are converted into chroma vectors to represent pitch class content. A graph-based harmonic model encodes possible chord transitions and overtone relationships. Deterministic tools handle chord detection, key inference, and progression suggestions. We did our literature review for this project to convert the theory to code. Here are some papers we read: A multi-level tonal interval space for modelling pitch relatedness and musical consonance Detecting Harmonic Change In Musical Audio A Computational Model of Tonal Tension Profile of Chord Progressions in the Tonal Interval Space Autochord An Efficient Algorithm for the Calculation of a Constant Q Transform Part 2: AI Features We wanted to think creatively about how to incorporate the best AI tools to enhance our product. The Perplexity Sonar API is queried to retrieve popular songs that also follow a similar chord progression. This helps with inspiration and enhancing how you think about future directions. We separated fast local harmonic analysis from slower AI reasoning to maintain responsiveness during live play.

Challenges we ran into

Real-time Translation Problem: Signal processing for instruments is noisy and still an open problem. Solution: Using direct MIDI input from our keyboard Chord ambiguity Problem: Translating chroma vectors into user input. Multiple harmonic interpretations can exist for the same pitch set. Problem: Avoiding random suggestions and diluting our product. Solution: Incorporating the Perplexity Sonar API and Claude Agent SDK in an intentional way. Synchronization

Accomplishments we're proud of

Real-time harmonic graph visualization driven by live chroma data. Generating backing tracks dynamically from user-generated chord progressions. Integrating signal processing and graph theory into one coherent system. Building a system that feels like collaboration, not automation. All of the musical friends we made along the way!

What we learned

Creative AI tools are most compelling when they enhance, not replace, human expression. Harmony is surprisingly graph-like — functional relationships translate naturally into network structures. AI works best as a decision layer on top of deterministic musical analysis. Real-time systems require architectural separation between fast analysis and slower reasoning.

What's next

Personalized skill modeling and long-term learning for music. Support for even more musical instruments, like guitars. Expanded harmonic graph modeling using learned embeddings. Voice-based interaction (“make it darker,” “more tension”)

JASS-APP

Real-time MIDI chord detection and suggestion engine. Play chords on a MIDI piano and get intelligent next-chord suggestions streamed to a web UI via WebSocket.

Prerequisites

Python 3.10+
Node.js 18+
A MIDI controller (optional — the backend can run without one)

Setup

Backend

cd backend
pip install -r requirements.txt

Frontend

cd frontend
npm install

Running

1. Start the backend

python backend/main.py

The WebSocket server starts on ws://localhost:8000/ws.

To run without a MIDI device:

DISABLE_MIDI=1 python backend/main.py

2. Start the frontend

cd frontend
npm run dev

Opens at http://localhost:3000.

Configuration

Environment Variables

Create a .env file in the backend directory with the following variables:

# Perplexity API Key (required for song recommendations)
PERPLEXITY_API_KEY=your_api_key_here

# Spotify API Credentials (optional - for album art and artist info)
SPOTIFY_CLIENT_ID=your_client_id_here
SPOTIFY_CLIENT_SECRET=your_client_secret_here

# Optional: Disable MIDI input
DISABLE_MIDI=1

Getting Perplexity API Key

Visit https://www.perplexity.ai/
Sign up or log in
Navigate to your account settings and generate an API key

Getting Spotify Credentials

Visit https://developer.spotify.com/dashboard
Log in or create a developer account
Create a new application
Copy the Client ID and Client Secret
Add them to your .env file

Note: Spotify credentials are optional. Without them, song recommendations will still work but won't display album art or artist information.

Project Structure

backend/
  main.py              # FastAPI WebSocket server + MIDI capture
  jass/                 # Chord suggestion & tonal tension engine
  pianomidi/            # MIDI input & chord detection utilities
  requirements.txt

frontend/              # Next.js + React + Tailwind CSS
  src/app/

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

CSSIn code
Next.jsIn code
PythonIn code
ReactIn code
Tailwind CSSIn code
TypeScriptIn code
VercelClaimed

6 of 7 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

260 KB

Source files

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.