# Project export: Heart-BEATS

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: Generates music dynamically with heart beat changes with signal processing and AI.
- Devpost: https://devpost.com/software/heart-beats-1snate
- GitHub: https://github.com/shainotshy1/HeartBEAT_TreeHacks2025
- Team: 3 GitHub contributor(s) — Shai Dickman (19 commits), prestonfu (12 commits), Vatsal Ananthula (4 commits)

## Devpost submission (written by the team)

### Inspiration

Heart-BEATS was inspired by the need for a real-time, personalized way to help individuals manage panic attacks and anxiety. Many existing solutions offer guided meditation or generic relaxation music, but we wanted to create a system that adapts dynamically to the user’s physiological state, providing a more immersive and effective calming experience.

### What it does

Heart-BEATS listens to the user’s bodily signals, specifically their heartbeat, and uses real-time signal processing to generate custom music beats. Measuring several attributes of the heartbeat enables us to estimate a user’s emotions. Heart-BEATS adjusts the music accordingly, helping to restore a sense of calm and stability.

### How we built it

We integrated multiple components to bring Heart-BEATS to life: Vitals Monitoring: Capturing the user's heartbeat data using Arduino KY-039 sensors. Signal Processing: Analyzing the heart rate variations to detect stress or panic states. Sound Sample Database: A curated collection of sounds designed to promote relaxation. Custom Music Generation Software: Algorithmically generates music that syncs with the user's heartbeat. OpenAI-Guided Sample Construction: Leveraging AI-generated samples to enhance the experience.

### Challenges we ran into

Ensuring accurate real-time heartbeat detection and processing. Designing music that responds naturally and effectively to physiological changes.

### Accomplishments we're proud of

Successfully implementing a system that dynamically adjusts music based on heart rate. Combining signal processing and AI-driven music generation in a novel way. Providing a potential tool for individuals who experience anxiety or panic attacks.

### What we learned

The importance of real-time signal processing and latency optimization. How different musical elements can influence emotional states. The potential of AI in personalized mental health solutions.

### What's next

Expanding sensor compatibility to work with more wearable devices. Our current system makes use of IR pulse detection, similar to the mechanisms used in Apple Watches and other wearable devices for pulse detection. Enhancing the capabilities of our music generation models. Exploring clinical applications and potential collaborations with mental health professionals.

## README (from the GitHub repository)

# HeartBEAT_TreeHacks2025

## Setup
```bash
conda create -n heartbeat python=3.10
conda activate heartbeat
pip install -e .
```

## Data
```
mkdir data
cd data
wget https://www.hexawe.net/mess/200.Drum.Machines/drums.zip
unzip drums.zip
```

## Run

```bash
python heartbeat_scripts/generate.py
```

Currently this is set up so that we can test on the simulated heart rate data.

TODO (sort of in order):
* Tune the parameters so debugging in normal mode is less painful (haven't tested much, tbh have no idea if switching beats even works since I haven't run through this for more than a minute)
* Sounds weird cuz of some thread issue maybe? Or maybe my software is just bad.
* Implement synth (just uncomment stuff in `generate.py`)
* Add logging/visualization for original/filtered signal (so people have some idea of how we got to our emotion)
* Make it sound good (hard)
* Update arduino-related things

## Detected evidence (automated analysis)

Indexed codebase: 15 recognized source files, 53 KB.
- Python (language) — detected in the code
- OpenAI (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (21 of 21)

```
.gitignore
heartbeat/__init__.py
heartbeat/beat_construction/beat_constructor.py
heartbeat/beat_construction/beat_selector.py
heartbeat/beat_construction/drum_patterns.csv
heartbeat/beat_construction/synth_patterns.csv
heartbeat/beat_construction/utils.py
heartbeat/heartbeat_sensor/__init__.py
heartbeat/heartbeat_sensor/create_wav.py
heartbeat/heartbeat_sensor/emotion.py
heartbeat/heartbeat_sensor/heartbeat_sensor.ino
heartbeat/heartbeat_sensor/heartbeat_sensors.py
heartbeat/heartbeat_sensor/read_sensor.py
heartbeat/heartbeat_sensor/signal_processing.py
heartbeat/recommender/recommender.py
heartbeat/scripts/generate.py
heartbeat/scripts/test.py
heartbeat/tests/hp.ipynb
heartbeat/tests/signals.ipynb
README.md
setup.py
```

### Dependencies

No dependency index available.

### Recent commits (newest first)

- everythign sucks
- hf api
- openai
- kinda working
- big update
- manual merge
- Merge remote-tracking branch 'origin/main'
- fixed csv
- extended buffer, added nothing tracks
- Merge remote-tracking branch 'origin/main'
- bpm able to update dynamically
- beat selector
- added synth - attached file since its post processed
- cleaned
- build beat generator with random
- Merge remote-tracking branch 'origin/main'
- nice
- removed readme pip install
- moved to heartbeat
- full test with metronome

## Key source files (fetched from GitHub, selected and truncated for size)

### setup.py

```python
from setuptools import setup, find_packages

setup(
    name="heartbeat",
    version="0.1",
    packages=find_packages(),
    install_requires=[
        'pyserial>=3.5',
        'numpy>=1.24.0',
        'pandas>=1.5.2',
        'matplotlib>=3.6.2',
        'ipykernel>=6.23.1',
        'pygame>=2.6.1',
        'openai>=1.12.0',
        'scipy>=1.10.0',
        'python-dotenv>=1.0.1',
        'heartpy>=1.2.7',
        'tqdm>=4.67.1',
        'requests>=2.32.3',
    ],
)

```

### heartbeat/__init__.py

```python


```

### heartbeat/heartbeat_sensor/__init__.py

```python


```

### heartbeat/heartbeat_sensor/read_sensor.py

```python
import serial 
import time 
# read .env file
import os
from dotenv import load_dotenv

load_dotenv()

PORT = os.getenv("PORT")
BAUDRATE = os.getenv("BAUDRATE")

arduino = serial.Serial(port=PORT, baudrate=BAUDRATE, timeout=1) 

def read_sensor(): 
    time.sleep(0.05) 
    data = arduino.readline() 
    data = data.decode('utf-8').strip()
    print(data)

while True:
    read_sensor()

```

### heartbeat/heartbeat_sensor/emotion.py

```python
from enum import Enum

class Emotion(Enum):
    HIGH_STRESS_FEAR = "High Stress/Fear"
    ANXIOUS = "Anxious"
    DEEP_RELAXATION = "Deep Relaxation"
    CALM = "Calm"
    HAPPY_EXCITED = "Happy/Excited"
    NEUTRAL = "Neutral"
    MILD_AROUSAL = "Mild Arousal"
    MILD_RELAXATION = "Mild Relaxation"
    MILD_STRESS = "Mild Stress"
    FOCUS_CONCENTRATION = "Focus/Concentration"
    INCREASED_PARASYMPATHETIC_ACTIVITY = "Increased Parasympathetic Activity"
    INCREASED_SYMPATHETIC_ACTIVITY = "Decreased Sympathetic Activity"
    MIXED_EMOTIONAL_STATE = "Mixed Emotional State"
    
all_emotion_str = [e.value for e in Emotion]
```

### heartbeat/beat_construction/utils.py

```python
import os

def list_files_in_directory(directory, ext=None):
    file_list = []
    
    # Recursively go through the directory and subdirectories
    for item in os.listdir(directory):
        full_path = os.path.join(directory, item)
        
        if os.path.isdir(full_path):  # If it's a directory, recurse into it
            result = list_files_in_directory(full_path, ext)
            if result: 
                file_list.extend(result)
        else:  # If it's a file, add it to the list
            if ext is None:
                file_list.append(full_path)
            elif os.path.splitext(full_path)[1] == ext:
                file_list.append(full_path)
    return file_list


# def group_files_by_category(directory, ext=None):
#     """Wrapper on list_files_in_directory"""
#     list_files_result = list_files_in_directory(directory, ext)
#     res_dict = {}
#     for file in list_files_result:
#         category = file.split('/')[2]  # 'data/drums/Korg DDM110/MaxV - 110_snare.wav' -> 'Korg DDM110'
#         if category not in res_dict:
#             res_dict[category] = []
#         res_dict[category].append(file)
#     return res_dict


def keep_last_folders(path):
    # Normalize the path to handle mixed separators
    normalized_path = os.path.normpath(path)
    
    # Split the path into its components
    path_components = normalized_path.split(os.sep)
    
    # Check if there are at least two components
    if len(path_components) >= 3:
        # Join the last two components (last 2 folders and file name)
        result = os.path.join(path_components[-3], os.path.join(path_components[-2], path_components[-1]))
        return result
    else:
        return normalized_path  # If the path has less than two components, return as is

```

### heartbeat/scripts/test.py

```python
from heartbeat.beat_construction.beat_constructor import Note, LayerConfig, BPM_Manager, BeatConstructor, TimeSignature, PygameWavSample
import pandas as pd
from heartbeat.beat_construction.utils import list_files_in_directory

# Song Characteristics #
time_signature = TimeSignature(4, 4)
base_unit = Note.SIXTEENTH
########################

# LOAD DRUM PATTERNS #
drum_patterns = pd.read_csv("drum_patterns.csv").to_dict(orient='records')
drum_patterns = {item['label']: eval(item['pattern']) for item in drum_patterns}
######################

# LOAD SYNTH PATTERNS #
synth_patterns = pd.read_csv("synth_patterns.csv").to_dict(orient='records')
synth_patterns = {item['label']: eval(item['pattern']) for item in synth_patterns}
######################

# LOAD DRUM SAMPLES #
directory_path = '../../data/drums'
drum_tracks = list_files_in_directory(directory_path)
#####################

# LOAD SYNTH SAMPLES #
directory_path = '../../data/synths'
synth_tracks = list_files_in_directory(directory_path)
#####################

# Generate Metronome #
bpm = 120
metronome_fn = '00.wav'
metronome_sample = None#WavSample(Note.QUARTER, "Metronome", metronome_fn)
bpm_manager = BPM_Manager(bpm, metronome_sample=metronome_sample, beat_note=Note.QUARTER, base_unit=base_unit)
######################

# Generate Beat #
num_bars = 3
layer_configs = []
drum_layers = 2
synth_layers = 1
for _ in range(drum_layers):
    layer_configs.append(LayerConfig("drums", drum_tracks, drum_patterns))
for _ in range(synth_layers):
    layer_configs.append(LayerConfig("drums", synth_tracks, synth_patterns))
beat = BeatConstructor.build_beat(layer_configs, time_signature, num_bars, base_unit, "happy")
bpm_manager.add_child(beat)
#################

# Start Beat #
bpm_manager.start()
##############
```

### heartbeat/heartbeat_sensor/create_wav.py

```python
import numpy as np
import wave
import struct
import time
import serial
from scipy.io.wavfile import write
from heartbeat_sensors import ArduinoHeartbeatSensor, SimulatedHeartbeatSensor

class HeartbeatToWAV:
    def __init__(self, sensor, duration=5, filename="heartbeat.wav", sampling_rate=1000):
        self.sensor = sensor
        self.duration = duration
        self.filename = filename
        self.sampling_rate = sampling_rate  # Audio sampling rate (1000 Hz for better quality)
        self.signal_buffer = []

    def record_signal(self):
        """Records heartbeat signal for a given duration."""
        num_samples = self.duration * self.sampling_rate
        print(f"Recording {self.duration} seconds of heartbeat data...")

        for _ in range(num_samples):
            signal = self.sensor.read_signal()
            self.signal_buffer.append(signal)
            time.sleep(1 / self.sampling_rate)  # Wait based on the sample rate

        print("Recording complete.")

    def normalize_signal(self):
        """Normalizes the heartbeat signal to fit 16-bit PCM range."""
        signal_array = np.array(self.signal_buffer)
        
        # Normalize to fit in 16-bit PCM range (-32768 to 32767)
        signal_array -= np.mean(signal_array)  # Center around zero
        signal_array /= np.max(np.abs(signal_array))  # Scale to [-1, 1]
        signal_array *= 32767  # Scale to 16-bit range
        signal_array = signal_array.astype(np.int16)

        return signal_array

    def save_to_wav(self):
        """Saves the normalized heartbeat signal as a WAV file."""
        audio_signal = self.normalize_signal()
        write(self.filename, self.sampling_rate, audio_signal)
        print(f"Heartbeat signal saved as {self.filename}")

# -------- Example Usage --------

# Use Arduino sensor if connected, else fallback to simulation
try:
    sensor = ArduinoHeartbeatSensor(serial_port="COM3")  # Adjust for your Arduino port
    print("Using Arduino sensor...")
except:
    sensor = SimulatedHeartbeatSensor()
    print("Using simulated sensor...")

# Create WAV file from heartbeat data
heartbeat_recorder = HeartbeatToWAV(sensor, duration=5, filename="heartbeat.wav", sampling_rate=1000)
heartbeat_recorder.record_signal()
heartbeat_recorder.save_to_wav()

```

### heartbeat/beat_construction/beat_selector.py

```python
import os
import openai
from typing import Dict
from .test import beats, play_beat, play_beat_with_variations  # Import from test.py

class EmotionalBeatSelector:
    def __init__(self, api_key: str):
        openai.api_key = api_key
        self.beats = beats  # Use beats dictionary from test.py
        self.pattern_cache: Dict[str, str] = {}
        
    def get_beat_for_emotion(self, emotion: str, current_bpm: float) -> str:
        """
        Use ChatGPT to select an appropriate beat name for the detected emotion.
        Returns the name of the beat to play
        """
        # Check cache first
        if emotion in self.pattern_cache:
            return self.pattern_cache[emotion]

        # Prepare prompt for ChatGPT
        prompt = f"""
        Given the following emotional state: '{emotion}' and current BPM: {current_bpm},
        select the most appropriate beat pattern from this list to help normalize the person's emotional state:
        
        Available patterns:
        {list(self.beats.keys())}
        
        Consider these guidelines:
        - For high stress/anxiety: Choose slower, simpler patterns
        - For low energy: Choose upbeat, complex patterns
        - For neutral states: Choose moderate patterns
        
        Return only the exact name of one pattern from the list.
        """

        try:
            response = openai.ChatCompletion.create(
                model="gpt-3.5-turbo",
                messages=[
                    {"role": "system", "content": "You are a music therapy expert specializing in drum patterns."},
                    {"role": "user", "content": prompt}
                ],
                temperature=0.7,
                max_tokens=50
            )
            
            beat_name = response.choices[0].message.content.strip()
            
            # Verify the beat exists
            if beat_name in self.beats:
                # Cache the result
                self.pattern_cache[emotion] = beat_name
                return beat_name
            else:
                return list(self.beats.keys())[0]  # Return first beat as fallback
            
        except Exception as e:
            print(f"Error getting beat recommendation: {str(e)}")
            return list(self.beats.keys())[0]  # Return first beat as fallback

    def play_emotional_beat(self, emotion: str, bpm: float, with_variations: bool = False):
        """
        Select and play a beat based on emotion using the same functions as test.py
        """
        beat_name = self.get_beat_for_emotion(emotion, bpm)
        
        if with_variations:
            play_beat_with_variations(beat_name, bpm)
        else:
            play_beat(beat_name, bpm)
        
        return beat_name

```

### heartbeat/scripts/generate.py

```python
import asyncio
import pandas as pd
import time
from dataclasses import dataclass
from datetime import datetime

from heartbeat.beat_construction.beat_constructor import Note, LayerConfig, BPM_Manager, BeatConstructor, TimeSignature
from heartbeat.beat_construction.utils import list_files_in_directory
from heartbeat.heartbeat_sensor.heartbeat_sensors import HeartbeatSensor, ArduinoHeartbeatSensor, SimulatedHeartbeatSensor
from heartbeat.heartbeat_sensor.signal_processing import SignalProcessor
from heartbeat.heartbeat_sensor.emotion import Emotion


@dataclass
class MusicConfig:
    time_signature: TimeSignature
    base_unit: Note
    bpm: int
    num_bars: int
    drum_layers: int
    synth_layers: int
    emotion_run_length: float  # seconds
    sensor_buffer_size: int  # ticks
    sensor_signal_filter_chunk_len: int  # ticks
    

async def async_beat_builder(layer_configs, time_signature, num_bars, base_unit, emotion: Emotion):
    return BeatConstructor.build_beat(layer_configs, time_signature, num_bars, base_unit, emotion.value)

async def main_loop(config: MusicConfig, sensor: HeartbeatSensor, debug: bool = False, logging: bool = True):
    drum_patterns = pd.read_csv("heartbeat/beat_construction/drum_patterns.csv").to_dict(orient='records')
    drum_patterns = {item['label']: eval(item['pattern']) for item in drum_patterns}
    drum_tracks = list_files_in_directory('data/drums', '.wav')
    synth_patterns = pd.read_csv("heartbeat/beat_construction/synth_patterns.csv").to_dict(orient='records')
    synth_patterns = {item['label']: eval(item['pattern']) for item in synth_patterns}
    synth_tracks = list_files_in_directory('data/synths', '.wav')
    layer_configs = []
    for _ in range(config.drum_layers):
        layer_configs.append(LayerConfig("drums", drum_tracks, drum_patterns))
    for _ in range(config.synth_layers):
        layer_configs.append(LayerConfig("drums", synth_tracks, synth_patterns))

    signal_processor = SignalProcessor()
    
    existing_beat = None
    old_emotion = Emotion.NEUTRAL  # most recent emotion used for music generation
    existing_emotion = Emotion.NEUTRAL  # most recent computed emotion
    emotion_start_time = None
    last_logged_time = None
    should_toggle = True
    
    beat = BeatConstructor.build_beat(layer_configs, config.time_signature, config.num_bars, config.base_unit, existing_emotion)
    bpm_manager = BPM_Manager(config.bpm, metronome_sample=None, beat_note=Note.QUARTER, base_unit=config.base_unit)
    bpm_manager.add_child(beat)
    
    bpm_manager.start()

    i = 0  # sensor buffer index
    try:
        while True:
            signal_value, timestamp_str = sensor.read_signal()
            timestamp_dt = datetime.strptime(timestamp_str, '%Y-%m-%d %H:%M:%S.%f')
            if emotion_start_time is None:
                emotion_start_time = timestamp_dt
                last_logged_time = timestamp_dt
            if not debug:  # simulate heartbeat delay
                time.sleep((timestamp_dt - last_logged_time).total_seconds())
            signal_processor.update_signal(signal_value)
            
            if i > sensor.buffer_size and (i + 1) % config.sensor_signal_filter_chunk_len == 0:
                filtered_signal = signal_processor.filter_noise_ema(sensor.signal_values)
                sensor.process(filtered_signal, sensor.timestamps, timing=False)
                emotion = sensor.determine_emotion()

                # if emotion has run for long enough, change the music
                if should_toggle and (timestamp_dt - emotion_start_time).total_seconds() > config.emotion_run_length:
                    if logging:
                        print(f'==== Switching songs based on emotion: {old_emotion} -> {emotion.value}')
                    old_emotion = emotion.value
                    beat = await async_beat_builder(layer_configs, config.time_signature, config.num_bars, config.base_unit, emotion)
                    bpm_manager.remove_child(existing_beat)  # FIXME: maybe wrong
                    bpm_manager.add_child(beat)
                    existing_beat = beat
                    should_toggle = False
                    
                # mark new emotion
                if existing_emotion != emotion:
                    existing_emotion = emotion
                    emotion_start_time = timestamp_dt
                    should_toggle = True
            
            i += 1

    except KeyboardInterrupt:
        bpm_manager.stop()


def generate_synthetic(debug=False):
    config = MusicConfig(
        time_signature=TimeSignature(4, 4),
        base_unit=Note.SIXTEENTH,
        bpm=120,
        num_bars=1,
        drum_layers=1,
        synth_layers=0,
        emotion_run_length=1,
        sensor_buffer_size=1000,
        sensor_signal_filter_chunk_len=100
    )
    
    sensor = SimulatedHeartbeatSensor(buffer_size=config.sensor_buffer_size)
    asyncio.run(main_loop(config, sensor, debug))
    
    
def generate_arduino(debug=False):
    config = MusicConfig(
        time_signature=TimeSignature(4, 4),
        base_unit=Note.SIXTEENTH,
        bpm=120,
        num_bars=2,
        drum_layers=2,
        synth_layers=1,
        emotion_run_length=1,
        sensor_buffer_size=1000,
        sensor_signal_filter_chunk_len=100
    )
    
    sensor = ArduinoHeartbeatSensor(buffer_size=config.sensor_buffer_size)
    asyncio.run(main_loop(config, sensor, debug))
        

if __name__ == "__main__":
    generate_synthetic()
    # generate_arduino()
```

[4 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]