# Project export: Memody

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: Enhance reading comprehension and memory with our browser extension, which converts text into music. Engage auditory and visual memory, making information easier to absorb and recall.
- Devpost: https://devpost.com/software/memody
- GitHub: https://github.com/Axelmannen/memody
- Video: https://player.vimeo.com/video/1057271885?byline=0&portrait=0&title=0#t=
- Team: 1 GitHub contributor(s) — Axel Wennström (3 commits)

## Devpost submission (written by the team)

### Inspiration

The inspiration for Memody came from understanding how the human brain processes information through multiple sensory channels. Research shows that multi-modal learning - combining visual and auditory inputs - can significantly enhance memory retention and comprehension. This will be extraordinarily helpful for learners with dyslexia and ADHD. We were also inspired by how music has been used throughout history as a memorization tool, from ancient oral traditions to modern educational songs. By transforming text into music, we're creating a new way to experience written content that leverages these powerful cognitive connections.

### What it does

Memody transforms the reading experience by creating a synchronized audio-visual experience: Real-time text highlighting follows your natural reading pace, using a karaoke-style visualization Converts text into melodic patterns based on sentence structure, word importance, and semantic meaning Creates unique musical signatures for different types of content (articles, documentation, educational material) Offers customizable audio settings to match user preferences and content type Works seamlessly across any webpage through the browser extension

### How we built it

JavaScript-based browser extension framework for cross-browser compatibility Text analysis engine to process sentence structure and time words to music(music.ai) LLM to generate music corresponding to text Interpretative layer LLM for simplifying dense text

### Challenges we ran into

Sourcing an LLM to generate music corresponding to text Deciding on the optimal purpose of the application (enhancing learning and memorization with multi-modal learning vs. emphasis on interpreting the text into catchy tunes) challenges of matching natural sounding music to reading speed

### Accomplishments we're proud of

Created a novel approach to enhancing reading comprehension through music Developed a text-to-music system that produces coherent, pleasant melodies Built a smooth, intuitive user interface that works across various websites without significantly impacting user performance

### What we learned

the importance of the user experience in educational tools adjusting between performance and functionality, making tradeoffs The challenges of sourcing technology for cheap in a burgeoning field

### What's next

Real-time text tracking system for accurate highlighting Enhance the tool specifically for people with ADHD and dyslexia A custom algorithm that further aligns text patterns with music Build analytics to track reading comprehension improvements Add support for different languages Enhance the tool specifically for people with ADHD and dyslexia A custom algorithm that further aligns text patterns with music Build analytics to track reading comprehension improvements Add support for different languages

## README (from the GitHub repository)

No README available.

## Detected evidence (automated analysis)

Indexed codebase: 6 recognized source files, 16 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code

## Codebase structure (from repository index)

### Files (9 of 9)

```
.gitignore
background.js
index.html
lyrics.json
manifest.json
package.sh
README.md
script.js
styles.css
```

### Dependencies

No dependency index available.

### Recent commits (newest first)

- Third Commit
- Second commit
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### background.js

```javascript
// Listen for extension installation
chrome.runtime.onInstalled.addListener(() => {
  console.log('Memody Extension installed');
  
  // Create context menu item
  chrome.contextMenus.create({
    id: "openMemody",
    title: "Use Memody to memorize this",
    contexts: ["selection"]
  });
});

// Listen for extension icon click
chrome.action.onClicked.addListener((tab) => {
  chrome.tabs.create({
    url: chrome.runtime.getURL('index.html')
  });
});

// Listen for context menu clicks
chrome.contextMenus.onClicked.addListener((info, tab) => {
  if (info.menuItemId === "openMemody") {
    const selectedText = info.selectionText || '';
    if (selectedText) {
      chrome.storage.local.set({ 'selectedSongTitle': selectedText }, () => {
        chrome.tabs.create({
          url: chrome.runtime.getURL('index.html')
        });
      });
    }
  }
});
```

### index.html

```html
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Memody</title>
    <link rel="preconnect" href="https://fonts.googleapis.com">
    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
    <link href="https://fonts.googleapis.com/css2?family=Montserrat:wght@500;700&display=swap" rel="stylesheet">
    <link rel="stylesheet" href="styles.css">
</head>
<body>
    <div class="loading" id="loading">
        <div class="loading-spinner"></div>
        <p>Translating title...</p>
    </div>
    <div class="app-header">
        <img src="icons/logo.svg" alt="Memody Logo" class="logo">
        <h1 class="app-title">Memody</h1>
    </div>
    <div class="container content" id="main-content">
        <div class="header">
            <img src="image_example_eagle.jpeg" alt="Song Artwork" class="cover-image">
            <h1 class="song-title">The Civil War</h1>
            <p class="song-subtitle"></p>
        </div>
        <div class="player-container">
            <audio id="audio-player" controls>
                <source src="example_civil_war.mp3" type="audio/mpeg">
                Your browser does not support the audio element.
            </audio>
        </div>
        <div class="lyrics-container" id="lyrics-container">
            <!-- Lyrics will be inserted here by JavaScript -->
        </div>
    </div>
    <script src="script.js"></script>
</body>
</html>

```

### styles.css

```css
body {
    margin: 0;
    padding: 20px;
    font-family: 'Montserrat', sans-serif;
    background-color: #000000;
    color: #808080;
    font-weight: 500;
}

.app-header {
    display: flex;
    align-items: center;
    justify-content: center;
    gap: 12px;
    margin-bottom: 24px;
}

.logo {
    width: 32px;
    height: 32px;
}

.app-title {
    margin: 0;
    font-size: 24px;
    font-weight: 700;
    color: #ee85fe;
}

.container {
    max-width: 800px;
    margin: 0 auto;
    padding: 2rem;
    background: linear-gradient(135deg, #1a2535 0%, #1e2f45 100%);
    border-radius: 15px;
    box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
}

.player-container {
    text-align: center;
    margin-bottom: 20px;
}

#audio-player {
    width: 100%;
    max-width: 500px;
    margin: 20px 0;
}

.lyrics-container {
    line-height: 2.2;
    font-size: 20px;
    padding: 20px;
    letter-spacing: 0.5px;
}

.lyrics-line {
    padding: 8px;
    margin: 4px 0;
    border-radius: 4px;
}

.word {
    display: inline-block;
    padding: 2px 4px;
    margin: 0 2px;
    border-radius: 3px;
    transition: all 0.3s ease;
    color: #808080;
    font-weight: 700;
    cursor: pointer;
}

.word:hover {
    color: #ee85fe !important;
}

.word.active {
    color: white;
}

.word.sung {
    color: white;
}

.header {
    text-align: center;
    margin-bottom: 30px;
}

.cover-image {
    max-width: 300px;
    width: 100%;
    height: auto;
    border-radius: 10px;
    margin-bottom: 20px;
    box-shadow: 0 4px 20px rgba(255, 255, 255, 0.1);
}

.song-title {
    font-size: 32px;
    color: white;
    margin: 0 0 10px 0;
    font-weight: 700;
    letter-spacing: 1px;
}

.song-subtitle {
    font-size: 18px;
    color: #808080;
    margin: 0;
    font-weight: 500;
    letter-spacing: 0.5px;
}

/* Loading spinner */
.loading {
    display: none;
    position: fixed;
    top: 50%;
    left: 50%;
    transform: translate(-50%, -50%);
    text-align: center;
    width: 100%;
}

.loading p {
    margin-top: 24px;
    color: #ee85fe;
    font-weight: 500;
    font-size: 24px;
}

.loading-spinner {
    width: 80px;
    height: 80px;
    border: 6px solid rgba(238, 133, 254, 0.2);
    border-top: 6px solid #ee85fe;
    border-radius: 50%;
    animation: spin 1s linear infinite;
    margin: 0 auto;
}

@keyframes spin {
    0% { transform: rotate(0deg); }
    100% { transform: rotate(360deg); }
}

.loading.active {
    display: block;
}

.content.loading {
    opacity: 0.5;
    pointer-events: none;
}

```

### script.js

```javascript
document.addEventListener('DOMContentLoaded', async () => {
    const audioPlayer = document.getElementById('audio-player');
    const lyricsContainer = document.getElementById('lyrics-container');
    const songTitleElement = document.querySelector('.song-title');
    const loadingElement = document.getElementById('loading');
    const loadingMessage = loadingElement.querySelector('p');
    const mainContent = document.getElementById('main-content');

    const MUSIC_AI_API_KEY = 'b6c6aae5-652a-441f-af78-c1d2c69f045a'; 

    async function processAudioFile() {
        try {
            // First, get upload URL
            const uploadResponse = await fetch('https://api.music.ai/api/upload', {
                method: 'GET',
                headers: {
                    'Authorization': MUSIC_AI_API_KEY
                }
            });
            
            if (!uploadResponse.ok) {
                throw new Error('Failed to get upload URL');
            }
            
            const { uploadUrl, downloadUrl } = await uploadResponse.json();
            console.log('Got upload URL:', uploadUrl);
            console.log('Got download URL:', downloadUrl);

            // Upload the audio file
            const audioUrl = chrome.runtime.getURL('example_civil_war.mp3');
            const audioResponse = await fetch(audioUrl);
            const audioBlob = await audioResponse.blob();
            
            const uploadResult = await fetch(uploadUrl, {
                method: 'PUT',
                body: audioBlob,
                headers: {
                    'Content-Type': 'audio/mpeg'
                }
            });

            if (!uploadResult.ok) {
                throw new Error('Failed to upload audio file');
            }
            console.log('Successfully uploaded audio file');

            // Create a job to process the audio
            const jobResponse = await fetch('https://api.music.ai/api/job', {
                method: 'POST',
                headers: {
                    'Authorization': MUSIC_AI_API_KEY,
                    'Content-Type': 'application/json'
                },
                body: JSON.stringify({
                    name: "Process Audio for Lyrics",
                    workflow: "lyricstiming",
                    params: {
                        inputUrl: downloadUrl
                    }
                })
            });

            if (!jobResponse.ok) {
                throw new Error('Failed to create job');
            }

            const jobData = await jobResponse.json();
            console.log('Created job:', jobData);
            const jobId = jobData.id;

            // Poll for job completion
            while (true) {
                const statusResponse = await fetch(`https://api.music.ai/api/job/${jobId}/status`, {
                    headers: {
                        'Authorization': MUSIC_AI_API_KEY
                    }
                });
                
                const statusData = await statusResponse.json();
                console.log('Job status:', statusData);
                
                if (statusData.status === 'FAILED') {
                    throw new Error('Job processing failed');
                }
                
                if (statusData.status === 'SUCCEEDED') {
                    // Get the job results
                    const resultResponse = await fetch(`https://api.music.ai/api/job/${jobId}`, {
                        headers: {
                            'Authorization': MUSIC_AI_API_KEY
                        }
                    });
                    
                    const finalJobData = await resultResponse.json();
                    console.log('Final job data:', finalJobData);

                    // Fetch the lyrics from the result URL
                    if (finalJobData.result && finalJobData.result.lyrics) {
                        const lyricsResponse = await fetch(finalJobData.result.lyrics);
                        if (!lyricsResponse.ok) {
                            throw new Error('Failed to fetch lyrics data');
                        }
                        const lyricsData = await lyricsResponse.json();
                        console.log('Fetched lyrics data:', lyricsData);
                        return lyricsData;
                    } else {
                        throw new Error('No lyrics URL in job result');
                    }
                }
                
                // Wait 2 seconds before next poll
                await new Promise(resolve => setTimeout(resolve, 2000));
            }
        } catch (error) {
            console.error('Error processing audio:', error);
            throw error;
        }
    }

    async function translateText(text) {
        const response = await fetch('https://api.mistral.ai/v1/chat/completions', {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
                'Authorization': 'Bearer ZO2wrYPV2XQqCBDQjVvGDZ2Yb0414PpN'
            },
            body: JSON.stringify({
                model: "mistral-tiny",
                messages: [
                    {
                        role: "system",
                        content: "Return a suitable title for the following text. Approximately three words. Do not add quotees or write 'title:' or anything like that."
                    },
                    {
                        role: "user",
                        content: text
                    }
                ]
            })
        });

        if (!response.ok) {
            throw new Error('Translation failed');
        }

        const data = await response.json();
        return data.choices[0].message.content.trim();
    }
    
    try {
        // Show loading state
        loadingElement.classList.add('active');
        mainContent.classList.add('loading');
        if (loadingMessage) {
            loadingMessage.tex
[truncated — 5856 more characters]
```