# Project export: Duet: Brainwaves -> Live Music

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 11.0
- Tagline: Creativity is inherent, but expressing it is hard. Duet reads live brainwaves via EEG and uses AI to generate evolving soundscapes that reflect your emotions, giving voice to your inner symphony.
- Devpost: https://devpost.com/software/duet-0tbkxe
- GitHub: https://github.com/daniel04wawng/duet-frontend
- Demo: https://github.com/soapantelope/duet-backend
- Video: https://www.youtube.com/embed/wZucZ7wvL-w?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner (Cal Hacks: 1st Overall)
- Team: 3 GitHub contributor(s) — Justin Wu (4 commits), jjin18 (2 commits), Sophia Zhang (1 commits)

## Devpost submission (written by the team)

### Overview

Duet's music generation revolutionizes how we approach music therapy. We capture real-time brainwave data using Emotiv EEG technology, translating it into dynamic, personalized soundscapes live. Our platform, backed by machine learning, classifies emotional states and generates adaptive music that evolves with your mind. We are all intrinsically creative, but some—whether language or developmental barriers—struggle to convey it. We’re not just creating music; we’re using the intersection of art, neuroscience, and technology to let your inner mind shine. About the project

### Inspiration

Duet revolutionizes the way children with developmental disabilities—approximately 1 in 10 in the United States—express their creativity through music by harnessing EEG technology to translate brainwaves into personalized musical experiences. Daniel and Justin have extensive experience teaching music to children, but working with those who have developmental disabilities presents unique challenges: Identifying and adapting resources for non-verbal and special needs students. Integrating music therapy principles into lessons to foster creativity. Encouraging improvisation to facilitate emotional expression. Navigating the complexities of individual accessibility needs. Unfortunately, many children are left without the tools they need to communicate and express themselves creatively. That's where Duet comes in. By utilizing EEG technology, we aim to transform the way these children interact with music, giving them a voice and a means to share their feelings. At Duet, we are committed to making music an inclusive experience for all, ensuring that every child—and anyone who struggles to express themselves—has the opportunity to convey their true creative self!

### What it does

: Wear an EEG Experience your brain waves as music! Focus and relaxation levels will change how fast/exciting vs. slow/relaxing the music is.

### How we built it

: We started off by experimenting with Emotiv’s EEGs — devices that feed a stream of brain wave activity in real time! After trying it out on ourselves, the CalHacks stuffed bear, and the Ariana Grande cutout in the movie theater, we dove into coding. We built the backend in Python, leveraging the Cortex library that allowed us to communicate with the EEGs. For our database, we decided on SingleStore for its low latency, real-time applications, since our goal was to ultimately be able to process and display the brain wave information live on our frontend. Traditional live music is done procedurally, with rules manually fixed by the developer to decide what to generate. On the other hand, existing AI music generators often generate sounds through diffusion-like models and pre-set prompts. However, we wanted to take a completely new approach — what if we could have an AI be a live “composer”, where it decided based on the previous few seconds of live emotional data, a list of available instruments it can select to “play”, and what it previously generated to compose the next few seconds of music? This way, we could have live AI music generation (which, to our knowledge, does not exist yet). Powered by Google’s Gemini LLM, we crafted a prompt that would do just that — and it turned out to be not too shabby! To play our AI-generated scores live, we used Sonic Pi, a Ruby-based library that specializes in live music generation (think DJing in code). We fed this and our brain wave data to a frontend built in Next.js to display the brain waves from the EEG and sound spectrum from our audio that highlight the correlation between them. Challenges: Our biggest challenge was coming up with a way to generate live music with AI. We originally thought it was impossible and that the tech wasn’t “there” yet — we couldn’t find anything online about it, and even spent hours thinking about how to pivot to another idea that we could use our EEGs with. However, we eventually pushed through and came up with a completely new method of doing live AI music generation that, to our knowledge, doesn’t exist anywhere else! It was most of our first times working with this type of hardware, and we ran into many issues with getting it to connect properly to our computers — but in the end, we got everything to run smoothly, so it was a huge feat for us to make it all work!

### What's next

for Duet? Music therapy is on the rise – and Duet aims to harness this momentum by integrating EEG technology to facilitate emotional expression through music. With a projected growth rate of 15.59% in the music therapy sector, our mission is to empower kids and individuals through personalized musical experiences. We plan to implement our programs in schools across the states, providing students with a unique platform to express their emotions creatively. By partnering with EEG companies, we’ll ensure access to the latest technology, enhancing the therapeutic impact of our programs. Duet gives everyone a voice to express emotions and ideas that transcend words, and we are committed to making this future a reality! Built with: Emotiv EEG headset SingleStore real-time database Python Google Gemini Sonic Pi (Ruby library) Next.js

## README (from the GitHub repository)

This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app).

## Getting Started

First, run the development server:

```bash
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
```

Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.

You can start editing the page by modifying `app/page.js`. The page auto-updates as you edit the file.

This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel.

## Learn More

To learn more about Next.js, take a look at the following resources:

- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.

You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome!

## Deploy on Vercel

The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.

Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details.


## Detected evidence (automated analysis)

Indexed codebase: 17 recognized source files, 151 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Next.js (technology) — detected in the code
- React (technology) — detected in the code
- TypeScript (language) — detected in the code
- Google Gemini (technology) — claimed on Devpost, not found in the code
- Python (language) — claimed on Devpost, not found in the code
- Ruby (language) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (24 of 24)

```
.env
.eslintrc.json
.gitignore
app/api/alphabeta/route.js
app/brain-anim/brain-particles.tsx
app/brain-anim/brain-tubes.tsx
app/brain-anim/BrainAnim.tsx
app/brain-anim/data.ts
app/brain-anim/index.css
app/brain-anim/main.tsx
app/brain-anim/types.d.ts
app/brain-anim/utils.ts
app/brain-anim/vite-env.d.ts
app/EEGMusicInterface.js
app/globals.css
app/layout.js
app/page.js
app/page.module.css
index.html
jsconfig.json
next.config.mjs
package.json
README.md
tsconfig.json
```

### Dependencies

- package.json: @react-three/drei@^9.114.6, @react-three/fiber@^8.17.10, @types/node@22.7.7, @types/react@18.3.11, eslint@^8, eslint-config-next@14.2.15, mysql2@^3.11.3, next@14.2.15, react@^18, react-dom@^18, three@^0.169.0

### Recent commits (newest first)

- page layout
- alpha beta values
- brain animation
- working live visualizer based on database (hi im soap)
- Merge branch 'master' of https://github.com/daniel04wawng/duet-frontend
- Initial commit
- Updated Brain Tubes
- visual edits, glow, audio vis
- Update EEGMusicInterface.js
- Update brain-tubes.tsx
- init
- Initial commit from Create Next App

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "name": "duet-frontend",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "@react-three/drei": "^9.114.6",
    "@react-three/fiber": "^8.17.10",
    "mysql2": "^3.11.3",
    "next": "14.2.15",
    "react": "^18",
    "react-dom": "^18",
    "three": "^0.169.0"
  },
  "devDependencies": {
    "@types/node": "22.7.7",
    "@types/react": "18.3.11",
    "eslint": "^8",
    "eslint-config-next": "14.2.15"
  }
}

```

### app/page.js

```javascript
import Image from "next/image";
import styles from "./page.module.css";
import EEGMusicInterface from "./EEGMusicInterface";
import BrainAnim from "./brain-anim/BrainAnim";

export default function Home() {
  return (
    <div style={{ backgroundColor: "black" }}>
      <BrainAnim></BrainAnim>
      <EEGMusicInterface></EEGMusicInterface>
    </div>
  );
}

```

### app/layout.js

```javascript
import localFont from "next/font/local";
import "./globals.css";

const geistSans = localFont({
  src: "./fonts/GeistVF.woff",
  variable: "--font-geist-sans",
  weight: "100 900",
});
const geistMono = localFont({
  src: "./fonts/GeistMonoVF.woff",
  variable: "--font-geist-mono",
  weight: "100 900",
});

export const metadata = {
  title: "Create Next App",
  description: "Generated by create next app",
};

export default function RootLayout({ children }) {
  return (
    <html lang="en">
      <body className={`${geistSans.variable} ${geistMono.variable}`}>
        {children}
      </body>
    </html>
  );
}

```

### app/brain-anim/main.tsx

```typescript
import React from 'react';
import ReactDOM from 'react-dom/client';
import App from './BrainAnim.js';
import './index.css';

ReactDOM.createRoot(document.getElementById('root')!).render(
  <React.StrictMode>
    <App />
  </React.StrictMode>,
);

```

### app/api/alphabeta/route.js

```javascript
import mysql from 'mysql2/promise';

// Create a connection to SingleStore
const pool = mysql.createPool({
  host: process.env.SINGLESTORE_HOST, // Replace with your SingleStore host
  user: process.env.SINGLESTORE_USER, // Replace with your SingleStore user
  password: process.env.SINGLESTORE_PASSWORD, // Replace with your SingleStore password
  database: process.env.SINGLESTORE_DATABASE, // Replace with your SingleStore database name
  connectionLimit: 10,
  waitForConnections: true,
  queueLimit: 0,
});

export async function GET(req) {
  try {
    const connection = await pool.getConnection();

    // Query to get the most recent alpha and beta values
    const rows = await connection.query(
      'SELECT avg_alpha, avg_beta FROM brain_wave_data ORDER BY timestamp DESC LIMIT 1'
    );

    console.log(rows[0][0]);

    // Close the database connection
    connection.release();

    // If rows are returned, send the most recent one
    if (rows.length > 0) {
      return new Response(JSON.stringify(rows[0][0]), {
        status: 200,
        headers: { 'Content-Type': 'application/json' },
      });
    } else {
      return new Response(JSON.stringify({ message: 'No data found' }), {
        status: 404,
        headers: { 'Content-Type': 'application/json' },
      });
    }
  } catch (error) {
    console.error('Error fetching alpha/beta data:', error);
    return new Response(
      JSON.stringify({ message: 'Server error', error: error.message }),
      {
        status: 500,
        headers: { 'Content-Type': 'application/json' },
      }
    );
  }
}

```

### index.html

```html
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Thematic Scrolling Carousel with Laser Pen Cursor</title>

    <!-- Import Google Fonts -->
    <link href="https://fonts.googleapis.com/css2?family=Poppins:wght@300;700&display=swap" rel="stylesheet" />
    <link rel="stylesheet" href="style.css"> <!-- Link to the CSS file -->
  </head>

  <body>
    <div class="cursor" id="cursor"></div> <!-- Custom cursor element -->

    <div class="heading">Duet</div>

    <div class="for-container">
      <span class="for-text">for</span>
      <span class="carousel"></span>
    </div>

    <script src="script.js"></script> <!-- Link to the JavaScript file -->
  </body>
</html>


```

### app/globals.css

```css
:root {
  --background: #ffffff;
  --foreground: #171717;
}

@media (prefers-color-scheme: dark) {
  :root {
    --background: #0a0a0a;
    --foreground: #ededed;
  }
}

html,
body {
  max-width: 100vw;
  overflow-x: hidden;
}

body {
  color: var(--foreground);
  background: var(--background);
  font-family: Arial, Helvetica, sans-serif;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

* {
  box-sizing: border-box;
  padding: 0;
  margin: 0;
}

a {
  color: inherit;
  text-decoration: none;
}

@media (prefers-color-scheme: dark) {
  html {
    color-scheme: dark;
  }
}

```

### app/page.module.css

```css
.page {
  --gray-rgb: 0, 0, 0;
  --gray-alpha-200: rgba(var(--gray-rgb), 0.08);
  --gray-alpha-100: rgba(var(--gray-rgb), 0.05);

  --button-primary-hover: #383838;
  --button-secondary-hover: #f2f2f2;

  display: grid;
  background-color: black;
  grid-template-rows: 20px 1fr 20px;
  align-items: center;
  justify-items: center;
  min-height: 100svh;
  padding: 80px;
  gap: 64px;
  font-family: var(--font-geist-sans);
}

@media (prefers-color-scheme: dark) {
  .page {
    --gray-rgb: 255, 255, 255;
    --gray-alpha-200: rgba(var(--gray-rgb), 0.145);
    --gray-alpha-100: rgba(var(--gray-rgb), 0.06);

    --button-primary-hover: #ccc;
    --button-secondary-hover: #1a1a1a;
  }
}

.main {
  background-color: black;
  display: flex;
  flex-direction: column;
  gap: 32px;
  grid-row-start: 2;
}

.main ol {
  font-family: var(--font-geist-mono);
  padding-left: 0;
  margin: 0;
  font-size: 14px;
  line-height: 24px;
  letter-spacing: -0.01em;
  list-style-position: inside;
}

.main li:not(:last-of-type) {
  margin-bottom: 8px;
}

.main code {
  font-family: inherit;
  background: var(--gray-alpha-100);
  padding: 2px 4px;
  border-radius: 4px;
  font-weight: 600;
}

.ctas {
  display: flex;
  gap: 16px;
}

.ctas a {
  appearance: none;
  border-radius: 128px;
  height: 48px;
  padding: 0 20px;
  border: none;
  border: 1px solid transparent;
  transition: background 0.2s, color 0.2s, border-color 0.2s;
  cursor: pointer;
  display: flex;
  align-items: center;
  justify-content: center;
  font-size: 16px;
  line-height: 20px;
  font-weight: 500;
}

a.primary {
  background: var(--foreground);
  color: var(--background);
  gap: 8px;
}

a.secondary {
  border-color: var(--gray-alpha-200);
  min-width: 180px;
}

.footer {
  grid-row-start: 3;
  display: flex;
  gap: 24px;
}

.footer a {
  display: flex;
  align-items: center;
  gap: 8px;
}

.footer img {
  flex-shrink: 0;
}

/* Enable hover only on non-touch devices */
@media (hover: hover) and (pointer: fine) {
  a.primary:hover {
    background: var(--button-primary-hover);
    border-color: transparent;
  }

  a.secondary:hover {
    background: var(--button-secondary-hover);
    border-color: transparent;
  }

  .footer a:hover {
    text-decoration: underline;
    text-underline-offset: 4px;
  }
}

@media (max-width: 600px) {
  .page {
    padding: 32px;
    padding-bottom: 80px;
  }

  .main {
    align-items: center;
  }

  .main ol {
    text-align: center;
  }

  .ctas {
    flex-direction: column;
  }

  .ctas a {
    font-size: 14px;
    height: 40px;
    padding: 0 16px;
  }

  a.secondary {
    min-width: auto;
  }

  .footer {
    flex-wrap: wrap;
    align-items: center;
    justify-content: center;
  }
}

@media (prefers-color-scheme: dark) {
  .logo {
    filter: invert();
  }
}

```

### app/EEGMusicInterface.js

```javascript
'use client'

import React, { useEffect, useRef, useState } from 'react';

const EEGMusicInterface = () => {
  const canvasRef = useRef(null);
  const [alphaBetaData, setAlphaBetaData] = useState({ alpha: 1, beta: 1 });
  const [prevAlphaBetaData, setPrevAlphaBetaData] = useState({ alpha: 1, beta: 1 });
  const [offset, setOffset] = useState(0);
  const [interpolationProgress, setInterpolationProgress] = useState(0);

  useEffect(() => {
    const interval = setInterval(async () => {
      try {
        const res = await fetch('/api/alphabeta', { method: 'GET' });
        if (!res.ok) {
          throw new Error('Network response was not ok');
        }
        const data = await res.json();

        // Update the previous alpha/beta values and reset interpolation progress
        setPrevAlphaBetaData(alphaBetaData);
        setAlphaBetaData({ alpha: data.avg_alpha, beta: data.avg_beta });
        setInterpolationProgress(0);  // Start new interpolation cycle
      } catch (error) {
        console.error('Error fetching data:', error);
      }
    }, 200);

    return () => clearInterval(interval);
  }, []);

  useEffect(() => {
    const canvas = canvasRef.current;
    if (!canvas) {
      return;
    } 
    const ctx = canvas.getContext('2d');
    let animationFrameId;

    const resizeCanvas = () => {
      canvas.width = window.innerWidth;
      canvas.height = window.innerHeight;
    };

    const drawVisualizer = (interpolatedAlpha, interpolatedBeta) => {
      const barWidth = 1; 
      const notes = 50;
    
      ctx.shadowBlur = 20;
      ctx.shadowColor = '#5dbdf5'; 
      ctx.shadowOffsetX = 0;
      ctx.shadowOffsetY = 0;
    
      ctx.fillStyle = '#5dbdf5';
    
      for (let i = 0; i < notes; i++) {
        const barHeight = Math.abs((i / notes) - (interpolatedAlpha / 2)) * canvas.height * 0.2;
        ctx.fillRect(i * canvas.width / notes, canvas.height - barHeight - (i % 2) * 20, barWidth, barHeight + (i % 2) * 20);
      }

      ctx.shadowBlur = 0;
      ctx.shadowColor = 'transparent';
    };

    const drawText = (interpolatedAlpha, interpolatedBeta) => {
      ctx.font = '16px Arial';
      ctx.fillStyle = '#ffffff';
      ctx.textAlign = 'left';
      ctx.fillText(`Alpha: ${interpolatedAlpha.toFixed(2)}`, 10, 30);
      ctx.fillText(`Beta: ${interpolatedBeta.toFixed(2)}`, 10, 60);
    };

    window.addEventListener('resize', resizeCanvas);
    resizeCanvas();

    let audioData = new Uint8Array(128).fill(0);

    const drawWave = (y, amplitude, color, waveOffset) => {
      ctx.beginPath();
      ctx.moveTo(0, y);
      ctx.strokeStyle = color;
      ctx.lineWidth = 1;
      ctx.shadowBlur = 20; 
      ctx.shadowColor = '#5dbdf5'; 
      ctx.shadowOffsetX = 0;
      ctx.shadowOffsetY = 0;

      for (let i = 0; i < canvas.width; i++) {
        ctx.lineTo(i, y + Math.sin((i + waveOffset) * 0.02) * amplitude);
      }
      ctx.stroke();
      ctx.shadowBlur = 0;
      ctx.shadowColor = 'transparent';
    };

    const interpolate = (start, end, progress) => {
      return start + (end - start) * progress;
    };

    const animate = () => {
      ctx.fillStyle = 'rgba(0, 0, 0, 0.1)';
      ctx.fillRect(0, 0, canvas.width, canvas.height);

      // Calculate interpolation progress (between 0 and 1) over time
      const interpolatedAlpha = interpolate(prevAlphaBetaData.alpha, alphaBetaData.alpha, interpolationProgress);
      const interpolatedBeta = interpolate(prevAlphaBetaData.beta, alphaBetaData.beta, interpolationProgress);

      // Draw the waves with interpolated values
      drawWave(canvas.height / 2, Math.min(350, (interpolatedAlpha - 0.5) * 290), '#5dbdf5', offset);
      drawWave(canvas.height / 2, Math.min(350, (interpolatedBeta - 0.4) * 290), '#ffdc96', -offset);

      drawVisualizer(interpolatedAlpha, interpolatedBeta);

      // Draw the text with interpolated values
      drawText(interpolatedAlpha, interpolatedBeta);

      setOffset(offset + 0.2);
      setInterpolationProgress((prevProgress) => Math.min(prevProgress + 0.0075, 1));  // Smooth interpolation

      animationFrameId = requestAnimationFrame(animate);
    };

    animate();

    return () => {
      window.removeEventListener('resize', resizeCanvas);
      cancelAnimationFrame(animationFrameId);
    };
  }, [alphaBetaData, prevAlphaBetaData, interpolationProgress]);

  return <canvas ref={canvasRef} style={{ display: 'block' }} />;
};

export default EEGMusicInterface;
```

### app/brain-anim/vite-env.d.ts

```typescript
/// <reference types="vite/client" />

```

[6 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]