# Project export: DAWZY
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: UC Berkeley AI Hackathon 2025
- Tagline: DAWZY is a new paradigm for music creation. Whether you speak, hum, or type, DAWZY works alongside you in any digital audio workstation to help bring your musical ideas to life.
- Devpost: https://devpost.com/software/dawzy
- GitHub: https://github.com/rrisic/DAWZY
- Video: https://www.youtube.com/embed/R5PT1xjBsx0?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 4 GitHub contributor(s) β Ryder Risic (14 commits), PeterM (5 commits), Sanchit Singh (3 commits), waso (2 commits)
## Devpost submission (written by the team)
### Inspiration
π§ Music production software is incredibly powerful, but often intimidating. Many aspiring creators hear melodies in their heads but lack the technical knowledge or time to bring those ideas to life. We wanted to change that. Our goal was to build an assistant that empowers anyone, beginners & seasoned artists, to turn their musical ideas into reality with just their voice or a simple prompt.
### What it does
π€ DAWZY is an AI music production assistant that integrates with any DAW (digital audio workstation). It supports: Voice-to-Voice Communication: Have a natural conversation with the assistant about your music project. Text-to-Music Prompting: Describe what you want and let DAWZY generate instruments, tracks, and effects. Humming-to-MIDI Conversion: Hum a melody and have it transcribed into a playable MIDI part. DAW Control via Language: Tell DAWZY to add reverb, adjust EQs, or create new instruments, and it writes scripts to control your DAW environment directly. Contextual Memory: DAWZY keeps track of your session and helps build on ideas progressively.
### How we built it
π Frontend: A React-based chat interface with support for audio messages, speech input, and real-time feedback. Backend: A Flask server acting as a bridge between user input, language models, and Reaper (our current DAW prototype). LLMs: OpenAI models power the conversational and action logic, while Claude handles DAW scripting. Music Generation: We integrated Beatoven.ai for prompt-based music generation. Speech-to-Speech: We designed the architecture to allow natural voice dialogue with OpenAI's new speech model. Audio Conversion Tools: We implemented humming-to-MIDI by recording audio, converting it to WAV, and sending it to our MIDI processing logic. Reaper Integration: Using python-reapy, we remotely manipulate Reaper sessions from user commands.
### Challenges we ran into
β οΈ Audio Handling: Getting consistent recording formats and playback behavior across browsers was surprisingly complex. Audio Handling: Getting consistent recording formats and playback behavior across browsers was surprisingly complex. LLM Integration: Coordinating different models (chat, voice, and action) while maintaining session continuity took careful planning. LLM Integration: Coordinating different models (chat, voice, and action) while maintaining session continuity took careful planning. DAW Control: Reaper exposes a lot of functionality, but translating natural language into precise DAW scripting was a major challenge. DAW Control: Reaper exposes a lot of functionality, but translating natural language into precise DAW scripting was a major challenge. Latency and Feedback: Ensuring that the assistant felt responsive, even when handling complex tasks like MIDI conversion, required efficient message handling and is something we wish to improve on in the future Latency and Feedback: Ensuring that the assistant felt responsive, even when handling complex tasks like MIDI conversion, required efficient message handling and is something we wish to improve on in the future
### Accomplishments we're proud of
π A Working End-to-End Flow: From voice input to MIDI playback, we created a seamless creative loop. A Working End-to-End Flow: From voice input to MIDI playback, we created a seamless creative loop. Intelligent DAW Manipulation: DAWZY can write and run scripts to make edits inside a live DAW session. Intelligent DAW Manipulation: DAWZY can write and run scripts to make edits inside a live DAW session. Multimodal Interaction: Users can interact by typing, speaking, or even humming. Multimodal Interaction: Users can interact by typing, speaking, or even humming. User-Friendly UI: The chat interface feels intuitive and responsive even when performing behind-the-scenes audio and AI processing. User-Friendly UI: The chat interface feels intuitive and responsive even when performing behind-the-scenes audio and AI processing.
### What we learned
π How to integrate multiple AI services together in real time. How to integrate multiple AI services together in real time. The importance of UX when building tools for creative professionals. The importance of UX when building tools for creative professionals. Audio processing and format compatibility challenges across platforms. Audio processing and format compatibility challenges across platforms. Designing systems that adapt LLMs to real-world control tasks beyond text generation. Designing systems that adapt LLMs to real-world control tasks beyond text generation.
### What's next
π Full DAW Support: We plan to expand DAWZY beyond Reaper to support other popular DAWs like Ableton, Logic Pro, and FL Studio. Full DAW Support: We plan to expand DAWZY beyond Reaper to support other popular DAWs like Ableton, Logic Pro, and FL Studio. Real-Time MIDI Feedback: Users will be able to edit melodies or chord progressions conversationally while hearing live previews. Real-Time MIDI Feedback: Users will be able to edit melodies or chord progressions conversationally while hearing live previews. On-Device Voice Looping and Jamming: Think of DAWZY as your AI bandmate. On-Device Voice Looping and Jamming: Think of DAWZY as your AI bandmate. Plugin Suggestion and Effect Chain Design: DAWZY will recommend and auto-configure effect chains for genres or moods. Plugin Suggestion and Effect Chain Design: DAWZY will recommend and auto-configure effect chains for genres or moods. Deeper VAPI Integration: Fully enable two-way natural voice conversations. Deeper VAPI Integration: Fully enable two-way natural voice conversations. Our long-term vision is to make DAWZY the ultimate collaborator for music producersβcreative, technical, and always ready to jam.
## README (from the GitHub repository)
## πΆ DAWZY
**A new paradigm for music creation**
DAWZY is an AI-powered music assistant that connects to your DAW and brings your creative ideas to life, whether you speak them, type them, or hum them. From melody sketching to full track manipulation, DAWZY blends intelligent conversation with powerful DAW control to help anyone create the music they hear in their head.
### π‘Built at Berkeley's AI Hackathon 2025
### π οΈ Built With
- OpenAI API: Chat + voice-to-voice conversation
- Claude AI: DAW scripting logic
- Beatoven.ai: AI music generation from prompts
- Python (Flask): Backend server and DAW control
- React + Tailwind: Frontend chat UI and audio interface
- python-reapy: Live control over Reaper DAW
- Web Audio API: Recording and playback in-browser
### π§ Why It Matters
Making music is powerful, but modern DAWs can be intimidating. Many people never produce the melodies they imagine because the tools are complex, expensive, or inaccessible. DAWZY removes that barrier.
Whether youβre a beginner with a melody in your head or a seasoned artist trying to iterate faster, DAWZY helps you move from idea to reality, effortlessly.
### π§ What It Does
##### DAWZY is your AI collaborator in the studio.
- π£ Voice-to-Voice Chat: Talk to DAWZY like you would a co-producer. It listens, responds, and acts.
- π΅ Humming-to-MIDI: Hum a tune. DAWZY converts it into MIDI and plays it back with an instrument of your choice.
- π Prompt-Based Music Generation: Describe what you want - βlofi chill beat with pianoβ - and DAWZY builds it using Beatoven.
- π DAW Scripting via Language: Say βAdd reverb to track 2β or βIncrease the bassβ β DAWZY generates scripts to modify your DAW.
- π§ Session Memory: DAWZY remembers context from your conversation and your DAW.
### π¬ Interaction Modes
#### π Voice Button
Start a conversation with the DAWZY AI assistant using your mic
#### ποΈ Record Button
Hum or sing a melody
- Converts to MIDI for easy DAW integration
- π§ Audio Chat Playback
Messages with attached audio
- Play/pause controls and progress bar
### π§ͺ Prototype Demo
β‘ *[Check out our demo video!](https://www.youtube.com)* (coming soon)
DAWZY currently works with Reaper DAW and supports:
- π£ Voice and text chat
- π Live DAW editing
- π΅ Melody-to-MIDI conversion
- πΌ Music generation via prompt
- π Interactive audio buttons and playback
### πͺ What's Next
- π Support for Ableton, Logic, FL Studio
- 𧡠Real-time looping and jamming
- π Plugin/effect chain suggestions
- π£οΈ Full VAPI voice pipeline
- β±οΈ Faster, local MIDI preview generation
### π€ Team
Built with creativity by Ryder Risic, Sanchit Singh, Peter Ma, Sawyer Blankenship
π Berkeley's AI Hackathon 2025 Submission
## Detected evidence (automated analysis)
Indexed codebase: 25 recognized source files, 187 KB.
- CSS (language) β detected in the code
- Flask (technology) β detected in the code
- HTML (language) β detected in the code
- JavaScript (language) β detected in the code
- OpenAI (technology) β detected in the code
- Python (language) β detected in the code
- React (technology) β detected in the code
- Tailwind CSS (technology) β detected in the code
## Codebase structure (from repository index)
### Files (31 of 31)
```
.gitignore
backend/app.py
backend/requirements.txt
backend/test_beatoven.py
BEETHOVEN_SETUP.md
file.txt
generating_code_rag.py
hardcode.py
index.html
main.js
package.json
preload.js
rag_results.json
README.md
requirements.txt
setup_env.py
src/App.jsx
src/components/AIVoiceResponse.jsx
src/components/AudioMessage.jsx
src/components/ChatWindow.jsx
src/components/MusicGenerationResult.jsx
src/components/RecordButton.jsx
src/components/VAPIVoiceButton.jsx
src/components/VoiceButton.jsx
src/index.css
src/main.jsx
tailwind.config.js
test_beethoven.py
test_whisper.py
VAPI_SETUP.md
vite.config.js
```
### Dependencies
- backend/requirements.txt: flask@>=2.3.0, flask-cors@>=4.0.0, numpy@>=1.24.0, openai@>=1.0.0, python-dotenv@>=1.0.0, python-reapy@>=0.10.0, requests@>=2.31.0, websockets@>=11.0.0
- package.json: @vitejs/plugin-react@^4.0.0, cross-env@^7.0.3, electron@^27.0.0, electron-builder@^24.13.3, react@^18.2.0, react-dom@^18.2.0, tailwindcss@^3.3.0, vite@^5.0.0
- requirements.txt: openai, python-dotenv, python-reapy, requests
### Recent commits (newest first)
- Merge branch 'main' of https://github.com/rrisic/DAWZY
- flowchart
- Update README.md
- thumbnail
- Faster tool call
- beathoven integration working
- Merge branch 'main' of https://github.com/rrisic/DAWZY
- buttons2
- Midi notes interaction w backend
- buttons
- Merge branch 'main' of https://github.com/rrisic/DAWZY
- voice
- Tools for llm to adjust FX param
- Claude function call test
- vapi start
- UI record button, store locally
- Update file.txt
- changed model
- changed model
- Add llm and Reaper connection to backend
## Key source files (fetched from GitHub, selected and truncated for size)
### VAPI_SETUP.md
```markdown
# VAPI Integration Setup
This guide explains how to set up VAPI for real-time voice conversation in DAWZY.
## Prerequisites
1. **VAPI Account**: Sign up at [vapi.ai](https://vapi.ai)
2. **API Key**: Get your VAPI API key from the dashboard
3. **Assistant ID** (optional): Create an assistant in VAPI dashboard
## Environment Variables
Create a `.env` file in the `backend/` directory with the following variables:
```env
# OpenAI API Key for LLM processing
OPENAI_API_KEY=your-openai-api-key-here
# VAPI API Key for real-time voice processing
VAPI_API_KEY=your-vapi-api-key-here
# VAPI Assistant ID (optional - can be created via VAPI dashboard)
VAPI_ASSISTANT_ID=your-vapi-assistant-id-here
```
## Python Dependencies
Install the required Python packages:
```bash
cd backend
pip install -r requirements.txt
```
## VAPI Configuration
### Option 1: Use VAPI Assistant ID
1. Go to [VAPI Dashboard](https://console.vapi.ai)
2. Create a new assistant
3. Configure the assistant with:
- **Model**: GPT-4
- **System Prompt**: "You are a helpful REAPER assistant..."
- **Voice**: Choose your preferred voice
4. Copy the Assistant ID to your `.env` file
### Option 2: Let Backend Create Assistant
If you don't provide `VAPI_ASSISTANT_ID`, the backend will create a temporary assistant for each call.
## Usage
1. **Start DAWZY**: The backend will automatically start the WebSocket server on port 8765
2. **Voice Button**: Click the green microphone button (π€) to start a voice conversation
3. **Real-time Processing**: Speak naturally - VAPI will transcribe your speech and respond with voice
4. **Integration**: Voice transcripts are automatically sent to the chat and processed by the LLM
## Architecture
### Frontend (Electron)
- **VAPIVoiceButton**: Real-time audio capture and WebSocket communication
- **WebSocket Client**: Streams audio chunks to backend
- **UI Feedback**: Visual indicators for connection and recording status
### Backend (Python)
- **WebSocket Server**: Handles real-time audio streaming
- **VAPI Client**: Manages VAPI calls and audio processing
- **Audio Processor**: Converts and buffers audio data
- **LLM Integration**: Processes voice transcripts through existing chat system
### Data Flow
1. User clicks voice button β Frontend starts recording
2. Audio chunks β WebSocket β Backend
3. Backend β VAPI β Speech-to-Text
4. Transcript β LLM β Response
5. Response β Chat interface
## Troubleshooting
### WebSocket Connection Issues
- Check if backend is running
- Verify port 8765 is not blocked
- Check browser console for connection errors
### VAPI Errors
- Verify API key is correct
- Check VAPI account has sufficient credits
- Ensure assistant is properly configured
### Audio Issues
- Check microphone permissions
- Verify audio format compatibility
- Check browser audio settings
## Security Notes
- API keys are stored in `.env` file (not committed to git)
- WebSocket server runs on localhost only
- Audio data is processed locally be
[truncated β 59 more characters]
```
### BEETHOVEN_SETUP.md
```markdown
# Beatoven.ai Music Generation Setup
This guide will help you set up the Beatoven.ai music generation feature to replace Lyria RealTime.
## Prerequisites
1. **Beatoven.ai API Key**: You'll need a Beatoven.ai API key to access their music generation service
2. **Python Environment**: Make sure your backend virtual environment is activated
3. **REAPER**: For importing generated music tracks
## Setup Steps
### 1. Get Beatoven.ai API Key
1. Go to [Beatoven.ai](https://beatoven.ai)
2. Sign up for an account
3. Navigate to your API settings
4. Generate a new API key
5. Copy the generated API key
### 2. Configure Environment Variables
Create or update the `.env` file in the `backend/` directory:
```bash
# Existing OpenAI key
OPENAI_API_KEY=your_openai_api_key_here
# Beatoven.ai key (replace the old Google AI key)
BEATOVEN_AI_API_KEY=your_beatoven_ai_api_key_here
```
### 3. Install Dependencies
Make sure your virtual environment is activated, then install the updated dependencies:
```bash
cd backend
pip install -r requirements.txt
```
### 4. Test the Setup
1. Start the backend:
```bash
cd backend
python app.py
```
2. Start the frontend:
```bash
npm run dev
```
3. Test music generation by asking DAWZY to create music:
- "Create a techno track with heavy bass"
- "Generate a peaceful ambient melody"
- "Make a drum and bass beat at 140 BPM"
- "can you generate me a snare drumline please"
## How It Works
1. **User Input**: You ask DAWZY to create music through voice or text
2. **OpenAI Processing**: OpenAI GPT analyzes your request and creates structured instructions
3. **Beatoven.ai Generation**: Beatoven.ai generates the actual music track based on your request
4. **Async Processing**: The system polls for completion and downloads the generated track
5. **REAPER Integration**: The generated track is automatically imported into REAPER
6. **File Management**: Tracks are saved with descriptive filenames based on your request
## Current Implementation
The system now:
- β
Processes music generation requests through OpenAI GPT
- β
Creates structured instructions for Beatoven.ai
- β
Generates high-quality music using Beatoven.ai's async API
- β
Polls for completion with proper timeout handling
- β
Downloads and saves generated tracks
- β
Imports tracks into REAPER automatically
- β
Displays detailed track information in the chat
- β
Saves files with descriptive names based on user requests
## Beatoven.ai Features
Beatoven.ai provides:
- **High-quality music generation** with professional production quality
- **Dynamic, fluid audio** with rich harmonic content
- **Multiple genres and styles** support
- **Async processing** for complex compositions
- **Stem separation** (bass, chords, melody, percussion)
- **Studio-quality output** that sounds like professional production
## API Workflow
1. **Compose Request**: Send prompt to `/api/v1/tracks/compose`
2. **Get Task ID**: Receive task_id for tracking
3. **Poll
[truncated β 2407 more characters]
```
### requirements.txt
```
# REAPER Chatbot Dependencies
# Core dependencies
openai
python-dotenv
python-reapy
# Additional useful packages for development
requests
```
### package.json
```
{
"name": "music-assistant",
"version": "1.0.0",
"main": "main.js",
"scripts": {
"start": "electron .",
"dev": "vite",
"build": "vite build",
"preview": "vite preview",
"electron-dev": "cross-env NODE_ENV=development electron .",
"electron-prod": "npm run build && cross-env NODE_ENV=production electron .",
"package": "npm run build && electron-builder",
"dist": "npm run build && electron-builder --publish=never"
},
"dependencies": {
"electron": "^27.0.0",
"react": "^18.2.0",
"react-dom": "^18.2.0"
},
"devDependencies": {
"@vitejs/plugin-react": "^4.0.0",
"cross-env": "^7.0.3",
"electron-builder": "^24.13.3",
"tailwindcss": "^3.3.0",
"vite": "^5.0.0"
},
"build": {
"appId": "com.musicassistant.app",
"productName": "Music Production Assistant",
"directories": {
"output": "release"
},
"files": [
"main.js",
"preload.js",
"dist/**/*",
"backend/**/*",
"package.json"
],
"extraResources": [
{
"from": "backend",
"to": "backend"
}
],
"win": {
"target": "nsis",
"icon": "assets/icon.ico"
},
"mac": {
"target": "dmg",
"icon": "assets/icon.icns"
},
"linux": {
"target": "AppImage",
"icon": "assets/icon.png"
}
}
}
```
### backend/requirements.txt
```
openai>=1.0.0
python-dotenv>=1.0.0
flask>=2.3.0
flask-cors>=4.0.0
websockets>=11.0.0
requests>=2.31.0
python-reapy>=0.10.0
numpy>=1.24.0
```
### main.js
```javascript
const { app, BrowserWindow, ipcMain } = require('electron')
const path = require('path')
const fs = require('fs')
const { spawn } = require('child_process')
let mainWindow
let pythonProcess = null
let melodyCounter = 0
// Disable GPU acceleration to prevent GPU process crashes
app.disableHardwareAcceleration()
// Handle GPU process crashes gracefully
app.commandLine.appendSwitch('--disable-gpu-sandbox')
app.commandLine.appendSwitch('--disable-software-rasterizer')
app.commandLine.appendSwitch('--disable-dev-shm-usage')
app.commandLine.appendSwitch('--no-sandbox')
// Create recordings directory if it doesn't exist
const recordingsDir = path.join(__dirname, 'recordings')
if (!fs.existsSync(recordingsDir)) {
fs.mkdirSync(recordingsDir, { recursive: true })
} else {
// Clear the recordings directory on startup
try {
const files = fs.readdirSync(recordingsDir)
for (const file of files) {
const filePath = path.join(recordingsDir, file)
fs.unlinkSync(filePath)
console.log(`Deleted old recording: ${filePath}`)
}
console.log('Recordings folder cleared on startup')
} catch (error) {
console.error('Error clearing recordings folder:', error)
}
}
// Reset melody counter on startup
function resetMelodyCounter() {
melodyCounter = 0
}
function createWindow () {
mainWindow = new BrowserWindow({
width: 800,
height: 1000,
minHeight: 800,
webPreferences: {
preload: path.join(__dirname, 'preload.js'),
nodeIntegration: false,
contextIsolation: true,
// Disable hardware acceleration for renderer
enableRemoteModule: false,
webSecurity: true,
// Disable hardware acceleration
offscreen: false
},
// Make it look more professional for demos
titleBarStyle: 'default',
show: false, // Don't show until ready
icon: path.join(__dirname, 'assets/icon.png'), // Optional: add an icon
// Prevent flashing
backgroundColor: '#2e026d' // Match the gradient background
})
// Load the built React app
const isDev = process.env.NODE_ENV === 'development'
if (isDev) {
mainWindow.loadURL('http://localhost:5174')
} else {
mainWindow.loadFile(path.join(__dirname, 'dist/index.html'))
}
// Show window when ready to prevent visual flash
mainWindow.once('ready-to-show', () => {
mainWindow.show()
// Focus the window
mainWindow.focus()
})
// Handle window closed
mainWindow.on('closed', () => {
mainWindow = null
})
// Handle GPU process crashes
mainWindow.webContents.on('crashed', (event, killed) => {
console.log('Renderer process crashed:', killed)
// Reload the window if it crashes
if (!killed) {
mainWindow.reload()
}
})
// Optional: Open DevTools in development
if (isDev) {
mainWindow.webContents.openDevTools()
}
}
function startPythonBackend() {
// Start Python backend process
const pythonPath = process.platform === 'win32' ? 'python' : 'python3'
pythonProcess = spawn(pythonPath, ['backend/app.py'], {
stdio: ['pipe', 'pipe', 'pipe'],
cwd: __dirname
})
pythonProcess.stdout.on('data', (data) => {
console.log('Python backend:', data.toString())
})
pythonProcess.stderr.on('data', (data) => {
console.error('Python backend error:', data.toString())
})
pythonProcess.on('close', (code) => {
console.log('Python backend process exited with code:', code)
// Try to restart the backend if it crashes
if (mainWindow && !app.isQuitting) {
setTimeout(startPythonBackend, 2000)
}
})
pythonProcess.on('error', (error) => {
console.error('Failed to start Python backend:', error)
})
}
// IPC handlers for communication with renderer process
ipcMain.handle('send-to-backend', async (event, message) => {
if (!pythonProcess) {
throw new Error('Python backend not running')
}
return new Promise((resolve, reject) => {
// Set up response listener
const responseHandler = (data) => {
try {
const response = JSON.parse(data.toString())
if (response.type === 'response') {
pythonProcess.stdout.removeListener('data', responseHandler)
resolve({
reply: response.content,
success: response.success,
actions: response.actions || []
})
}
} catch (e) {
// Not a JSON response, ignore
}
}
pythonProcess.stdout.on('data', responseHandler)
// Send message to Python backend via stdin
try {
pythonProcess.stdin.write(JSON.stringify({ type: 'message', content: message }) + '\n')
// Timeout after 10 seconds
setTimeout(() => {
pythonProcess.stdout.removeListener('data', responseHandler)
reject(new Error('Backend response timeout'))
}, 10000)
} catch (error) {
pythonProcess.stdout.removeListener('data', responseHandler)
reject(error)
}
})
})
ipcMain.handle('send-voice-to-backend', async (event, audioData) => {
if (!pythonProcess) {
throw new Error('Python backend not running')
}
// Send voice data to Python backend
pythonProcess.stdin.write(JSON.stringify({ type: 'voice', content: audioData }) + '\n')
return {
reply: 'π€ Voice processing will be implemented soon!'
}
})
ipcMain.handle('get-backend-status', async () => {
return {
connected: pythonProcess !== null && !pythonProcess.killed,
pid: pythonProcess ? pythonProcess.pid : null
}
})
// Save audio file to recordings directory
ipcMain.handle('save-audio-file', async (event, buffer, filename) => {
try {
// Increment counter and create sequential filename
melodyCounter++
const safeFilename = `Melody_${melodyCounter}.wav`
const filePath = path.join(recordingsDir, safeFilename)
// Write the file
fs.writeFileSync(filePath, buffer)
console.log(`Audio file saved: ${filePath}`)
return {
success: true,
filePath: filePath,
file
[truncated β 1444 more characters]
```
### src/main.jsx
```javascript
import React from 'react'
import ReactDOM from 'react-dom/client'
import App from './App.jsx'
import './index.css'
ReactDOM.createRoot(document.getElementById('root')).render(
,
)
```
### src/App.jsx
```javascript
import ChatWindow from './components/ChatWindow'
export default function App() {
return (
)
}
```
### backend/app.py
```python
#!/usr/bin/env python3
"""
Music Production Assistant Backend
Handles LLM processing, OpenAI Whisper transcription, and REAPER integration using reapy
"""
import json
import sys
import logging
import threading
import time
import asyncio
import websockets
import base64
import io
from datetime import datetime
import random
import os
from typing import Dict, Any, List
import wave
import tempfile
import numpy as np
# Load environment variables from .env file
try:
from dotenv import load_dotenv
import os
# Load .env from the backend directory
backend_dir = os.path.dirname(os.path.abspath(__file__))
env_path = os.path.join(backend_dir, '.env')
load_dotenv(env_path)
except ImportError:
print("Warning: python-dotenv not found. Install with: pip install python-dotenv")
# OpenAI API
try:
from openai import OpenAI
except ImportError:
print("Error: openai not found. Please install with: pip install openai")
OpenAI = None
# Beatoven.ai for music generation
import requests
# Flask for HTTP endpoints
try:
from flask import Flask, request, jsonify
from flask_cors import CORS
except ImportError:
print("Error: flask not found. Please install with: pip install flask flask-cors")
Flask = None
# Import reapy instead of ReaScript API
try:
import reapy
from reapy import reascript_api as RPR
except ImportError:
print("Error: reapy not found. Please install with: pip install python-reapy")
# Fallback for testing outside REAPER
def RPR_ShowConsoleMsg(msg):
print(msg)
# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
# Initialize Flask app
app = Flask(__name__)
CORS(app) # Enable CORS for all routes
# Global conversation history (in production, you'd want to store this per user/session)
conversation_history = []
MAX_HISTORY = 10 # Keep last 10 messages for context
@app.route('/transcribe', methods=['POST'])
def transcribe_audio():
"""Transcribe audio using OpenAI Whisper"""
try:
data = request.get_json()
if not data or 'audio' not in data:
return jsonify({'success': False, 'error': 'No audio data provided'})
# Decode base64 audio
audio_data = base64.b64decode(data['audio'])
# Create temporary file
with tempfile.NamedTemporaryFile(suffix='.webm', delete=False) as temp_file:
temp_file.write(audio_data)
temp_file_path = temp_file.name
try:
# Transcribe using OpenAI Whisper
if not OpenAI:
return jsonify({'success': False, 'error': 'OpenAI not available'})
client = OpenAI(api_key=os.getenv('OPENAI_API_KEY'))
with open(temp_file_path, 'rb') as audio_file:
transcript = client.audio.transcriptions.create(
model="whisper-1",
file=audio_file,
response_format="text"
)
logger.info(f"Transcription successful: {transcript}")
return jsonify({
'success': True,
'transcript': transcript
})
finally:
# Clean up temporary file
os.unlink(temp_file_path)
except Exception as e:
logger.error(f"Transcription error: {e}")
return jsonify({'success': False, 'error': str(e)})
@app.route('/tts', methods=['POST'])
def text_to_speech():
"""Convert text to speech using OpenAI TTS"""
try:
data = request.get_json()
if not data or 'text' not in data:
return jsonify({'success': False, 'error': 'No text provided'})
text = data['text']
logger.info(f"Converting to speech: {text}")
if not OpenAI:
return jsonify({'success': False, 'error': 'OpenAI not available'})
client = OpenAI(api_key=os.getenv('OPENAI_API_KEY'))
# Generate speech using OpenAI TTS
response = client.audio.speech.create(
model="tts-1",
voice="alloy", # Options: alloy, echo, fable, onyx, nova, shimmer
input=text
)
# Convert to base64 for sending to frontend
audio_data = response.content
audio_base64 = base64.b64encode(audio_data).decode('utf-8')
logger.info(f"TTS successful for text: {text[:50]}...")
return jsonify({
'success': True,
'audio': audio_base64,
'text': text
})
except Exception as e:
logger.error(f"TTS error: {e}")
return jsonify({'success': False, 'error': str(e)})
@app.route('/chat', methods=['POST'])
def chat_message():
"""Handle chat messages with OpenAI GPT"""
global conversation_history # Declare as global
try:
data = request.get_json()
if not data or 'message' not in data:
return jsonify({'success': False, 'error': 'No message provided'})
message = data['message']
logger.info(f"Received chat message: {message}")
# Use OpenAI GPT for intelligent responses
if not OpenAI:
response = "I'm sorry, but I'm not able to process your request right now. Please try again later."
else:
try:
client = OpenAI(api_key=os.getenv('OPENAI_API_KEY'))
# Create a conversation context for music production
system_prompt = """You are DAWZY, a helpful music production assistant specializing in the REAPER DAW. You can help with:
- REAPER workflow questions and tutorials
- Audio production techniques and best practices
- Track management and organization
- Effects and processing chains
- MIDI and audio editing
- Automation and mixing
- Music theory and composition
- Sound design and sy
[truncated β 35966 more characters]
```
### tailwind.config.js
```javascript
module.exports = {
content: [
'./index.html',
'./src/**/*.{js,jsx,ts,tsx}',
],
theme: {
extend: {},
},
plugins: [],
}
```
[16 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]