Project Info

NeuroPlot

This project did not submit a description on Devpost.

Parametric Drawing Generator

Transform natural language and voice prompts into mathematical parametric curves and rendered images using AI.

Built for Cal Hacks 12.0

What It Does

Describe an image with text or voice, and watch AI generate it using parametric equations:

Natural Language to Math - "Draw a butterfly" → parametric curves
Voice Input Support - Record or upload audio descriptions
Iterative Refinement - AI self-improves drawings through multi-agent evaluation
Robot-Ready Output - Generate programs for physical drawing robots
Beautiful Visualization - High-quality rendered images

Quick Start (Run the Full Application)

Prerequisites

Python 3.11+ with pip
Node.js 18+ with npm
Anthropic API Key (Get one here)

1. Clone & Set Up Environment

# Clone the repository
git clone <repository-url>
cd CalHacks12

# Set up environment variables
cp .env.example .env
# Edit .env and add your ANTHROPIC_API_KEY

2. Install Dependencies

Backend:

cd backend
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt
cd ..

Frontend:

cd frontend
npm install
cd ..

3. Run the Application

Option A: Two Terminal Windows (Recommended)

Terminal 1 - Backend:

cd backend
source venv/bin/activate  # On Windows: venv\Scripts\activate
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

Terminal 2 - Frontend:

cd frontend
npm run dev

Option B: Using tmux (Advanced)

# Start backend in background pane
tmux new-session -d -s calhacks 'cd backend && source venv/bin/activate && uvicorn app.main:app --reload'
# Start frontend in foreground
tmux split-window -h 'cd frontend && npm run dev'
tmux attach -t calhacks

4. Open Your Browser

Frontend: http://localhost:3000
Backend API Docs: http://localhost:8000/docs
Health Check: http://localhost:8000/health

5. Try It Out!

Type a prompt: "Draw a spiral flower with 5 petals"
Or record audio: Click the microphone button and describe your image
Hit "Generate Drawing"
Watch the AI create parametric equations and render your image!

📁 Project Structure

CalHacks12/
├── frontend/                    # Next.js web application
│   ├── app/
│   │   ├── page.tsx            # Main page component
│   │   └── api/draw/route.ts   # API proxy to backend
│   ├── components/
│   │   ├── drawing-input.tsx   # Input form (text/voice)
│   │   └── drawing-results.tsx # Results display
│   ├── types/drawing.ts        # TypeScript interfaces
│   └── package.json
│
├── backend/                     # FastAPI backend
│   ├── app/
│   │   ├── main.py             # FastAPI application
│   │   ├── pipeline.py         # Main orchestration pipeline
│   │   ├── schemas.py          # Pydantic data models
│   │   ├── claude_client.py    # Claude AI integration
│   │   ├── renderer_agent.py   # Image rendering
│   │   ├── evaluator_agent.py  # Quality evaluation
│   │   └── utils_relative.py   # Robot coordinate transforms
│   ├── static/                 # Generated images (runtime)
│   ├── exports/                # Robot programs (runtime)
│   ├── requirements.txt
│   └── Dockerfile
│
├── hardware/                    # Robot control (optional)
│   ├── robot_plotter.py        # Differential drive robot controller
│   ├── README.md               # Hardware documentation
│   └── requirements.txt
│
├── tests/                       # Test suite
├── .env.example                 # Environment template
├── .env                         # Your secrets (git-ignored)
└── README.md                    # This file

Development Setup

Backend Setup

Create virtual environment:

cd backend
python -m venv venv
source venv/bin/activate

Install dependencies:
```
pip install -r requirements.txt
```

Configure environment variables (in project root .env):

# Required
ANTHROPIC_API_KEY=sk-ant-...

# Optional
VAPI_API_KEY=your_vapi_key      # For voice transcription
LETTA_API_KEY=your_letta_key    # For persistent memory
PORT=8000                        # Backend port (default: 8000)

Run backend:

# Development mode with auto-reload
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

# Or use the provided script:
bash scripts/run_server.sh

Verify it's running:
- API: http://localhost:8000
- Docs: http://localhost:8000/docs
- Health: http://localhost:8000/health

Frontend Setup

Install dependencies:
```
cd frontend
npm install
```
Configure backend URL (optional):

Create frontend/.env.local if you need to change the backend URL:
```
BACKEND_URL=http://localhost:8000
```
Default: If not set, it uses http://localhost:8000
Run frontend:
```
npm run dev
```
Verify it's running:
- Frontend: http://localhost:3000

Running Both Together

Recommended workflow:

Start backend first (wait for "Application startup complete")
Start frontend second
Frontend will automatically connect to backend at http://localhost:8000

Troubleshooting Connection:

Backend running? Check http://localhost:8000/health
Frontend running? Check http://localhost:3000
CORS enabled? (Backend automatically allows all origins in dev mode)
Ports not in use? Change with --port or PORT env var

How to Use

Text Input

Open http://localhost:3000
Type your prompt: "Draw a heart shape"
Optionally toggle "Use Letta Memory" for contextual awareness
Click "Generate Drawing"
View your image, parametric equations, and AI evaluation score!

Voice Input

Click the "Record Audio" button (grant microphone permission)
Describe your image: "Draw a butterfly with rainbow wings"
Click "Stop Recording"
The system will transcribe and generate your drawing
See the transcribed prompt displayed with your image

Or upload an audio file:

Click "Upload Audio"
Select a .wav, .mp3, or other audio file
Generate!

Architecture

System Flow

┌─────────────────────────────────────────────────────────────┐
│                        FRONTEND                             │
│  (Next.js 16 + React 19 + Tailwind CSS + shadcn/ui)       │
│                                                             │
│  ┌──────────────┐      ┌──────────────┐                   │
│  │ Text Input   │      │ Voice Input  │                   │
│  │  Component   │      │  Component   │                   │
│  └──────┬───────┘      └──────┬───────┘                   │
│         │                     │                            │
│         └─────────┬───────────┘                            │
│                   │                                        │
│         ┌─────────▼──────────┐                             │
│         │   API Route        │                             │
│         │  /api/draw         │                             │
│         └─────────┬──────────┘                             │
└───────────────────┼─────────────────────────────────────┘
                    │ HTTP POST
                    │
┌───────────────────▼─────────────────────────────────────┐
│                      BACKEND                             │
│            (FastAPI + Python 3.11)                       │
│                                                           │
│  ┌─────────────┐         ┌─────────────┐                │
│  │ POST /draw  │         │POST /draw/  │                │
│  │   (text)    │         │   audio     │                │
│  └──────┬──────┘         └──────┬──────┘                │
│         │                       │                        │
│         │    ┌──────────────────┘                        │
│         │    │                                           │
│    ┌────▼────▼────┐                                      │
│    │   Pipeline   │                                      │
│    └────┬─────────┘                                      │
│         │                                                │
│    ┌────▼──────────────────────────────────┐            │
│    │  Phase 1: Prompt Interpretation       │            │
│    │           (Claude)                    │            │
│    └────┬──────────────────────────────────┘            │
│         │                                                │
│    ┌────▼──────────────────────────────────┐            │
│    │  Phase 2: Equation Generation         │            │
│    │           (Claude)                    │            │
│    └────┬──────────────────────────────────┘            │
│         │                                                │
│    ┌────▼──────────────────────────────────┐            │
│    │  Phase 3: Multi-Agent Refinement      │            │
│    │   ┌────────────────────────┐          │            │
│    │   │ Render → Evaluate →    │          │            │
│    │   │ Refine (up to 3x)      │          │            │
│    │   └────────────────────────┘          │            │
│    └────┬──────────────────────────────────┘            │
│         │                                                │
│    ┌────▼──────────────────────────────────┐            │
│    │  Phase 4: Relative Program Gen        │            │
│    │   (Robot coordinate transforms)       │            │
│    └────┬──────────────────────────────────┘            │
│         │                                                │
│    ┌────▼──────────────────────────────────┐            │
│    │  Phase 5: Return Results              │            │
│    │  - Image (base64)                     │            │
│    │  - Parametric equations               │            │
│    │  - Robot program                      │            │
│    │  - Evaluation score                   │            │
│    └───────────────────────────────────────┘            │
└──────────────────────────────────────────────────────────┘

Technology Stack

Frontend:

Framework: Next.js 16 (App Router) + React 19
Styling: Tailwind CSS 4 + shadcn/ui components
Audio: Web Audio API + MediaRecorder
Type Safety: TypeScript 5

Backend:

Framework: FastAPI (Python 3.11)
AI: Anthropic Claude Sonnet 4.5
Rendering: Matplotlib
Voice (Optional): Vapi API
Memory (Optional): Letta Cloud
Data Validation: Pydantic v2

API Reference

Frontend → Backend Flow

Text Input Flow:

POST /api/draw
Content-Type: application/json

{
  "prompt": "Draw a circle",
  "use_letta": false
}
↓
Backend: POST http://localhost:8000/draw
↓
Response: DrawingResponse

Audio Input Flow:

POST /api/draw
Content-Type: multipart/form-data

FormData {
  file: Blob (audio),
  use_letta: "false"
}
↓
Backend: POST http://localhost:8000/draw/audio
FormData {
  audio: Blob,
  use_letta: "false"
}
↓
Response: DrawingResponse (with transcribed prompt)

Backend Endpoints

Full API documentation: http://localhost:8000/docs

`POST /draw`

Create drawing from text prompt.

Request:

{
  "prompt": "Draw a spiral galaxy",
  "use_letta": false
}

Response:

{
  "success": true,
  "prompt": "Draw a spiral galaxy",
  "image_base64": "data:image/png;base64,...",
  "relative_program": {
    "segments": [
      {
        "name": "spiral_arm",
        "x_rel": "t*cos(t)",
        "y_rel": "t*sin(t)",
        "t_min": 0,
        "t_max": 12.566,
        "pen": { "color": "#000000" }
      }
    ]
  },
  "evaluation_score": 8.5,
  "iterations": 2,
  "processing_time": 4.2,
  "stats": {
    "run_id": "abc123...",
    "export_path": "exports/relative_program_abc123.json"
  }
}

`POST /draw/audio`

Create drawing from audio file.

Request:

curl -X POST http://localhost:8000/draw/audio \
  -F "audio=@recording.wav" \
  -F "use_letta=false"

Response: Same as /draw, with transcribed prompt

`GET /robot/{run_id}`

Fetch robot program for physical drawing.

Response:

{
  "run_id": "abc123...",
  "prompt": "Draw a star",
  "relative_program": { ... }
}

`GET /health`

Check system status.

Response:

{
  "status": "healthy",
  "services": {
    "anthropic_claude": "configured",
    "vapi_voice": "not_configured",
    "letta_memory": "not_configured"
  }
}

Understanding the Output

Generated Image

The AI creates a mathematical drawing rendered as a PNG image, displayed in the frontend.

Prompt Display

Shows the exact prompt used (especially useful for voice input to confirm transcription).

Parametric Equations

Relative Program Segments (preferred for robots):

spiral_arm:
  x(t) = t*cos(t)
  y(t) = t*sin(t)
  t ∈ [0.00, 12.57]
  Color: Black

Each segment is expressed in local coordinates relative to the previous segment's end position. Perfect for robots without global localization!

Evaluation Score

AI scores the drawing from 0-10 based on how well it matches the prompt:

9-10: Excellent match
7-8: Good match
5-6: Acceptable
<5: Needs improvement

Iterations

Number of refinement cycles the AI performed (max 3).

🧪 Testing

Backend Testing

cd backend
source venv/bin/activate

# Run all tests
python -m pytest tests/

# Test a specific prompt
python tests/test_sample_prompt.py "Draw a heart"

# Interactive mode
python tests/test_sample_prompt.py --interactive

Frontend Testing

cd frontend
npm run build  # Ensure no build errors
npm run lint   # Check for linting issues

End-to-End Testing

Start both backend and frontend
Test Text Input:
- Enter: "Draw a circle"
- Verify image appears
- Check parametric equations displayed
- Confirm score and iterations shown
Test Voice Input:
- Click "Record Audio"
- Say: "Draw a flower with 5 petals"
- Stop recording
- Verify transcription appears
- Check drawing generated
Test Error Handling:
- Stop backend
- Try to generate a drawing
- Verify user-friendly error message appears
- Restart backend
- Verify recovery

Configuration

Environment Variables

Create a .env file in the project root:

# === REQUIRED ===
ANTHROPIC_API_KEY=sk-ant-...

# === OPTIONAL ===
# Voice transcription (Vapi)
VAPI_API_KEY=your_vapi_key

# Persistent memory across sessions (Letta)
LETTA_API_KEY=your_letta_key

# Backend port
PORT=8000

Frontend Configuration

Create frontend/.env.local (optional):

# Backend URL (default: http://localhost:8000)
BACKEND_URL=http://localhost:8000

# For production deployment:
# BACKEND_URL=https://your-backend.com

Docker Deployment

Backend Only

cd backend
docker build -t parametric-drawing-backend .
docker run -d -p 8000:8000 \
  -e ANTHROPIC_API_KEY=sk-ant-... \
  parametric-drawing-backend

Full Stack (docker-compose)

Create docker-compose.yml:

version: '3.8'
services:
  backend:
    build: ./backend
    ports:
      - "8000:8000"
    environment:
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
    volumes:
      - ./backend/static:/app/static
      - ./backend/exports:/app/exports

  frontend:
    build: ./frontend
    ports:
      - "3000:3000"
    environment:
      - BACKEND_URL=http://backend:8000
    depends_on:
      - backend

Run:

docker-compose up -d

Robot Integration

The system generates relative motion programs for physical drawing robots. A complete hardware control implementation is available in the hardware/ directory.

How It Works

Generate a drawing via /draw endpoint
Backend returns stats.run_id and exports program to exports/
Robot fetches program via GET /robot/{run_id}
Robot executes segments sequentially in local coordinates

Using the Included Robot Plotter

A complete differential drive robot controller is provided in hardware/:

# Install dependencies (on Raspberry Pi)
cd hardware
pip install -r requirements.txt

# Run with a generated drawing
python robot_plotter.py <run_id>

# Or test in simulation mode
python robot_plotter.py <run_id> --simulate

Features:

Accurate differential drive kinematics
Synchronized dual stepper motor control (28BYJ-48)
Pen up/down handling for color changes
Works with or without Raspberry Pi hardware (simulation mode)

See hardware/README.md for complete documentation.

Custom Robot Integration

import requests
import math

# 1. Generate drawing on server
response = requests.post("http://server:8000/draw",
                         json={"prompt": "Draw a star"})
data = response.json()
run_id = data["stats"]["run_id"]

# 2. On robot: Fetch program
program = requests.get(f"http://server:8000/robot/{run_id}").json()

# 3. Execute each segment
for segment in program["relative_program"]["segments"]:
    if segment["pen"]["color"] == "none":
        # Pen up - travel move
        move_without_drawing(segment)
    else:
        # Pen down - draw with specified color
        set_pen_color(segment["pen"]["color"])
        draw_parametric_curve(
            x_expr=segment["x_rel"],
            y_expr=segment["y_rel"],
            t_min=segment["t_min"],
            t_max=segment["t_max"]
        )
    # Local frame automatically resets to (0,0,0) for next segment

Pen Colors (Normalized)

All colors are automatically mapped to:

"none" - Pen up (no drawing)
"#000000" - Black pen
"#0000FF" - Blue pen

Perfect for dual-pen robot systems!

📚 Example Prompts

Prompt	Complexity	Expected Result
`"Draw a circle"`	⭐	Perfect circle
`"Draw a heart shape"`	⭐⭐	Symmetric heart
`"Draw a butterfly with symmetric wings"`	⭐⭐⭐	Butterfly with mirrored wings
`"Draw a flower with 5 petals"`	⭐⭐⭐	5-petal radial flower
`"Draw a spiral galaxy"`	⭐⭐⭐⭐	Logarithmic spiral
`"Draw a Celtic knot"`	⭐⭐⭐⭐⭐	Intricate interwoven pattern

🐛 Troubleshooting

Frontend won't connect to backend

Symptom: "Failed to generate drawing" error

Solutions:

Check backend is running: curl http://localhost:8000/health
Check BACKEND_URL in frontend/.env.local (default: http://localhost:8000)
Check CORS in backend logs (should allow all origins in dev)
Check browser console for network errors

"ANTHROPIC_API_KEY not found"

Solution:

# Check .env file exists in project root
cat .env

# Should contain:
ANTHROPIC_API_KEY=sk-ant-...

# Restart backend after adding key

Port Already in Use

Symptom: Address already in use error

Solutions:

# Find process using port 8000
lsof -i :8000
kill -9 <PID>

# Or use different port
uvicorn app.main:app --port 8001

Microphone Access Denied

Solution:

Check browser permissions (URL bar → lock icon → permissions)
Allow microphone access
Refresh page

"Module not found" errors

Backend:

cd backend
source venv/bin/activate
pip install -r requirements.txt

Frontend:

cd frontend
rm -rf node_modules package-lock.json
npm install

Images not appearing

Solution:

Check backend/static/ directory exists and is writable
Check backend logs for rendering errors
Verify image data in browser DevTools → Network tab
Check console for image loading errors

Production Deployment

Environment Setup

Backend:

Set ANTHROPIC_API_KEY in production environment
Use gunicorn or uvicorn with production settings
Configure proper CORS origins (not wildcard)
Set up HTTPS with reverse proxy (nginx/Caddy)

Frontend:

Set BACKEND_URL to production backend URL
Build with npm run build
Serve with npm start or deploy to Vercel/Netlify

Recommended Stack

Backend: Railway, Render, Fly.io, or AWS ECS
Frontend: Vercel, Netlify, or Cloudflare Pages
Storage: Mount persistent volumes for static/ and exports/

🎓 How It Works (Under the Hood)

Phase 1: Prompt Interpretation

Claude Sonnet 4.5 analyzes your prompt to extract:

Visual components (e.g., "wings", "body", "petals")
Symmetry type (vertical, horizontal, radial, none)
Complexity rating (1-5 scale)
Detailed structural description

Phase 2: Equation Generation

Claude generates mathematical parametric equations:

x(t) and y(t) expressions using trig functions
Parameter range [t_min, t_max]
Color information

Example (Circle):

{
  "name": "circle",
  "x": "cos(t)",
  "y": "sin(t)",
  "t_min": 0,
  "t_max": 6.283185307179586
}

Phase 3: Multi-Agent Refinement

Iterative improvement loop (up to 3 iterations):

Renderer Agent: Plots equations using Matplotlib
Evaluator Agent: Scores image 0-10, provides feedback
Refinement Agent: Adjusts equations based on feedback
Repeat until score ≥ 9 or max iterations reached

Phase 4: Relative Program Generation

Transforms absolute curves into robot-ready format:

Compute End Poses: Calculate (x, y, θ) at end of each curve
Local Frame Transform: Express each curve relative to previous end pose
Pen Control: Assign normalized colors ("none", "#000000", "#0000FF")

Mathematical Transform:

[x_rel]   [cos(-θ)  -sin(-θ)] ([x(t)]   [x_prev])
[y_rel] = [sin(-θ)   cos(-θ)] ([y(t)] - [y_prev])

Phase 5: Output

Returns comprehensive results:

Rendered image (base64 PNG with data URI)
Relative program (robot-ready segments)
Evaluation score and feedback
Processing metadata

Additional Resources

Backend API Docs: http://localhost:8000/docs (when running)
Anthropic Claude: https://www.anthropic.com/claude
Next.js Documentation: https://nextjs.org/docs
FastAPI Documentation: https://fastapi.tiangolo.com/

Acknowledgments

Built with technologies from Cal Hacks 12.0 sponsors:

Anthropic - Claude Sonnet 4.5 AI
Vapi - Voice-to-text API
Letta - Persistent memory
Fetch.ai - Agent framework concepts
Composio - Tool integration patterns

License

MIT License - See LICENSE file for details

Contributing

This is a hackathon project, but contributions, suggestions, and feedback are welcome! Feel free to:

Open issues for bugs or feature requests
Submit pull requests
Share your generated drawings!

Support

For questions or issues:

Check the Troubleshooting section
Review backend logs: tail -f backend/logs/app.log
Open an issue on GitHub
Contact the development team

Analysis

Compare with all teams

View

Metric

pranavsant
11
Claude
4

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

AnthropicIn code
CSSIn code
FastAPIIn code
Next.jsIn code
PythonIn code
ReactIn code
Tailwind CSSIn code
TypeScriptIn code
JavaScriptClaimed

8 of 9 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

Claude CodeCommits

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

533 KB

Source files

106

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.

NeuroPlot

Parametric Drawing Generator

What It Does

Quick Start (Run the Full Application)

Prerequisites

1. Clone & Set Up Environment

2. Install Dependencies

Backend:

Frontend:

3. Run the Application

4. Open Your Browser

5. Try It Out!

📁 Project Structure

Development Setup

Backend Setup

Frontend Setup

Running Both Together

How to Use

Text Input

Voice Input

Architecture

System Flow

Technology Stack

API Reference

Frontend → Backend Flow

Text Input Flow:

Audio Input Flow:

Backend Endpoints

POST /draw

POST /draw/audio

GET /robot/{run_id}

GET /health

Understanding the Output

Generated Image

Prompt Display

Parametric Equations

Evaluation Score

Iterations

🧪 Testing

Backend Testing

Frontend Testing

End-to-End Testing

Configuration

Environment Variables

Frontend Configuration

Docker Deployment

Backend Only

Full Stack (docker-compose)

Robot Integration

How It Works

Using the Included Robot Plotter

Custom Robot Integration

Pen Colors (Normalized)

📚 Example Prompts

🐛 Troubleshooting

Frontend won't connect to backend

"ANTHROPIC_API_KEY not found"

Port Already in Use

Microphone Access Denied

"Module not found" errors

Images not appearing

Production Deployment

Environment Setup

Recommended Stack

🎓 How It Works (Under the Hood)

Phase 1: Prompt Interpretation

Phase 2: Equation Generation

Phase 3: Multi-Agent Refinement

Phase 4: Relative Program Generation

Phase 5: Output

Additional Resources

Acknowledgments

License

Contributing

Support

Technology

AI coding agents

Codebase size

Structure

Interface

`POST /draw`

`POST /draw/audio`

`GET /robot/{run_id}`

`GET /health`