# Project export: souma

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 12.0
- Tagline: An intelligent, real-time coaching overlay for League of Legends that provides actionable guidance to players through AI-powered game state analysis and directive coaching commands.
- Devpost: https://devpost.com/software/calhacks-25
- GitHub: https://github.com/ethannyang/calhacks-25
- Demo: https://souma.base44.app/
- Video: https://www.youtube.com/embed/Dh-jWRhHM00?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 2 GitHub contributor(s) — Claude (6 commits), Ethan Yang (1 commits)

## Devpost submission (written by the team)

No Devpost description available.

## README (from the GitHub repository)

# 🧠 Souma — Real-Time AI League of Legends Coach

> “Don’t just play League — learn it while you play.”

Souma is an **AI-powered real-time coaching overlay** for *League of Legends*.  
It analyzes your live in-game state and provides **specific, actionable advice** — helping players improve their game sense, mechanics, and decision-making while they play.

---

## 🌟 Inspiration

Most existing League tools only help *before* or *after* your game — with builds, stats, or post-match reviews.  
But what about **during** the match, when decisions matter most?

League has an incredibly high skill floor — both strategically and mechanically.  
Many players give up before learning how to actually enjoy the game.  

So we asked:  
> What if an AI coach could guide you live, teaching better habits and strategy in real time?

That’s how **Souma** was born.

---

## ⚙️ What It Does

Souma watches your gameplay and reacts just like a live coach would.  
It reads your **health, mana, gold, items, minimap state, and lane conditions**, and gives guidance like:

- 🩸 *“Back — low HP, enemy jungler nearby.”*  
- 💰 *“Recall now — major item spike available.”*  
- 🧠 *“Freeze the wave under tower.”*  
- 🗺️ *“Rotate to dragon — 30 seconds to spawn.”*  

This helps players **build instincts** and **learn strategy faster**, turning frustration into confidence and wins.

---

## 🏗️ How It Works

### 🔧 Backend
- **FastAPI** — Asynchronous API backend for real-time data processing.  
- **OpenCV** — Captures game frames and extracts ROIs (health bar, mana, gold, minimap).  
- **Tesseract / EasyOCR** — Optical character recognition for reading in-game text.  
- **Riot API Client** — Fetches live match data (rate-limited).

### 🧠 AI Engines
- **Rule Engine (F1, F6)** — For deterministic events like low-health alerts and recall timing.  
- **LLM Engine (F2, F4)** — For reasoning-based advice such as wave management and objective control.  

### 🖥️ Frontend
- **Electron + React + TypeScript** — Cross-platform overlay interface.  
- **Zustand** — Lightweight state management.  
- **TailwindCSS** — Clean and adaptive styling.  
- **WebSocket** — Real-time connection between backend and overlay.

---

## 🧩 Challenges

- Building stable **image processing** for a live, animated game environment was a huge challenge.  
- **Audio synchronization** for abilities and cues was complex to crossmatch in real time.  
- Integrating **voice input → LLM** interactions took multiple rewrites, but it made the coach feel truly alive.

---

## 🏆 Accomplishments

- Developed a **working live computer vision pipeline** for League of Legends.  
- Implemented **ROI-specific logic** for gold, health, mana, and minimap awareness.  
- Built an AI that **prioritizes coaching advice** intelligently (e.g., safety > objectives > wave control).  
- Watching Souma *call out a gank before it happened* was an unforgettable moment.

---

## 💡 What I Learned

- **Never give up.** Debugging real-time systems tests patience like nothing else.  
- **Image processing for AI in games** still has huge room to grow.  
- **Audio-visual fusion** is key to capturing complex game states.  
- **Voice-driven LLM input** opens up new frontiers for interactive, adaptive AI experiences.

---

## 🚀 Next Steps

- 🧩 Champion-specific advice modules (e.g., different coaching for Garen vs. Fiora).  
- 🔊 Real-time audio feedback from the AI coach.  
- 🧠 Personalized learning paths based on player history.  
- 🕹️ Expand to other esports titles (Dota, Valorant, etc.).

---

## ⚡ Running Locally

### Clone & Install
```bash
git clone https://github.com/yourname/souma.git
cd souma
npm install


## Detected evidence (automated analysis)

Indexed codebase: 69 recognized source files, 348 KB.
- Anthropic (technology) — detected in the code
- CSS (language) — detected in the code
- FastAPI (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers

## Codebase structure (from repository index)

### Files (76 of 76)

```
.gitignore
backend/.env.example
backend/calibrate_rois.py
backend/COMBAT_COACHING_SETUP.md
backend/debug/check_permissions.py
backend/debug/debug_rois.py
backend/debug/debug_websocket.py
backend/debug/find_riot_id.py
backend/game_loop.py
backend/main.py
backend/package.json
backend/PIPELINE_IMPLEMENTATION.md
backend/QUICKSTART.md
backend/requirements.txt
backend/src/__init__.py
backend/src/ai_engine/__init__.py
backend/src/ai_engine/build_tracker.py
backend/src/ai_engine/command_manager.py
backend/src/ai_engine/llm_engine.py
backend/src/ai_engine/rule_engine.py
backend/src/capture/__init__.py
backend/src/capture/base.py
backend/src/capture/macos.py
backend/src/combat_vision/__init__.py
backend/src/combat_vision/audio_detector.py
backend/src/combat_vision/audio_template_detector.py
backend/src/combat_vision/combat_capture.py
backend/src/combat_vision/combat_coach_module.py
backend/src/combat_vision/darius_vs_garen_coach.py
backend/src/combat_vision/garen_detector.py
backend/src/models/__init__.py
backend/src/models/game_state.py
backend/src/ocr/__init__.py
backend/src/ocr/extractor.py
backend/src/riot_api/__init__.py
backend/src/riot_api/client.py
backend/src/riot_api/game_client_api.py
backend/src/riot_api/live_game_manager.py
backend/test_pipeline.py
backend/test_voice_cooldowns.py
backend/tests/test_audio_detection.py
backend/tests/test_audio_template.py
backend/tests/test_capture.py
backend/tests/test_garen_detection.py
backend/tests/test_garen_realtime.py
backend/tests/test_live_game.py
backend/tests/test_ocr.py
backend/tests/test_voice_input.py
backend/voice_proxy.js
CLAUDE.md
frontend/electron/main.js
frontend/electron/preload.js
frontend/index.html
frontend/package.json
frontend/postcss.config.js
frontend/public/pcm-resampler-worklet.js
frontend/src/App.tsx
frontend/src/components/CommandCard.tsx
frontend/src/components/ConnectionStatus.tsx
frontend/src/components/CooldownDisplay.tsx
frontend/src/components/VoiceInput.tsx
frontend/src/index.css
frontend/src/main.tsx
frontend/src/services/voiceStreaming.ts
frontend/src/services/websocket.ts
frontend/src/store/coachingStore.ts
frontend/src/vite-env.d.ts
frontend/tailwind.config.js
frontend/tsconfig.json
frontend/tsconfig.node.json
frontend/vite.config.ts
MACOS_SETUP.md
MVP_STATUS.md
README.md
TECHNICAL_PRD.md
VOICE_INPUT_SETUP.md
```

### Dependencies

- backend/package.json: dotenv@^17.2.3, nodemon@^3.0.1, ws@^8.18.3
- backend/requirements.txt: aiohttp@==3.9.1, anthropic@>=0.40.0, fastapi@==0.104.1, loguru@==0.7.2, numpy@>=1.24.0, openai@==1.3.7, opencv-python@==4.8.1.78, pillow@==10.1.0, PyAudio@>=0.2.13, pydantic@==2.5.2, pyobjc-framework-Quartz@>=12.0, pytesseract@==0.3.10, python-dotenv@==1.0.0, python-multipart@==0.0.6, scipy@>=1.10.0, uvicorn[standard]@==0.24.0, websockets@==12.0
- frontend/package.json: @types/react@^18.2.45, @types/react-dom@^18.2.18, @vitejs/plugin-react@^4.2.1, autoprefixer@^10.4.16, concurrently@^8.2.2, electron@^28.3.3, electron-builder@^24.9.1, postcss@^8.4.32, react@^18.2.0, react-dom@^18.2.0, tailwindcss@^3.4.0, typescript@^5.3.3, uiohook-napi@^1.5.4, vite@^5.0.8, wait-on@^7.2.0, zustand@^4.4.7

### Recent commits (newest first)

- Revise README for Souma AI Coaching Overlay
- Enhance voice input for natural language cooldown tracking
- Update Garen ability audio templates with new sound files
- Integrate audio-based combat coaching into game loop
- Add audio-based ability detection system for Garen
- Add combat vision system for Darius vs Garen matchup
- Fix Electron IPC handlers initialization issue
- Add unified start script and improve overlay window level
- Add game loop integration and full backend pipeline
- fixed roi
- base
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### MACOS_SETUP.md

```markdown
# macOS Setup Guide

## Screen Recording Permissions

On macOS, apps need explicit permission to capture the screen. You need to grant permission to **Terminal** (or whatever terminal app you're using).

### Steps to Grant Permission:

1. Open **System Settings** (System Preferences on older macOS)
2. Go to **Privacy & Security**
3. Click **Screen Recording** (or **Screen & System Audio Recording**)
4. Click the **lock icon** to make changes (enter your password)
5. Find and enable **Terminal** (or iTerm, VS Code, etc.)
6. **Restart your terminal application** for changes to take effect

### Verifying Permissions

Run the test script to verify screen capture works:

```bash
cd backend
python3 test_capture.py
```

If permissions are working, you should see a list of open windows. If you only see 1-2 windows (like Dock), permissions may not be granted yet.

## Testing Screen Capture

### Without League of Legends

You can test screen capture with any other window:

```bash
python3 test_capture.py
```

When prompted, select `y` to test with another window, then choose a window number from the list.

This will create `test_capture.png` if capture is working.

### With League of Legends

1. Launch League of Legends and start a game (Practice Tool is easiest)
2. Run the test script:
```bash
python3 test_capture.py
```

If LoL is detected, it will:
- Capture the game window
- Extract ROIs (gold, CS, HP/mana, minimap, etc.)
- Save images for inspection

Check these files:
- `captured_frame.png` - Full game window capture
- `roi_gold.png` - Gold counter area
- `roi_cs.png` - CS counter area
- `roi_player_hp.png` - HP bar area
- `roi_player_mana.png` - Mana bar area
- `roi_minimap.png` - Minimap area
- `roi_game_time.png` - Game timer area

## Common Issues

### "Only 1 window found"
- Screen recording permissions not granted
- Terminal app needs to be restarted after granting permissions

### "League of Legends window not found"
- Make sure LoL is running (in-game, not just client)
- Try during an actual game or Practice Tool
- The test script will show all available windows

### "Failed to capture window"
- Some apps block screen capture for security
- Try with a different window to verify capture works

## Next Steps

Once screen capture is working:

1. **Test with Live Game**: Play a League game with the test script running
2. **Verify OCR**: Check if gold, CS, and timer can be read from the ROI images
3. **Full Pipeline**: Integrate into main backend service
4. **Real-time Coaching**: Start receiving AI coaching commands in the overlay!

## Dependencies Installed

- `pyobjc-framework-Quartz` - macOS screen capture APIs
- `opencv-python` - Image processing
- `pytesseract` - OCR for text extraction
- `easyocr` - Fallback OCR system

## Performance Notes

- Screen capture runs at **1-2 FPS** (configurable in .env)
- Target **<5% CPU usage**
- ROI extraction is fast (<10ms)
- OCR processing takes 50-200ms per ROI
- Total latency target: **<500ms** from ca
[truncated — 26 more characters]
```

### VOICE_INPUT_SETUP.md

```markdown
# Voice Input Setup Guide

This guide explains how to set up and use the cloud-based voice recognition system for ability tracking.

## Why Cloud ASR?

**The Web Speech API doesn't work in Electron** because Chromium's speech backend requires Google API keys that aren't shipped with Electron. You'll see `SpeechRecognitionErrorEvent { error: 'network' }` - the API exists but immediately fails.

This implementation uses **Deepgram** (cloud ASR) with AudioWorklet streaming for:
- ✅ **Low latency**: ~300ms (faster than Web Speech API)
- ✅ **Live transcription**: See partial results as you speak
- ✅ **Better accuracy**: Gaming vocabulary boosting (Q, W, E, R, Flash, etc.)
- ✅ **Actually works in Electron**

## Architecture

```
[T key pressed] → Electron captures mic
                → AudioWorklet resamples to 16kHz PCM (20ms frames)
                → WebSocket to local proxy (ws://localhost:8787)
                → Proxy authenticates + forwards to Deepgram
                → Partial/final transcripts flow back
                → Frontend parses abilities and sends to backend
```

---

## Setup Instructions

### 1. Get Deepgram API Key

1. Sign up at https://deepgram.com (free tier: $200 credit, ~46,000 minutes)
2. Go to **API Keys** in console
3. Create a new API key
4. Copy the key (starts with something like `abc123...`)

### 2. Add API Key to Environment

Create or update your `.env` file in the **backend** directory:

```bash
# backend/.env
DEEPGRAM_API_KEY=your_api_key_here
```

**IMPORTANT:** Never commit this file to git. Add it to `.gitignore`.

### 3. Install Node.js Dependencies for Proxy

The voice proxy is a Node.js server (separate from your Python backend). Install dependencies:

```bash
cd backend

# Option A: Use the package.json
mv voice-proxy-package.json package.json
npm install

# Option B: Install dependencies manually
npm install ws dotenv
```

### 4. Start the Voice Proxy Server

In a **separate terminal** (keep this running):

```bash
cd backend
node voice_proxy.js
```

You should see:
```
🎤 Voice proxy server starting on port 8787...
✅ Voice proxy server listening on http://localhost:8787
   WebSocket endpoint: ws://localhost:8787/stt
   Deepgram API key: abc123...
```

**Keep this terminal running** while using voice input.

### 5. Start Your Electron App

In another terminal:

```bash
cd frontend
npm run electron:dev
```

### 6. Test Voice Input

1. Press and hold **T** key
2. Say an ability: "Garen Q", "flash", "enemy W", etc.
3. You should see:
   - **LISTENING...** indicator
   - Live transcript in quotes (interim results)
   - Green confirmation when command is recognized

---

## How It Works

### Files Created

1. **`frontend/public/pcm-resampler-worklet.js`**
   - AudioWorklet processor
   - Converts mic input (48kHz) → 16kHz PCM (what Deepgram expects)
   - Sends 20ms frames for low latency

2. **`frontend/src/services/voiceStreaming.ts`**
   - Voice streaming client
   - Manages mic, AudioWorklet, WebSocket connecti
[truncated — 4480 more characters]
```

### backend/package.json

```
{
  "name": "voice-proxy",
  "version": "1.0.0",
  "description": "WebSocket proxy for Deepgram speech-to-text",
  "main": "voice_proxy.js",
  "scripts": {
    "start": "node voice_proxy.js",
    "dev": "nodemon voice_proxy.js"
  },
  "dependencies": {
    "ws": "^8.18.3",
    "dotenv": "^17.2.3"
  },
  "devDependencies": {
    "nodemon": "^3.0.1"
  }
}

```

### backend/requirements.txt

```
# FastAPI and ASGI server
fastapi==0.104.1
uvicorn[standard]==0.24.0
python-multipart==0.0.6
websockets==12.0

# Computer Vision and OCR
opencv-python==4.8.1.78
pillow==10.1.0
pytesseract==0.3.10
pyobjc-framework-Quartz>=12.0  # macOS screen capture

# Audio Processing (for combat vision)
numpy>=1.24.0
scipy>=1.10.0
PyAudio>=0.2.13

# HTTP Client for Riot API
aiohttp==3.9.1

# AI/LLM SDKs
anthropic>=0.40.0
openai==1.3.7

# Data Validation
pydantic==2.5.2

# Utilities
python-dotenv==1.0.0
loguru==0.7.2

```

### frontend/package.json

```
{
  "name": "lol-ai-coaching-overlay",
  "version": "0.1.0",
  "description": "League of Legends AI Coaching Overlay",
  "main": "electron/main.js",
  "scripts": {
    "dev": "vite",
    "build": "tsc && vite build",
    "electron:dev": "concurrently \"vite\" \"wait-on http://localhost:5173 && electron .\"",
    "electron:build": "npm run build && electron-builder",
    "preview": "vite preview",
    "start": "concurrently -n \"backend,vite,electron\" -c \"cyan,green,yellow\" \"cd ../backend && python3 main.py\" \"vite\" \"wait-on http://localhost:5173 && electron .\""
  },
  "keywords": [
    "lol",
    "league-of-legends",
    "coaching",
    "overlay",
    "electron"
  ],
  "author": "",
  "license": "MIT",
  "dependencies": {
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "uiohook-napi": "^1.5.4",
    "zustand": "^4.4.7"
  },
  "devDependencies": {
    "@types/react": "^18.2.45",
    "@types/react-dom": "^18.2.18",
    "@vitejs/plugin-react": "^4.2.1",
    "autoprefixer": "^10.4.16",
    "concurrently": "^8.2.2",
    "electron": "^28.3.3",
    "electron-builder": "^24.9.1",
    "postcss": "^8.4.32",
    "tailwindcss": "^3.4.0",
    "typescript": "^5.3.3",
    "vite": "^5.0.8",
    "wait-on": "^7.2.0"
  },
  "build": {
    "appId": "com.lolaicoaching.overlay",
    "productName": "LoL AI Coaching",
    "files": [
      "dist/**/*",
      "electron/**/*",
      "package.json"
    ],
    "directories": {
      "output": "release"
    },
    "mac": {
      "target": "dmg",
      "icon": "assets/icon.icns"
    },
    "win": {
      "target": "nsis",
      "icon": "assets/icon.ico"
    },
    "linux": {
      "target": "AppImage",
      "icon": "assets/icon.png"
    }
  }
}

```

### backend/main.py

```python
"""
League of Legends AI Coaching Overlay - Backend Entry Point
FastAPI server with WebSocket support for real-time coaching commands
"""

import asyncio
from contextlib import asynccontextmanager
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from fastapi.middleware.cors import CORSMiddleware
from loguru import logger
import sys

from game_loop import GameLoop
from src.models.game_state import CoachingCommand

# Configure logger
logger.remove()
logger.add(sys.stderr, level="INFO")

# Global game loop instance
game_loop: GameLoop = None
game_loop_task: asyncio.Task = None


@asynccontextmanager
async def lifespan(app: FastAPI):
    """Application lifespan handler"""
    global game_loop, game_loop_task

    logger.info("Starting LoL AI Coaching Backend...")

    # Start game loop in background
    game_loop = GameLoop()
    game_loop_task = asyncio.create_task(game_loop.run())
    logger.info("Game loop started in background")

    yield

    # Stop game loop
    logger.info("Shutting down LoL AI Coaching Backend...")
    if game_loop:
        game_loop.stop()
    if game_loop_task:
        await game_loop_task


app = FastAPI(
    title="LoL AI Coaching Backend",
    description="Real-time coaching overlay backend for League of Legends",
    version="0.1.0",
    lifespan=lifespan
)

# CORS middleware for Electron frontend
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # In production, specify exact origins
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)


class ConnectionManager:
    """Manages WebSocket connections"""

    def __init__(self):
        self.active_connections: list[WebSocket] = []

    async def connect(self, websocket: WebSocket):
        await websocket.accept()
        self.active_connections.append(websocket)
        logger.info(f"Client connected. Total connections: {len(self.active_connections)}")

    def disconnect(self, websocket: WebSocket):
        self.active_connections.remove(websocket)
        logger.info(f"Client disconnected. Total connections: {len(self.active_connections)}")

    async def broadcast(self, message: dict):
        """Send message to all connected clients"""
        for connection in self.active_connections:
            try:
                await connection.send_json(message)
            except Exception as e:
                logger.error(f"Error broadcasting message: {e}")


manager = ConnectionManager()


@app.get("/health")
async def health_check():
    """Health check endpoint"""
    return {
        "status": "healthy",
        "service": "lol-ai-coaching-backend",
        "version": "0.1.0"
    }


@app.get("/test-command")
async def test_command():
    """Send a test coaching command to verify overlay is working"""
    import time
    test_cmd = CoachingCommand(
        priority="high",
        category="safety",
        icon="⚠️",
        message="TEST: Overlay is working! You should see this message.",
        duration=10,
        timestamp=time.time()
    )
    await manager.broadcast({
        "type": "command",
        "data": {
            "priority": test_cmd.priority,
            "category": test_cmd.category,
            "icon": test_cmd.icon,
            "message": test_cmd.message,
            "duration": test_cmd.duration,
            "timestamp": test_cmd.timestamp
        }
    })
    return {"status": "test command sent"}


async def broadcast_command(command: CoachingCommand):
    """Callback for game loop to broadcast commands to all connected clients"""
    message = {
        "type": "command",
        "data": {
            "priority": command.priority,
            "category": command.category,
            "icon": command.icon,
            "message": command.message,
            "duration": command.duration,
            "timestamp": command.timestamp
        }
    }
    await manager.broadcast(message)


@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
    """
    WebSocket endpoint for real-time coaching commands
    Client receives: {"type": "command", "data": {...}}
    Client sends: {"type": "config", "data": {...}}
    """
    global game_loop

    await manager.connect(websocket)

    # Connect game loop to broadcast commands
    if game_loop and not game_loop.on_command:
        game_loop.set_command_callback(broadcast_command)
        logger.info("Game loop connected to WebSocket broadcast")

    try:
        while True:
            # Receive messages from client
            data = await websocket.receive_json()
            logger.info(f"Received from client: {data}")

            # Handle different message types
            msg_type = data.get("type")

            if msg_type == "ability_used":
                # Manual ability reporting from voice input
                ability_data = data.get("data", {})
                ability = ability_data.get("ability")
                target = ability_data.get("target", "enemy")

                logger.info(f"Voice input: {target} used {ability}")

                # Forward to combat coach module
                if game_loop and game_loop.combat_coach:
                    game_loop.combat_coach.manual_report_ability(ability, target)
                    logger.info(f"Reported {ability} to combat coach")

                    # Get updated cooldowns
                    cooldowns = game_loop.combat_coach.audio_detector.get_ability_cooldowns()

                    # Broadcast cooldowns to all clients
                    await manager.broadcast({
                        "type": "cooldowns",
                        "data": cooldowns
                    })

                    # Send acknowledgment
                    await websocket.send_json({
                        "type": "ack",
                        "message": f"Tracked {target} {ability}",
                        "data": ability_data
                    })
                else:
                    await websocket.send_json({
  
[truncated — 808 more characters]
```

### frontend/src/main.tsx

```typescript
import React from 'react'
import ReactDOM from 'react-dom/client'
import App from './App.tsx'
import './index.css'

ReactDOM.createRoot(document.getElementById('root')!).render(
  <React.StrictMode>
    <App />
  </React.StrictMode>,
)

```

### frontend/src/App.tsx

```typescript
import { useEffect, useState } from 'react';
import CommandCard from './components/CommandCard';
import ConnectionStatus from './components/ConnectionStatus';
import VoiceInput from './components/VoiceInput';
import CooldownDisplay from './components/CooldownDisplay';
import { useCoachingStore } from './store/coachingStore';
import { connectWebSocket } from './services/websocket';

function App() {
  const { currentCommand, isConnected } = useCoachingStore();
  const [isVoiceListening, setIsVoiceListening] = useState(false);

  useEffect(() => {
    // Connect to backend WebSocket
    connectWebSocket('ws://localhost:8000/ws');

    // Listen for click-through toggle events
    if (window.electron) {
      window.electron.onClickThroughToggled((isEnabled: boolean) => {
        console.log('Click-through toggled:', isEnabled);
      });

      // Listen for voice input toggle events from Electron
      window.electron.onVoiceInputToggle((isActive: boolean) => {
        console.log('Voice input toggled:', isActive);
        setIsVoiceListening(isActive);
      });
    }
  }, []);

  return (
    <div className="w-full h-full flex flex-col items-center justify-start pt-4 px-4">
      {/* Connection status indicator */}
      <ConnectionStatus isConnected={isConnected} />

      {/* Main coaching command display */}
      {currentCommand && <CommandCard command={currentCommand} />}

      {/* Demo mode indicator (remove in production) */}
      {!currentCommand && (
        <div className="text-white/50 text-sm font-mono bg-black/30 px-3 py-2 rounded">
          Waiting for coaching data...
        </div>
      )}

      {/* Voice input component */}
      <VoiceInput isListening={isVoiceListening} onListeningChange={setIsVoiceListening} />

      {/* Cooldown tracker */}
      <CooldownDisplay />
    </div>
  );
}

export default App;

```

### frontend/electron/main.js

```javascript
/**
 * Electron Main Process
 * Creates transparent, always-on-top, click-through overlay window
 */

const { app, BrowserWindow, screen, ipcMain, globalShortcut } = require('electron');
const path = require('path');
const { uIOhook, UiohookKey } = require('uiohook-napi');

let mainWindow;
let isClickThrough = false;  // Start with click-through disabled so window is more visible
let isVoiceInputActive = false;  // Track voice input state

// Handle EPIPE errors gracefully to prevent app crashes
process.on('uncaughtException', (error) => {
  if (error.code === 'EPIPE') {
    // Ignore EPIPE errors from console.log
    return;
  }
  // Log other errors but don't crash
  console.error('Uncaught exception:', error);
});

// Safe console logging wrapper
function safeLog(...args) {
  try {
    console.log(...args);
  } catch (e) {
    // Silently ignore console errors
  }
}

function createWindow() {
  // Get primary display dimensions
  const primaryDisplay = screen.getPrimaryDisplay();
  const { width, height } = primaryDisplay.workAreaSize;

  mainWindow = new BrowserWindow({
    width: 500,
    height: 300,
    x: width - 520,  // Position in top-right (20px from right edge)
    y: 20,  // 20px from top
    frame: true,  // Show frame for easier visibility
    transparent: false,  // Disable transparency to make it more visible
    alwaysOnTop: true,
    skipTaskbar: false,  // Show in taskbar for now (easier to close during dev)
    resizable: true,
    visibleOnAllWorkspaces: true,  // Show on all spaces/desktops
    fullscreenable: false,  // Prevent fullscreen mode
    hasShadow: true,  // Add shadow to make window more visible
    opacity: 1.0,  // Full opacity initially
    backgroundColor: '#1a1a1a',  // Dark background so it's visible
    webPreferences: {
      nodeIntegration: false,
      contextIsolation: true,
      preload: path.join(__dirname, 'preload.js'),
      // Enable media access for speech recognition
      enableRemoteModule: false,
      sandbox: false,  // Disable sandbox to allow speech API
    }
  });

  // Critical: Set highest window level to show over fullscreen apps/games
  // Try 'screen-saver' first (highest level), fallback to 'pop-up-menu' or 'floating'
  // Note: For true fullscreen games, this may not work - game needs to be in Borderless mode
  try {
    mainWindow.setAlwaysOnTop(true, 'screen-saver', 1);
  } catch (e) {
    safeLog('Could not set screen-saver level, trying pop-up-menu:', e);
    mainWindow.setAlwaysOnTop(true, 'pop-up-menu', 1);
  }
  mainWindow.setVisibleOnAllWorkspaces(true);

  // Periodically ensure window stays on top (every 2 seconds)
  setInterval(() => {
    if (mainWindow && !mainWindow.isDestroyed()) {
      try {
        mainWindow.setAlwaysOnTop(true, 'screen-saver', 1);
      } catch (e) {
        mainWindow.setAlwaysOnTop(true, 'pop-up-menu', 1);
      }
      mainWindow.showInactive(); // Show without stealing focus
    }
  }, 2000);

  // Set click-through by default (can be toggled)
  if (isClickThrough) {
    mainWindow.setIgnoreMouseEvents(true, { forward: true });
  }

  // Grant permission for media devices (microphone for speech recognition)
  mainWindow.webContents.session.setPermissionRequestHandler((webContents, permission, callback) => {
    if (permission === 'media') {
      // Approve microphone access for speech recognition
      callback(true);
    } else {
      callback(false);
    }
  });

  // Load the app
  if (process.env.NODE_ENV === 'development' || !app.isPackaged) {
    mainWindow.loadURL('http://localhost:5173');
    // Open DevTools in development
    // mainWindow.webContents.openDevTools({ mode: 'detached' });
  } else {
    mainWindow.loadFile(path.join(__dirname, '../dist/index.html'));
  }

  // Handle window events
  mainWindow.on('closed', () => {
    mainWindow = null;
  });

  // Register global shortcuts
  registerShortcuts();
}

function registerShortcuts() {
  // Toggle click-through: Ctrl+Shift+C
  globalShortcut.register('CommandOrControl+Shift+C', () => {
    if (mainWindow) {
      isClickThrough = !isClickThrough;
      mainWindow.setIgnoreMouseEvents(isClickThrough, { forward: true });
      mainWindow.webContents.send('click-through-toggled', isClickThrough);
      safeLog(`Click-through: ${isClickThrough ? 'enabled' : 'disabled'}`);
    }
  });

  // Toggle DevTools: Ctrl+Shift+I
  globalShortcut.register('CommandOrControl+Shift+I', () => {
    if (mainWindow) {
      mainWindow.webContents.toggleDevTools();
    }
  });

  // Reload: Ctrl+Shift+R
  globalShortcut.register('CommandOrControl+Shift+R', () => {
    if (mainWindow) {
      mainWindow.reload();
    }
  });
}

// IPC handlers - must be set up before app.whenReady()
function setupIPCHandlers() {
  ipcMain.handle('get-display-size', () => {
    const primaryDisplay = screen.getPrimaryDisplay();
    return primaryDisplay.workAreaSize;
  });

  ipcMain.handle('set-window-position', (event, x, y) => {
    if (mainWindow) {
      mainWindow.setPosition(x, y);
    }
  });

  ipcMain.handle('set-window-size', (event, width, height) => {
    if (mainWindow) {
      mainWindow.setSize(width, height);
    }
  });

  ipcMain.handle('set-opacity', (event, opacity) => {
    if (mainWindow) {
      mainWindow.setOpacity(opacity);
    }
  });
}

function setupKeyboardMonitoring() {
  // Monitor T key for push-to-talk
  uIOhook.on('keydown', (e) => {
    // T key
    if (e.keycode === UiohookKey.T) {
      if (!isVoiceInputActive) {
        isVoiceInputActive = true;
        if (mainWindow) {
          mainWindow.webContents.send('voice-input-toggle', true);
        }
        safeLog('Voice input activated');
      }
    }
  });

  uIOhook.on('keyup', (e) => {
    // T key
    if (e.keycode === UiohookKey.T) {
      if (isVoiceInputActive) {
        isVoiceInputActive = false;
        if (mainWindow) {
          mainWindow.webContents.send('voice-input-toggle', false);
        }
        safeLog('Voice input deactivated');
     
[truncated — 1486 more characters]
```

### frontend/postcss.config.js

```javascript
export default {
  plugins: {
    tailwindcss: {},
    autoprefixer: {},
  },
}

```

[61 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]