# Project export: Brain Crush This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: A BCI-powered robotic arm that knows before you do - Devpost: https://devpost.com/software/brain-crush - GitHub: https://github.com/cc13985/treehacks - Demo: http://braincrush.vercel.app/ - Video: https://www.youtube.com/embed/NFhLC1fv3Ms?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 4 GitHub contributor(s) — Cursor (8 commits), Jeremy Hur (5 commits), sarah888 (2 commits), daniel04wawng (1 commits) ## Devpost submission (written by the team) ### Overview About the Project ### Inspiration Technology is supposed to make communication easier, but face-to-face connection harder than ever. Around 80% of Gen Zs report feeling lonely, and some are even turning to AI companions. We wanted to explore whether we could help people act on their real-world impulses. What We Built We developed a brain computer interface (BCI) pipeline that uses your brain activity to control a robotic arm and generates a custom message for that someone you might be interested in. Using Emotiv BCIs, Brain Crush reads raw EEG signals including alpha, low beta, high beta, and theta frequency bands. We then created our own interpretation layer that filters and processes these signal bands to convert them into robotic input commands we could send to our LeRobot. At the same time, we used Perplexity to create a prompt we could feed into HeyGen. We were able to train an AI avatar based on Jeremy for our demo that would deliver a message to the person we wanted to use Brain Crush on. Finally, Brain Crush is deployed on Vercel. Conceptually, the pipeline is: $$ \text{Brain Waves} \rightarrow \text{Signal Processing} \rightarrow \text{AI Interpretation} \rightarrow \text{Discrete Intent} \rightarrow \text{Robot Action} \rightarrow \text{AI Video Generation} $$ Because neural signals are noisy and inconsistent, we converted a continuous signal into a small set of discrete outputs. Basically, this meant mapping a high-dimensional time series into a few categories (e.g., calm, focused, excited, or neutral): $$ f: \mathbb{R}^n \rightarrow {A_1, A_2, A_3, A_4} $$ Each emotion activates a different robotic behaviour. How We Built It Our system consists of five main layers: 1) Brain Signal Collecting We used Emotiv Epoc X and Impulse BCIs to get real-time waveform data (EEGs). 2) EEG Processing We used data we collected from medical research to filter and isolate brain waves that are the most indicative of romantic interest. In our case, this would be the Beta High waves. We set our threshold to be around 300 microvolts squared for interest, and under 5 microvolts squared for a sustained 5 seconds for non-interest. 3) Intent Mapping To ensure reliability, we constrained the output to a small number of discrete actions. Instead of attempting control, we implemented a classification system that maps continuous inputs into a set of commands we programmed to the LeRobot. 4) Robotic Execution The robotic arm performs a physical task corresponding to the detected intent. For example: Extend the arm to hand candy over to your romantic interest Shake its arm no in the case that you didn’t feel too interested 4) HeyGen We used Perplexity to interpret the rest of the data available to us from the BCI to create a prompt for HeyGen. The prompt is made up of: Combination of the Alpha, Beta, and Gamma waves Rates of change (derivatives) of the fluctuations in brain activity The video that is uploaded of the user talking into the camera Then, HeyGen creates a message with an AI avatar trained to look and sound exactly like you. Basically, if you’re too shy to talk to someone at the cafe, our HeyGen avatar can do it for you. Data and Simulation We used data from a variety of sources. The main input was obviously real-time EEG data from our BCIs. To interpret our EEG data, we used Bright Data’s site scrapers to figure out which specific band of wavelength was most effective in determining romantic interest. Edge cases were also accounted for by using direct data injection in Node-Red, making sure any inconsistencies with our BCI were smoothed out. Challenges We Faced 1) EEG Outputs Dry electrodes produce noisy signals, so saline solution is used to improve conductivity for the BCI nodes. Since we didn’t have any saline solution, we had to grab salt shakers from a dining hall to create our own. 2) Output Translation We had to isolate individual brain waves and EEG data points and convert it to a set of outputs. We had to research and design our own logic system that linked specific bands of brain wavelength to romantic interest and excitement. 3) Real-Time Integration We had to optimize our system to reduce latency as much as possible, since we’re essentially controlling an entire AI processing, computer vision, and robotics system with our thoughts (mind control). What We Learned This was our first time working with LeRobot robotic arms, and Sarah, Jeremy, and Claire’s first experience using EEG technology. We had to quickly learn how to configure the LeRobot environment and interpret what the raw EEG outputs meant in practice. Something cool that all of us also learned was how to pet a Llama (shoutout to Munay!) Built with: Emotiv EEG headsets (insight and epocx) LeRobot Robotic Arms Node-RED Next.js Perplexity HeyGen BrightData Vercel ## README (from the GitHub repository) No README available. ## Detected evidence (automated analysis) Indexed codebase: 14 recognized source files, 142 KB. - CSS (language) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - Vercel (technology) — claimed on Devpost, not found in the code - AI coding agent: Cursor — evidence: commit authorship or trailers ## Codebase structure (from repository index) ### Files (24 of 24) ``` .gitignore extract_band_power.py frontend/app.js frontend/index.html frontend/README-wave-nodered.md frontend/style.css frontend/wave-monitor.html heygen/extract_band_power.py heygen/HEYGEN_README.md heygen/heygen_requirements.txt heygen/heygen_server.py heygen/heygen_static/index.html heygen/README.md lerobot/keyboard_drive_v2.py lerobot/src/lerobot.egg-info/dependency_links.txt lerobot/src/lerobot.egg-info/entry_points.txt lerobot/src/lerobot.egg-info/PKG-INFO lerobot/src/lerobot.egg-info/requires.txt lerobot/src/lerobot.egg-info/SOURCES.txt lerobot/src/lerobot.egg-info/top_level.txt lerobot/udp_listener_test.py node-red_arm-verdict-lock-flow.json node-red_ARM-VERDICT-README.md node-red_bci-wave-flow.json ``` ### Dependencies No dependency index available. ### Recent commits (newest first) - Brain Crush: live brainwave, Continue to result, fallback thresholds (150/3), result screen fix - successful integration of heygen + social media share links. now need to connect bci results - Add HeyGen avatar video generation app - Merge pull request #2 from cc13985/frontend-wave-monitor - feat: add EmailJS email sending + heart pulse animation at 60bpm - merge main: keep friend's index.html, add braincrush app - lsog' - Merge origin/main: keep keyboard_drive_v2.py - BCI wave, verdict (no lock), arm UDP, frontend Brain Crush - feat: full Brain Crush single-page app with 6-screen flow - Add BCI wave monitor frontend - pushing working - Add EEG band power extraction script - yay - Merge branch 'main' of https://github.com/cc13985/treehacks - Initial commit: trehacks + lerobot - Initial commit ## Key source files (fetched from GitHub, selected and truncated for size) ### node-red_ARM-VERDICT-README.md ```markdown # Arm verdict + lock (release when user returns to home) ## Node-RED quick checklist (to support Brain Crush + Continue to result) 1. **Import BCI wave flow** Import **`node-red_bci-wave-flow.json`**. You get: Format beta high for wave, Send to wave simulator, Wave clients connect here, **Robot done (port 5006)**, **Robot done → frontend**, Test inject (disable it if using real sensor). 2. **Import arm verdict flow** Import **`node-red_arm-verdict-lock-flow.json`**. You get: **Verdict (no lock)** (or Verdict and lock, depending on file). 3. **Wire BCI → wave** Connect your **sensor / high-beta output** to **Format beta high for wave** (already wired to **Send to wave simulator**). That gives the live wave and μV² in Brain Crush. 4. **Wire decision → arm and frontend** Connect **function 1** (APPROACH/REFUSE) → **Verdict (no lock)** (or Verdict and lock). - **First output** of Verdict → your **UDP out** node set to **127.0.0.1:5005** (arm). - **Second output** of Verdict → **Send to wave simulator** (so the app gets the verdict and shows “Robot moving…”). 5. **Robot done (Continue to result)** The BCI wave flow already has **Robot done (port 5006)** and **Robot done → frontend** wired to **Send to wave simulator**. When the Python arm sends ROBOT_DONE to port 5006, Node-RED sends `{ robotDone: true }` to the app and the Continue button enables. No extra wiring. 6. **Release (if you use the lock flow)** If your verdict flow has **Release lock** and **UDP RELEASE to arm**: connect **Wave clients connect here** → **Release lock (from frontend)** → **UDP RELEASE to arm** (127.0.0.1:5005). 7. **Deploy** Click **Deploy**. ## Behaviour 1. When your flow decides **APPROACH** or **REFUSE**, that decision runs **once** and is sent to the arm and frontend. 2. The arm and Node-RED are **locked**: no new decision is sent to the arm until the user **returns to the home screen** in Brain Crush. 3. When the user clicks **"Keep analyzing my brain"** (Reset) and goes back to the home screen, the frontend sends **release**; Node-RED clears its lock and sends **RELEASE** to the Python arm so it can accept the next decision. ## Node-RED setup ### Step 1: Import the lock nodes 1. In Node-RED: **Menu (≡) → Import → select file** (or paste from clipboard). 2. Choose **`node-red_arm-verdict-lock-flow.json`** and import. 3. You should see three nodes: **"Verdict and lock"**, **"Release lock (from frontend)"**, and **"UDP RELEASE to arm"**. ### Step 2: Wire Verdict and lock between function 1 and UDP 1. **Remove** any existing wire from **function 1** to **udp 127.0.0.1:5005**. 2. **Connect function 1 → Verdict and lock** - Drag from the **output** of **function 1** to the **input** of **"Verdict and lock"**. 3. **Connect Verdict and lock → UDP** - **"Verdict and lock"** has **two outputs**. Drag from the **first (top) output** to the **input** of **udp 127.0.0.1:5005**. 4. **Connect Verdict and lock → fro [truncated — 3712 more characters] ``` ### heygen/HEYGEN_README.md ```markdown # HeyGen Text → Video Simple app: **prompt on the homescreen** → your avatar speaks it → **video**. Uses the [HeyGen API Quick Start](https://docs.heygen.com/docs/quick-start) (Option C: Digital Twin / Photo Avatar). ## Setup 1. **Install** ```bash pip install -r heygen_requirements.txt ``` 2. **API key** Get it from [HeyGen → Settings → API](https://app.heygen.com/settings?nav=API). Put in `.env`: ```env HEYGEN_API_KEY=your_api_key ``` (If you already have `LIVEAVATAR_API_KEY` in `.env`, the app will use that if `HEYGEN_API_KEY` is not set.) 3. **Run** ```bash python heygen_server.py ``` 4. Open **http://127.0.0.1:5000**, enter what the avatar should talk about, click **Generate video**. The video appears when ready (usually 1–2 minutes). ## Config - **Avatar** default: look ID `160e3fd51deb4be180f90a491c4b6c9b` as a **Photo Avatar look** (`talking_photo`). Override with `HEYGEN_AVATAR_ID` in `.env`. If your ID is a Digital Twin / public avatar, set `HEYGEN_AVATAR_TYPE=avatar` in `.env`. - **Original voice**: The app uses the avatar/look’s **default voice** when HeyGen has one (from List Avatars V2 or from avatar groups’ `default_voice_id`). If none is set, it falls back to a default voice. To use the avatar’s voice, assign a default voice to the look in the HeyGen app if possible, or set `HEYGEN_VOICE_ID` in `.env` to that voice’s ID. - **Voice**: override with `HEYGEN_VOICE_ID` in `.env` (from [List Voices V2](https://docs.heygen.com/reference/list-voices-v2)). ## API - **POST /api/video/generate** — Body: `{ "prompt": "what to talk about" }` → returns `{ "video_id": "..." }`. - **GET /api/video/status/** — Returns `{ "status", "video_url", "error" }`. Poll until `status === "completed"`. ``` ### frontend/app.js ```javascript // ═══════════════════════════════════════════════════════ // BCI Wave Monitor — Frontend // ═══════════════════════════════════════════════════════ // // USAGE (from your start button, Node-RED, or any JS): // // // Start with simulated data (demo mode): // BCI.start(); // // // Start with external data (you push samples manually): // BCI.start({ mode: 'external', timeLimit: 30 }); // BCI.pushSample(0.42); // call this each time new data arrives // // // Start with WebSocket (auto-connects and reads data): // BCI.start({ mode: 'websocket', url: 'ws://localhost:1880/bci' }); // // // Stop manually: // BCI.stop(); // // // Listen for stop events (time limit or trajectory hit): // BCI.onStop = (reason) => console.log('Stopped:', reason); // // ═══════════════════════════════════════════════════════ (() => { // ── DOM Elements ────────────────────────────────── const stopBtn = document.getElementById('stopBtn'); const wavePanel = document.getElementById('wavePanel'); const statusBadge = document.getElementById('statusBadge'); const canvas = document.getElementById('waveCanvas'); const ctx = canvas.getContext('2d'); const amplitudeDisplay = document.getElementById('amplitudeValue'); const frequencyDisplay = document.getElementById('frequencyValue'); const elapsedDisplay = document.getElementById('elapsedValue'); // ── Config (defaults, overridden by BCI.start(options)) ── const config = { mode: 'simulated', // 'simulated' | 'external' | 'websocket' url: 'ws://localhost:1880/bci', // WebSocket URL (only used in 'websocket' mode) timeLimit: 0, // seconds, 0 = no limit trajectoryThreshold: 0, // amplitude threshold, 0 = disabled speed: 1.8, // sweep speed (pixels per frame) eraseWidth: 40, // gap ahead of sweep }; // ── Internal State ──────────────────────────────── let running = false; let animId = null; let elapsedTime = 0; let sweepX = 0; let waveBuffer = []; let W = 0; let H = 0; const DPR = window.devicePixelRatio || 1; const LINE_WIDTH = 2.5; const TICKER_RADIUS = 7; const CYAN = '#00D2FF'; // Data queue — external sources push here, renderer consumes let dataQueue = []; let lastSample = 0; // WebSocket reference let ws = null; // Simulated signal state let signalPhase = 0; let ampTarget = 0.6; let ampCurrent = 0.6; let driftTimer = 0; const FIXED_FREQ = 0.7; // ── Simulated Signal Generator ──────────────────── function generateSimulatedSample(dt) { signalPhase += dt; driftTimer -= dt; if (driftTimer <= 0) { ampTarget = 0.3 + Math.random() * 0.7; driftTimer = 2.0 + Math.random() * 4.0; } ampCurrent += (ampTarget - ampCurrent) * 0.008; const primary = Math.sin(signalPhase * FIXED_FREQ * 2 * Math.PI) * ampCurrent; const harmonic = Math.sin(signalPhase * FIXED_FREQ * 4 * Math.PI) * ampCurrent * 0.12; return primary + harmonic; } // ── Get Next Sample ─────────────────────────────── // Returns a normalized value roughly in [-1, 1] function getNextSample(dt) { if (config.mode === 'simulated') { return generateSimulatedSample(dt); } // For external / websocket: consume from queue, or hold last value if (dataQueue.length > 0) { lastSample = dataQueue.shift(); } return lastSample; } // ── WebSocket ───────────────────────────────────── function connectWebSocket() { if (ws) ws.close(); ws = new WebSocket(config.url); ws.onopen = () => console.log('[BCI] WebSocket connected:', config.url); ws.onclose = () => console.log('[BCI] WebSocket closed'); ws.onerror = (e) => console.error('[BCI] WebSocket error:', e); ws.onmessage = (event) => { try { const data = JSON.parse(event.data); // Accept { amplitude: number } or just a raw number const value = typeof data === 'number' ? data : (data.amplitude ?? data.value ?? 0); dataQueue.push(value); } catch { // Try as plain number const num = parseFloat(event.data); if (!isNaN(num)) dataQueue.push(num); } }; } function disconnectWebSocket() { if (ws) { ws.close(); ws = null; } } // ── Resize ──────────────────────────────────────── function resize() { const rect = canvas.parentElement.getBoundingClientRect(); W = rect.width; H = rect.height; canvas.width = W * DPR; canvas.height = H * DPR; canvas.style.width = W + 'px'; canvas.style.height = H + 'px'; ctx.setTransform(DPR, 0, 0, DPR, 0, 0); waveBuffer = new Array(Math.ceil(W)).fill(null); sweepX = 0; } window.addEventListener('resize', resize); // ── Drawing: Grid ───────────────────────────────── function drawGrid() { ctx.save(); ctx.strokeStyle = 'rgba(37, 38, 54, 0.8)'; ctx.lineWidth = 1; const hCount = 8; for (let i = 1; i < hCount; i++) { const y = (H / hCount) * i; ctx.beginPath(); ctx.moveTo(0, y); ctx.lineTo(W, y); ctx.stroke(); } const vSpacing = 80; for (let x = vSpacing; x < W; x += vSpacing) { ctx.beginPath(); ctx.moveTo(x, 0); ctx.lineTo(x, H); ctx.stroke(); } // Center baseline ctx.strokeStyle = 'rgba(37, 38, 54, 1)'; ctx.lineWidth = 1.5; ctx.beginPath(); ctx.moveTo(0, H / 2); ctx.lineTo(W, H / 2); ctx.stroke(); ctx.restore(); } // ── Drawing: Wave + Ticker ──────────────────────── function drawWave(tickerPosX, tickerPosY) { const midY = H / 2; ctx.save(); ctx.lineWidth = LINE_WIDTH; ctx.lineCap = 'round'; ctx.lineJoin = 'round'; const eraseStart = sweepX; const eraseEnd = (sweepX + config.eraseWidth) % W; // Trail ctx.beginPath(); let drawing = false; for (let x = 0; x < W; x++) { const inErase = eraseEnd > eraseStart ? (x >= eraseStart && x <= eraseEnd) : (x >= eraseStart || x <= eraseEnd) [truncated — 6260 more characters] ``` ### extract_band_power.py ```python """ Extract band power features from the s52.mat EEG dataset. Mirrors the band power output that Emotiv Cortex provides (theta, alpha, betaL, betaH, gamma). """ import scipy.io import numpy as np from scipy.signal import welch # ---- Load data ---- print("Loading s52.mat...") mat = scipy.io.loadmat(r'c:\Users\User\Downloads\s52.mat') eeg = mat['eeg'][0, 0] srate = int(eeg['srate'][0, 0]) # 512 Hz n_channels = 68 print(f"Sampling rate: {srate} Hz") print(f"Channels: {n_channels}") # ---- Define frequency bands (same as Emotiv) ---- BANDS = { 'theta': (4, 8), 'alpha': (8, 12), 'betaL': (12, 16), # Emotiv "low beta" 'betaH': (16, 25), # Emotiv "high beta" 'gamma': (25, 45), } print(f"\nBand definitions (matching Emotiv):") for name, (lo, hi) in BANDS.items(): print(f" {name}: {lo}-{hi} Hz") def compute_band_power(eeg_data, srate, window_sec=2.0): """ Compute band power for each channel using Welch's method. Parameters ---------- eeg_data : ndarray, shape (n_channels, n_samples) Raw EEG data srate : int Sampling rate in Hz window_sec : float Window length for Welch PSD estimation Returns ------- band_powers : dict {band_name: array of shape (n_channels,)} with power in each band freqs : ndarray Frequency axis from Welch psd : ndarray Full PSD, shape (n_channels, n_freqs) """ nperseg = int(window_sec * srate) # samples per segment # Welch PSD for all channels freqs, psd = welch(eeg_data, fs=srate, nperseg=nperseg, axis=1) # Extract power in each band band_powers = {} for band_name, (f_low, f_high) in BANDS.items(): # Find frequency indices within this band idx = np.where((freqs >= f_low) & (freqs <= f_high))[0] # Mean power in the band for each channel band_powers[band_name] = np.mean(psd[:, idx], axis=1) return band_powers, freqs, psd # ---- Extract from resting state (baseline) ---- print("\n" + "="*60) print("RESTING STATE - Band Power") print("="*60) rest_data = eeg['rest'] # shape (68, 34048) print(f"Rest data shape: {rest_data.shape} ({rest_data.shape[1]/srate:.1f} seconds)") rest_powers, freqs, rest_psd = compute_band_power(rest_data, srate) print(f"\nBand power per channel (first 5 channels):") print(f"{'Channel':<10}", end="") for band in BANDS: print(f"{band:>12}", end="") print() print("-" * 70) for ch in range(5): print(f"Ch {ch:<7}", end="") for band in BANDS: print(f"{rest_powers[band][ch]:>12.1f}", end="") print() # ---- Extract from a single imagery trial ---- print("\n" + "="*60) print("SINGLE IMAGERY TRIAL (left hand, trial #1) - Band Power") print("="*60) imagery_left = eeg['imagery_left'] # shape (68, 358400) frame = eeg['frame'].flatten() # [-2000, 5000] ms n_trials = int(eeg['n_imagery_trials'][0, 0]) # Data is pre-epoched: 100 trials x 3584 samples concatenated trial_samples = int((frame[1] - frame[0]) / 1000 * srate) pre_samples = int(abs(frame[0]) / 1000 * srate) # 1024 samples (2s before onset) print(f"Trial length: {trial_samples} samples ({trial_samples/srate:.1f}s, from {frame[0]}ms to {frame[1]}ms)") print(f"Trials: {n_trials}, Total samples: {n_trials} x {trial_samples} = {n_trials * trial_samples}") print(f"Event onset within each trial: sample {pre_samples} (t=0)") # Extract trial 1 (first 3584 samples) trial_idx = 0 trial_start = trial_idx * trial_samples trial_end = trial_start + trial_samples trial_data = imagery_left[:, trial_start:trial_end] print(f"Trial 1: samples [{trial_start}:{trial_end}]") print(f"Trial data shape: {trial_data.shape}") trial_powers, _, _ = compute_band_power(trial_data, srate) print(f"\nBand power for trial 1 (first 5 channels):") print(f"{'Channel':<10}", end="") for band in BANDS: print(f"{band:>12}", end="") print() print("-" * 70) for ch in range(5): print(f"Ch {ch:<7}", end="") for band in BANDS: print(f"{trial_powers[band][ch]:>12.1f}", end="") print() # ---- Compare: How Emotiv would output this ---- print("\n" + "="*60) print("EMOTIV-STYLE OUTPUT (simulated)") print("="*60) print(""" Emotiv Cortex streams 'pow' data like this per sample: [AF3/theta, AF3/alpha, AF3/betaL, AF3/betaH, AF3/gamma, T7/theta, T7/alpha, T7/betaL, T7/betaH, T7/gamma, Pz/theta, Pz/alpha, Pz/betaL, Pz/betaH, Pz/gamma, T8/theta, T8/alpha, T8/betaL, T8/betaH, T8/gamma, AF4/theta, AF4/alpha, AF4/betaL, AF4/betaH, AF4/gamma] The .mat file has 68 channels. Mapping approximate equivalents: AF3 ~ Ch 0, T7 ~ Ch 10, Pz ~ Ch 31, T8 ~ Ch 40, AF4 ~ Ch 3 """) # Approximate channel mapping (68-ch montage to Emotiv Insight positions) emotiv_map = { 'AF3': 0, 'T7': 10, 'Pz': 31, 'T8': 40, 'AF4': 3, } print("Simulated Emotiv pow output from trial 1:") print(f"{'Electrode':<10}", end="") for band in BANDS: print(f"{band:>10}", end="") print() print("-" * 60) emotiv_pow_vector = [] for elec, ch_idx in emotiv_map.items(): print(f"{elec:<10}", end="") for band in BANDS: val = trial_powers[band][ch_idx] emotiv_pow_vector.append(round(val, 3)) print(f"{val:>10.3f}", end="") print() print(f"\nAs a flat array (like Emotiv streams it):") print(f" {emotiv_pow_vector}") # ---- Sliding window: band power over time (like real-time Emotiv) ---- print("\n" + "="*60) print("TIME-SERIES BAND POWER (2s sliding windows, like Emotiv real-time)") print("="*60) window_samples = 2 * srate # 2 second window step_samples = srate // 2 # 0.5s step (2 Hz update, similar to Emotiv) n_windows = (trial_data.shape[1] - window_samples) // step_samples + 1 print(f"Window: 2s ({window_samples} samples), Step: 0.5s, Windows: {n_windows}") print(f"\nAlpha power over time for mapped Emotiv channels:") print(f"{'Time (s)':<10}", end="") for elec in emotiv_map: print(f"{elec:>10}", end="") p [truncated — 471 more characters] ``` ### lerobot/udp_listener_test.py ```python #!/usr/bin/env python3 """Minimal UDP listener on 127.0.0.1:5005. Run this to verify Node-RED is sending. Usage: python udp_listener_test.py Then in Node-RED: inject APPROACH into your UDP-out node. You should see lines here.""" import socket UDP_IP = "127.0.0.1" UDP_PORT = 5005 sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) sock.bind((UDP_IP, UDP_PORT)) print(f"Listening on {UDP_IP}:{UDP_PORT}. Inject APPROACH/REFUSE/RELEASE from Node-RED...") print("-" * 50) while True: data, addr = sock.recvfrom(1024) raw = data.decode("utf-8", errors="replace").strip() print(f"RECV: {raw!r} (from {addr})") ``` ### frontend/wave-monitor.html ```html BCI Wave Monitor
Idle
EEG Signal Brain Wave Activity
Amplitude 0.00 μV
Frequency 0.0 Hz
Elapsed 0.0s
``` ### frontend/style.css ```css /* ── Reset & Base ──────────────────────────────────── */ *, *::before, *::after { margin: 0; padding: 0; box-sizing: border-box; } :root { --bg: #0a0b10; --surface: #12131a; --surface-2: #1a1b25; --border: #252636; --text: #e4e4ed; --text-dim: #8585a0; --accent: #6C63FF; --accent-light: #8B83FF; --cyan: #00D2FF; --green: #00E09E; --red: #FF4D6A; --wave-color: #00D2FF; --wave-glow: rgba(0, 210, 255, 0.3); --ticker-color: #fff; --font: 'Inter', -apple-system, BlinkMacSystemFont, sans-serif; --mono: 'JetBrains Mono', 'Fira Code', monospace; } html, body { height: 100%; background: var(--bg); color: var(--text); font-family: var(--font); -webkit-font-smoothing: antialiased; overflow: hidden; } /* ── App Layout ───────────────────────────────────── */ .app { display: flex; flex-direction: column; height: 100vh; } /* ── Header ───────────────────────────────────────── */ .header { display: flex; align-items: center; justify-content: space-between; padding: 16px 28px; border-bottom: 1px solid var(--border); background: var(--surface); backdrop-filter: blur(20px); z-index: 10; } .header-left { display: flex; align-items: center; gap: 16px; } .logo { display: flex; align-items: center; gap: 12px; } .logo h1 { font-size: 18px; font-weight: 600; letter-spacing: -0.3px; background: linear-gradient(135deg, #e4e4ed, #8585a0); -webkit-background-clip: text; -webkit-text-fill-color: transparent; } .status-badge { display: flex; align-items: center; gap: 8px; padding: 6px 14px; border-radius: 20px; background: var(--surface-2); border: 1px solid var(--border); font-size: 13px; font-weight: 500; color: var(--text-dim); transition: all 0.3s ease; } .status-badge.active { border-color: rgba(0, 224, 158, 0.3); color: var(--green); } .status-dot { width: 8px; height: 8px; border-radius: 50%; background: var(--text-dim); transition: background 0.3s ease; } .status-badge.active .status-dot { background: var(--green); box-shadow: 0 0 8px var(--green); animation: pulse-dot 2s ease infinite; } @keyframes pulse-dot { 0%, 100% { opacity: 1; } 50% { opacity: 0.5; } } /* ── Main ─────────────────────────────────────────── */ .main { flex: 1; display: flex; flex-direction: column; position: relative; overflow: hidden; } /* ── Wave Panel ───────────────────────────────────── */ .wave-panel { flex: 1; display: flex; flex-direction: column; opacity: 1; } .panel-header { display: flex; align-items: center; justify-content: space-between; padding: 14px 28px; border-bottom: 1px solid var(--border); background: var(--surface); } .channel-info { display: flex; align-items: baseline; gap: 12px; } .channel-label { font-size: 15px; font-weight: 600; color: var(--cyan); } .channel-sub { font-size: 12px; color: var(--text-dim); } .panel-controls { display: flex; align-items: center; gap: 24px; } .metric { display: flex; align-items: baseline; gap: 6px; } .metric-label { font-size: 11px; font-weight: 500; color: var(--text-dim); text-transform: uppercase; letter-spacing: 0.5px; } .metric-value { font-family: var(--mono); font-size: 16px; font-weight: 500; color: var(--text); min-width: 50px; text-align: right; } .metric-unit { font-size: 11px; color: var(--text-dim); } .stop-btn { display: flex; align-items: center; gap: 6px; padding: 7px 16px; border: 1px solid rgba(255, 77, 106, 0.3); border-radius: 8px; background: rgba(255, 77, 106, 0.08); color: var(--red); font-family: var(--font); font-size: 13px; font-weight: 500; cursor: pointer; transition: all 0.2s ease; } .stop-btn:hover { background: rgba(255, 77, 106, 0.15); border-color: rgba(255, 77, 106, 0.5); } /* ── Canvas ────────────────────────────────────────── */ .canvas-wrapper { flex: 1; position: relative; background: var(--bg); overflow: hidden; } canvas { display: block; width: 100%; height: 100%; } /* ── Scrollbar ────────────────────────────────────── */ ::-webkit-scrollbar { width: 6px; } ::-webkit-scrollbar-track { background: transparent; } ::-webkit-scrollbar-thumb { background: var(--border); border-radius: 3px; } ``` ### lerobot/keyboard_drive_v2.py ```python import sys import time import socket import json from scservo_sdk import * # feetech-servo-sdk # --- 1. CONFIGURATION --- DEVICENAME = '/dev/cu.usbmodem58FA0920121' BAUDRATE = 1000000 # UDP Configuration (Must match your Node-RED UDP Node) UDP_IP = "127.0.0.1" UDP_PORT = 5005 # Port where Node-RED listens for "robot finished" (UDP in node) ROBOT_DONE_PORT = 5006 # Motor IDs BASE = 1 SHOULDER = 2 ELBOW = 3 WRIST = 4 GRIPPER = 6 # --- 2. YOUR CALIBRATED POSES --- POSES = { "HOME": { BASE: 2059, SHOULDER: 1821, ELBOW: 1317, WRIST: 3051, GRIPPER: 2475 }, "EXTEND": { BASE: 2043, SHOULDER: 2477, ELBOW: 1373, WRIST: 2275, GRIPPER: 2475 # Holding the object }, "RIGHT": { BASE: 2568, SHOULDER: 1821, ELBOW: 1317, WRIST: 3051, GRIPPER: 2438 }, "LEFT": { BASE: 1544, SHOULDER: 1821, ELBOW: 1317, WRIST: 3051, GRIPPER: 2438 } } # Gripper Settings GRIPPER_HOLD = 2475 GRIPPER_OPEN = 3000 # Based on your finding that 1800 was too tight # --- 3. HARDWARE INITIALIZATION --- portHandler = PortHandler(DEVICENAME) packetHandler = PacketHandler(0) if not portHandler.openPort() or not portHandler.setBaudRate(BAUDRATE): print("❌ Connection Failed. Check USB cable and DEVICENAME.") sys.exit(1) # Enable Torque for all motors for id in [BASE, SHOULDER, ELBOW, WRIST, GRIPPER]: packetHandler.write1ByteTxRx(portHandler, id, 40, 1) # Tracker for current motor positions current_pos = {} for id in [BASE, SHOULDER, ELBOW, WRIST, GRIPPER]: p, res, err = packetHandler.read2ByteTxRx(portHandler, id, 56) current_pos[id] = p # --- 4. ANIMATION ENGINE --- def write_motor(motor_id, pos): """Sends command to motor and updates internal tracker.""" pos = int(max(0, min(4096, pos))) packetHandler.write2ByteTxRx(portHandler, motor_id, 42, pos) current_pos[motor_id] = pos def smooth_move(target_pose, duration_sec=2.0): """Interpolates movement for cinematic smoothness.""" hz = 50 steps = int(duration_sec * hz) start_snapshot = {mid: current_pos[mid] for mid in target_pose} for step in range(steps): progress = step / steps for mid, target in target_pose.items(): start = start_snapshot[mid] new_val = start + (target - start) * progress write_motor(mid, new_val) time.sleep(1.0 / hz) for mid, target in target_pose.items(): write_motor(mid, target) # --- 5. BEHAVIORS --- def _notify_robot_done(): """Tell Node-RED (and thus Brain Crush) that the robot finished moving.""" try: out = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) out.sendto(b"ROBOT_DONE", (UDP_IP, ROBOT_DONE_PORT)) out.close() except Exception: pass def behavior_hand_over(): print("\n🤖 Behavior: Give Candy (Attraction Detected)") # 1. Extend Arm target = POSES["EXTEND"].copy() target[GRIPPER] = GRIPPER_HOLD smooth_move(target, duration_sec=2.5) time.sleep(0.5) # 2. Release item print(" Releasing item...") write_motor(GRIPPER, GRIPPER_OPEN) time.sleep(1.0) # 3. Retract to Home print(" Retracting...") smooth_move(POSES["HOME"], duration_sec=2.0) print("✅ Ready for next command.") _notify_robot_done() def behavior_refuse(): print("\n🤖 Behavior: Reject (Low Attraction)") # Shake head No (Left -> Right -> Left -> Home) smooth_move(POSES["LEFT"], duration_sec=0.4) smooth_move(POSES["RIGHT"], duration_sec=0.4) smooth_move(POSES["LEFT"], duration_sec=0.4) smooth_move(POSES["HOME"], duration_sec=1.0) print("✅ Ready for next command.") _notify_robot_done() # --- 6. MAIN LOOP (UDP LISTENER) --- print("-----------------------------------------") print(f"🧠 BCI RECEIVER ACTIVE ON PORT {UDP_PORT}") print(" - High Beta: HAND OVER | Low Beta (5s): REFUSE") print("-----------------------------------------") # Setup Socket sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) sock.bind((UDP_IP, UDP_PORT)) sock.setblocking(False) try: while True: try: data, addr = sock.recvfrom(1024) message = data.decode('utf-8', errors='replace').strip() if message.startswith('{'): try: obj = json.loads(message) message = obj.get('payload', obj.get('message', message)) if isinstance(message, str): message = message.strip().upper() except Exception: pass else: message = message.upper() if message == "APPROACH": print(f"\rCurrent Brain State: APPROACH ", end="", flush=True) behavior_hand_over() elif message == "REFUSE": print(f"\rCurrent Brain State: REFUSE ", end="", flush=True) behavior_refuse() except BlockingIOError: pass time.sleep(0.01) except KeyboardInterrupt: print("\nShutting down...") finally: portHandler.closePort() print("Disconnected.") ``` [4 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]