# Project export: Cadence- A voice that sounds like you
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: UC Berkeley AI Hackathon 2026
- Tagline: 4-agent streaming pipeline: Deepgram hears the partner, Claude drafts emotion-tuned replies from memory, ElevenLabs speaks in your cloned voice. Real-time AAC for tens of millions who can't speak.
- Devpost: https://devpost.com/software/cadence-a-voice-that-sounds-like-you
- GitHub: https://github.com/Yxp23/cadence
- Video: https://www.youtube.com/embed/helXwrHwf90?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) β Yxp23 (9 commits)
## Devpost submission (written by the team)
### Overview
π
### Inspiration
Picture this. You're at dinner with your family. Your daughter tells a joke. You laugh β but the only voice you can use to say "that's funny" sounds like a GPS. Your mom asks how you're feeling. You want to say "a little tired, but okay." By the time you've tapped it out on a stiff menu of pre-built phrases, the conversation has moved on. When the voice finally speaks, it sounds flat β like a stranger reading your words. You wanted to tell your dad you love him before bed. The robot did it for you. This is the daily reality for roughly \(97{,}000{,}000\) people worldwide who rely on augmentative and alternative communication (AAC): \(2\text{M}+\) Americans with aphasia after stroke β thought intact, retrieval broken \(\sim 30{,}000\) Americans with ALS β losing the voice they once had, progressively \(\frac{1}{36}\) children diagnosed with autism (CDC, 2023) β roughly \(30\%\) minimally verbal \(1\text{M}+\) with cerebral palsy whose motor variability makes typing painful The tools they're given today were designed in the 1990s. The voices sound robotic. The menus take minutes to navigate. The emotion is completely missing. We built Cadence because "I love you" should never sound like a GPS announcement. π What It Does Cadence is a real-time AAC tool. Four live AI agents turn a heard conversation into a reply spoken in your own cloned voice, with the right feeling, in seconds β using ElevenLabs Instant Voice Cloning + emotion-tuned voice_settings for the output. The full loop: $$ \text{partner speech} \;\rightarrow\; \text{Listener (Deepgram)} \;\rightarrow\; \text{transcript} \;\rightarrow\; \text{Tiles + Memory} \;\rightarrow\; \text{tile grid} \;\rightarrow\; \text{taps} \;\rightarrow\; \text{Generator} \;\rightarrow\; {c_1, c_2, c_3} \;\rightarrow\; \text{pick} \;\rightarrow\; \text{ElevenLabs} \;\rightarrow\; \text{your voice} $$ Each candidate \(c_i\) is a tuple: $$ c_i = (\text{text}i,\ e_i,\ \mathbf{S}{e_i}) $$ where \(e_i\) is one of 13 emotion labels and \(\mathbf{S}_{e_i}\) is the hand-tuned ElevenLabs voice_settings vector for that emotion. π§ How We Built It Four AI agents + a Memory layer + a Voice layer Four decision-making AI agents β Listener, Tiles, Suggester, Generator β coordinate over a Redis memory layer, with ElevenLabs as the voice output layer. ElevenLabs Voice Cloning β the identity layer For users who can still speak (ALS pre-diagnosis, autistic adults, anyone before a stroke), we record \(\sim 60\) seconds of phoneme-balanced Harvard Sentences in the browser and POST to ElevenLabs Instant Voice Cloning. The returned voice_id is stored per-session in Redis under cadence:session:{sid}:voice so it survives page refresh. From that moment, every TTS call uses the user's voice, not a stock voice. Latency budget Cadence has to feel as fast as natural conversation: $$ T_{\text{total}} = T_{\text{stt}} + T_{\text{generate}} + T_{\text{select}} + T_{\text{tts}} $$ Empirically: \(T_{\text{stt}} \approx 200\text{ms}\), \(T_{\text{generate}} \approx 600\text{ms}\), \(T_{\text{tts}} \approx 700\text{ms}\). ElevenLabs is tuned for low first-byte latency: output_format=mp3_44100_64 optimize_streaming_latency=2 model_id=eleven_turbo_v2_5 turbo_v2_5 was picked over flash_v2_5 because cloned voices need its better prosody handling; mode 2 keeps prosody intact while streaming the first byte fast. Pre-warming (the biggest latency win) The moment the Generator returns the 3 candidates, the frontend issues 3 parallel ElevenLabs TTS requests before the user has even read them: $$ T_{\text{perceived}} = T_{\text{generate}} + \max\left( T_{\text{select}},\ \max_i T_{\text{tts}}(c_i) \right) $$ Because users spend \(T_{\text{select}} \approx 2\text{s}\) reading and deciding, we cache the audio during that window, so: $$ T_{\text{tap-to-sound}} \approx 0\text{ms} $$ Savings vs. no pre-warming: \(\Delta T \approx 700\text{ms}\) per turn. Over a 20-turn conversation, \(\sim 14\) seconds of accumulated waiting removed. Emotion engine β the moat Cloning a voice is easy now. Making an ElevenLabs cloned voice express emotion is what nobody else does for AAC. Each emotion \(e\) maps to a voice_settings vector: $$ \mathbf{S}_e = (s_e,\ b_e,\ y_e,\ v_e) $$ where \(s_e\) = stability (lower = more variation), \(b_e\) = similarity_boost (anchors the cloned identity), \(y_e\) = style, \(v_e\) = speed multiplier. Selected profiles after 13 hand-tuned iterations: $$ \mathbf{S}{\text{excited}} = (0.35,\ 0.80,\ 0.55,\ 0.98) $$ $$ \mathbf{S}{\text{warm}} = (0.45,\ 0.80,\ 0.40,\ 0.92) $$ $$ \mathbf{S}{\text{tender}} = (0.55,\ 0.82,\ 0.40,\ 0.88) $$ $$ \mathbf{S}{\text{neutral}} = (0.55,\ 0.80,\ 0.25,\ 0.92) $$ Critical constraint discovered empirically: \(s_e \geq 0.35\) for all \(e\). Drop below this floor and the cloned voice warbles. The Generator returns only the label \(e\) (from a closed vocabulary of 13); the backend looks up \(\mathbf{S}_e\) and passes it to ElevenLabs. Numbers are locked; expression is flexible. Turn-taking state machine A 2-state machine (Listening / Composing) with timer-driven transitions: Listening β Composing on a final transcript followed by \(\Delta t > 3.5\text{s}\); Composing β Listening when a candidate is picked and playback ends + a 500ms buffer; auto-return to Listening if idle \(> 120\text{s}\) with no taps. AudioWorklet-level mic gating Mic state is read synchronously in the worklet, not via React state: When listening is false, audio frames are dropped at the worklet level β they never reach Deepgram. No re-render lag, no echo from ElevenLabs playback feeding back, no room-noise pollution of memory. Durability β surviving long conversations Deepgram closes idle WS connections after \(\sim 10\text{s}\). During "My turn" no audio flows, so we send a KeepAlive every 5s (\(5\text{s} < 10\text{s}\)): Dwell-click for cerebral palsy users A progress value \(p(t) = \min(1,\ (t - t_0) / T_{\text{dwell}})\) with \(T_{\text{dwell}} = 1.1\text{s}\) by default, rendered with requestAnimationFrame. Reach \(p = 1\) to activate the tile β no physical tap required. Per-profile feature differentiation ALS gets the voice-banking prompt because voice cloning is most emotionally loaded for them β recording now, before speech is lost. Every value overridable in Settings. π οΈ Challenges We Faced ElevenLabs cloned voices warbling at low stability. Early high-emotion profiles used \(s_e \in [0.15, 0.25]\) and warbled. Fix: floor stability at \(s_e \geq 0.35\), compensate with higher style and anchor identity with \(b_e \geq 0.80\). Deepgram dropping connections during composing. WS dies silently after 10s idle. Fix: explicit KeepAlive every 5s. Echo loop from speakers. TTS playback got captured by the mic and transcribed as a partner turn. Fix: AudioWorklet-level mic gating during my-turn + 500ms post-playback buffer. Profile differentiation felt cosmetic. Fix: built real per-profile features β picture+word tiles, ALS voice-banking flow, dwell-click for CP. Generator returning unreliable numeric voice settings. Claude's free-form numbers drifted out of range. Fix: Generator picks only a label \(e\) from 13; backend maps \(e \mapsto \mathbf{S}_e\) deterministically. π What We Learned Voice cloning is the easy part. Emotion is the moat. Making a cloned voice feel requires hand-tuning, not auto-generation. Pre-warming is the cheapest latency win. Generate TTS while the user reads candidates and tap-to-sound feels instant. Real-time systems are 80% durability work. KeepAlives, heartbeats, mic gating β the difference between a demo and a product. AAC needs vary wildly even within one diagnosis. Profiles set defaults; Settings override everything. Empathy first, tech second. Every decision flowed from: would this make someone feel more like a person? π What's Next Deployment β Vercel + Render/Fly so anyone can try it Screen-reader ARIA pass β accessibility audit SLP partnership β speech-language pathologist review of the defaults Switch/scanning input β single-switch users (severe CP, late-stage ALS) Real user testing β paid co-design with AAC users from each of the four audiences Cadence is a working prototype today β but the dream is that one day, someone who can't speak will look up from their tablet, hear their own voice say "I love you" with real warmth, and the person across the table will hear them. Really hear them. For the first time in years β maybe ever. That's why we built this. π
## README (from the GitHub repository)
# Cadence β a voice that sounds like you
Real-time AAC that lets people who can't speak keep up with a conversation, in their own voice.
Built solo in 24 hours at the UC Berkeley AI Hackathon 2026.
## The problem
Nearly 100 million people worldwide can't rely on their own voice β ALS, stroke, autism, cerebral palsy. The cruelest part of a condition like aphasia is that the person knows exactly what they want to say and just can't get it out. And the augmentative and alternative communication (AAC) tools they're given today were designed in the 1990s: robotic voices, slow menus that take minutes to navigate, zero emotion.
Cadence is built to change that.
## What it does
Cadence listens to the conversation live. When someone taps a few concept tiles, it turns them into a full, natural sentence β spoken in a clone of the person's own voice, with the right emotion, in seconds.
It's not autocomplete. The wedge is that every other AAC tool predicts in a vacuum β Cadence is the first to ground every reply in what the other person just said. The same two taps after "Are you hungry?" and after "Did you like the food?" produce completely different sentences, because it heard the difference.
And it always proposes, never speaks for you β the user picks the candidate before anything is said aloud. Agency stays with the person.
## How it works
A 4-agent real-time pipeline, plus a cloned-voice layer:
| Layer | Tech | Role |
| --- | --- | --- |
| Listener | Deepgram streaming WS | Real-time partner transcription with endpointing + KeepAlive |
| Tiles agent | Claude Haiku | Picks the most contextually relevant tiles after each partner turn |
| Suggester | Claude Haiku | Proactive reply predictions with no taps needed |
| Generator | Claude Haiku | Fuses heard context + taps + memory into 3 emotion-tagged candidates |
| Memory | Redis (per session) | Persistent conversation log; recent turns feed back into the Generator |
| Voice | ElevenLabs (Instant Voice Cloning) | Speaks the chosen candidate in the user's cloned voice, with per-emotion settings |
Four decision-making AI agents β Listener, Tiles, Suggester, Generator β coordinate over a Redis memory layer, with ElevenLabs as the voice output.
### Key engineering details:
* **TTS pre-warming** β the moment the Generator returns candidates, all audio is fetched in parallel so tap-to-speech feels near-instant.
* **AudioWorklet-level mic gating** β audio is dropped at the worklet when it isn't the partner's turn, preventing the app from transcribing its own spoken output (echo) or polluting memory with room noise.
* **13 hand-tuned emotion profiles** β the hard part isn't cloning a voice, it's making a cloned voice feel. The Generator picks an emotion label; the backend maps it to locked, hand-tuned ElevenLabs voice settings (stability floored at 0.35 to avoid warbling).
* **Turn-taking state machine** plus Deepgram KeepAlive so the connection survives long conversational pauses.
* **Four accessibility profiles** β autistic, ALS, aphasia, cerebral palsy β each transforming the UI (picture+word tiles, voice banking, dwell-click), all overridable in Settings.
## Tech stack
* **Frontend:** React + Vite + Tailwind + framer-motion Β· Web Audio API
* **Backend:** FastAPI + WebSockets (Python)
* **Speech-to-text:** Deepgram (streaming)
* **Reasoning:** Claude Haiku
* **Memory:** Redis
* **Voice:** ElevenLabs (Instant Voice Cloning, turbo_v2_5)
## Running locally
You'll need API keys for Deepgram, Anthropic, ElevenLabs, and a Redis URL.
### 1. Backend
```bash
cd backend
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
Create `backend/.env` with your keys:
```
DEEPGRAM_API_KEY=your_key
ANTHROPIC_API_KEY=your_key
ELEVENLABS_API_KEY=your_key
REDIS_URL=your_redis_url
```
Then run:
```bash
python main.py
```
### 2. Frontend
```bash
cd frontend
npm install
npm run dev
```
Open `http://localhost:5173`, click Connect, grant microphone access, and start a conversation.
> **No mic handy?** Use the Simulate panel to feed a partner phrase as text and test the full pipeline without audio.
## Links
* Demo and full writeup: [Devpost](https://devpost.com/software/cadence-a-voice-that-sounds-like-you)
## Note
This is a working prototype built at a hackathon β not a medical device. For any clinical use, consult a speech-language pathologist. Voices are cloned with consent and stored privately per session.
Built with Deepgram, Claude, ElevenLabs, and Redis.
## Detected evidence (automated analysis)
Indexed codebase: 15 recognized source files, 165 KB.
- Anthropic (technology) β detected in the code
- CSS (language) β detected in the code
- FastAPI (technology) β detected in the code
- HTML (language) β detected in the code
- JavaScript (language) β detected in the code
- Python (language) β detected in the code
- React (technology) β detected in the code
- Redis (technology) β detected in the code
- Tailwind CSS (technology) β detected in the code
## Codebase structure (from repository index)
### Files (19 of 19)
```
.gitignore
backend/.env.example
backend/main.py
backend/requirements.txt
frontend/index.html
frontend/package.json
frontend/postcss.config.js
frontend/public/audio-processor.js
frontend/src/App.jsx
frontend/src/Demo.jsx
frontend/src/index.css
frontend/src/Landing.jsx
frontend/src/main.jsx
frontend/src/symbols.js
frontend/src/Tutorial.jsx
frontend/src/VoiceRecorder.jsx
frontend/tailwind.config.js
frontend/vite.config.js
README.md
```
### Dependencies
- backend/requirements.txt: anthropic@>=0.40.0, fastapi@==0.111.0, httpx@>=0.27.0, python-dotenv@==1.0.1, python-multipart@>=0.0.9, redis@>=5.0.0, uvicorn[standard]@==0.29.0, websockets@==12.0
- frontend/package.json: @vitejs/plugin-react@^4.3.1, autoprefixer@^10.4.19, framer-motion@^12.40.0, postcss@^8.4.38, react@^18.3.1, react-dom@^18.3.1, tailwindcss@^3.4.4, vite@^5.3.1
### Recent commits (newest first)
- docs: add devpost link
- trigger github contributors update
- docs: add README.md
- docs(demo): honest agent count β "4 AI agents, 1 memory, 1 voice"
- fix(demo): A/B sample uses stock voice for AAC, cloned voice for Cadence
- feat(demo): A/B audio comparison β flat AAC vs Cadence emotional
- feat(frontend): polished 8-step tutorial walkthrough
- feat(backend): surface Redis + ElevenLabs status on /health
- chore(deps): add framer-motion for page transitions and UI animations
- feat(frontend): calm AAC UI with profiles, voice banking, and cinematic intro
- feat(backend): 5-agent AAC pipeline with voice cloning and durability
- feat: Step 0+1 β scaffold + live listening proof
## Key source files (fetched from GitHub, selected and truncated for size)
### backend/requirements.txt
```
fastapi==0.111.0
uvicorn[standard]==0.29.0
websockets==12.0
python-dotenv==1.0.1
anthropic>=0.40.0
httpx>=0.27.0
redis>=5.0.0
python-multipart>=0.0.9
```
### frontend/package.json
```
{
"name": "cadence-frontend",
"private": true,
"version": "0.0.1",
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview"
},
"dependencies": {
"framer-motion": "^12.40.0",
"react": "^18.3.1",
"react-dom": "^18.3.1"
},
"devDependencies": {
"@vitejs/plugin-react": "^4.3.1",
"autoprefixer": "^10.4.19",
"postcss": "^8.4.38",
"tailwindcss": "^3.4.4",
"vite": "^5.3.1"
}
}
```
### frontend/src/main.jsx
```javascript
import React from 'react'
import ReactDOM from 'react-dom/client'
import App from './App.jsx'
import './index.css'
ReactDOM.createRoot(document.getElementById('root')).render(
)
```
### backend/main.py
```python
"""
Cadence backend β Step 2: Fusion Engine (Generator)
Flow:
browser mic (PCM16) β WebSocket β here β Deepgram raw WS β transcripts β back to browser
browser taps β here β Claude (context + taps) β 3 candidate sentences β back to browser
The Generator is the core of Cadence: same taps + different heard context = different candidates.
"""
import asyncio
import json
import logging
import os
from collections import deque
import time
from typing import Optional
import anthropic
import httpx
import redis.asyncio as redis_async
import uvicorn
import websockets as ws_lib
from dotenv import load_dotenv
from fastapi import FastAPI, File, Form, HTTPException, Request, UploadFile, WebSocket, WebSocketDisconnect
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import StreamingResponse
load_dotenv()
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s %(message)s")
log = logging.getLogger("cadence")
app = FastAPI(title="Cadence")
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
DEEPGRAM_API_KEY = os.getenv("DEEPGRAM_API_KEY", "")
DEEPGRAM_BASE = "wss://api.deepgram.com/v1/listen"
# NOTE: verify these query params at https://developers.deepgram.com/reference/streaming
# if behaviour seems wrong after a Deepgram API update.
DEEPGRAM_PARAMS = (
"model=nova-2"
"&language=en-US"
"&encoding=linear16"
"&channels=1"
"&interim_results=true"
"&endpointing=500" # ms of silence to finalize a turn
"&utterance_end_ms=1000" # fire UtteranceEnd event after this much extra silence
)
# -- Anthropic (Claude) for the Generator ------------------------------------
ANTHROPIC_API_KEY = os.getenv("ANTHROPIC_API_KEY", "")
# Using claude-3-5-haiku for speed (latency-critical path)
CLAUDE_MODEL = "claude-haiku-4-5"
# Create async client (lazy β won't fail if key is missing until actually called)
anthropic_client = anthropic.AsyncAnthropic(api_key=ANTHROPIC_API_KEY) if ANTHROPIC_API_KEY else None
# -- ElevenLabs (TTS) β Step 3 ---------------------------------------------------
ELEVENLABS_API_KEY = os.getenv("ELEVENLABS_API_KEY", "")
# Default demo voice: "Rachel" β clear, natural-sounding English. Replace with cloned voice later.
ELEVENLABS_VOICE_ID = os.getenv("ELEVENLABS_VOICE_ID", "21m00Tcm4TlvDq8ikWAM")
# Flash v2.5 = lowest-latency model (~75ms). Use eleven_turbo_v2_5 or _multilingual for quality.
ELEVENLABS_MODEL = "eleven_turbo_v2_5" # better expression than flash; still fast
# Default neutral voice settings β Emotion layer overrides these per-sentence
DEFAULT_VOICE_SETTINGS = {
"stability": 0.5,
"similarity_boost": 0.8,
"style": 0.3,
"use_speaker_boost": True,
"speed": 0.92, # slightly slower than 1.0 = more natural, less rushed
}
# Hand-tuned per-emotion settings for a CLONED voice. Cloned voices need higher
# stability + similarity_boost than stock voices, or they warble. style stays
# moderate so emotion comes through without sounding over-acted.
# stability β = more variation in pitch/rhythm (more expressive)
# similarity_boost β = stay closer to the cloned voice identity
# style β = lean harder into the speaker's stylistic quirks
EMOTION_PROFILES = {
"neutral": {"stability": 0.55, "similarity_boost": 0.80, "style": 0.25, "speed": 0.92},
"warm": {"stability": 0.45, "similarity_boost": 0.80, "style": 0.40, "speed": 0.92},
"happy": {"stability": 0.40, "similarity_boost": 0.80, "style": 0.50, "speed": 0.95},
"excited": {"stability": 0.35, "similarity_boost": 0.80, "style": 0.55, "speed": 0.98},
"playful": {"stability": 0.40, "similarity_boost": 0.80, "style": 0.50, "speed": 0.95},
"thoughtful": {"stability": 0.60, "similarity_boost": 0.80, "style": 0.30, "speed": 0.88},
"tender": {"stability": 0.55, "similarity_boost": 0.82, "style": 0.40, "speed": 0.88},
"tired": {"stability": 0.65, "similarity_boost": 0.82, "style": 0.30, "speed": 0.85},
"sad": {"stability": 0.55, "similarity_boost": 0.82, "style": 0.45, "speed": 0.86},
"anxious": {"stability": 0.40, "similarity_boost": 0.80, "style": 0.45, "speed": 0.94},
"frustrated": {"stability": 0.55, "similarity_boost": 0.85, "style": 0.35, "speed": 0.88},
"firm": {"stability": 0.60, "similarity_boost": 0.85, "style": 0.28, "speed": 0.87},
"apologetic": {"stability": 0.55, "similarity_boost": 0.82, "style": 0.40, "speed": 0.88},
}
# -- Memory agent (Redis) --------------------------------------------------------
REDIS_URL = os.getenv("REDIS_URL", "")
redis_client: Optional[redis_async.Redis] = None
if REDIS_URL:
try:
redis_client = redis_async.from_url(REDIS_URL, decode_responses=True)
except Exception as e:
log.error(f"Redis init failed: {e}")
redis_client = None
class MemoryAgent:
"""
Stores conversation history per session in Redis.
Each session has a list of turns: {role: 'partner'|'user', text, ts}.
The Generator queries recent history to ground candidates further.
"""
def __init__(self, client: Optional[redis_async.Redis]):
self.client = client
def _key(self, session_id: str) -> str:
return f"cadence:session:{session_id}:turns"
async def save_turn(self, session_id: str, role: str, text: str):
if not self.client or not session_id or not text.strip():
return
entry = json.dumps({"role": role, "text": text.strip(), "ts": int(time.time())})
try:
await self.client.rpush(self._key(session_id), entry)
await self.client.expire(self._key(session_id), 60 * 60 * 24) # 24h TTL
except Exception as e:
log.warning(f"Memory save failed: {e}")
async def get_history(self, session_id: str, limit: int = 20) -> list[dict]:
if not self.client or not session_id:
return []
[truncated β 34982 more characters]
```
### frontend/postcss.config.js
```javascript
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
}
```
### frontend/vite.config.js
```javascript
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
export default defineConfig({
plugins: [react()],
server: {
port: 5173,
},
})
```
### frontend/tailwind.config.js
```javascript
/** @type {import('tailwindcss').Config} */
export default {
content: [
'./index.html',
'./src/**/*.{js,jsx,ts,tsx}',
],
theme: {
extend: {},
},
plugins: [],
}
```
### frontend/index.html
```html
Cadence
```
### frontend/public/audio-processor.js
```javascript
/**
* AudioWorklet processor β runs on the audio thread.
* Accumulates Float32 mic samples into 4096-sample chunks,
* converts to Int16 PCM, and posts the raw ArrayBuffer to the main thread.
* We send the browser's native sample rate to Deepgram (no downsampling needed).
*/
class AudioProcessor extends AudioWorkletProcessor {
constructor() {
super()
this._buf = new Float32Array(4096)
this._idx = 0
}
process(inputs) {
const channel = inputs[0]?.[0]
if (!channel) return true
for (let i = 0; i < channel.length; i++) {
this._buf[this._idx++] = channel[i]
if (this._idx >= this._buf.length) {
const pcm16 = new Int16Array(this._buf.length)
for (let j = 0; j < this._buf.length; j++) {
const s = Math.max(-1, Math.min(1, this._buf[j]))
pcm16[j] = s < 0 ? s * 32768 : s * 32767
}
// Transfer ownership of the buffer β zero-copy send
this.port.postMessage(pcm16.buffer, [pcm16.buffer])
this._idx = 0
}
}
return true
}
}
registerProcessor('audio-processor', AudioProcessor)
```
### frontend/src/symbols.js
```javascript
/*
* Word β symbol mapping for tiles in symbolMode (autistic + aphasia profiles).
*
* Why emoji and not bespoke PECS pictograms? PECS imagery is proprietary, and
* Unicode emoji render universally on every device without bundling assets.
* If/when we partner with a symbol set provider (ARASAAC is open-source AAC
* imagery), swap this for an asset URL map.
*
* Matching is normalized (lowercase, trimmed, single-spaced). Multi-word
* concepts get checked first, then individual words.
*/
const MAP = {
// Core acknowledgements
'yes': 'β
', 'no': 'β', 'maybe': 'π€', 'okay': 'π', 'ok': 'π',
"i don't know": 'π€·', 'not sure': 'π€·', 'not really': 'π
',
// Politeness
'thank you': 'π', 'thanks': 'π', 'please': 'π', 'sorry': 'π',
"you're welcome": 'π',
// Needs
'help': 'π', 'water': 'π§', 'food': 'π½οΈ', 'hungry': 'π½οΈ', 'thirsty': 'π§',
'bathroom': 'π»', 'toilet': 'π»', 'medicine': 'π', 'rest': 'ποΈ', 'sleep': 'π',
'more': 'β', 'stop': 'π', 'wait': 'βΈοΈ', 'go': 'βΆοΈ',
// Feelings
'happy': 'π', 'sad': 'π’', 'tired': 'π΄', 'sleepy': 'π΄',
'angry': 'π ', 'frustrated': 'π€', 'scared': 'π¨', 'anxious': 'π',
'excited': 'π', 'calm': 'π', 'pain': 'π£', 'hurt': 'π€',
'love': 'β€οΈ', 'good': 'π', 'bad': 'π', 'fine': 'π',
'feeling better': 'π', 'not great': 'π',
// People & places
'family': 'π¨βπ©βπ§', 'mom': 'π©', 'dad': 'π¨', 'sister': 'π§', 'brother': 'π¦',
'doctor': 'π©Ί', 'nurse': 'π©Ί', 'friend': 'π«', 'home': 'π ', 'school': 'π«',
'work': 'πΌ', 'phone': 'π±',
// Time
'now': 'β°', 'later': 'β³', 'today': 'π
', 'tomorrow': 'π', 'yesterday': 'π',
'in a bit': 'β³', 'soon': 'β³', 'morning': 'π
', 'night': 'π',
// Common words
'with you': 'π«', 'with me': 'π«', 'alone': 'πΆ', 'together': 'π€',
'i': 'π€', 'you': 'π', 'we': 'π«', 'us': 'π«',
'eat': 'π½οΈ', 'drink': 'π₯€', 'play': 'π²', 'read': 'π', 'watch': 'π',
'walk': 'πΆ', 'sit': 'πͺ', 'stand': 'π§',
}
const normalize = (s) => s.toLowerCase().trim().replace(/\s+/g, ' ')
export function symbolFor(text) {
if (!text) return null
const key = normalize(text)
if (MAP[key]) return MAP[key]
// Try first significant word as fallback
const words = key.split(' ').filter(w => w.length > 1)
for (const w of words) {
if (MAP[w]) return MAP[w]
}
return null
}
```
[5 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]