# Project export: Lucid Voice This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: Lucid Voice gives people who can't speak their voice back: a few taps become a full sentence in their own voice, shaped to who they're talking to, and fully private on-device. - Devpost: https://devpost.com/software/lucid-voice - GitHub: https://github.com/dbhargav-uw/Lucid-Voice - Video: https://www.youtube.com/embed/3OAKS2ppySo?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Result: winner (GRAND PRIZE - Ddoski's World Track; Finalist) - Team: 3 GitHub contributor(s) — Claude Opus 4.8 (38 commits), Dhruv Bhargava (28 commits), Dhruv Bhargava (14 commits) ## Devpost submission (written by the team) ### Inspiration An estimated five million Americans and nearly a hundred million people worldwide cannot rely on their own voice, from ALS, stroke, autism, or cerebral palsy. The cruelest part of conditions like aphasia is that the person knows exactly what they want to say but cannot get the words out, and today's tools are slow and robotic. We wanted to build something that knows the person and gives them their own voice back. ### What it does Lucid Voice turns two or three tapped words into a full, natural sentence, spoken in a clone of the person's own voice. It is not autocomplete: it understands who they are talking to, so the same words become "sweetie" for a daughter and "mijo" for a grandson, and it learns their style the more they use it. Behind it is a personal "brain," a living model of their world that you can watch think, learn, and grow, and the whole thing runs on-device and offline so their private words never leave their hands. ### How we built it Lucid Voice is local-first. A personal knowledge graph (Kuzu) holds the person's people, routines, and preferences; a hybrid retrieval pipeline traverses that graph and uses a submodular optimization to pick the most relevant facts, which steer a frozen local model (Gemma via MLX on Apple Silicon) to write the candidates. Two learning systems sit on top: the graph reinforces associations the way memory does, and a style model learns the user's phrasing from which suggestions they pick, using the same preference formulation behind RLHF. Voice cloning (XTTS) and speech recognition (Whisper) run on-device too, with Redis for local vector search, Deepgram, Sentry, and Claude Code in the stack, and nothing on the core path touches the internet. ### Challenges we ran into On-device voice cloning sent us down a brutal dependency chain through torchaudio, torchcodec, and FFmpeg. Responses took sixty seconds until we found the model was burning thousands of reasoning tokens we then discarded, and turning that off made it near-instant. We also fixed a race that resurrected stale suggestions against the wrong line of conversation. The hardest part was conceptual: making the system truly understand the person rather than guess, and solving the empty-graph cold-start, which led to the mode where the assistant interviews the user to build their brain. ### Accomplishments we're proud of We built a complete real-time AI system, graph reasoning, generation, voice cloning, and speech recognition, that runs entirely on a laptop with the internet off, which is rare in a year when capable AI usually means a data center, and it keeps a vulnerable person's words private. We are proud that the context-awareness works, that the same input yields the right different sentence per person, and that the learning is real and visible on screen. And we kept the person in control by always proposing and letting them choose, never speaking for them. ### What we learned We went deep on running large models locally and fast, hybrid graph-and-vector retrieval, optimization, and preference learning. The bigger lesson was that sophistication only matters if people can see it, so we put the reasoning, learning, and growth on screen, and that preserving the user's agency and dignity matters more than any clever trick. ### What's next Richer automatic context like identifying who is speaking from their voice, more input methods such as eye-gaze and switch access for later-stage ALS, and real testing with the AAC and disability community and clinicians. Longer term, an opt-in cloud quality tier, deeper personalization as the brain grows, and getting it into the hands of the millions who could use it. ## README (from the GitHub repository)
Lucid Voice ### Speak again — in your own voice. **A local-first AAC app that turns 2–3 tapped words into a full, situationally‑correct sentence — spoken in a cloned version of the user's own voice.**
The Lucid Voice memory graph, firing The user's personal memory graph, rendered as a living brain. The exact memories that grounded a reply light up in real time.
--- ## What it is People who can't speak fluently — after a stroke (aphasia), with ALS, or autism — communicate on tap‑boards at roughly **8 words per minute**. Ordinary speech is about **150**. Lucid Voice closes that gap. The person taps a couple of word‑fragments; an AI **reconstructs several full, register‑correct candidate sentences** grounded in their personal life‑context; they **select one**, and it's spoken aloud in a clone of their own voice. Everything that matters runs **on‑device**, so it keeps working in airplane mode. > **AAC** = Augmentative & Alternative Communication. ## The thesis — and the moat > **The same minimal input produces a *different, correctly‑registered* reply depending on _who is listening_.** That's the whole idea, and it's the part a generic LLM prompt can't reliably do from two words: | She taps | Partner | Lucid Voice speaks | |---|---|---| | `tired · maybe` | **Sofia** (daughter) | *"I'd love to, **sweetie**, but I've been so tired lately. Can I tell you Saturday?"* | | `tired · maybe` | **Mateo** (grandson, 4) | *"I'm a little tired right now, **mijo**. Maybe after my nap? I love you."* | Identical input. The wording, the term of endearment, even the promise of a nap — all pulled from a **personal knowledge graph**, not invented. The moat is the **personal‑memory layer + on‑device privacy + the user's own preserved voice**, not the text generation. ## How it works ``` ┌── Tap ──────────┐ ┌── Ground ─────────────┐ ┌── Reconstruct ───────┐ ┌── Speak ──────────┐ │ 2–3 vocab tiles │ │ Hybrid GraphRAG over │ │ Claude writes 3 │ │ You select one → │ │ (or the partner's│ → │ your personal graph + │ → │ register-correct │ → │ it plays in your │ │ line via STT) │ │ partner detection │ │ candidates, grounded │ │ cloned voice — and │ │ │ │ → abstain if unsure │ │ in your facts │ │ the graph learns │ └──────────────────┘ └────────────────────────┘ └───────────────────────┘ └─────────────────────┘ ``` 1. **Tap** word‑fragments on a calm vocabulary board, or capture what the partner said with the mic (speech‑to‑text). Composition is tiles‑first — the assisted user never *has* to type (an optional type‑to‑add field is there if they want it). 2. **Ground** — `retrieval.py` runs a hybrid **GraphRAG**: anchor the fragments to graph nodes, detect *who* is being spoken to, expand the neighbourhood, blend in vector search, re‑rank, and select a low‑redundancy fact set. If confidence is too low it **abstains** ("add one more word") rather than guessing. 3. **Reconstruct** — `generation.py` asks the LLM for exactly three candidates at different lengths/registers, grounded in the retrieved facts. A learned **personal style model** re‑ranks them (server‑side) to sound like *you*, and a **tone dial** (warm / even / direct / playful) lets you nudge which register floats to the top. 4. **Speak** — selecting a candidate plays it aloud (cloned‑audio‑first, with an on‑device system‑voice fallback so it's never silent), fires `/confirm`, and **reinforces the memory graph** so it gets better over time. ## Features - 🧠 **Personal Knowledge Graph (PKG)** — an embedded [Kuzu](https://kuzudb.com/) graph of the people, places, routines, preferences and phrases that make replies sound like you. - 🎭 **Register that fits the listener** — partner detection drives the term of address ("sweetie" / "mijo" / "mi amor") and tone. - 🗣️ **Your own voice** — zero‑shot voice cloning (Coqui XTTS‑v2), cache‑first so confirmed lines replay instantly and offline. Playback is cloned‑audio‑first; until a voice is enrolled it gracefully uses the best on‑device system voice. - 🌀 **Show your work** — a 3D **hologram‑brain** view (`/graph`) where the retrieved memories *fire* the path that grounded each reply. Plus **"Build your brain,"** where a warm AI interviews you and each answer **blooms** a new memory onto the graph, live. - 📈 **Online learning loop** — every confirmation reinforces the graph; a consolidation pass promotes recurring patterns into durable preferences; and a scheduled decay pass (`run_decay`) fades what goes unused. - ✈️ **On‑device & airplane‑mode capable** — local LLM, STT, TTS, embeddings and graph. Cloud providers are strictly opt‑in. ## Architecture ``` Frontend (Vite · React · TypeScript · Tailwind · three.js) :5173 ├── Conversation / the unified turn-loop (capture → tap → generate → select → speak) └── Graph /graph the 3D hologram brain + "Build your brain" │ fetch /api/* (Vite proxies /api → :8000) Backend (FastAPI · Python) :8000 ├── retrieval.py hybrid GraphRAG (anchor → expand → vector → rerank → submodular → abstain) ├── generation.py LLM candidate generation (strict JSON, repair retry, fallback) ├── style.py per-user learned communication-style model ├── graph.py Personal Knowledge Graph (Kuzu) ├── learning.py online learning (/confirm reinforce · /consolidate · decay) ├── cache.py cache-first /speak (pre-rendered demo audio) └── providers/ local-first LLM · embedding · STT · TTS (cloud opt-in) ``` Every backend service is built **lazily and degrades gracefully** — the app boots and serves end‑to‑end even with no LLM / graph / TTS installed (returning correctly‑shaped placeholders), which is what makes it demo‑safe. ### Tech stack | Layer | Default (on‑device) | Opt‑in cloud | |---|---|---| | **LLM** | LM Studio (OpenAI‑compatible, `:1234`) | Anthropic **Claude** | | **Embeddings** | sentence‑transformers (`bge‑small`) | — | | **Speech‑to‑text** | faster‑whisper | **Deepgram** | | **Voice clone / TTS** | Coqui **XTTS‑v2** | **ElevenLabs** | | **Graph DB** | Kuzu (embedded) | — | Cloud providers are reached **only** when you set their provider env var and supply an API key; with zero keys, everything runs locally. ### API `/health` · `/generate` · `/speak` · `/confirm` · `/assistant_turn` · `/consolidate` · `/style/{person_id}` · `/stt` · `/enroll` · `/graph/{person_id}` · `/trace/latest` ## Quickstart ```bash cd aac ./run.sh ``` `run.sh` creates/reuses a virtualenv, installs backend + frontend deps if needed, copies `.env.example` → `.env`, and starts both servers together. - **Frontend:** http://localhost:5173 - **Backend API:** http://localhost:8000 ### Prerequisites - **Python 3.10+** and **Node 18+** - **FFmpeg** — required by local voice synthesis (`brew install ffmpeg`). Without it, `/speak` still serves cached audio and degrades gracefully. - **LM Studio** serving any instruct model on `:1234` — *only* needed for live (non‑`DEMO_MODE`) generation. > First run pulls heavy ML weights (XTTS‑v2 ≈ 1.8 GB); they cache after that. Or set `DEMO_MODE=true` in `aac/backend/.env` to skip all of it. ### Manual run ```bash # Backend cd aac/backend && python3 -m venv .venv && source .venv/bin/activate pip install -r requirements.txt && cp .env.example .env uvicorn app.main:app --reload --port 8000 # Frontend (second terminal) cd aac/frontend && npm install && npm run dev ``` ### Giving someone a cloned voice ```bash cd aac/backend python -m data.enroll_voice elena path/to/elena.wav # a clean ~10s reference python -m data.prerender_demo elena # pre-render demo lines into the cache ``` ## Project structure ``` aac/ ├── backend/ │ ├── app/ │ │ ├── main.py [README truncated for size] ## Detected evidence (automated analysis) Indexed codebase: 60 recognized source files, 456 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Redis (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - PyTorch (technology) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers ## Codebase structure (from repository index) ### Files (72 of 72) ``` .claude/launch.json .gitattributes .gitignore aac/.gitignore aac/backend/.env.example aac/backend/app/__init__.py aac/backend/app/config.py aac/backend/app/main.py aac/backend/app/models.py aac/backend/app/observability.py aac/backend/app/providers/__init__.py aac/backend/app/providers/embedding.py aac/backend/app/providers/llm.py aac/backend/app/providers/stt.py aac/backend/app/providers/tts.py aac/backend/app/services/__init__.py aac/backend/app/services/cache.py aac/backend/app/services/generation.py aac/backend/app/services/graph.py aac/backend/app/services/learning.py aac/backend/app/services/redis_store.py aac/backend/app/services/retrieval.py aac/backend/app/services/style.py aac/backend/app/tracing.py aac/backend/data/demo_fixtures.json aac/backend/data/enroll_voice.py aac/backend/data/prerender_demo.py aac/backend/data/run_decay.py aac/backend/data/seed_graph.py aac/backend/reports/grounding_eval.jsonl aac/backend/reports/grounding_eval.md aac/backend/requirements.txt aac/backend/scripts/eval_grounding.py aac/backend/scripts/phoenix_grounding_proof.py aac/backend/scripts/verify_deepgram.py aac/backend/scripts/verify_redis_e2e.py aac/backend/scripts/verify_redis.py aac/backend/scripts/verify_sentry.py aac/frontend/DESIGN.md aac/frontend/index.html aac/frontend/package.json aac/frontend/postcss.config.js aac/frontend/PRODUCT.md aac/frontend/src/App.tsx aac/frontend/src/assets/voice-pulse.json aac/frontend/src/components/BuildBrainPanel.tsx aac/frontend/src/components/CandidateCard.tsx aac/frontend/src/components/ConstructionStrip.tsx aac/frontend/src/components/ForceGraph.tsx aac/frontend/src/components/HologramBrain.tsx aac/frontend/src/components/NextFragments.tsx aac/frontend/src/components/QuickPhrases.tsx aac/frontend/src/components/StateIndicator.tsx aac/frontend/src/components/ToneDial.tsx aac/frontend/src/components/VocabBoard.tsx aac/frontend/src/components/Waveform.tsx aac/frontend/src/hooks/useSpeak.ts aac/frontend/src/index.css aac/frontend/src/lib/api.ts aac/frontend/src/lib/demo.ts aac/frontend/src/lib/motion.ts aac/frontend/src/main.tsx aac/frontend/src/views/ConversationView.tsx aac/frontend/src/views/GraphView.tsx aac/frontend/src/vite-env.d.ts aac/frontend/tailwind.config.js aac/frontend/tsconfig.json aac/frontend/tsconfig.node.json aac/frontend/vite.config.ts aac/README.md aac/run.sh README.md ``` ### Dependencies - aac/backend/requirements.txt: anthropic@==0.42.0, arize-phoenix@==7.9.0, arize-phoenix-otel@==0.7.0, coqui-tts@==0.27.5, fastapi@==0.115.6, faster-whisper@==1.1.1, httpx@==0.28.1, kuzu@==0.7.1, numpy@==2.2.1, openinference-semantic-conventions@==0.1.12, opentelemetry-exporter-otlp-proto-http, opentelemetry-sdk, pydantic@==2.10.4, pydantic-settings@==2.7.1, python-dotenv@==1.0.1, redis@==5.2.1, sentence-transformers@==3.3.1, sentry-sdk[fastapi]@==2.20.0, torchaudio@==2.11.0, torchcodec@==0.14.0, uvicorn[standard]@==0.34.0 - aac/frontend/package.json: @phosphor-icons/react@^2.1.10, @types/react@^18.3.5, @types/react-dom@^18.3.0, @types/three@^0.169.0, @vitejs/plugin-react@^4.3.1, autoprefixer@^10.4.20, framer-motion@^11.5.4, lottie-react@^2.4.1, postcss@^8.4.47, react@^18.3.1, react-dom@^18.3.1, react-force-graph-2d@^1.25.5, react-router-dom@^6.26.2, tailwindcss@^3.4.12, three@^0.169.0, typescript@^5.5.4, vite@^5.4.6 ### Recent commits (newest first) - Merge pull request #2 from dbhargav-uw/feat/backend-sponsor-integrations - feat(backend): add Deepgram, Sentry, Redis Stack, and Arize Phoenix integrations - README: add logo + brain hero, drop the demo walkthrough - Remove the presentation/ pitch deck from the project - Brand: swap header + favicon to the head-profile logo - Add project README, brand assets, and pitch/demo deck - Conversation: give "Listen to partner" more room from the right edge - Build Your Brain: use assistant's question-specific Next suggestions - Build Your Brain: assistant_turn returns question-specific answer words - Conversation: nudge "Listen to partner" status off the card edge - Build Your Brain: frame the question into the answer generation - Build Your Brain: transcript, input-overflow fix, better next words, equal cards - Build Your Brain: match reconstruction + legend card sizes - Build Your Brain: dock the reconstruction overlay bottom-right, compact - Build Your Brain: live Reconstruction stats overlay on the brain - Build Your Brain: side-by-side layout, predictive words, typed input - Build Your Brain frontend: live graph growth in the Graph tab - Build Your Brain backend: /assistant_turn + /confirm new_nodes/new_edges - Predictive next-words: 8 options, combine all matching intents - Disable Gemma-4 thinking via reasoning_effort to cut /generate latency ## Key source files (fetched from GitHub, selected and truncated for size) ### aac/frontend/PRODUCT.md ```markdown # Lucid Voice — PRODUCT.md **Register:** product (design serves the task; the tool should disappear, with one crafted moment). ## What it is A local-first AAC (augmentative & alternative communication) web app. A person who can't speak fluently taps 2–3 word fragments; on-device AI proposes a few complete, context-correct sentences; the person **explicitly selects one** (choose-one / reject-two) and only then is it spoken aloud in their **own cloned voice**. Never auto-speaks. ## Primary user & scene **Elena, 67**, expressive aphasia after a stroke. Uses it on a tablet at home and in conversation. Low energy, word-finding difficulty. Needs **large targets, high legibility, calm pacing, zero surprise.** A care partner or (for the hackathon demo) judges watch a second "reasoning" surface that makes the AI's choice legible. ## The one moment **Fragments → a chosen full sentence → spoken in the person's voice.** Everything else is quiet, trustworthy console; this single transition is where craft and delight concentrate. ## Surfaces - **Speak** (hero): vocab tiles → construction strip → "Suggest replies" → candidate sentences → select → speak. Integrated **reasoning rail** (heard context, what taps signal, profile facts, confidence, grounded memories). - **Graph** (Phase 5, teammate-owned): live PKG visualization with retrieval highlighting. - **Conversation**: partner transcript / dictation (secondary). ## Constraints - Accessibility is non-negotiable: WCAG AA+ contrast, visible focus, full keyboard path, `prefers-reduced-motion` honored, color never the only signal. - Local-first / offline; live backend with deterministic demo fallback. Never auto-speak. ## Design direction (see DESIGN.md for tokens) Dark "ink" console. **Warm = the human** (their words, voice, primary actions); **cool = the machine** (reasoning, confidence). Calm and familiar (Linear/Raycast-grade trust), with the single signature speak moment. Type: one sans superfamily for UI + data, one reading serif for the spoken human utterance only. Motion conveys state, never decorates. ``` ### aac/frontend/DESIGN.md ```markdown # Lucid Voice — Design System (Speaker view) > Design lead: **🖌️ Gigi**. Implemented by **🎨 Fiona** in `frontend/`. > Direction chosen by the CEO: **bold & premium showpiece**, accessible for a user with > aphasia, impressive for hackathon judges. Surface for this build: **the Speaker view** > (the full AAC loop) with an integrated **reasoning rail**. Data: **live backend + demo fallback**. ## Thesis — "The Console of a Returning Voice" A few tapped fragments become a full sentence, spoken in the person's own voice. The UI dramatizes exactly that transformation and makes the AI's reasoning legible. Every choice encodes the product's core duality: - **Warm amber = the human** — their words, their cloned voice, the primary actions. - **Cool aqua = the machine's mind** — the reasoning trail, confidence, the "thinking" state. This two‑tone semantic (not decoration) is the spine of the whole design. ## Color tokens (dark "ink" canvas) Deep, warm indigo‑black — premium and calm, never clinical, never pure black. | Token | Hex | Use | |---|---|---| | `ink` | `#14121C` | app canvas | | `ink-raised` | `#1E1B29` | cards, strip, rail surfaces | | `ink-sunken` | `#100E16` | wells (vocab board, transcript) | | `ink-line` | `#2C2838` | hairline borders/dividers | | `text` | `#F4EFE9` | primary text (warm off‑white, "paper") | | `text-muted` | `#A39DB0` | secondary text/labels (lavender‑grey) | | `text-faint` | `#6E6880` | placeholders, disabled | | **`voice`** | `#FF9E5E` | THE HUMAN — primary CTAs, speak, chosen utterance glow | | `voice-deep` | `#F0743E` | voice gradient end / pressed | | `voice-soft` | `#3A2A24` | voice tint on dark (selected card bg) | | **`mind`** | `#5FE3D2` | THE MACHINE — reasoning rail accents, confidence, "thinking" | | `mind-soft` | `#1B2E30` | mind tint on dark (rail panel bg) | **Register triad** (tone of a candidate sentence — always paired with a text label, never color‑only): | Register | Hex | Feel | |---|---|---| | `warm` | `#FFB778` | amber — affectionate | | `neutral` | `#B9A8FF` | soft violet — even | | `direct` | `#6FE3D2` | aqua — concise | Contrast: `text` on `ink` ≈ 14:1; dark text (`ink`) on `voice` ≈ 8:1. All AA+. ## Typography — a tri‑face system (each face means something) Load via Google Fonts (no npm deps). `` in `index.html`. - **Fraunces** (`opsz`, weights 400/500/600, soft optical) → **the human utterance**: candidate sentences, the live "now speaking" line. Literary, warm, emotional. `font-utter`. - **Bricolage Grotesque** (400/500/600/700) → **the interface**: headings, tiles, labels, body. `font-ui`. - **IBM Plex Mono** (400/500) → **the machine**: reasoning rail, confidence %, trace, eyebrows/kbd. `font-mono`. ### Type scale | Role | Family | Size (clamp) | Weight | Notes | |---|---|---|---|---| | Stage utterance (speaking) | Fraunces | `clamp(2rem, 4.5vw, 3.25rem)` | 500 | line-height 1.15 | | Candidate sentence | Fraunces | `clamp(1.5rem, 2.4vw, 2rem)` | 500 | line-height 1.25 | [truncated — 10508 more characters] ``` ### aac/frontend/package.json ``` { "name": "lucid-voice-frontend", "private": true, "version": "0.1.0", "type": "module", "scripts": { "dev": "vite", "build": "tsc && vite build", "preview": "vite preview" }, "dependencies": { "@phosphor-icons/react": "^2.1.10", "framer-motion": "^11.5.4", "lottie-react": "^2.4.1", "react": "^18.3.1", "react-dom": "^18.3.1", "react-force-graph-2d": "^1.25.5", "react-router-dom": "^6.26.2", "three": "^0.169.0" }, "devDependencies": { "@types/react": "^18.3.5", "@types/react-dom": "^18.3.0", "@types/three": "^0.169.0", "@vitejs/plugin-react": "^4.3.1", "autoprefixer": "^10.4.20", "postcss": "^8.4.47", "tailwindcss": "^3.4.12", "typescript": "^5.5.4", "vite": "^5.4.6" } } ``` ### aac/backend/requirements.txt ``` # Lucid Voice — backend dependencies # # Web / core (lightweight; safe to import at module top-level): fastapi==0.115.6 uvicorn[standard]==0.34.0 pydantic==2.10.4 pydantic-settings==2.7.1 python-dotenv==1.0.1 httpx==0.28.1 numpy==2.2.1 # --------------------------------------------------------------------------- # Heavy / optional local-ML dependencies. # These pull in large native wheels and model frameworks (torch, etc.) and can # take significant time/disk to install. They are LAZY-IMPORTED inside the # functions that need them, so the app imports and runs even when these are # absent (e.g. for `python -m py_compile` or DEMO_MODE). # --------------------------------------------------------------------------- kuzu==0.7.1 # embedded graph database sentence-transformers==3.3.1 # local embeddings faster-whisper==1.1.1 # local STT # Local TTS / XTTS-v2 voice clone. Use the MAINTAINED idiap fork "coqui-tts" # (NOT the archived "TTS" package). torchaudio + torchcodec provide audio IO, # and torchcodec needs FFmpeg installed on the system (e.g. `brew install ffmpeg` # on macOS). XTTS runs on CPU on Apple Silicon (MPS is unreliable for XTTS). coqui-tts==0.27.5 # Coqui XTTS-v2 voice clone (idiap fork) torchaudio==2.11.0 # audio IO for coqui-tts torchcodec==0.14.0 # audio decode/encode (requires system FFmpeg) anthropic==0.42.0 # optional cloud LLM provider (opt-in via env) # --------------------------------------------------------------------------- # Sponsor integrations. All OPT-IN and lazy-imported/guarded, so the app still # imports and runs fully offline (airplane mode) when these are absent or the # integration is disabled via env. Install to demo the respective feature. # --------------------------------------------------------------------------- sentry-sdk[fastapi]==2.20.0 # Sentry error monitoring (no-op without SENTRY_DSN) redis==5.2.1 # Redis Stack client (vector KNN + cache + memory) # Arize Phoenix — LOCAL LLM tracing + eval (opt-in via PHOENIX_ENABLED). Pulls in # OpenTelemetry + OpenInference. Also needs a system Redis STACK (RediSearch) for # the Redis vector index, installed separately (see .env.example). arize-phoenix==7.9.0 arize-phoenix-otel==0.7.0 openinference-semantic-conventions==0.1.12 opentelemetry-sdk opentelemetry-exporter-otlp-proto-http ``` ### aac/frontend/src/main.tsx ```typescript import React from "react"; import ReactDOM from "react-dom/client"; import { BrowserRouter } from "react-router-dom"; import App from "./App"; import "./index.css"; ReactDOM.createRoot(document.getElementById("root") as HTMLElement).render( ); ``` ### aac/frontend/src/App.tsx ```typescript import { NavLink, Navigate, Route, Routes, useLocation } from "react-router-dom"; import { AnimatePresence, MotionConfig, motion } from "framer-motion"; import { CaretDown } from "@phosphor-icons/react"; import ConversationView from "./views/ConversationView"; import GraphView from "./views/GraphView"; import { DUR, EASE_OUT } from "./lib/motion"; import logoMark from "./assets/logo-mark.png"; const NAV_ITEMS = [ { to: "/", label: "Conversation", end: true }, { to: "/graph", label: "Graph", end: false }, ]; function TopBar() { return (
{/* Brand — the Lucid Voice head mark (assets/logo-mark.png) + wordmark. */}
Lucid Voice
{/* Person switcher (hidden on the narrowest screens to keep one row). */} {/* Quiet pill nav. */} {/* Honest status chip — non-interactive (hidden below lg). */}
on-device · airplane-ok
); } export default function App() { const location = useLocation(); return ( // reducedMotion="user" makes every Framer Motion component honor the OS // "reduce motion" setting (disables transform/layout animation, keeps // opacity) — covers the candidate bloom + selection choreography that CSS // alone can't reach.
{/* Light-touch route transitions. */} } /> } /> } />
); } ``` ### aac/backend/app/main.py ```python """FastAPI application entrypoint for Lucid Voice. This is the Phase 1 SCAFFOLD. Every endpoint returns a correctly-SHAPED placeholder response so the app is runnable end-to-end. Real logic lands in later phases (see BUILD ORDER). Heavy/optional providers and services are imported and instantiated lazily so the app starts even when on-device dependencies (kuzu, sentence-transformers, TTS/coqui, faster-whisper) or optional cloud SDKs are missing. """ from __future__ import annotations import logging import time from fastapi import FastAPI from fastapi.middleware.cors import CORSMiddleware from app.config import settings from app.observability import init_sentry, add_breadcrumb, set_tag, span from app.tracing import init_tracing from app.models import ( AssistantTurnRequest, AssistantTurnResponse, ConfirmRequest, ConfirmResponse, ConsolidateRequest, ConsolidateResponse, EnrollRequest, EnrollResponse, GenerateRequest, GenerateResponse, GraphResponse, HealthResponse, SpeakRequest, SpeakResponse, STTRequest, STTResponse, StyleProfile, Candidate, RetrievalInfo, ) logger = logging.getLogger("lucid_voice") # Initialize Sentry as early as possible (before the app/middleware are built) so # the FastAPI/Starlette integrations capture every request. No-op without a DSN. init_sentry() # Initialize Phoenix/OpenInference LLM tracing. No-op unless PHOENIX_ENABLED. init_tracing() app = FastAPI(title="Lucid Voice") # CORS for the Vite dev server. app.add_middleware( CORSMiddleware, allow_origins=[ "http://localhost:5173", "http://127.0.0.1:5173", ], allow_credentials=True, allow_methods=["*"], allow_headers=["*"], ) # Most recent /generate trace, served by GET /trace/latest. Kept in-process as # the fallback; mirrored to Redis (agent/session memory) when Redis is enabled. latest_trace: dict = {} def _store_trace(trace: dict) -> None: """Persist the latest trace in-process AND in Redis (when available).""" global latest_trace latest_trace = trace rs = _get("redis") if rs is not None and getattr(rs, "available", False): try: rs.set_latest_trace(trace) except Exception as exc: # pragma: no cover - defensive logger.warning("redis set_latest_trace failed: %s", exc) def _read_trace() -> dict: """Read the latest trace from Redis when available, else in-process.""" rs = _get("redis") if rs is not None and getattr(rs, "available", False): try: v = rs.get_latest_trace() if v is not None: return v except Exception as exc: # pragma: no cover - defensive logger.warning("redis get_latest_trace failed: %s", exc) return latest_trace # --- Lazy service / provider wiring ----------------------------------------- # # Services and providers are built lazily so a missing heavy dependency never # crashes import. Each helper caches its result in `_services` and degrades to # None (with a warning) on failure; endpoints stay shaped regardless. _singletons: dict = {} def _get(name: str): """Lazily construct and cache a provider/service with its dependencies. Returns the instance, or None if construction fails (logged). Endpoints degrade to correctly-shaped placeholders in the None case, so a missing heavy dependency never crashes the API. """ if name in _singletons: return _singletons[name] inst = None try: if name == "graph": from app.services.graph import GraphService inst = GraphService() inst.connect() # opens/creates Kuzu + schema elif name == "embedding": from app.providers import get_embedding_provider inst = get_embedding_provider() elif name == "llm": from app.providers import get_llm_provider inst = get_llm_provider() elif name == "redis": from app.services.redis_store import get_redis_store inst = get_redis_store() elif name == "retrieval": from app.services.retrieval import RetrievalService inst = RetrievalService( _get("graph"), _get("llm"), _get("embedding"), redis_store=_get("redis") ) elif name == "generation": from app.services.generation import GenerationService inst = GenerationService(_get("llm")) elif name == "learning": from app.services.learning import LearningService inst = LearningService(_get("graph"), _get("llm"), _get("embedding")) elif name == "cache": from app.services.cache import CacheService inst = CacheService() elif name == "tts": from app.providers import get_tts_provider inst = get_tts_provider() elif name == "stt": from app.providers import get_stt_provider inst = get_stt_provider() elif name == "style": from app.services.style import StyleService inst = StyleService(_get("graph")) except Exception as exc: # pragma: no cover - defensive guard logger.warning("Could not initialize %r: %s", name, exc) inst = None _singletons[name] = inst return inst def get_retrieval_service(): return _get("retrieval") def get_generation_service(): return _get("generation") def get_graph_service(): return _get("graph") def get_learning_service(): return _get("learning") def get_cache_service(): return _get("cache") def _provider_status() -> dict: """Best-effort report of the configured providers for /health. Reflects the env-selected provider names without instantiating heavy providers, so this stays cheap and import-safe. """ status = { "llm": settings.llm_provider, "embedding": settings.embedding_pro [truncated — 15137 more characters] ``` ### aac/run.sh ```shell #!/usr/bin/env bash set -euo pipefail # Lucid Voice — local dev launcher. # Starts the FastAPI backend and the Vite frontend together. # Run from anywhere; paths are resolved relative to this script. SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" BACKEND_DIR="$SCRIPT_DIR/backend" FRONTEND_DIR="$SCRIPT_DIR/frontend" VENV_DIR="$BACKEND_DIR/.venv" # --- Prerequisite checks ----------------------------------------------------- # The ML/web deps require Python >= 3.10. macOS often ships an older stock # python3 (e.g. 3.9), so probe known names and pick the first >= 3.10. PYTHON="" for cand in python3.13 python3.12 python3.11 python3.10 python3; do if command -v "$cand" >/dev/null 2>&1; then if "$cand" -c 'import sys; sys.exit(0 if sys.version_info[:2] >= (3, 10) else 1)' 2>/dev/null; then PYTHON="$cand" break fi fi done if [ -z "$PYTHON" ]; then echo "ERROR: no Python >= 3.10 found. Install Python 3.11+ and try again." >&2 echo " (found: $(python3 --version 2>&1 || echo 'none'))" >&2 exit 1 fi echo "==> Using $PYTHON ($("$PYTHON" --version 2>&1))" if ! command -v node >/dev/null 2>&1; then echo "ERROR: node not found. Install Node 18+ and try again." >&2 exit 1 fi if ! command -v npm >/dev/null 2>&1; then echo "ERROR: npm not found. Install Node 18+ (which provides npm) and try again." >&2 exit 1 fi # FFmpeg is required by Coqui XTTS-v2 (torchcodec) for local voice synthesis. # Not fatal: /speak still serves cached audio and falls back gracefully without it. if ! command -v ffmpeg >/dev/null 2>&1; then echo "WARNING: ffmpeg not found — local XTTS voice synthesis will be unavailable" >&2 echo " (cached + ElevenLabs paths still work). Install: brew install ffmpeg" >&2 fi # --- Backend: virtualenv ----------------------------------------------------- if [ ! -d "$VENV_DIR" ]; then echo "==> Creating Python virtualenv at backend/.venv" "$PYTHON" -m venv "$VENV_DIR" else echo "==> Reusing existing virtualenv at backend/.venv" fi # shellcheck disable=SC1091 source "$VENV_DIR/bin/activate" # --- Backend: dependencies --------------------------------------------------- REQUIREMENTS="$BACKEND_DIR/requirements.txt" STAMP="$VENV_DIR/.requirements.stamp" if [ ! -f "$STAMP" ] || [ "$REQUIREMENTS" -nt "$STAMP" ]; then echo "==> Installing backend dependencies (this can take a while; ML deps are heavy)" pip install --upgrade pip >/dev/null pip install -r "$REQUIREMENTS" touch "$STAMP" else echo "==> Backend dependencies up to date; skipping install" fi # --- Backend: .env ----------------------------------------------------------- if [ ! -f "$BACKEND_DIR/.env" ]; then echo "==> Creating backend/.env from .env.example" cp "$BACKEND_DIR/.env.example" "$BACKEND_DIR/.env" else echo "==> backend/.env already exists; leaving it untouched" fi # --- Launch backend in the background ---------------------------------------- echo "==> Starting backend on http://localhost:8000" ( cd "$BACKEND_DIR" exec uvicorn app.main:app --reload --port 8000 ) & BACKEND_PID=$! # Kill the backend when this script exits for any reason. cleanup() { echo "" echo "==> Shutting down backend (pid $BACKEND_PID)" kill "$BACKEND_PID" >/dev/null 2>&1 || true wait "$BACKEND_PID" 2>/dev/null || true } trap cleanup EXIT INT TERM # --- Frontend: dependencies -------------------------------------------------- if [ ! -d "$FRONTEND_DIR/node_modules" ]; then echo "==> Installing frontend dependencies" (cd "$FRONTEND_DIR" && npm install) else echo "==> Frontend dependencies present; skipping npm install" fi # --- Launch frontend in the foreground --------------------------------------- echo "==> Starting frontend on http://localhost:5173" echo " (Ctrl-C to stop both servers)" (cd "$FRONTEND_DIR" && npm run dev) ``` ### aac/frontend/postcss.config.js ```javascript export default { plugins: { tailwindcss: {}, autoprefixer: {}, }, }; ``` ### aac/frontend/vite.config.ts ```typescript import { defineConfig } from "vite"; import react from "@vitejs/plugin-react"; // https://vitejs.dev/config/ export default defineConfig({ plugins: [react()], server: { port: 5173, proxy: { // Frontend calls "/api/..." which is forwarded to the FastAPI backend. "/api": { target: "http://localhost:8000", changeOrigin: true, rewrite: (path) => path.replace(/^\/api/, ""), }, }, }, }); ``` [50 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]