# Project export: Séance This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: Point your phone at any object and it wakes up as a character you can have a live voice conversation with. - Devpost: https://devpost.com/software/seance-gpa2xy - GitHub: https://github.com/hethb/Seance - Video: https://www.youtube.com/embed/LGDYokNOhnc?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 5 GitHub contributor(s) — Claude Opus 4.8 (1M context) (42 commits), Nandini Tiwari (27 commits), gandhiishan777 (26 commits), aishanisingh (24 commits), Heth Bhatt (17 commits) ## Devpost submission (written by the team) ### Overview Point your phone at any object and it wakes up as a character with its own voice and personality. Argue with your water bottle. Get sass from a stapler. Have a real conversation with the junk on your desk. Problem & ### Inspiration People talk to objects all the time. They yell at printers, thank vending machines, and apologize to chairs they walk into. We kept coming back to the obvious question: what if the object answered back? The honest truth is that talking to your stuff sounds like a gimmick until you actually try it. The moment you point a phone at a water bottle and it complains, in its own voice, that you left it in a hot car, something clicks. It's not a chatbot in a box. It's this object, with this history, reacting to you. The same trick works on a stapler, a mug, or a backpack, and every one of them becomes a different character with a different grievance. We built Séance around two rules we refused to break. It only awakens objects, never people, because an app that invents personalities should never put words in a real person's mouth. And it stays comedic rather than sentimental, because a grumpy stapler is funnier and more honest than a supportive one. Those two rules are what make the demo land instead of feeling uncomfortable. What It Does Séance turns any object in front of you into a character you can hold a live, spoken conversation with, and it remembers that character the next time you see it. For the person holding the phone, the flow is five steps and takes seconds. You capture a photo of an object. A short awaken sequence plays while the character is generated. The object is revealed as a character with a generated portrait, a name, a one-line tagline, and the first thing it says when it notices you. You talk to it out loud, in a voice picked to match its personality. And when you scan the same object again, it remembers: same character, same grievances, same conversation right where you left off. For the world it builds, two features give it depth beyond a single joke. The Ledger is a gallery of every object you've awakened, each showing its portrait, name, and last line; tapping one reopens that conversation. Encounters let you awaken one object and introduce it to a second one, so two characters can talk to each other while you watch. How We Built It Séance is four stages chained together: see the object, write the character, paint it, and give it a voice. Each stage is one sponsor's technology, and each lives in its own file so it can be swapped or mocked on its own. Getting all four to run live on a phone, in a few seconds, on venue Wi-Fi, is the part we're most proud of. Anthropic Claude Opus 4.8 is the brain that sees and writes. We send Claude the captured photo, and it identifies the object and authors the entire character. We force structured output through a single tool, emit_persona, so Claude has to return a complete object every time (name, tagline, backstory, opening line, voice, an in-character system prompt, and an image prompt) and can never fall back to loose prose or a refusal. Claude assigns each object one of 30 comedic archetypes (grumpy elder, dramatic diva, deadpan stoic, anxious overachiever) and writes to that archetype, which made the comedy far funnier and more consistent than fully freeform personas. Getting that constrained, validated generation right is something we're really proud of. During the conversation the generated system prompt keeps Claude in voice, and we send only the most recent turns so replies stay fast as the chat grows. Deepgram gives every object a real voice. The conversation runs on Deepgram's Voice Agent, which handles speech-to-text, the model turn, and text-to-speech in one real-time stream with barge-in, so you can interrupt an object mid-sentence and it stops to listen. Each character picks its own Deepgram Aura voice as part of its persona, so a self-important object booms and an anxious one goes soft. The phone never holds the API key: the backend mints short-lived scoped Deepgram tokens through the auth grant endpoint, and the client connects with those. A REST path (Nova-3 transcription, a Claude reply, Aura speech) backs up the live agent so the object can always talk, even when the real-time connection drops. The portrait is what turns an object into a character on screen. Claude writes an image prompt for each persona, and we render it as an expressive portrait that always shows up no matter what. If Claude was unsure about the object we generate a Pollinations "mystery creature" instead, and as a final fallback we re-skinned the captured photo into a stylized character portrait. The point was reliability over polish: the reveal never lands on a broken image. Redis is what makes objects feel like recurring characters instead of one-off jokes. Each object is a JSON session stored under a normalized key (seance:object:) with a 7-day expiry, so the same red stapler always maps to the same character. A separate Redis hash indexes every object as a short summary, and that index powers the Ledger and the resume-conversation flow. Every Redis call is time-boxed with a short connect timeout, no infinite reconnect, and a per-command deadline, so an unreachable cache fails over to an in-process store instead of hanging. That's exactly what kept the app responsive when the venue network turned hostile. The client is an Expo and React Native app that handles camera capture and the full Capture → Awaken → Reveal → Conversation → Ledger flow. It talks to a TypeScript and Express backend that orchestrates the four stages and reports on startup which integrations are live versus mocked. Challenges & Accomplishments Four sequential API calls add up, and latency was our first enemy. We cut it down by sending Claude only recent turns, time-boxing every external call, and using the "waking up" animation to absorb the wait the user would otherwise feel. Failing gracefully was the second challenge: any stage can fail in front of a judge, so each one has a fallback (a playable persona, a fallback portrait, text when speech fails, and an in-memory store when Redis is down). The real-time Deepgram integration needs a native module Expo Go can't load, which pushed us into a custom dev build and a long detour through Xcode signing, developer mode, and a managed-device restriction. And the venue network nearly beat us: our laptop IP changed three times and Redis behaved differently on each one, so we tethered to a single phone hotspot for a stable IP and made the client surface a real error instead of spinning forever. What we're proudest of is that the whole thing never hard-fails. A complete multi-model pipeline (vision, structured writing, image generation, and real-time voice) runs live on a phone and reads in a few seconds with no explanation. With no API keys it runs fully mocked, and each stage switches to live the moment its key is added. What We Learned Forcing structured output through a tool is dramatically more reliable than parsing JSON out of prose. Constraining Claude to a fixed set of archetypes improved both the comedy and the consistency, which was the opposite of what we expected going in. Designing every single stage to fail softly is the only reason a live demo on bad Wi-Fi was possible at all. And we relearned that most of the distance between a working simulator and a working phone app isn't the AI. It's signing, native modules, and networking. Brainstorming & Process Séance didn't start as a finished idea. It started from a behavior we kept noticing: people already talk to their stuff, so the question was never "will anyone talk to an object," it was "what makes the object worth talking back." That framing drove every decision that followed, and most of them changed shape at least once before they landed. The biggest pivot was how the character gets written. Our first version let Claude invent personas completely freeform, and the results were fine but mushy: every object drifted toward the same friendly, agreeable voice, and the jokes rarely had an edge. We rewrote it around a fixed set of 30 comedic archetypes (grumpy elder, dramatic diva, deadpan stoic, anxious overachiever) and made Claude commit to one and write to it. Constraining the model made the comedy sharper and far more consistent, which was the opposite of what we assumed going in. That result is what convinced us archetypes were worth keeping. The second decision was reliability as a design principle, not an afterthought. We made an early call that no stage was allowed to hard-fail in front of a judge, and we built backward from that: structured output forced through a single emit_persona tool so Claude can never return loose prose or a refusal, a portrait that always renders even when the object is unclear, a REST voice path behind the real-time one, and an in-process store behind Redis. Each of those fallbacks exists because we asked "what does the user see when this specific thing breaks" before we wrote the happy path. The venue network failing repeatedly during the event was the test we didn't ask for, and the fallbacks are the reason the demo survived it. We also deliberately kept the four stages decoupled, each in its own file with its own mock, so we could build, test, and swap them independently instead of wiring one giant pipeline and hoping it held together. That structure is why we could iterate on the persona prompt without touching voice, and tune voice without touching memory. Ethical Considerations Because Séance invents personalities and gives them a voice, we treated the boundaries as a core part of the design rather than a disclaimer bolted on at the end. It awakens objects, never people. This is enforced in the generation step, not just promised in the pitch. Claude is instructed to set objectRecognized to false whenever a frame is dominated by a person rather than a thing, and when that happens the app falls back to a generic "mystery" character instead of inventing a personality for a real human. An app that puts words in a real person's mouth is a different and more dangerous product, and we drew that line on purpose. It stays comedic, not emotionally manipulative. We kept every persona grumpy, theatrical, and clearly a bit, rather than warm and attached. A supportive object that wants you to keep talking to it is the kind of thing that quietly encourages parasocial dependency, and we steered away from it deliberately. The humor is the safeguard: nobody mistakes a sarcastic stapler for a friend. Photos are ephemeral. A captured image is sent to Claude for vision analysis in a single request and is never written to disk or stored in Redis. We persist the generated persona and a portrait URL, not the user's photo. The system is built to remember the character, not to keep a record of where someone was or what their camera saw. Data minimization and key safety. Object sessions are stored with a 7-day expiry rather than kept forever, so the memory that makes the demo charming is also short-lived by default. The phone never holds an API key: the backend mints short-lived, scoped Deepgram tokens that expire in an hour, so a captured client can't leak long-lived credentials. Compute awareness. Personas and portraits are generated once per object and reused on every later scan, and conversations send only the most recent turns instead of the full transcript. Those choices started as latency fixes, but they also cut redundant model calls and the energy that comes with them, which matters when the whole experience is four AI models chained together. What's Next Animated portraits that move while the character talks. Multi-object scenes so a whole group of objects can argue at once. Shared memory so the same kind of object remembers topics across different people, using the Redis Agent Memory Server. And a growing collection of the funniest awakened objects, because the best ones deserve to be saved. Built at HackBerkeley AI Hackathon 2026, Ddoski's Playground track. ## README (from the GitHub repository) # 🔮 Séance **Point your phone at *any* object and it wakes up as a character you can have a live voice conversation with.** Built at Cal AI Hacks 2026. --- ## The pitch > Séance is a spirit medium for objects. Point the camera at a stapler, a mug, a backpack — Claude looks at it, invents the larger-than-life character secretly living inside, paints its portrait, gives it a voice, and lets you talk to it out loud. It *remembers you* the next time you point the camera at the same thing. Bring two objects together and they meet each other — their relationship is saved too. --- ## How it works ``` 📷 camera frame │ ▼ 🧠 Claude (vision) forced tool-use → exactly 3 ranked Persona objects │ (name, archetype, voice model, opening line, portrait prompt) ▼ 🎨 portrait gen paints the character portrait │ Gemini Imagen 3 / Adobe Firefly / Pollinations (mystery objects) ▼ 🗣️ Deepgram you speak → STT; Claude replies → TTS in the character's own voice │ ▼ 🧠 Claude (chat) replies in character, in persona, 1-2 sentences per turn │ ▼ 💾 Redis object remembers you across sessions (keyed by objectKey) pair dynamic memory for two-object encounters ``` --- ## Features - **30 archetypes** — grumpy elder, dramatic diva, conspiracy theorist, mob boss, and 26 more. Claude picks the best fit for the object and commits hard to its voice. - **Persona picker** — Claude ranks 3 personas by fit on the reveal screen; user can swap before starting the conversation. The chosen persona is saved without resetting history. - **Object memory** — scan the same object tomorrow and it picks up where it left off. Keyed by a normalized `objectKey` in Redis with a 7-day TTL. - **Two-object encounters** — bring two awakened objects together, set a dynamic ("rivals", "old friends", etc.), and Claude generates a 6-line scripted scene between them in both characters' voices. The pair dynamic is saved so their relationship evolves. - **AliveAvatar** — the portrait moves. Archetype-specific idle/talking motion, blinking eyes, drifting pupils, eyebrow raises, and lip sync — all procedural React Native `Animated` with the native driver. Zero design assets, zero native deps. - **Awaken progress** — atmospheric log lines tick during the API call so a 30s response never feels broken. --- ## Stack | Layer | Tech | |---|---| | Mobile client | React Native (Expo), Expo Router | | API server | Node.js / Express (TypeScript) | | AI — persona + chat | Claude API (Anthropic) — `claude-opus-4-8` / `claude-sonnet-4-6` | | Speech | Deepgram STT + TTS (Aura-2 voice models) | | Portrait generation | Gemini Imagen 3 (`GEMINI_API_KEY`) · Adobe Firefly · Pollinations.ai (fallback) | | Memory | Redis (7-day TTL) · in-process Map fallback for local dev | --- ## Quickstart ```bash npm install cp .env.example .env # works empty — every key is optional npm run dev # → API on http://localhost:3000 ``` The API server runs immediately. The phone client is the Expo app in `mobile/` — run it with `cd mobile && npx expo start`. > **Runs with zero API keys.** With no `.env`, persona is a canned fallback and portrait is the captured photo. Add keys one at a time — the server logs which capabilities are live on boot. ### Environment variables ``` ANTHROPIC_API_KEY real personalities and conversation DEEPGRAM_API_KEY spoken voice in and out IMAGE_PROVIDER gemini | firefly | mock (default: mock) GEMINI_API_KEY required when IMAGE_PROVIDER=gemini GEMINI_IMAGE_MODEL default: gemini-2.5-flash-image ADOBE_FIREFLY_CLIENT_ID required when IMAGE_PROVIDER=firefly ADOBE_FIREFLY_CLIENT_SECRET REDIS_URL persistent memory across restarts PORT default: 3000 ``` --- ## Project layout ``` src/ server.ts Express API — all endpoints config.ts env reading + capability flags types.ts Persona / Turn / SessionState / Archetype lib/ claude.ts awakenAll() · reply() · generateEncounter() · archetypeCatalog() deepgram.ts transcribe() · speak() imagegen.ts paintPortrait() · generateMysteryPortrait() memory.ts loadState() · saveState() · loadPairDynamic() · savePairDynamic() history.ts recordSession() · listSessions() · getSession() mobile/ app/ index.tsx capture screen — camera / library picker awaken.tsx loading screen — progress log lines while API processes reveal.tsx persona reveal — auto-TTS opening line, persona picker conversation.tsx voice conversation screen — hold-to-talk, chat transcript encounter.tsx two-object scene — scripted exchange with replay + exit CTAs src/ api.ts typed fetch wrappers for all endpoints sessionStore.ts module-level handoff for large data (image, awaken/encounter results) components/ AliveAvatar.tsx procedural portrait animation + face overlay hooks/ useConverse.ts voice session state machine scripts/ test-ux-fixes.ts UX fix test suite (32 assertions) test-persona-picker.ts test-encounter.ts ``` --- ## API reference | Method | Path | Body | Returns | |---|---|---|---| | `POST` | `/api/awaken` | `{ image, objectKey? }` | `{ persona, personas[], portraitUrl, encounters, returning, history }` | | `POST` | `/api/select-persona` | `{ objectKey, persona }` | `{ ok }` | | `POST` | `/api/converse` | multipart: `objectKey`, `audio?`, `text?` | `{ userText, replyText, audio? }` | | `POST` | `/api/encounter` | `{ objectKey1, objectKey2, dynamic? }` | `{ lines, relationship, persona1, persona2, portraitUrl1, portraitUrl2 }` | | `POST` | `/api/tts` | `{ text, voiceModel? }` | `{ audio }` | | `POST` | `/api/turns` | `{ objectKey, turns[] }` | `{ ok }` | | `GET` | `/api/history` | — | `{ sessions[] }` | | `GET` | `/api/history/:objectKey` | — | `{ persona, portraitUrl, history, encounters }` | | `GET` | `/api/archetypes` | — | `{ archetypes[] }` | | `GET` | `/api/status` | — | capability flags | --- ## Running tests ```bash npm run test:ux-fixes # 32 assertions across all UX fixes npm run test:persona-picker # persona picker + select-persona endpoint npm run test:encounter # encounter + pair dynamic memory ``` --- Go talk to your stapler. ## Detected evidence (automated analysis) Indexed codebase: 95 recognized source files, 452 KB. - Anthropic (technology) — detected in the code - Express (technology) — detected in the code - JavaScript (language) — detected in the code - React (technology) — detected in the code - Redis (technology) — detected in the code - TypeScript (language) — detected in the code - Node.js (technology) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: commit authorship or trailers ## Codebase structure (from repository index) ### Files (120 of 122) ``` .agents/skills/iris-development/.cursor-plugin/plugin.json .agents/skills/iris-development/references/ltm-bulk-create.md .agents/skills/iris-development/references/ltm-organize.md .agents/skills/iris-development/references/ltm-search.md .agents/skills/iris-development/references/promotion-overview.md .agents/skills/iris-development/references/session-add-event.md .agents/skills/iris-development/references/session-retrieval.md .agents/skills/iris-development/references/session-when-to-use.md .agents/skills/iris-development/references/setup-auth-token.md .agents/skills/iris-development/references/setup-cloud-service.md .agents/skills/iris-development/SKILL.md .agents/skills/redis-clustering/.cursor-plugin/plugin.json .agents/skills/redis-clustering/references/hash-tags.md .agents/skills/redis-clustering/references/read-replicas.md .agents/skills/redis-clustering/SKILL.md .agents/skills/redis-connections/.cursor-plugin/plugin.json .agents/skills/redis-connections/references/blocking.md .agents/skills/redis-connections/references/client-cache.md .agents/skills/redis-connections/references/pipelining.md .agents/skills/redis-connections/references/pooling.md .agents/skills/redis-connections/references/timeouts.md .agents/skills/redis-connections/SKILL.md .agents/skills/redis-core/.cursor-plugin/plugin.json .agents/skills/redis-core/evals/core/baselines/aggregate-benchmark.json .agents/skills/redis-core/evals/core/baselines/aggregate-benchmark.md .agents/skills/redis-core/evals/core/baselines/baseline.json .agents/skills/redis-core/evals/core/baselines/model-matrix.json .agents/skills/redis-core/evals/core/baselines/README.md .agents/skills/redis-core/evals/core/evals.json .agents/skills/redis-core/evals/core/model-matrix.json .agents/skills/redis-core/references/choose-data-structure.md .agents/skills/redis-core/references/key-naming.md .agents/skills/redis-core/SKILL.md .agents/skills/redis-observability/.cursor-plugin/plugin.json .agents/skills/redis-observability/references/commands.md .agents/skills/redis-observability/references/metrics.md .agents/skills/redis-observability/SKILL.md .agents/skills/redis-query-engine/.cursor-plugin/plugin.json .agents/skills/redis-query-engine/references/dialect.md .agents/skills/redis-query-engine/references/field-types.md .agents/skills/redis-query-engine/references/index-creation.md .agents/skills/redis-query-engine/references/index-management.md .agents/skills/redis-query-engine/references/query-optimization.md .agents/skills/redis-query-engine/references/skip-initial-scan.md .agents/skills/redis-query-engine/SKILL.md .agents/skills/redis-security/.cursor-plugin/plugin.json .agents/skills/redis-security/references/acls.md .agents/skills/redis-security/references/auth.md .agents/skills/redis-security/references/network.md .agents/skills/redis-security/SKILL.md .agents/skills/redis-semantic-cache/.cursor-plugin/plugin.json .agents/skills/redis-semantic-cache/references/best-practices.md .agents/skills/redis-semantic-cache/references/langcache-usage.md .agents/skills/redis-semantic-cache/SKILL.md .agents/skills/redis-vector-search/.cursor-plugin/plugin.json .agents/skills/redis-vector-search/references/algorithm-choice.md .agents/skills/redis-vector-search/references/hybrid-search.md .agents/skills/redis-vector-search/references/index-creation.md .agents/skills/redis-vector-search/references/rag-pattern.md .agents/skills/redis-vector-search/SKILL.md .env.example .gitignore app/.gitignore app/app.json app/App.tsx app/babel.config.js app/index.ts app/package.json app/README.md app/src/api/client.ts app/src/config.ts app/src/navigation.tsx app/src/screens/AwakeningScreen.tsx app/src/screens/CaptureScreen.tsx app/src/screens/ConversationScreen.tsx app/src/screens/EncounterScreen.tsx app/src/screens/RevealScreen.tsx app/src/theme.ts app/src/types.ts app/tsconfig.json mobile/.env.example mobile/.gitignore mobile/app.config.js mobile/app/_layout.tsx mobile/app/awaken.tsx mobile/app/conversation.tsx mobile/app/encounter.tsx mobile/app/history.tsx mobile/app/index.tsx mobile/app/reveal.tsx mobile/babel.config.js mobile/package.json mobile/src/api.ts mobile/src/components/AliveAvatar.tsx mobile/src/constants.ts mobile/src/hooks/useConverse.ts mobile/src/sessionStore.ts mobile/src/theme.ts mobile/src/types.ts mobile/tsconfig.json package.json README.md scripts/prebake.ts scripts/README.md scripts/test-encounter.ts scripts/test-gemini.ts scripts/test-persona-picker.ts scripts/test-persona.ts scripts/test-ux-fixes.ts skills-lock.json src/config.ts src/lib/claude.ts src/lib/deepgram.ts src/lib/history.ts src/lib/imagegen.ts src/lib/memory.ts src/server.ts src/types.ts tasks/todo.md test/claude.test.ts [2 more files omitted for size] ``` ### Dependencies - app/package.json: @react-navigation/native@^7.0.14, @react-navigation/native-stack@^7.2.0, @types/react@~19.1.0, babel-preset-expo@~54.0.10, expo@^54.0.35, expo-asset@~12.0.13, expo-av@~16.0.8, expo-camera@~17.0.10, expo-constants@~18.0.13, expo-image-manipulator@~14.0.8, expo-image-picker@~17.0.11, expo-status-bar@~3.0.9, react@19.1.0, react-native@0.81.5, react-native-safe-area-context@~5.6.0, react-native-screens@~4.16.0, typescript@^5.3.3 - mobile/package.json: @babel/core@^7.25.2, @expo-google-fonts/dm-mono@^0.4.2, @expo-google-fonts/instrument-serif@^0.4.1, @types/react@~18.3.12, expo@~52.0.0, expo-asset@~11.0.5, expo-av@~15.0.2, expo-blur@~14.0.3, expo-camera@~16.0.0, expo-font@~13.0.4, expo-image-picker@~16.0.6, expo-linear-gradient@~14.0.2, expo-linking@~7.0.5, expo-router@~4.0.0, expo-splash-screen@~0.29.24, expo-status-bar@~2.0.1, react@18.3.1, react-native@0.76.5, react-native-safe-area-context@4.12.0, react-native-screens@~4.4.0, react-native-svg@15.8.0, typescript@~5.3.3 - package.json: @anthropic-ai/sdk@^0.70.0, @types/express@^5.0.0, @types/multer@^2.0.0, @types/node@^24.0.0, dotenv@^17.0.0, express@^5.0.0, multer@^2.0.0, redis@^5.0.0, tsx@^4.19.0, typescript@^5.6.0 ### Recent commits (newest first) - docs: rewrite README to reflect shipped project - fix(encounter): normalize speaker by line index so dialogue alternates - Merge pull request #20 from hethb/feature/avatar-eyes - feat(avatar): show the eyes + mouth on reveal, thinking, and encounter screens - Merge pull request #19 from hethb/feature/avatar-eyes - feat(avatar): bigger chat avatar with moving pupils and eyebrows - feat(avatar): give the chat-screen avatar blinking eyes and talking lips - Merge pull request #18 from hethb/feature/alive-avatar - Merge origin/main into feature/alive-avatar - feat(converse): cap character replies at 1-2 sentences - Merge pull request #17 from hethb/chore/ui-theme-cleanup - chore(mobile): unify screens on the design-system tokens - chore: stop tracking DEVPOST.md (keep local only) - chore(mobile): apply expo-cli generated tsconfig/gitignore sync - docs: add Devpost writeup - fix(awaken): restore object recognition and halve persona latency - Merge pull request #16 from hethb/feature/ux-fixes - feat(mobile): bring the awakened object to life with archetype motion - feat(awaken): keep progress lines ticking during long API calls - fix(ux): resolve 5 post-ship issues — 26/26 tests passing ## Key source files (fetched from GitHub, selected and truncated for size) ### tasks/todo.md ```markdown # Task — object-matched voices + stage-direction tone Branch: `feature/voices-and-tone` ## Decisions (from user) - Stage directions `*...*`: **hide completely** — strip from BOTH speech (TTS) and the on-screen transcript. Claude is prompted to produce them (shapes expressive delivery), then they're removed. - Voices: **deterministic + varied** — stable per-object hash into archetype-matched aura-2 pools. No dependence on Claude's pick; never collapses to one default. ## Plan - [x] `claude.ts`: archetype→voice pools (validated aura-2 ids) + `pickVoice(archetype, objectKey)` deterministic hash; export it. - [x] `claude.ts` `reply()`: append a delivery note so replies sound natural and include <=1 short `*stage direction*`. - [x] `deepgram.ts`: `stripStageDirections()` + apply inside `speak()` (never vocalize `*...*`; skip TTS if nothing left). - [x] `server.ts` `/api/awaken`: after objectKey is finalized, set `persona.voiceModel = pickVoice(...)`. - [x] `mobile/app/conversation.tsx`: strip `*...*` in `ChatBubble` render, fall back to raw if a turn is only a direction. - [x] Verify: 38 aura-2 voices validated; typecheck clean; live test passed. ## Review - Voice variety proven: two objects of the same archetype (dramatic_diva) → `athena` vs `cordelia`. Deterministic + stable per objectKey. - Stage directions: raw reply `*gasps dramatically* Who am I? Darling...` → spoken/shown as `Who am I? Darling...`; audio (118KB) synthesized from the cleaned text. No asterisks vocalized. - History keeps RAW replies (Claude sees its own style); display + TTS strip at the edges. - Returning objects reuse `prior.persona`, so an established object's voice never changes. ``` ### .agents/skills/redis-core/SKILL.md ```markdown --- name: redis-core description: Core Redis modeling guidance — choose the right data structure (String, Hash, List, Set, Sorted Set, JSON, Stream, Vector Set) and use consistent colon-separated key names. Use when designing a Redis data model, caching objects, deciding between Hash and JSON, building counters, leaderboards, membership sets, or session stores, or when reviewing/cleaning up Redis key naming. license: MIT metadata: author: Redis, Inc. version: "0.1.0" --- # Redis Core Foundational guidance for modeling data in Redis. Covers data-type selection and key-name conventions — the two decisions that most directly drive memory, performance, and maintainability. ## When to apply - Caching objects, sessions, or per-user state. - Counters, leaderboards, recent-items lists, unique-membership sets. - Reviewing or refactoring Redis key names. - Deciding between a Redis Hash and a JSON document for an entity. ## 1. Choose the right data structure Pick the type that matches the *access pattern*, not just the shape of the data. | Use case | Recommended type | Why | |---|---|---| | Simple values, counters | String | Atomic `INCR`/`DECR`, `SET`/`GET` | | Object with independently updated fields | Hash | Per-field reads/writes, no whole-object rewrite | | Queue, recent-N items | List | O(1) push/pop at ends | | Unique items, membership checks | Set | O(1) `SADD`/`SISMEMBER`/`SCARD` | | Rankings, score-based ranges | Sorted Set | Score-ordered; `ZADD`/`ZRANGE`/`ZRANK` | | Nested / hierarchical data | JSON | Path-level updates, nested arrays, RQE indexing | | Event log, fan-out messaging | Stream | Persistent, consumer groups | | Vector similarity | Vector Set | Native vector storage with HNSW | **Common anti-pattern:** stuffing a flat object into a serialized string. Updating one field means fetch + parse + mutate + rewrite. Use a Hash instead. See [references/choose-data-structure.md](references/choose-data-structure.md) for full rationale and Python/Java examples. ## 2. Use consistent key names Use `colon-separated` segments with a stable hierarchy: ``` {entity}:{id}:{attribute} user:1001:profile user:1001:settings order:2024:items session:abc123 article:987:likes game:space-invaders:leaderboard ``` Rules of thumb: - **Lowercase, colon-separated.** No spaces, no mixed casing (`User_1001_Profile` is bad). - **Keep keys short but readable** — keys live in memory and appear in every command. - **Don't use full URLs or long strings as keys.** Extract a short identifier, or use a hash digest of the URL. - **Prefix for multi-tenancy** (`tenant:42:user:7:cart`) so scans and ACLs can target a tenant cleanly. - **Be consistent.** Pick one convention per service and apply it across all keys. See [references/key-naming.md](references/key-naming.md) for cleanup examples and edge cases. ## References - [Redis: Choosing the right data type](https://redis.io/docs/latest/develop/data-types/compare-data-types/) - [Redis: Keys](https://redis.io/d [truncated — 34 more characters] ``` ### package.json ``` { "name": "seance", "version": "0.1.0", "private": true, "type": "module", "description": "Point your camera at any object and it wakes up as a character you can have a live voice conversation with.", "scripts": { "dev": "tsx watch src/server.ts", "start": "tsx src/server.ts", "typecheck": "tsc --noEmit", "test": "ANTHROPIC_API_KEY= DEEPGRAM_API_KEY= REDIS_URL= node --import tsx --test test/*.test.ts", "test:persona": "tsx scripts/test-persona.ts", "test:persona-picker": "tsx scripts/test-persona-picker.ts", "test:ux-fixes": "tsx scripts/test-ux-fixes.ts", "test:encounter": "tsx scripts/test-encounter.ts", "test:gemini": "tsx scripts/test-gemini.ts", "prebake": "tsx scripts/prebake.ts" }, "dependencies": { "@anthropic-ai/sdk": "^0.70.0", "dotenv": "^17.0.0", "express": "^5.0.0", "multer": "^2.0.0", "redis": "^5.0.0" }, "devDependencies": { "@types/express": "^5.0.0", "@types/multer": "^2.0.0", "@types/node": "^24.0.0", "tsx": "^4.19.0", "typescript": "^5.6.0" } } ``` ### app/package.json ``` { "name": "seance-app", "version": "0.1.0", "private": true, "description": "Séance mobile client — point your phone at any object and talk to the character inside it.", "main": "index.ts", "scripts": { "start": "expo start", "android": "expo run:android", "ios": "expo run:ios", "typecheck": "tsc --noEmit" }, "dependencies": { "@react-navigation/native": "^7.0.14", "@react-navigation/native-stack": "^7.2.0", "expo": "^54.0.35", "expo-asset": "~12.0.13", "expo-av": "~16.0.8", "expo-camera": "~17.0.10", "expo-constants": "~18.0.13", "expo-image-manipulator": "~14.0.8", "expo-image-picker": "~17.0.11", "expo-status-bar": "~3.0.9", "react": "19.1.0", "react-native": "0.81.5", "react-native-safe-area-context": "~5.6.0", "react-native-screens": "~4.16.0" }, "devDependencies": { "@types/react": "~19.1.0", "babel-preset-expo": "~54.0.10", "typescript": "^5.3.3" } } ``` ### mobile/package.json ``` { "name": "seance-mobile", "version": "0.1.0", "main": "expo-router/entry", "scripts": { "start": "expo start", "ios": "expo run:ios", "android": "expo run:android", "prebuild": "expo prebuild" }, "dependencies": { "@expo-google-fonts/dm-mono": "^0.4.2", "@expo-google-fonts/instrument-serif": "^0.4.1", "expo": "~52.0.0", "expo-asset": "~11.0.5", "expo-av": "~15.0.2", "expo-blur": "~14.0.3", "expo-camera": "~16.0.0", "expo-font": "~13.0.4", "expo-image-picker": "~16.0.6", "expo-linear-gradient": "~14.0.2", "expo-linking": "~7.0.5", "expo-router": "~4.0.0", "expo-splash-screen": "~0.29.24", "expo-status-bar": "~2.0.1", "react": "18.3.1", "react-native": "0.76.5", "react-native-safe-area-context": "4.12.0", "react-native-screens": "~4.4.0", "react-native-svg": "15.8.0" }, "devDependencies": { "@babel/core": "^7.25.2", "@types/react": "~18.3.12", "typescript": "~5.3.3" } } ``` ### app/index.ts ```typescript import { registerRootComponent } from "expo"; import App from "./App"; // Expo's entry point — registers the root component for both Expo Go and builds. registerRootComponent(App); ``` ### app/App.tsx ```typescript import { NavigationContainer, DefaultTheme } from "@react-navigation/native"; import { StatusBar } from "expo-status-bar"; import { SafeAreaProvider } from "react-native-safe-area-context"; import { RootNavigator } from "./src/navigation"; import { colors } from "./src/theme"; // Dark séance theme so there's no white flash between screens. const navTheme = { ...DefaultTheme, dark: true, colors: { ...DefaultTheme.colors, background: colors.bg, card: colors.bg, text: colors.text, primary: colors.accent, border: colors.border, }, }; export default function App() { return ( ); } ``` ### src/server.ts ```typescript import express from "express"; import multer from "multer"; import { config, caps, logCapabilities } from "./config.js"; import { awaken, awakenAll, reply, generateEncounter, archetypeCatalog, type ImageInput } from "./lib/claude.js"; import { paintPortrait, generateMysteryPortrait } from "./lib/imagegen.js"; import { transcribe, speak } from "./lib/deepgram.js"; import { loadState, saveState, loadPairDynamic, savePairDynamic } from "./lib/memory.js"; import { recordSession, listSessions, getSession } from "./lib/history.js"; import type { SessionState } from "./types.js"; // API only — the client is the Expo phone app in app/. No static web frontend. /** Normalize objectKey so Redis lookups are stable regardless of Claude's exact casing/spacing. */ function normalizeKey(raw: string): string { return raw.trim().toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-|-$/g, ""); } const app = express(); const upload = multer({ storage: multer.memoryStorage(), limits: { fileSize: 25 * 1024 * 1024 } }); app.use(express.json({ limit: "15mb" })); /** * GET /api/archetypes * The personality catalog (key + label + description) for the UI picker. */ app.get("/api/archetypes", (_req, res) => { res.json({ archetypes: archetypeCatalog() }); }); /** * GET /api/history * The "past chats" gallery — every remembered object, most recent first. */ app.get("/api/history", async (_req, res) => { try { res.json({ sessions: await listSessions() }); } catch (err) { console.error("history list failed:", err); res.status(500).json({ error: String(err) }); } }); /** * GET /api/history/:objectKey * Reopen a past chat: the persona + portrait + full transcript to revisit. */ app.get("/api/history/:objectKey", async (req, res) => { try { const state = await getSession(req.params.objectKey); if (!state) return res.status(404).json({ error: "That memory has faded — awaken it again." }); res.json({ persona: state.persona, portraitUrl: state.portraitUrl, encounters: state.encounters, history: state.history, }); } catch (err) { console.error("history fetch failed:", err); res.status(500).json({ error: String(err) }); } }); /** * POST /api/awaken * Body: { image: "data:image/jpeg;base64,..." | "https://...", archetype?: string } * Pipeline: photo → Claude invents persona → paint portrait → load/save memory. * Omit `archetype` to get Claude's recommendation; pass one of /api/archetypes * to force the personality the user picked. * Returns the persona + portrait + how many times this object has been met. */ app.post("/api/awaken", async (req, res) => { try { const image: string = req.body.image; let input: ImageInput; if (image?.startsWith("data:")) { const [, mediaType = "image/jpeg", base64 = ""] = image.match(/^data:([^;]+);base64,(.+)$/) ?? []; input = { base64, mediaType }; } else if (/^https?:\/\//.test(image ?? "")) { input = { url: image }; } else { return res.status(400).json({ error: "Expected { image: dataURL | https URL }" }); } // 1. Channel 3 ranked personas from the photo. const personas = await awakenAll(input); for (const p of personas) { p.objectKey = normalizeKey(p.objectKey); } // Optional override: pin a stable objectKey so the same rehearsed object // reliably "remembers you" across scans. if (typeof req.body.objectKey === "string" && req.body.objectKey.trim()) { const key = normalizeKey(req.body.objectKey); for (const p of personas) p.objectKey = key; } // 2. Has this object been awakened before? // awakenAll always returns ≥1 persona (falls back to fallbackPersona on error). const primaryPersona = personas[0]!; const prior = await loadState(primaryPersona.objectKey); // 3. Paint the portrait once (skip if returning; use first persona's prompts). const portraitUrl = prior?.portraitUrl ?? (primaryPersona.objectRecognized ? await paintPortrait(primaryPersona, image) : await generateMysteryPortrait(image)); // Save the top-ranked persona as the active one. const state: SessionState = { persona: prior?.persona ?? primaryPersona, portraitUrl, history: prior?.history ?? [], encounters: (prior?.encounters ?? 0) + 1, }; await saveState(state); await recordSession(state); // index it for the "past chats" gallery res.json({ persona: state.persona, // All 3 ranked personas so the client can offer a picker. // Returning objects get only their saved persona (they already have history). personas: prior ? [state.persona] : personas, portraitUrl: state.portraitUrl, encounters: state.encounters, returning: Boolean(prior), history: state.history, }); } catch (err) { console.error("awaken failed:", err); res.status(500).json({ error: String(err) }); } }); /** * POST /api/select-persona * Body: { objectKey, persona: Persona } * Swaps the active persona for an awakened object without resetting history. * Called when the user picks one of the alt personas from the reveal screen picker. */ app.post("/api/select-persona", async (req, res) => { try { const { objectKey, persona } = req.body as { objectKey?: string; persona?: unknown }; if (!objectKey || !persona) { return res.status(400).json({ error: "objectKey and persona are required." }); } const key = normalizeKey(objectKey); const state = await loadState(key); if (!state) return res.status(404).json({ error: "Unknown object — awaken it first." }); state.persona = persona as typeof state.persona; await saveState(state); res.json({ ok: true }); } catch (err) { console.error("select-persona failed:", err); res.status(500).json({ error: String(err) }); } }); /** * POST /api/converse (multipart) * Fields: objectKey (text), audio (file, optional) [truncated — 7982 more characters] ``` ### mobile/app/_layout.tsx ```typescript import { InstrumentSerif_400Regular, InstrumentSerif_400Regular_Italic, } from '@expo-google-fonts/instrument-serif'; import { DMMono_400Regular, DMMono_500Medium, } from '@expo-google-fonts/dm-mono'; import { useFonts } from 'expo-font'; import { Stack } from 'expo-router'; import { StatusBar } from 'expo-status-bar'; import { View } from 'react-native'; import { C } from '../src/theme'; export default function RootLayout() { const [fontsLoaded] = useFonts({ InstrumentSerif_400Regular, InstrumentSerif_400Regular_Italic, DMMono_400Regular, DMMono_500Medium, }); if (!fontsLoaded) { return ; } return ( <> ); } ``` ### mobile/app/index.tsx ```typescript /** * Capture Screen — Séance * Warm cream/sepia theme. User photographs an object to awaken its spirit. */ import * as ImagePicker from 'expo-image-picker'; import { LinearGradient } from 'expo-linear-gradient'; import { router, useFocusEffect, useLocalSearchParams } from 'expo-router'; import { useCallback, useEffect, useRef, useState } from 'react'; import { sessionStore } from '../src/sessionStore'; import { fetchHistory, type HistoryItem } from '../src/api'; import { ActivityIndicator, Animated, Image, Pressable, ScrollView, StyleSheet, Text, TouchableOpacity, View, } from 'react-native'; import { SafeAreaView } from 'react-native-safe-area-context'; import { C, FONTS, R, SP } from '../src/theme'; // ── Ledger accent palette (one per card) ─────────────────────────────────────── const LEDGER_TONES = [C.tealDeep, C.amber, C.red]; // ── Corner bracket decoration ───────────────────────────────────────────────── function CornerBracket({ position, }: { position: 'tl' | 'tr' | 'bl' | 'br'; }) { const iTop = position === 'tl' || position === 'tr'; const isLeft = position === 'tl' || position === 'bl'; return ( ); } // ── Main screen ─────────────────────────────────────────────────────────────── export default function CaptureScreen() { const challengerResult = sessionStore.getChallenger(); const [photo, setPhoto] = useState(null); const [loading, setLoading] = useState(false); const [error, setError] = useState(null); const [ledger, setLedger] = useState([]); const btnScale = useRef(new Animated.Value(1)).current; const scanAnim = useRef(new Animated.Value(0)).current; const redDotOpacity = useRef(new Animated.Value(0.5)).current; // Idempotent navigation: reset every time this screen regains focus, so a // double-tap can't push two awaken screens (and two awaken() calls). const navLock = useRef(false); useFocusEffect(useCallback(() => { navLock.current = false; // Refresh the ledger whenever we return here (e.g. after awakening a new object). fetchHistory().then(setLedger).catch(() => {}); }, [])); useEffect(() => { Animated.loop( Animated.timing(scanAnim, { toValue: 1, duration: 3600, useNativeDriver: true }) ).start(); Animated.loop( Animated.sequence([ Animated.timing(redDotOpacity, { toValue: 1, duration: 650, useNativeDriver: true }), Animated.timing(redDotOpacity, { toValue: 0.5, duration: 650, useNativeDriver: true }), ]) ).start(); }, [scanAnim, redDotOpacity]); async function pickImageFromLibrary() { const { status } = await ImagePicker.requestMediaLibraryPermissionsAsync(); if (status !== 'granted') { setError('Library access denied'); return; } const result = await ImagePicker.launchImageLibraryAsync({ mediaTypes: ['images'], base64: true, quality: 0.5, }); if (!result.canceled && result.assets[0]) { const asset = result.assets[0]; if (asset.base64) { setPhoto(`data:image/jpeg;base64,${asset.base64}`); setError(null); } } } async function pickImageFromCamera() { const { status } = await ImagePicker.requestCameraPermissionsAsync(); if (status !== 'granted') { // Fall back to library await pickImageFromLibrary(); return; } const result = await ImagePicker.launchCameraAsync({ base64: true, quality: 0.5, }); if (!result.canceled && result.assets[0]) { const asset = result.assets[0]; if (asset.base64) { setPhoto(`data:image/jpeg;base64,${asset.base64}`); setError(null); } } } function onPressBtn() { Animated.sequence([ Animated.timing(btnScale, { toValue: 0.96, duration: 80, useNativeDriver: true }), Animated.timing(btnScale, { toValue: 1, duration: 80, useNativeDriver: true }), ]).start(); } function handleSummon() { if (!photo) { setError('Photograph an object first'); return; } if (navLock.current) return; // ignore double-taps navLock.current = true; onPressBtn(); setError(null); setLoading(true); sessionStore.setImage(photo); router.push('/awaken'); } return ( {/* ── Header ────────────────────────────────────── */} A SPIRIT MEDIUM FOR OBJECTS Séance {/* Separator row — or rival banner when in introduction mode */} {challengerResult ? ( ✦ INTRODUCING{" "} {challengerResult.persona.name.toUpperCase()} {" "}TO… ) : ( POI [truncated — 11546 more characters] ``` [84 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]