Project Info

Lullow

Inspiration

Tonight, somewhere, a four-year-old is lying awake in the dark — a little scared, missing a parent who is a thousand miles away on a factory floor, working a night shift, or simply too exhausted to tell one more story. So much of what we later call anxiety, insecurity, the "difficult kid," even the struggles we carry into adulthood, traces back to one quiet thing: the love, the secure attachment, and the gentle guidance a child did — or did not — receive at the very beginning. And that love has never been handed out fairly. The science is clear: Attachment theory (Bowlby & Ainsworth): a child's early bond with a responsive caregiver hardens into an attachment pattern that predicts lifelong emotional regulation and resilience. Bedtime — darkness plus separation — is when that attachment system lights up brightest. The landmark CDC–Kaiser ACE study found childhood adversity (including emotional neglect and parental absence) raises the odds of adult depression, addiction, and suicide attempts 4–12×. And the gap falls hardest on the families with the least flexibility: China's tens of millions of left-behind children (50–80% higher risk of suicidal ideation), the 52% of US families where both parents work full-time, and the 1 in 12 U.S. children who lose a parent or sibling by age 18. Being tucked in should be a right. Too often, it's a privilege. Lullow is our swing at closing that gap — from the cradle.

What it does

Lullow is a voice-first, parent-controlled bedtime companion: It sits with the child and settles the feeling. The child speaks how they feel — "I'm scared of the dark and I miss mom." Lullow answers in a warm voice, reflects the feeling gently, and turns it into a personalized, low-stimulation story that ends in a calming goodnight. It carries the parent's own love, values, and lessons into the room — "I want my daughter know that no one should touch her private part, and she should tell mom if that happens.Make a story about this" even on the nights the parent can't be there. This is the heart of Lullow. A physical mood lamp glows along with the story. Each scene's atmosphere maps to a bedtime-safe color — warm gold for calm, deep purple for fear, gold for victory — so the AI story becomes something you can feel in the room. It grows with the child, remembering characters, fears, and the lessons a parent has taught, all inside parent-set safety boundaries. Crucially, Lullow is the parent's messenger, not a replacement — the attachment, the value, and the love stay with the parent.

How we built it

Claude (Anthropic) — the reasoning layer: emotion extraction, safe story planning, personalized story writing, and per-scene mood tagging. Deepgram — voice-first: speech-to-text for the child, soft Aura text-to-speech narration. Gemini — image-first picture-book pages with a locked character reference for consistency. Pika — gentle low-motion animation of each page. Redis — family memory, story-world continuity, and semantic caching. Govee smart light (v2 API) — the physical mood lamp; the backend maps each scene's mood to a color and drives the lamp live as the story plays.

Challenges we ran into

A hidden voice deadlock. A busy state guard silently aborted the check-in right after speech-to-text — STT returned 200 but the UI froze forever. It only surfaced when we tested with a real microphone (everyone had been testing by typing). Govee's API. Our newer H612F strip wasn't supported by the legacy v1 API; we migrated to the v2 capability-based API. RGBWW strips also wash out color, so we hand-tuned a bedtime-safe palette. Live latency. Running everything for real (several sequential Claude calls plus per-scene image and TTS) stacks up — we learned exactly where the pipeline needs parallelizing. Hardware, the night of. ESP32s sold out and sockets had the wrong base — we pivoted fast. And we merged four parallel workstreams onto one backend.

Accomplishments we're proud of

A real, physical mood lamp that changes color live with the AI story's emotional arc. A genuinely child-safe design: a danger-escalation gate, no diagnosis, no secrecy, parent-in-the-loop, and full mock fallbacks. A product with a soul — it doesn't try to replace parents; it carries their value into the room.

What we learned

The hardest, most meaningful problems — childhood emotional development, the fairness of love — are worth swinging at, even in a weekend. Voice plus ambient light makes an AI feel present in a way a screen never does. Real-device testing surfaces bugs that mocks never will.

What's next

Let parents leave lessons and a goodnight in their own recorded (or cloned) voice. Parallelize and stream the pipeline for instant, gapless playback. Pilot with night-shift and migrant-worker families, and measure children's emotional-literacy growth over time.

Lullow 🌙

A gentle glow for big feelings at bedtime.

Lullow is a voice-first bedtime comfort companion for children ages 3–8. It listens to a child's nighttime feelings, turns them into a gentle personalized lullaby story, narrates it in a calming voice while a physical mood lamp follows each scene, leaves a soft picture-book keepsake to read afterwards, remembers each child's emotional growth, and stays inside parent-approved safety boundaries.

Lullow is not a therapist or a generic story generator. It is a bedtime emotional support companion built around voice, memory, safety, and calm.

Status: Integrated and verified — 173 backend tests + 40 frontend tests passing, frontend builds clean, and the full pipeline runs with zero API keys (every integration has a graceful mock fallback). Add keys to .env to go live. This branch combines four workstreams — a RAG backend, the live voice/visual pipeline, a moonlit dreamscape UI, and a physical mood lamp — into one app (see Contributors).

How to run
Architecture
Features (what's built)
Safety design
Project structure
API & environment
Demo flow
Operational notes
Contributors
Credits & assets

How to run

You need two terminals — backend (FastAPI, port 8000) and frontend (Vite, port 5173). The Vite dev server proxies /api → http://localhost:8000, so no CORS setup is needed. It runs fully with no API keys thanks to mock fallbacks; add keys to .env to go live.

Prereqs: Python 3.12, Node ≥ 18 (tested on Node 25 / npm 11).

1. Backend (terminal 1)

cd "backend"

# First time only — create venv + install deps:
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

# (Optional) add API keys to go live — the app works without this:
cp ../.env.example ../.env     # then edit ../.env

# Run the API:
.venv/bin/uvicorn app.main:app --reload --port 8000

API docs (Swagger): http://localhost:8000/docs
Live-vs-mock status: http://localhost:8000/api/status
On startup it seeds a demo parent account for login (demo_parent / lullow-demo). No demo child is seeded — every child profile is created by the family, so stories use the child's own name and world.

A .venv with all deps may already exist in backend/ — if so, skip the create/install steps and just run the last command. Note: .env is not watched by --reload; restart the server after changing keys/voice.

2. Frontend (terminal 2)

cd "frontend"
npm install        # first time only
npm run dev        # → http://localhost:5173

Open http://localhost:5173 → the moonlit profile picker. Create a child profile, then open it to enter Child Bedtime Mode (/child). The Parent Dashboard is at /parent. The frontend signs in to the demo parent account transparently, so there's no login screen for the single-family demo.

3. Tests

cd "backend"  && .venv/bin/pytest        # 173 tests, deterministic (mock mode)
cd "frontend" && npx vitest run          # 40 component/context tests
cd "frontend" && npm run build           # type-checks + production build

4. Health monitor (optional)

.venv/bin/python scripts/healthcheck.py  # end-to-end check of every integration

Architecture

Frontend (React + Vite + TS + Tailwind v4)        Backend (FastAPI, Python 3.12)
  ├── Profile picker  (route /)                     ├── Auth        → parent login + sessions
  │     moonlit dreamscape landing                  ├── Voice       → Deepgram (STT + Aura-2 TTS)
  ├── Child Bedtime Mode  (route /child)            ├── Story brain → Claude  ⇄  Fetch.ai ASI One
  │     voice/text check-in → calming audio story   │     emotion → plan → safety → story body
  │     → goodnight → silent picture-book keepsake, │     → safety eval → review trail
  │     looping lullaby BGM, "find a grown-up"      ├── RAG memory  → Redis (split app/profile DBs)
  └── Parent Dashboard  (route /parent)             │     vector store + agent memory
        profile, safety settings, story-world       ├── Visual      → image model → Pika clips
        memory, history + review trail + storybook,  └── Mood lamp   → Govee (physical, per scene)
        journal

Two design principles run through everything:

Graceful mocks. Every integration is wrapped in a client with a mock fallback and lazy SDK import, so a missing key or package never crashes the app — it degrades to deterministic mock output. GET /api/status reports which integrations are live vs. mocked.
Switchable LLM. The story brain runs on Claude (Anthropic) by default and can switch to Fetch.ai ASI One via PROMPT_PROVIDER, behind one prompt_agent facade — so the rest of the pipeline never changes.

Features (what's built)

Moonlit profile picker (landing)

A dark "dreamscape" landing — glowing animated portraits, a sleeping moon, drifting stars/mist/clouds — for choosing or creating a child profile.
Per-device roster (localStorage); each child's stories + memory live on the backend. Honors prefers-reduced-motion.

Child Bedtime Mode

Voice-first emotional check-in (hold-to-talk mic → Deepgram STT) with a text fallback and keyboard support.
Gentle, voice/tone-compliant reflection of the child's feeling.
Audio-first bedtime. The story is delivered as continuous, calming narration with a gentle female voice (Deepgram Aura-2 cora), slowed and pitch-preserved; long stories are stitched into one clip so narration never plays in disjointed segments. Eyes can stay closed — no screen to watch.
Looping lullaby BGM (soft piano) that starts on the first tap and plays at a steady level under the narration, with a mute toggle.
Physical mood lamp (Govee) follows each scene's atmosphere as the story plays, then fades off at goodnight. Safe no-op without hardware.
Silent picture-book keepsake. While the audio plays, soft illustrations (2–3 quiet pages) are painted in the background. They surface only after goodnight as an optional "📖 Read last night's storybook" — a self-paced, silent reader (no narration, no lamp). Generation never blocks or interrupts the child; the button stays soft-disabled until the art is ready.
Persistent "Find a grown-up" help button on every screen, plus an automatic warm escalation screen on danger signals.

Parent Dashboard

Child profile editor; parent safety settings (blocked topics/words, visual mode, max length, toggles).
Family memory / story-world editor (recurring character + setting + past themes), incl. the character's master reference-image thumbnail.
Story history with the full review trail (child said / memory used / safety constraints / avoided topics / parent edits / status), safety scores, and a "📖 View storybook" to read any story's illustrated pages.
Parent revise (e.g. "make softer") and approve (writes back to memory).
Growth journal (emotion counts + helpful elements + non-diagnostic reflection).

RAG backend & memory

Redis with a split keyspace (app/RAG in DB0, profiles in DB1), optional JSON compression, and an in-memory fallback when Redis is absent.
A vector store + agent memory (semantic / episodic / procedural / safety / working) used to ground each story in the child's history and constraints.

Backend pipeline emotion → load memory + parent constraints → safety/escalation gate → plan → story body → safety evaluation (regenerates once / safe fallback) → review trail → save. Visuals are a separate, background call so the audio story starts instantly and the picture-book paints itself afterwards.

Safety design

Child-safety is enforced, not just prompted:

Escalation gate on danger signals (physical harm, self-harm, intruder, abuse, alone-and-unsafe, medical). Triggered by both a keyword screen and the model's safety_flag. When triggered, no story is generated or shown — /api/story/generate returns story: null + an escalation block, and the UI shows a warm "find a trusted grown-up / press help" screen.
"Need help" is context-aware: "I need help falling asleep" → normal story; "help me, I'm scared and alone" → escalates.
Safety evaluation gates output: a failing story is regenerated once, then falls back to a guaranteed-safe story. Hard scary terms (kill/blood/monster…) hard-fail; scans use word boundaries (no "begun"→"gun" false positives).
No secrecy, no diagnosis, no therapist/AI tells — enforced in every prompt and checked in tests.
Image prompts are safety-filtered and always get bedtime-safe style modifiers.
Parent endpoints require login; lamp control is the only public bedtime endpoint.

Project structure

Lullow/
├── README.md                  ← you are here
├── API_CONTRACT.md            ← full endpoint contract (source of truth for FE/BE)
├── .env.example               ← all sponsor keys documented (copy → .env)
├── scripts/healthcheck.py     ← end-to-end integration health monitor
├── backend/
│   ├── requirements.txt  pytest.ini
│   ├── app/
│   │   ├── main.py            ← FastAPI app, CORS, /health, /status, router mounts
│   │   ├── config.py          ← settings + live-vs-mock feature_status()
│   │   ├── models/schemas.py  ← Pydantic data model (the shared contract)
│   │   ├── prompts/prompts.py ← prompts (voice/tone + safety + scene/mood rules)
│   │   ├── integrations/      ← anthropic, fetchai, deepgram/voice, redis
│   │   │                         (app/profile) + vector_store + embedding,
│   │   │                         image (gemini/openai/midjourney), pika, govee,
│   │   │                         arize, terac
│   │   ├── services/          ← emotion, comfort_strategy, planner, safety, story,
│   │   │                         story_retrieval, visual, journal, memory,
│   │   │                         prompt_agent, review_trail, auth, asset_cache
│   │   └── routers/           ← auth, session, story, voice, visual, profile,
│   │   │                         settings, journal, lamp, rag, admin
│   └── tests/                 ← 173 pytest tests (deterministic, mock mode)
└── frontend/
    └── src/
        ├── api.ts             ← typed client mirroring schemas.py (+ auto-login)
        ├── App.tsx  main.tsx
        ├── pages/             ← ProfilePicker (dreamscape), ChildMode, ParentDashboard,
        │                         profile create/edit
        ├── components/        ← NightSky, NinoFox, MicButton, HelpScreen,
        │                         StorybookReader (silent), BgmToggle, StatusBadge,
        │                         ProfileSwitcher, WarmBackground, Brand, …
        ├── lib/               ← bgm.ts (looping lullaby, steady level + mute),
        │                         audioFade.ts (eased volume ramps), profileStore.ts
        ├── context/           ← ProfileContext (per-device roster + active child)
        └── hooks/useAudio.ts  ← shared unlocked audio el + narration rate/volume

API & environment

Full API in API_CONTRACT.md. Key endpoints: POST /api/auth/login, POST /api/session/checkin, POST /api/story/generate, POST /api/story/revise, POST /api/visual/generate, POST /api/voice/stt, POST /api/voice/tts, POST /api/lamp/mood · /api/lamp/off, GET /api/journal/{child_id}, profile/settings CRUD, and GET /api/status (live-vs-mock badges).
Keys (all optional) live in .env (see .env.example). Anything left blank runs on its mock:
- PROMPT_PROVIDER — anthropic (default) or fetchai.
- ANTHROPIC_API_KEY (Claude) · FETCHAI_API_KEY / ASI_ONE_API_KEY (ASI One).
- DEEPGRAM_API_KEY · DEEPGRAM_TTS_MODEL (default aura-2-cora-en).
- REDIS_URL (+ optional split REDIS_APP_URL / REDIS_PROFILE_URL).
- IMAGE_PROVIDER + GEMINI_API_KEY / OPENAI_API_KEY / MIDJOURNEY_API_KEY (image) · PIKA_API_KEY (clips).
- GOVEE_API_KEY / GOVEE_DEVICE / GOVEE_SKU (physical lamp).
- ARIZE_*, TERAC_* (observability / annotation).

Demo flow

Profile picker (/) → create a child (name, age, favorites) → open it.
Child mode → "Hi, {name}" → say/type "I'm scared of the dark and I miss my mom." → hear the gentle reflection. Lullaby BGM fades in.
Continue → a personalized story plays as calming audio (eyes closed); the mood lamp shifts with each scene → goodnight (lamp fades off).
At goodnight, tap "📖 Read last night's storybook" → a silent, self-paced picture book of the same story (painted in the background while it played).
Try a danger phrase (e.g. "someone is hurting me") → it does not tell a story; it shows the warm help screen.
Parent dashboard (/parent) → review the story's review trail + safety scores, open the storybook, revise it, approve it (watch the story world remember the theme), and browse the growth journal.
Note the live/mock badge showing which integrations are wired.

Operational notes

.env is not hot-reloaded. uvicorn --reload watches code, not .env; restart the server after changing keys or the TTS voice.
Switchable LLM: set PROMPT_PROVIDER=fetchai to route the story brain through Fetch.ai ASI One instead of Claude. Both are tested behind prompt_agent.
Lamp is optional hardware: with no GOVEE_* keys, /api/lamp/* and the per-scene calls are silent no-ops; calls run in a background thread and swallow errors so the lamp can never slow or break the bedtime flow.
Live image paths (Gemini / OpenAI / Midjourney reference-image, set via IMAGE_PROVIDER) are only exercised with a real key — verify character consistency manually once keys are set.
Generated artifacts are written under backend/generated/ (git-ignored).

See API_CONTRACT.md for the full endpoint contract shared by frontend and backend.

Contributors

Lullow was built at the 2026 UC Berkeley AI Hackathon by four people:

Eugene Gu — @eugenegujing
Srinivas Rao Chavan — @srinivas1698
Thinh Nguyen — @mthinhngn
Ella Wu — @gualle

Credits & assets

Lullaby background music — three royalty-free piano tracks from Pixabay (no attribution required; credited here with thanks): atlasaudio, leberch, and the_mountain. Files live in frontend/public/bgm/; one is chosen at random per page load and looped.
AI / sponsor services — Anthropic Claude, Fetch.ai ASI One, Deepgram (Aura-2 voice), Google Gemini (image), Pika (clips), Govee (lamp), Redis, Arize, Terac.

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

AnthropicIn code
CSSIn code
FastAPIIn code
HTMLIn code
JavaScriptIn code
PythonIn code
ReactIn code
RedisIn code
Tailwind CSSIn code
TypeScriptIn code
Google GeminiClaimed

10 of 11 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

Claude CodeCommits

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

558 KB

Source files

127

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.