# Project export: Pebble This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: A purpose-built AI learning device for kids. Voice-first with on-device speech recognition, age-calibrated safety guardrails, a real-time parent dashboard, and LMS integration. - Devpost: https://devpost.com/software/pebble-90j1q6 - GitHub: https://github.com/zane-programs/treehacks-pebble - Demo: https://meet.pebble.you/ - Video: https://www.youtube.com/embed/TZiivZ8zc6k?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — Zane St. John (12 commits), Claude Opus 4.6 (1 commits) ## Devpost submission (written by the team) ### Inspiration 30% of US teens use AI chatbots daily. A third of them prefer talking to AI over talking to people for personal conversations. Common Sense Media found that every major chatbot fails to appropriately handle mental health conditions in young people. A 14-year-old died by suicide after forming an emotional bond with an AI companion. 83% of parents think their children's mental health is getting worse. Kids are already using AI unsupervised, at scale, and nothing out there was built for them. Parents have no visibility. No control. The $70 billion tutoring market is still gated by cost, and the "kid-safe" AI tools that exist are dumbed-down wrappers that children outgrow fast. Pebble started as a voice-first study companion for college students. A dedicated Android device, Canvas LMS integration, on-device speech recognition, cloud inference for tutoring. That was the plan going into TreeHacks, and for most of the 36 hours, that's what I built. Then, past 1 AM on Sunday, with submissions due at 9:30 that morning, I pivoted to children. I added a full parent management system, a real-time web dashboard, age-calibrated guardrails across four tiers (elementary, middle, high, college+), content blocking, device locking, and a multi-layered safety architecture. The college experience stayed intact as the top tier, but the product became something bigger: an AI learning device that parents can actually trust. How I built it Solo build. Three components, 36 hours: a Node.js/TypeScript backend, a native Kotlin/Jetpack Compose Android app, and a React parent dashboard. The Android device runs ONNX Whisper for speech-to-text entirely locally. No child's voice ever leaves the phone. But I didn't try to force the device to do more than it should. The real inference (Socratic tutoring, safety evaluation, tool orchestration) runs through foundation models in the cloud. Claude (Haiku) handles primary chat, with delegation to Sonnet and Opus for harder tasks. The system also routes to OpenAI (GPT) and Google (Gemini) via OpenRouter, so the agent picks the best model for a given task. Parents can select their child's preferred provider from device settings. Regardless of which model is working, all safety guardrails are enforced before any delegation happens. The agentic loop itself is built on Anthropic's Claude Agent SDK, which manages the full tool-use cycle, subagent delegation, and streaming. All 20+ Pebble tools (Canvas data, web scraping, study sessions, quizzes, persistent memory) are exposed to the SDK via an MCP server, and custom hooks bridge SDK events into SSE for the Android client in real-time. The AI teaches through active recall and Socratic questioning, never just giving answers. Canvas integration went smooth thanks to Laura Schauer's OpenAPI spec. Safety This is the part I care about most. If you're building AI for children, safety can't be a system prompt disclaimer. It has to be architectural. Pebble uses three layers. First, heuristic detection: after every response, the backend scans for redirect phrases as a fallback, catching safety events the model didn't explicitly flag. Second, an explicit tool (flag_safety_redirect) the model can call to formally log when it redirects away from a blocked topic, with structured metadata that flows to the parent dashboard. Third, emergency lockout: for critical violations, the model calls emergency_lock_device, which immediately locks the device, flags the conversation for parent review, and pushes a real-time alert via SSE. No further interaction until a parent unlocks it. Every interaction is also shaped by age-calibrated guardrails. Parents can block custom topics and set daily time limits. And the parent web dashboard shows every conversation, every study session, every safety redirect, plus AI-generated weekly summaries, mastery tracking, Canvas grades, and proactive alerts. The dashboard isn't a token gesture. It's the product's other half. Challenges SSE streaming with agentic tool-use loops was the hardest part. The model calls multiple tools mid-response, and the client needs to know what's happening at every step. I built incremental streaming that emits tool status, text deltas, heading tags, quiz payloads, study session updates, and safety events over a single connection. Most of the bugs I squashed during my all-nighter lived here: the model starting a study session but forgetting to quiz (solved with continuation nudging), text deltas arriving mid-tag, the client disconnecting mid-tool-loop. Each one was small, but they stacked up. The kid pivot at 1 AM was its own challenge. Not because the code was hard, but because the scope was enormous. Parent accounts, a web dashboard, real-time alerts, device locking, activity logging, age-calibrated prompts, and a safety architecture I could stand behind. All in under eight hours. What I learned The right abstraction boundaries make ambitious projects possible. On-device STT, cloud inference. Architectural safety, not prompt-level. The parent dashboard as a first-class component, not a bolt-on. Each decision made the system easier to reason about, even at 4 AM when nothing was compiling. Building for kids forces you to think harder. When your user is a 7-year-old, "it usually works" isn't good enough. ## README (from the GitHub repository) # Pebble A dedicated AI learning device for kids. Voice-first. Parent-managed. Built solo at [TreeHacks 2026](https://www.treehacks.com/) in 36 hours. ## Why [30% of US teens](https://www.pewresearch.org/internet/2025/12/09/teens-social-media-and-ai-chatbots-2025/) use AI chatbots daily. A third of them [prefer talking to AI over talking to people](https://techcrunch.com/2025/12/09/three-in-ten-u-s-teens-use-ai-chatbots-every-day-but-safety-concerns-are-growing/). [Common Sense Media found](https://www.commonsensemedia.org/press-releases/common-sense-media-finds-major-ai-chatbots-unsafe-for-teen-mental-health-support) that every major chatbot fails to safely handle young people's mental health. A [14-year-old died by suicide](https://www.npr.org/2025/12/29/nx-s1-5646633/teens-ai-chatbot-sex-violence-mental-health) after bonding with an AI companion. Kids are already using AI unsupervised, at scale, and nothing out there was built for them. Parents have no visibility. No control. The "kid-safe" AI tools that exist are dumbed-down wrappers that children outgrow fast. Pebble is a purpose-built Android device that runs a voice-first AI tutor, managed entirely by parents. It's not an app. It's an appliance for learning. ## What It Does **For kids:** Tap the orb, ask a question, hear an answer. Pebble connects to Canvas LMS, knows your courses and deadlines, and teaches through Socratic questioning and interactive quizzes. It doesn't give answers. It teaches. **For parents:** A real-time web dashboard showing every conversation, every study session, every time the AI redirected away from something unsafe. Set the age group (elementary, middle, high, college+), block specific topics, set daily time limits. The dashboard isn't a token gesture. It's the product's other half. **For college students:** The same powerful tutor, minus the parental guardrails. Full Canvas integration, guided study sessions, mastery tracking, proactive deadline alerts. Self-managed. ## Safety This is the part I care about most. If you're building AI for children, safety can't be a system prompt disclaimer. It has to be architectural. Pebble uses three layers: 1. **Heuristic detection** - After every response, the backend scans for redirect phrases as a fallback, catching safety events the model didn't explicitly flag. 2. **Explicit tooling** - The model can call `flag_safety_redirect` to formally log when it redirects away from a blocked topic, with structured metadata that flows to the parent dashboard. 3. **Emergency lockout** - For critical violations, `emergency_lock_device` immediately locks the device, flags the conversation for parent review, and pushes a real-time alert. No further interaction until a parent unlocks it. Every interaction is also shaped by age-calibrated guardrails. A 7-year-old gets simple words, lots of encouragement, one concept at a time. A 16-year-old gets full vocabulary, Socratic questioning, college prep focus. Parents can block custom topics on top of the defaults. And all of this is enforced before any model delegation happens, so safety guardrails hold regardless of which model is doing the work. ## Architecture Three components: ``` Android App (Kotlin/Compose) Backend (Node.js/TypeScript) Web Dashboard (React) ├── On-device ONNX Whisper STT ├── Express API server ├── Parent auth ├── Voice orb UI (4 states) ├── Claude agentic tool-use loop ├── Child device linking ├── Interactive quiz overlay ├── Canvas LMS proxy ├── Activity feed ├── Study session progress ├── Multi-model delegation ├── Progress tracking ├── Drawing canvas + vision ├── Web scraper (external syllabi) ├── Safety redirect log ├── Kiosk mode lockdown ├── Parent account system ├── Guardrail settings └── ElevenLabs TTS (system fallback) ├── Safety enforcement pipeline └── Proactive alerts ├── PostgreSQL persistence └── SSE streaming ``` **Privacy:** Speech recognition runs entirely on-device with Whisper. No child's voice ever leaves the phone. **Model delegation:** Claude Haiku handles primary chat (fast, cheap). It delegates to Sonnet for moderate complexity and Opus for hard problems. GPT and Gemini are also available via OpenRouter. Haiku always acts as the safety gatekeeper before any delegation. **Canvas integration:** Courses, assignments, grades, announcements, modules, pages. External course websites (like math department sites that Canvas doesn't know about) are detected and scraped automatically. ## Voice Conversation Flow 1. Kid taps the orb. Audio recorded at 16kHz mono. 2. ONNX Whisper transcribes on-device. Transcript sent to backend. 3. Claude runs an agentic tool-use loop (up to 10 iterations, 20+ tools available). 4. SSE streams back: tool status, text deltas, quiz events, study session updates, safety events. 5. Response spoken via ElevenLabs TTS (or system TTS as fallback). 6. Orb transitions: idle -> listening -> thinking -> speaking -> idle. ## Study Sessions Say "help me study for math" and Pebble pulls real assignments from Canvas, identifies weak topics from past mastery data, and walks you through active recall. Interactive quizzes with instant grading. It remembers what you got wrong last time and comes back to it. Mastery scores persist across sessions. ## Running It ### Backend ```bash cd backend npm install npm run db:migrate # PostgreSQL migrations (idempotent) npm run dev # Development server with verbose logging ``` Needs a `.env` with: `ANTHROPIC_API_KEY`, `DATABASE_URL` (PostgreSQL), `PORT` (default 3001). Optional: `OPENROUTER_API_KEY` for GPT/Gemini delegation. ### Android Open `android/` in Android Studio. Set `BACKEND_URL`, `ELEVENLABS_API_KEY`, and `ELEVENLABS_VOICE_ID` in `android/local.properties`. Default backend URL for emulator: `http://10.0.2.2:3001`. ### Web Dashboard ```bash cd web npm install npm run dev # Vite dev server npm run build # Builds to dist/ (served by backend as static files) ``` ### Demo UI (Splash Site) ```bash cd demo-ui npm install npm run dev ``` ## Stack - **Android:** Kotlin 2.1, Jetpack Compose, ONNX Runtime, Retrofit, minSdk 26 - **Backend:** Node.js, TypeScript (strict), Express, PostgreSQL, Anthropic SDK, Claude Agent SDK - **Web:** React 19, TypeScript, Vite, Tailwind CSS - **AI:** Claude (Haiku/Sonnet/Opus), GPT-4o, Gemini 2.5 Pro via OpenRouter - **Speech:** ONNX Whisper tiny.en (on-device STT), ElevenLabs (cloud TTS) ## Background This started as a voice-first study companion for college students. Canvas integration, on-device speech recognition, cloud inference for tutoring. That was the plan going into TreeHacks. Then, past 1 AM on Sunday, with submissions due at 9:30 that morning, I pivoted to children. I added a full parent management system, a real-time web dashboard, age-calibrated guardrails across four tiers, content blocking, device locking, and a multi-layered safety architecture. The college experience stayed intact as the top tier, but the product became something bigger: an AI learning device that parents can actually trust. Building for kids forces you to think harder. When your user is a 7-year-old, "it usually works" isn't good enough. ## Built With Built solo by [Zane St. John](https://github.com/zanestjohn) at TreeHacks 2026 (Stanford, Feb 14-16). ## License MIT. See [LICENSE](LICENSE). ## Detected evidence (automated analysis) Indexed codebase: 171 recognized source files, 2470 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - Express (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Kotlin (language) — detected in the code - OpenAI (technology) — detected in the code - PostgreSQL (technology) — detected in the code - React (technology) — detected in the code - SQL (language) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - Google Gemini (technology) — claimed on Devpost, not found in the code - Node.js (technology) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers ## Codebase structure (from repository index) ### Files (120 of 195) ``` .dockerignore .gitignore android/.gitignore android/app/build.gradle.kts android/app/src/main/AndroidManifest.xml android/app/src/main/java/com/pebble/app/BootReceiver.kt android/app/src/main/java/com/pebble/app/data/local/PreferencesManager.kt android/app/src/main/java/com/pebble/app/data/model/AuthModels.kt android/app/src/main/java/com/pebble/app/data/model/ChatMessage.kt android/app/src/main/java/com/pebble/app/data/model/ChatModels.kt android/app/src/main/java/com/pebble/app/data/model/OnboardingModels.kt android/app/src/main/java/com/pebble/app/data/model/OrbState.kt android/app/src/main/java/com/pebble/app/data/model/ProactiveAlert.kt android/app/src/main/java/com/pebble/app/data/model/ProactiveModels.kt android/app/src/main/java/com/pebble/app/data/model/QuizModels.kt android/app/src/main/java/com/pebble/app/data/model/WifiModels.kt android/app/src/main/java/com/pebble/app/data/remote/AuthInterceptor.kt android/app/src/main/java/com/pebble/app/data/remote/PebbleApi.kt android/app/src/main/java/com/pebble/app/data/remote/RetrofitClient.kt android/app/src/main/java/com/pebble/app/data/remote/SseClient.kt android/app/src/main/java/com/pebble/app/data/remote/TokenRefreshAuthenticator.kt android/app/src/main/java/com/pebble/app/domain/audio/AudioRecorderImpl.kt android/app/src/main/java/com/pebble/app/domain/audio/PebbleAudioRecorder.kt android/app/src/main/java/com/pebble/app/domain/notifications/ProactiveNotificationManager.kt android/app/src/main/java/com/pebble/app/domain/notifications/ProactiveNotificationReceiver.kt android/app/src/main/java/com/pebble/app/domain/stt/PebbleSpeechRecognizer.kt android/app/src/main/java/com/pebble/app/domain/stt/WhisperRecognizer.kt android/app/src/main/java/com/pebble/app/domain/tts/ElevenLabsTts.kt android/app/src/main/java/com/pebble/app/domain/tts/PebbleTextToSpeech.kt android/app/src/main/java/com/pebble/app/domain/tts/SystemTtsFallback.kt android/app/src/main/java/com/pebble/app/domain/wifi/WifiController.kt android/app/src/main/java/com/pebble/app/KioskService.kt android/app/src/main/java/com/pebble/app/MainActivity.kt android/app/src/main/java/com/pebble/app/ui/components/AnimatedTitleBar.kt android/app/src/main/java/com/pebble/app/ui/components/CaptivePortalWebView.kt android/app/src/main/java/com/pebble/app/ui/components/ChatMenuDrawer.kt android/app/src/main/java/com/pebble/app/ui/components/ChatMessageList.kt android/app/src/main/java/com/pebble/app/ui/components/HeadingDisplay.kt android/app/src/main/java/com/pebble/app/ui/components/ProactivityDialog.kt android/app/src/main/java/com/pebble/app/ui/components/ProactivityTicker.kt android/app/src/main/java/com/pebble/app/ui/components/QuizOverlay.kt android/app/src/main/java/com/pebble/app/ui/components/ResponseBubble.kt android/app/src/main/java/com/pebble/app/ui/components/StudySessionIndicator.kt android/app/src/main/java/com/pebble/app/ui/components/TextInputBar.kt android/app/src/main/java/com/pebble/app/ui/components/ThreadDrawer.kt android/app/src/main/java/com/pebble/app/ui/components/TranscriptScreen.kt android/app/src/main/java/com/pebble/app/ui/components/VoiceOrb.kt android/app/src/main/java/com/pebble/app/ui/components/WifiHiddenNetworkDialog.kt android/app/src/main/java/com/pebble/app/ui/components/WifiNetworkItem.kt android/app/src/main/java/com/pebble/app/ui/components/WifiPasswordDialog.kt android/app/src/main/java/com/pebble/app/ui/components/WordCloud.kt android/app/src/main/java/com/pebble/app/ui/navigation/PebbleNavGraph.kt android/app/src/main/java/com/pebble/app/ui/screens/home/HomeScreen.kt android/app/src/main/java/com/pebble/app/ui/screens/onboarding/OnboardingScreen.kt android/app/src/main/java/com/pebble/app/ui/screens/settings/AboutScreen.kt android/app/src/main/java/com/pebble/app/ui/screens/settings/AccountsScreen.kt android/app/src/main/java/com/pebble/app/ui/screens/settings/DeviceScreen.kt android/app/src/main/java/com/pebble/app/ui/screens/settings/NotificationSettingsScreen.kt android/app/src/main/java/com/pebble/app/ui/screens/settings/SettingsScreen.kt android/app/src/main/java/com/pebble/app/ui/screens/settings/WifiDetailScreen.kt android/app/src/main/java/com/pebble/app/ui/screens/settings/WifiScreen.kt android/app/src/main/java/com/pebble/app/ui/screens/setup/PairingScreen.kt android/app/src/main/java/com/pebble/app/ui/screens/setup/SetupScreen.kt android/app/src/main/java/com/pebble/app/ui/theme/Color.kt android/app/src/main/java/com/pebble/app/ui/theme/Theme.kt android/app/src/main/java/com/pebble/app/ui/theme/Type.kt android/app/src/main/java/com/pebble/app/util/RootShell.kt android/app/src/main/java/com/pebble/app/viewmodel/PebbleViewModel.kt android/app/src/main/java/com/pebble/app/viewmodel/WifiViewModel.kt android/app/src/main/res/drawable/ic_notification.xml android/app/src/main/res/raw/whisper_cpu_int8_model.onnx android/app/src/main/res/values/strings.xml android/app/src/main/res/values/themes.xml android/app/src/main/res/xml/network_security_config.xml android/build.gradle.kts android/gradle.properties android/gradle/libs.versions.toml android/gradle/wrapper/gradle-wrapper.properties android/gradlew android/gradlew.bat android/settings.gradle.kts android/subplans/STT.md android/subplans/TTS.md android/subplans/UI.md backend/.gitignore backend/canvas_openapi3.yaml backend/migrations/001_schema.sql backend/migrations/002_proactive_schedules.sql backend/migrations/003_assignment_summaries.sql backend/migrations/004_personalization.sql backend/migrations/005_pairing_codes.sql backend/migrations/006_memories.sql backend/migrations/007_study_sessions.sql backend/migrations/008_parent_accounts.sql backend/migrations/009_inference_provider.sql backend/migrations/010_safety_lockout.sql backend/migrations/011_flagged_threads.sql backend/openapi.yaml backend/package.json backend/PLAN_BACKEND.md backend/scripts/seed-demo.ts backend/src/index.ts backend/src/middleware/auth.ts backend/src/middleware/devLogger.ts backend/src/middleware/parentAuth.ts backend/src/migrate.ts backend/src/routes/auth.ts backend/src/routes/chat.ts backend/src/routes/onboarding.ts backend/src/routes/pairing.ts backend/src/routes/parent.ts backend/src/routes/proactive.ts backend/src/routes/threads.ts backend/src/sdk/agent-chat.ts backend/src/sdk/hooks.ts backend/src/sdk/pebble-mcp-server.ts backend/src/sdk/sse-bridge.ts backend/src/services/auth.service.ts backend/src/services/canvas.service.ts backend/src/services/content.service.ts [75 more files omitted for size] ``` ### Dependencies - backend/package.json: @anthropic-ai/claude-agent-sdk@^0.2.42, @anthropic-ai/sdk@^0.74.0, @ngrok/ngrok@^1.7.0, @types/bcryptjs@^2.4.6, @types/cors@^2.8.17, @types/express@^5.0.0, @types/html-to-text@^9.0.4, @types/jsdom@^21.1.7, @types/jsonwebtoken@^9.0.10, @types/pg@^8.16.0, @types/qrcode-terminal@^0.12.2, bcryptjs@^3.0.3, cors@^2.8.5, dotenv@^16.4.7, express@^4.21.2, html-to-text@^9.0.5, jsdom@^26.0.0, jsonwebtoken@^9.0.3, openai@^6.22.0, pg@^8.18.0, qrcode-terminal@^0.12.0, tsx@^4.19.2, typescript@^5.7.3, zod@^4.3.6 - demo-ui/package.json: @tailwindcss/vite@^4.1.18, @types/react@^19.2.7, @types/react-dom@^19.2.3, @vitejs/plugin-react@^5.1.1, framer-motion@^12.0.0, react@^19.2.0, react-dom@^19.2.0, tailwindcss@^4.1.18, typescript@~5.9.3, vite@^7.3.1 - web/package.json: @eslint/js@^9.39.1, @tailwindcss/vite@^4.1.18, @types/node@^24.10.1, @types/react@^19.2.7, @types/react-dom@^19.2.3, @vitejs/plugin-react@^5.1.1, eslint@^9.39.1, eslint-plugin-react-hooks@^7.0.1, eslint-plugin-react-refresh@^0.4.24, globals@^16.5.0, input-otp@^1.4.2, lucide-react@^0.564.0, react@^19.2.0, react-dom@^19.2.0, react-router-dom@^7.13.0, tailwindcss@^4.1.18, typescript@~5.9.3, typescript-eslint@^8.48.0, vite@^7.3.1 ### Recent commits (newest first) - Last thing - fix lock polling - Quiz mode fixes - Increase font sizes - Add README so folks know what is up :) - Add slideshow mode to splash UI - License and about screen - Claude Agents SDK integration + polish and parental safety feature - UI for splash website - Fix pairing codes staying locked after incomplete web setup - Cloud setup for Render et al. - Study sessions with active recall, web pairing for better UX. - More agentic features, more realtime (SSE/streaming) - Full-stack tool use improvements + multi-chat in DB - Initial backend with basic tool usage - Initial stab at Android UI - Initial commit - agent planning ## Key source files (fetched from GitHub, selected and truncated for size) ### CLAUDE.md ```markdown # CLAUDE.md This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. ## Project Overview Pebble is a voice-first AI study companion for Android that integrates with Canvas LMS. It consists of three components: a **Node.js/TypeScript backend**, a **Kotlin/Jetpack Compose Android app**, and a **React web pairing UI**. The backend orchestrates Claude API tool-use loops against Canvas data, while the Android app handles voice I/O (on-device Whisper STT, ElevenLabs TTS) and UI. ## Development Commands ### Backend ```bash cd backend npm install npm run dev # Run with tsx (NODE_ENV=development enables verbose color logging) npm run build # Compile TypeScript to dist/ npm start # Run compiled JS (production) npm run db:migrate # Run all SQL migrations from migrations/ npm run build:web # Build the web pairing UI (cd ../web && npm run build) ``` Required env vars in `backend/.env`: `ANTHROPIC_API_KEY`, `DATABASE_URL` (PostgreSQL connection string), `PORT` (default 3001). ### Android Open `android/` in Android Studio and build/run from there. The app reads `BACKEND_URL`, `ELEVENLABS_API_KEY`, and `ELEVENLABS_VOICE_ID` from `android/local.properties` (injected as BuildConfig fields). Default backend URL for emulator is `http://10.0.2.2:3001`. ### Web (Pairing UI) ```bash cd web npm install npm run dev # Vite dev server npm run build # Build to dist/ (served by backend as static files) ``` React 19 + TypeScript + Vite + Tailwind CSS. Three-step pairing flow: code entry → Canvas settings → success. ## Architecture ### Three-Component System - **Backend** (`backend/src/`): Express server that proxies Canvas LMS APIs, runs Claude agentic tool-use loops, manages sessions/auth, and generates proactive alerts. - **Android App** (`android/app/src/main/java/com/pebble/app/`): Native Kotlin app with voice orb UI, on-device speech-to-text, cloud/system TTS, kiosk mode support. - **Web UI** (`web/src/`): React pairing flow that lets users enter a 6-digit code and Canvas credentials from a browser. ### Backend Key Paths - `routes/chat.ts` — Main conversation endpoint with SSE streaming - `routes/auth.ts` — Canvas token validation and JWT session setup - `routes/proactive.ts` — Proactive alert generation (deadlines, grades, exams) - `routes/threads.ts` — Thread/conversation management - `routes/onboarding.ts` — Onboarding pipeline trigger and term polling - `routes/pairing.ts` — 6-digit pairing code flow (create, verify, bond, status) - `services/llm.service.ts` — Claude agentic tool-use loop orchestration with model delegation - `services/canvas.service.ts` — Canvas REST API wrapper (courses, assignments, grades, modules, pages) - `services/proactive.service.ts` — Alert engine: deadlines, low grades, exams, announcements - `services/db.service.ts` — PostgreSQL operations (sessions, threads, messages, study sessions, memories, pairing codes) - `services/study.service.t [truncated — 8253 more characters] ``` ### backend/PLAN_BACKEND.md ```markdown # Sub-Plan B: Backend — Canvas Proxy, LLM Orchestration & Proactive Engine > **For the full system architecture, integration order, API contracts, and demo narrative, read `../CENTRAL.md` before beginning work.** --- ## Your Mission Build Pebble's brain — the Node.js/TypeScript backend server that handles all cloud-side logic. This server is the intermediary between the Android app and every external service: Canvas LMS, Claude API, and external course websites. It also runs the proactive alert engine that makes Pebble feel alive. The Android app sends simple HTTP requests; this server does all the heavy lifting. --- ## Tech Stack - **Runtime:** Node.js 20+ - **Language:** TypeScript (strict mode) - **Framework:** Express.js - **LLM:** `@anthropic-ai/sdk` — Claude Sonnet 4.5 for conversation, Claude Haiku 4.5 for fast classification - **Scraping:** `jsdom` (HTML parsing) + `html-to-text` (clean text extraction) - **State:** In-memory `Map` objects (no database — this is a single-user hackathon demo) - **IDs:** `uuid` for session generation --- ## Project Structure ``` pebble-backend/ ├── src/ │ ├── index.ts # Express app entry, CORS, route mounting │ ├── routes/ │ │ ├── auth.ts # POST /api/auth/setup │ │ ├── chat.ts # POST /api/chat │ │ └── proactive.ts # GET /api/proactive/check │ ├── services/ │ │ ├── canvas.service.ts # Canvas REST API wrapper │ │ ├── llm.service.ts # Claude orchestration with agentic tool-use loop │ │ ├── proactive.service.ts # Alert generation engine │ │ ├── scraper.service.ts # External course website scraper + cache │ │ └── content.service.ts # Unified content layer (Canvas + scraped) │ ├── tools/ │ │ └── pebble-tools.ts # Claude tool definitions + execution dispatch │ └── types/ │ └── index.ts # Shared TypeScript interfaces ├── package.json ├── tsconfig.json └── .env # ANTHROPIC_API_KEY, PORT ``` --- ## API Routes ### `POST /api/auth/setup` - **Body:** `{ canvasToken: string, canvasBaseUrl: string }` - **Action:** Store the token in-memory (keyed by a new session ID). Validate by calling Canvas `GET /users/self/profile`. On success, kick off background content loading (fetch all courses, detect external syllabi, pre-scrape). - **Response:** `{ success: true, sessionId: string, profile: { name, email } }` ### `POST /api/chat` - **Body:** `{ message: string, sessionId: string }` - **Action:** Run the agentic Claude tool-use loop (see LLM Orchestration below). Maintain conversation history per session. - **Response:** `{ response: string }` ### `GET /api/proactive/check` - **Query:** `?sessionId=xxx` - **Action:** Run the full proactive alert check. - **Response:** `{ alerts: ProactiveAlert[] }` ### `GET /api/health` - **Response:** `{ status: "ok" }` Enable CORS for all origins (hackathon demo — no need for restrictions). --- ## Canvas API Service Wr [truncated — 8310 more characters] ``` ### Dockerfile ``` # ── Stage 1: Build web UI ── FROM node:20-alpine AS web-build WORKDIR /app/web COPY web/package.json web/package-lock.json ./ RUN npm ci COPY web/ ./ RUN npm run build # ── Stage 2: Build backend ── FROM node:20-alpine AS backend-build WORKDIR /app/backend COPY backend/package.json backend/package-lock.json ./ RUN npm ci COPY backend/ ./ RUN npm run build # ── Stage 3: Production ── FROM node:20-alpine WORKDIR /app/backend COPY --from=backend-build /app/backend/dist ./dist COPY --from=backend-build /app/backend/package.json ./ COPY --from=backend-build /app/backend/package-lock.json ./ COPY --from=backend-build /app/backend/migrations ./migrations RUN npm ci --omit=dev COPY --from=web-build /app/web/dist /app/web/dist ENV NODE_ENV=production ENV WEB_DIST_PATH=/app/web/dist EXPOSE 3001 CMD ["sh", "-c", "node dist/migrate.js && node dist/index.js"] ``` ### demo-ui/package.json ``` { "name": "pebble-demo", "private": true, "version": "0.0.0", "type": "module", "scripts": { "dev": "vite", "build": "tsc -b && vite build", "preview": "vite preview" }, "dependencies": { "@tailwindcss/vite": "^4.1.18", "framer-motion": "^12.0.0", "react": "^19.2.0", "react-dom": "^19.2.0", "tailwindcss": "^4.1.18" }, "devDependencies": { "@types/react": "^19.2.7", "@types/react-dom": "^19.2.3", "@vitejs/plugin-react": "^5.1.1", "typescript": "~5.9.3", "vite": "^7.3.1" } } ``` ### web/package.json ``` { "name": "web", "private": true, "version": "0.0.0", "type": "module", "scripts": { "dev": "vite", "build": "tsc -b && vite build", "lint": "eslint .", "preview": "vite preview" }, "dependencies": { "@tailwindcss/vite": "^4.1.18", "input-otp": "^1.4.2", "lucide-react": "^0.564.0", "react": "^19.2.0", "react-dom": "^19.2.0", "react-router-dom": "^7.13.0", "tailwindcss": "^4.1.18" }, "devDependencies": { "@eslint/js": "^9.39.1", "@types/node": "^24.10.1", "@types/react": "^19.2.7", "@types/react-dom": "^19.2.3", "@vitejs/plugin-react": "^5.1.1", "eslint": "^9.39.1", "eslint-plugin-react-hooks": "^7.0.1", "eslint-plugin-react-refresh": "^0.4.24", "globals": "^16.5.0", "typescript": "~5.9.3", "typescript-eslint": "^8.48.0", "vite": "^7.3.1" } } ``` ### backend/package.json ``` { "name": "pebble-backend", "version": "1.0.0", "description": "Pebble AI study companion backend", "main": "dist/index.js", "scripts": { "dev": "NODE_ENV=development npx tsx src/index.ts", "build": "tsc", "start": "node dist/index.js", "db:migrate": "npx tsx src/migrate.ts", "build:web": "cd ../web && npm run build", "seed:demo": "npx tsx scripts/seed-demo.ts" }, "dependencies": { "@anthropic-ai/claude-agent-sdk": "^0.2.42", "@anthropic-ai/sdk": "^0.74.0", "@ngrok/ngrok": "^1.7.0", "@types/bcryptjs": "^2.4.6", "@types/jsonwebtoken": "^9.0.10", "@types/pg": "^8.16.0", "bcryptjs": "^3.0.3", "cors": "^2.8.5", "dotenv": "^16.4.7", "express": "^4.21.2", "html-to-text": "^9.0.5", "jsdom": "^26.0.0", "jsonwebtoken": "^9.0.3", "openai": "^6.22.0", "pg": "^8.18.0", "qrcode-terminal": "^0.12.0", "zod": "^4.3.6" }, "devDependencies": { "@types/cors": "^2.8.17", "@types/express": "^5.0.0", "@types/html-to-text": "^9.0.4", "@types/jsdom": "^21.1.7", "@types/qrcode-terminal": "^0.12.2", "tsx": "^4.19.2", "typescript": "^5.7.3" } } ``` ### demo-ui/src/main.tsx ```typescript import { StrictMode } from 'react' import { createRoot } from 'react-dom/client' import './index.css' import App from './App.tsx' createRoot(document.getElementById('root')!).render( , ) ``` ### web/src/main.tsx ```typescript import { StrictMode } from 'react' import { createRoot } from 'react-dom/client' import './index.css' import App from './App.tsx' createRoot(document.getElementById('root')!).render( , ) ``` ### demo-ui/src/App.tsx ```typescript import { useMemo } from 'react' import { SlideEngine } from './engine/SlideEngine.tsx' import { useSlideNavigation } from './engine/useSlideNavigation.ts' import { getSlideshowPositions } from './engine/slideConfig.ts' import { ProgressDots } from './components/ProgressDots.tsx' import { BlankSlide } from './slides/BlankSlide.tsx' import { HeroSlide } from './slides/HeroSlide.tsx' import { ProblemSlide } from './slides/ProblemSlide.tsx' import { WhatIsPebbleSlide } from './slides/WhatIsPebbleSlide.tsx' import { VoiceFirstSlide } from './slides/VoiceFirstSlide.tsx' import { SafetySlide } from './slides/SafetySlide.tsx' import { StudySlide } from './slides/StudySlide.tsx' import { ParentSlide } from './slides/ParentSlide.tsx' import { PersonalSlide } from './slides/PersonalSlide.tsx' import { CTASlide } from './slides/CTASlide.tsx' const isSlideshow = new URLSearchParams(window.location.search).has('slideshow') export default function App() { const slideshowPositions = useMemo(() => isSlideshow ? getSlideshowPositions() : undefined, []) const { activeSlide, navigate } = useSlideNavigation( isSlideshow ? { slideCount: 11, arrowKeysOnly: true } : {} ) if (isSlideshow) { return ( ) } return ( <> navigate(i)} /> ) } ``` ### backend/src/index.ts ```typescript import "dotenv/config"; import express from "express"; import path from "path"; import cors from "cors"; import authRouter from "./routes/auth.js"; import chatRouter from "./routes/chat.js"; import proactiveRouter from "./routes/proactive.js"; import threadsRouter from "./routes/threads.js"; import onboardingRouter from "./routes/onboarding.js"; import pairingRouter from "./routes/pairing.js"; import parentRouter from "./routes/parent.js"; import { devLogger } from "./middleware/devLogger.js"; const app = express(); const PORT = process.env.PORT ?? 3001; app.use(cors()); app.use(express.json({ limit: "10mb" })); app.use(devLogger); // Routes app.use("/api/auth", authRouter); app.use("/api/chat", chatRouter); app.use("/api/proactive", proactiveRouter); app.use("/api/threads", threadsRouter); app.use("/api/onboarding", onboardingRouter); app.use("/api/pairing", pairingRouter); app.use("/api/parent", parentRouter); // Serve web UI static files const webDistPath = process.env.WEB_DIST_PATH || path.join(process.cwd(), "../web/dist"); app.use(express.static(webDistPath)); // Health check app.get("/api/health", (_req, res) => { res.json({ status: "ok" }); }); // SPA fallback: serve index.html for non-API GET requests app.get("*", (req, res, next) => { if (req.path.startsWith("/api/")) return next(); res.sendFile(path.join(webDistPath, "index.html"), (err) => { if (err) next(); }); }); // Global error handler app.use( ( err: Error, _req: express.Request, res: express.Response, _next: express.NextFunction ) => { console.error("Unhandled error:", err); res.status(500).json({ error: "Internal server error" }); } ); app.listen(PORT, async () => { const isDev = process.env.NODE_ENV === "development"; console.log(`Pebble backend listening on port ${PORT}${isDev ? " \x1b[33m[DEV MODE — verbose logging enabled]\x1b[0m" : ""}`); if (isDev) { try { const ngrok = await import("@ngrok/ngrok"); const qrcode = (await import("qrcode-terminal")).default; const fs = await import("fs"); const os = await import("os"); const path = await import("path"); // Read authtoken from ngrok CLI config file let authtoken: string | undefined; const configPaths = [ path.join(os.homedir(), "Library/Application Support/ngrok/ngrok.yml"), path.join(os.homedir(), ".config/ngrok/ngrok.yml"), path.join(os.homedir(), ".ngrok2/ngrok.yml"), ]; for (const p of configPaths) { try { const content = fs.readFileSync(p, "utf-8"); const match = content.match(/authtoken:\s*(.+)/); if (match) { authtoken = match[1].trim(); break; } } catch { /* try next */ } } const listener = await ngrok.forward({ addr: Number(PORT), ...(authtoken ? { authtoken } : {}), }); const url = listener.url()!; console.log(`\n\x1b[36m🌐 ngrok tunnel: ${url}\x1b[0m\n`); qrcode.generate(url, { small: true }, (code: string) => { console.log(code); }); } catch (err) { console.warn("\x1b[33m⚠ Failed to start ngrok tunnel:\x1b[0m", (err as Error).message); } } }); ``` [161 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]