# Project export: CareSync

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2026
- Tagline: Autonomous AI agents that watch over your grandparents' health 24/7, so you don't have to worry.
- Devpost: https://devpost.com/software/test-zdxlf0
- GitHub: https://github.com/SpartaKushK/treehacks2026
- Demo: https://treehacks2026-nine.vercel.app/
- Video: https://www.youtube.com/embed/0mPj7B3E9ec?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 7 GitHub contributor(s) — Brian Tong (30 commits), EdisonZhangUCSD (13 commits), Kush Khamesra (8 commits), Claude Opus 4.6 (4 commits), platawa (4 commits), Cursor (3 commits), EdisonZhangUCSD (2 commits)

## Devpost submission (written by the team)

### Inspiration

We were inspired by the challenge faced by elderly citizens, like our grandparents, who live alone and for whom small changes in health vitals can go unnoticed — even when they are early warning signs of serious complications. Given that 65 million Americans wear a smartwatch that collects heart rate, sleep, and activity data, we wanted to create a personal assistant tool that can take actions based on health trends. Healthcare is slow, and by the time someone notices a problem or books an appointment, it’s often too late for effective intervention. This inspired us to create an autonomous way to facilitate doctor and patient interactions for elderly citizens. What It Does CareSync gives every patient an autonomous AI care agent, exposed as a secure API endpoint, that monitors their health and coordinates care on their behalf. For Patients, Your Agent Ingests real Apple Watch data via an iOS + WatchOS app connected to HealthKit (9 data types including steps, heart rate, sleep stages, workouts, and irregular rhythm events). Detects anomalies automatically using a 28-day rolling baseline (mean ± std), flagging events like resting heart rate spikes and sleep drops with a 0–100 severity score. Makes autonomous decisions through a Secretary Agent using Claude's Agent SDK that weighs severity, urgency, and medical context by querying PubMed and FDA sources and escalates to a doctor’s receptionist agent. Books appointments through agent-to-agent negotiation via a 3-turn propose/counter/confirm protocol, with Ed25519 cryptographic signatures on every message. Communicates through HeyGen streaming avatars in 12+ languages (Deepgram STT + ElevenLabs TTS), allowing a Mandarin-speaking grandmother to have face-to-face conversations with her AI doctor. For Doctors, Your Agent Receives patient escalations and triages them through a LangGraph state machine using Claude. Checks Google Calendar availability and proposes appointment slots. Proposes and books appointments with full clinical context. The Platform Provides A registry where every person has a canonical API endpoint (/u/:handle). Ed25519 cryptographic signatures on every agent-to-agent request with replay protection. Policy engine support for caller whitelists, scope-based permissions, and payment gating. Full execution traces showing exactly what agents did and why. CareSync is a multi-agent orchestration and healthcare platform where each person becomes an agent endpoint. We transform static wearable health data into actionable outcomes with structured summaries, risk modeling, and scheduling. This reduces missed appointments, preventable deterioration, administrative stress, and care delays. How We Built It Architecture Monorepo with three main layers: Next.js 14 frontend (deployed on Vercel) Python Doctor Agent microservice (FastAPI) Shared type system (@people/shared) that keeps both sides synchronized through Pydantic models and Zod schemas Web Platform (Next.js 14) React 18 TailwindCSS Clerk authentication Supabase via Prisma The centerpiece is a "Secretary Agent,” an LLM orchestrator using Claude Sonnet 4.5 or GPT-4o with function-calling to: Analyze anomalies Triage patients Schedule appointments Generate health summaries Doctor Agent (FastAPI + LangGraph) A state machine modeling triage as a multi-turn workflow: receive alert → classify severity → ask follow-ups → decide action → dispatch Claude handles clinical reasoning in a multi-tool loop where it: Checks for life-threatening conditions Classifies severity on a four-tier scale Determines appointment type Checks real Google Calendar availability HeyGen Streaming Interface Instead of notifications, patients can interact with real-time streaming agents (Deepgram STT + ElevenLabs TTS) that explain anomalies and walk through next steps in 12+ languages. Claude’s reasoning outputs feed into HeyGen’s API for personalized, conversational explanations. Agent-to-Agent Communication Ed25519 cryptographic signatures on every request Canonically serialized JSON (sorted keys) Signed and verified with nonce-based replay protection 5-minute timestamp windows Policy engine validates permissions before execution Pipeline Biometric data arrives → 28-day personal baseline computed (mean ± std). Patient agent calculates anomaly score (0–100) and flags concerns. Claude evaluates anomaly, decides escalation, and drafts clinic messages. Doctor agent runs multi-turn intake, checks availability, and books appointments. Full trace persisted: alert, decision, triage outcome, calendar event. Challenges We Ran Into Incorporating multi-agent orchestration and negotiation amongst agents. A singular LLM endpoint became a bottleneck during early development. Google Calendar configuration issues: the bot created events on an invisible calendar due to hallucinated state; we learned we must explicitly specify the correct Google Calendar each time. After booking confirmation, appointments sometimes did not appear or double-booked. Our first trace system only logged "triage: ok" instead of a full traceback, limiting interpretability. Time constraints prevented building full tracing infrastructure like Langfuse. The AI makes decisions inside its own internal loop, so we had to hook into every step to capture reasoning, tool calls, and responses. Context management was difficult due to frequent pivots in use case and workflow. Accomplishments We’re Proud Of We built CareSync because our grandparents deserve the same quality of monitoring that a hospital ICU provides — but from the comfort of their home, in their own language, through a face they can trust. We built an end-to-end pipeline that actually works: A simulated Apple Watch anomaly goes in → a real Google Calendar appointment comes out. The patient agent detects the problem. The doctor agent triages it across multiple turns. A real event with clinical context is proposed and booked on the doctor's calendar. We built platform infrastructure for a healthcare agent ecosystem. The “People API” concept includes a registry, policy engine, capability discovery, and permission model. Another developer could register an agent today, define capabilities, set access policies, and plug into the network. We bridged the gap between passive data and active intervention — turning watch data into a doctor’s appointment through anomaly detection, clinical reasoning, scheduling negotiation, and calendar integration in a multi-turn agent reasoning framework. What’s Next for People API Multi-doctor routing: match patients to the right specialist based on triage outcome, availability, insurance, and location. Patient consent controls: granular settings for what your agent can share, with whom, and under what conditions. HIPAA compliance: privacy and security for clinical deployment. Tech Stack Frontend Next.js 14 (App Router) React TailwindCSS Backend Next.js API Routes FastAPI (Python) LangGraph Database Supabase PostgreSQL via Prisma Auth Clerk LLMs Anthropic Claude Sonnet (primary) OpenAI GPT-4o-mini Agent Framework Claude Agent SDK with MCP tool server Voice & Streaming Deepgram (STT) ElevenLabs (TTS) HeyGen (streaming avatar) Mobile Swift / SwiftUI HealthKit WatchOS Crypto Ed25519 (tweetnacl) for agent-to-agent signing Clinical Data Sources PubMed (NCBI) openFDA AHA / CDC / WHO guidelines Google Calendar API Deployment & Infrastructure Vercel pnpm workspaces (monorepo) Zod schemas Pydantic v2 Canonical JSON serialization

## README (from the GitHub repository)

# CareSync — AI-Powered Elderly Care Platform

**TreeHacks 2026**

CareSync is an end-to-end healthcare platform where AI agents autonomously monitor patient health, detect anomalies, triage alerts, consult patients via a talking avatar, and book appointments — all with full observability and cryptographic trust.

## Architecture

```
/
  apps/web/              Next.js 14 (App Router) — Dashboard + Patient UI + API
  apps/api/              Python FastAPI — Doctor Agent backend
  ios/                   iOS + Apple Watch app (HealthKit integration)
  packages/shared/       Shared TypeScript types, Zod schemas, LLM abstraction
  mcp-server/            Model Context Protocol server for healthcare tools
  Doctor_Agent_for_TreeHacks/   Doctor Agent (scheduling, triage, calendar)
```

## Quick Start

```bash
pnpm install
cd apps/web && npx prisma db push && cd ../..
pnpm dev
```

Open [http://localhost:3000](http://localhost:3000).

The database auto-seeds on first API request with users (pari, alex, dr_smith) and 30 days of health metrics.

## Demo Flow

### 1. Watch Trigger
Tap **"Trigger Alert"** on the Apple Watch. Haptic feedback confirms. Synthetic health data (low sleep, high resting HR, irregular heart rhythm) uploads to the backend via `POST /api/health-data`.

### 2. Clinical Dashboard — Anomaly Detection
On the Clinical Dashboard (`/dashboard/anomaly`), the anomaly appears in real-time with:
- Severity score (0–100)
- Flags: `SLEEP_DROP`, `RHR_SPIKE`, `STEPS_DROP`, `SYMPTOM_SPIKE`
- Agent's autonomous triage decision and urgency level
- Live patient metrics (auto-refreshes every 30s)

### 3. Trace Inspection
Click into any alert to view the full chain:
1. **Anomaly Detection** — flag detection and scoring from wearable data
2. **Secretary Agent Reasoning** — LLM-powered analysis with tool-calling
3. **PubMed Evidence Lookup** — clinical studies and guidelines from NCBI
4. **Triage Decision** — intake questions, urgency scoring, escalation
5. **Appointment Booking** — slot negotiation and calendar integration

Each step is logged with actor, event type, timestamp, and full data payload.

### 4. Patient Voice Consultation
At `/patient/call`, the patient speaks with **Dr. Smith** — a HeyGen streaming avatar:
- Context-aware greeting referencing the specific anomaly (elevated HR, low sleep)
- Real-time voice conversation via Deepgram STT + HeyGen TTS
- AI doctor responses generated by Claude with the full health context
- Live transcript and clinical entity extraction (symptoms, medications, allergies)
- Red flag detection and automated triage actions
- 12-language support

### 5. Appointment Booking
The patient's agent and doctor's agent negotiate appointment slots autonomously:
- Scheduler Agent checks Google Calendar availability
- Doctor Agent proposes slots from the doctor's calendar
- Patient Agent accepts and confirms
- Booking appears on Google Calendar and in the patient's appointments view at `/patient/appointments`

## Environment Variables

Copy `.env.example` to `.env.local`:

```bash
cp apps/web/.env.example apps/web/.env.local
```

| Variable | Required | Description |
|---|---|---|
| `ANTHROPIC_API_KEY` | Yes | Claude LLM for agent reasoning, voice responses, entity extraction |
| `OPENAI_API_KEY` | No | Alternative LLM provider (GPT-4o-mini) |
| `HEYGEN_API_KEY` | Yes | Streaming avatar for Dr. Smith voice consultations |
| `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` | Yes | Clerk authentication |
| `CLERK_SECRET_KEY` | Yes | Clerk server-side auth |
| `SUPABASE_URL` | Yes | Supabase project URL for health data storage |
| `SUPABASE_SERVICE_ROLE_KEY` | Yes | Supabase service role key |
| `DATABASE_URL` | Yes | Postgres connection string (Supabase) |
| `GOOGLE_CLIENT_ID` | No | Google Calendar OAuth |
| `GOOGLE_CLIENT_SECRET` | No | Google Calendar OAuth |
| `NEXT_PUBLIC_BASE_URL` | No | Defaults to `http://localhost:3000` |

## Deploy to Vercel

1. Push to GitHub
2. Import in [vercel.com/new](https://vercel.com/new)
3. Set root directory to `apps/web`
4. Add environment variables (see table above)
5. Deploy — the build runs `prisma generate && next build`

## Key API Endpoints

| Endpoint | Method | Description |
|---|---|---|
| `/api/health-data` | POST | Receive health data from iOS/Watch, run anomaly detection |
| `/api/trigger` | POST | Secretary Agent — route health triggers to sub-tools |
| `/api/anomaly/live` | GET | Live patient metrics for the clinical dashboard |
| `/api/anomaly/history` | GET | Alert history with filtering and pagination |
| `/api/voice/respond` | POST | Generate doctor responses + extract clinical entities |
| `/api/heygen/token` | POST | Create HeyGen streaming session tokens |
| `/api/calendar/events` | GET | Fetch calendar events for a patient |
| `/api/patient/agent` | POST | Patient agent stub (receives slot proposals from Doctor Agent) |
| `/api/demo/trace/:traceId` | GET | Retrieve full trace for inspection |

## Secretary Agent

The Secretary Agent is an LLM-powered orchestrator that receives health triggers and autonomously chains tools:

- `analyze_anomaly` — evaluates health anomaly severity and urgency
- `lookup_clinical_evidence` — searches PubMed and clinical guidelines
- `triage_patient` — runs intake questions, books appointment if needed
- `get_health_summary` — retrieves 30-day health trends
- `schedule_appointment` — finds available slots and books via Google Calendar
- `notify_doctor_agent` — alerts the Doctor Agent for scheduling negotiation

Supports both Anthropic (Claude) and OpenAI (GPT-4o) with automatic fallback.

## iOS / Apple Watch App

- **HealthKit Integration** — reads steps, heart rate, sleep, active energy, workouts, health events
- **Trigger Alert** — sends synthetic anomaly data to the backend for pipeline testing
- **Haptic Feedback** — success/failure haptics on the Watch
- **Auto-Sync** — periodic health data uploads

## Tech Stack

- **Monorepo**: pnpm workspaces
- **Web**: Next.js 14 (App Router), React 18, TypeScript, Tailwind CSS
- **Auth**: Clerk
- **DB**: Supabase (Postgres) + Prisma ORM
- **LLM**: Anthropic Claude (Messages API + Agent SDK), OpenAI GPT-4o
- **Voice**: HeyGen Streaming Avatar, Deepgram STT, ElevenLabs TTS
- **iOS**: Swift, SwiftUI, HealthKit, WatchKit
- **Signing**: Ed25519 via tweetnacl
- **Clinical**: PubMed E-utilities, AHA/CDC/WHO guidelines
- **Calendar**: Google Calendar API (OAuth 2.0)
- **Schemas**: Zod


## Detected evidence (automated analysis)

Indexed codebase: 248 recognized source files, 1377 KB.
- Anthropic (technology) — detected in the code
- CSS (language) — detected in the code
- FastAPI (technology) — detected in the code
- JavaScript (language) — detected in the code
- LangChain (technology) — detected in the code
- Next.js (technology) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- SQL (language) — detected in the code
- Supabase (technology) — detected in the code
- Swift (language) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- PostgreSQL (technology) — claimed on Devpost, not found in the code
- Vercel (technology) — claimed on Devpost, not found in the code
- AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers
- AI coding agent: Cursor — evidence: config files committed to the repository; commit authorship or trailers

## Codebase structure (from repository index)

### Files (120 of 282)

```
.claude/settings.local.json
.env.example
.gitignore
.npmrc
apps/api/.env.example
apps/api/app/__init__.py
apps/api/app/canonical_json.py
apps/api/app/capabilities/__init__.py
apps/api/app/capabilities/dispatcher.py
apps/api/app/capabilities/health.py
apps/api/app/capabilities/scheduling.py
apps/api/app/capabilities/triage.py
apps/api/app/config.py
apps/api/app/crypto.py
apps/api/app/db.py
apps/api/app/deps.py
apps/api/app/llm/__init__.py
apps/api/app/llm/base.py
apps/api/app/llm/claude.py
apps/api/app/llm/deterministic.py
apps/api/app/llm/openai_planner.py
apps/api/app/main.py
apps/api/app/models.py
apps/api/app/personas/__init__.py
apps/api/app/personas/doctor_receptionist.py
apps/api/app/personas/patient_health.py
apps/api/app/policy.py
apps/api/app/routers/__init__.py
apps/api/app/routers/agents.py
apps/api/app/routers/anomaly.py
apps/api/app/routers/auth.py
apps/api/app/routers/calendar.py
apps/api/app/routers/demo.py
apps/api/app/routers/google_oauth.py
apps/api/app/routers/health_data.py
apps/api/app/routers/heygen.py
apps/api/app/routers/invoke.py
apps/api/app/routers/registry.py
apps/api/app/seed.py
apps/api/app/trace.py
apps/api/pyproject.toml
apps/api/schema.sql
apps/api/supabase/.gitignore
apps/api/supabase/config.toml
apps/api/supabase/migrations/20240101000000_init.sql
apps/api/treehacks_api.egg-info/dependency_links.txt
apps/api/treehacks_api.egg-info/PKG-INFO
apps/api/treehacks_api.egg-info/requires.txt
apps/api/treehacks_api.egg-info/SOURCES.txt
apps/api/treehacks_api.egg-info/top_level.txt
apps/web/.env.example
apps/web/.gitignore
apps/web/app/api/agents/[handle]/config/route.ts
apps/web/app/api/agents/route.ts
apps/web/app/api/anomaly/[alertId]/dismiss/route.ts
apps/web/app/api/anomaly/[alertId]/resolve/route.ts
apps/web/app/api/anomaly/history/route.ts
apps/web/app/api/anomaly/live/route.ts
apps/web/app/api/auth/webhook/route.ts
apps/web/app/api/calendar/events/route.ts
apps/web/app/api/calendar/sync/route.ts
apps/web/app/api/chat/clear/route.ts
apps/web/app/api/chat/history/route.ts
apps/web/app/api/chat/review/history/route.ts
apps/web/app/api/chat/review/route.ts
apps/web/app/api/chat/route.ts
apps/web/app/api/demo/anomaly/route.ts
apps/web/app/api/demo/health/route.ts
apps/web/app/api/demo/schedule/route.ts
apps/web/app/api/demo/seed/route.ts
apps/web/app/api/demo/trace/[traceId]/route.ts
apps/web/app/api/doctor/alert/route.ts
apps/web/app/api/evidence/route.ts
apps/web/app/api/google/callback/route.ts
apps/web/app/api/google/connect/route.ts
apps/web/app/api/health-data/route.ts
apps/web/app/api/health-records/route.ts
apps/web/app/api/heygen/avatars/route.ts
apps/web/app/api/heygen/token/route.ts
apps/web/app/api/patient/agent/route.ts
apps/web/app/api/registry/lookup/[handle]/route.ts
apps/web/app/api/registry/register/route.ts
apps/web/app/api/trigger/route.ts
apps/web/app/api/u/[handle]/caps/route.ts
apps/web/app/api/u/[handle]/invoke/route.ts
apps/web/app/api/voice/respond/route.ts
apps/web/app/api/voice/speech-to-text/route.ts
apps/web/app/api/voice/summary/route.ts
apps/web/app/api/voice/text-to-speech/route.ts
apps/web/app/dashboard/agents/[handle]/page.tsx
apps/web/app/dashboard/agents/page.tsx
apps/web/app/dashboard/anomaly/[traceId]/page.tsx
apps/web/app/dashboard/anomaly/page.tsx
apps/web/app/dashboard/calendar/page.tsx
apps/web/app/dashboard/chat/page.tsx
apps/web/app/dashboard/demo/page.tsx
apps/web/app/dashboard/layout.tsx
apps/web/app/dashboard/page.tsx
apps/web/app/dashboard/records/page.tsx
apps/web/app/dashboard/voice-demo/page.tsx
apps/web/app/globals.css
apps/web/app/layout.tsx
apps/web/app/page.tsx
apps/web/app/patient/alert/page.tsx
apps/web/app/patient/appointments/page.tsx
apps/web/app/patient/booking/page.tsx
apps/web/app/patient/call/page.tsx
apps/web/app/patient/call/summary/page.tsx
apps/web/app/patient/components/PatientNav.tsx
apps/web/app/patient/evidence/page.tsx
apps/web/app/patient/family/page.tsx
apps/web/app/patient/health/page.tsx
apps/web/app/patient/history/page.tsx
apps/web/app/patient/layout.tsx
apps/web/app/patient/onboarding/page.tsx
apps/web/app/patient/page.tsx
apps/web/app/patient/settings/page.tsx
apps/web/app/patient/triage/page.tsx
apps/web/app/patient/voice-demo/page.tsx
apps/web/app/patient/voice-integration-example/page.tsx
[162 more files omitted for size]
```

### Dependencies

- apps/api/pyproject.toml: anthropic@>=0.40.0, fastapi[standard]@>=0.115.0, httpx@>=0.27.0, openai@>=1.50.0, pydantic-settings@>=2.0.0, pyjwt[crypto]@>=2.9.0, pynacl@>=1.5.0, supabase@>=2.0.0, uvicorn[standard]@>=0.30.0
- apps/web/package.json: @anthropic-ai/claude-agent-sdk@^0.2.42, @clerk/nextjs@^6.37.4, @heygen/streaming-avatar@^2.1.0, @people/shared@workspace:*, @prisma/client@^5.9.1, @radix-ui/react-avatar@^1.1.11, @radix-ui/react-dialog@^1.1.15, @radix-ui/react-label@^2.1.8, @radix-ui/react-progress@^1.1.8, @radix-ui/react-radio-group@^1.3.8, @radix-ui/react-separator@^1.1.8, @radix-ui/react-slot@^1.2.4, @radix-ui/react-switch@^1.2.6, @radix-ui/react-tabs@^1.1.13, @supabase/supabase-js@^2.95.3, @types/node@^20.11.5, @types/qrcode@^1.5.6, @types/react@^18.2.48, @types/react-dom@^18.2.18, @types/uuid@^9.0.7, autoprefixer@^10.4.24, class-variance-authority@^0.7.1, clsx@^2.1.1, framer-motion@^12.34.0, lucide-react@^0.564.0, next@^14.1.0, postcss@^8.5.6, prisma@^5.9.1, qrcode@^1.5.4, react@^18.2.0, react-dom@^18.2.0, react-markdown@^10.1.0, tailwind-merge@^3.4.0, tailwindcss@^3.4.19, tailwindcss-animate@^1.0.7, tsx@^4.7.0, tweetnacl@^1.0.3, tweetnacl-util@^0.15.1, typescript@^5.3.3, uuid@^9.0.0, zod@^3.22.4
- Doctor_Agent_for_TreeHacks/requirements.txt: anthropic@>=0.40.0, fastapi@>=0.115.0, google-api-python-client@>=2.140.0, google-auth-httplib2@>=0.2.0, google-auth-oauthlib@>=1.2.0, httpx@>=0.27.0, langchain@>=0.3.0, langchain-core@>=0.3.0, langgraph@>=0.2.0, modal@>=0.64.0, pydantic@>=2.7.0, python-dotenv@>=1.0.0, sendgrid@>=6.11.0, uvicorn[standard]@>=0.30.0
- mcp-server/package.json: @modelcontextprotocol/sdk@^1.26.0, @types/node@^20.11.5, poke@^0.2.0, tsx@^4.7.0, typescript@^5.3.3, zod@^3.22.4
- packages/shared/package.json: typescript@^5.3.3, zod@^3.22.4

### Recent commits (newest first)

- Prefer Anthropic over OpenAI across all LLM paths
- final summary handling
- readme
- don't use agentsdk
- call flags
- build
- handle voice transcript
- views
- route
- Merge pull request #11 from SpartaKushK/edison
- Fix dashboard layout after merge
- Update .gitignore
- Merge branch 'main' into edison
- calendar works
- asjgka
- styling
- fix inf load
- added changes
- build
- env changes

## Key source files (fetched from GitHub, selected and truncated for size)

### VOICE_QUICK_START.md

```markdown
# 🎤 Voice AI - Quick Start

## 1️⃣ Get API Keys (5 minutes)

### Required
```bash
# ElevenLabs (Text-to-Speech)
# Sign up: https://elevenlabs.io
ELEVENLABS_API_KEY=sk_xxxxx
```

### Choose One
```bash
# Option A: Deepgram (Recommended - Faster)
# Sign up: https://deepgram.com
DEEPGRAM_API_KEY=xxxxx

# Option B: OpenAI Whisper
# Sign up: https://platform.openai.com
OPENAI_API_KEY=sk-xxxxx
```

## 2️⃣ Add to .env.local

```bash
# Copy example file
cp .env.example .env.local

# Add your keys
DEEPGRAM_API_KEY=your_key_here
ELEVENLABS_API_KEY=your_key_here
```

## 3️⃣ Test It (2 minutes)

```bash
# Start dev server
npm run dev

# Visit these pages:
# 1. Voice Demo
open http://localhost:3000/patient/voice-demo

# 2. Simple Voice Chat
open http://localhost:3000/patient/voice-simple

# 3. Enhanced Chat
open http://localhost:3000/dashboard/chat
```

## 4️⃣ Use in Your Code

### Add Voice to Any Page

```tsx
import VoiceChat from "@/components/VoiceChat";

function MyPage() {
  return (
    <VoiceChat
      onTranscript={(text) => console.log(text)}
      language="en"
    />
  );
}
```

### Add Speak Button to Text

```tsx
import VoiceOutput from "@/components/VoiceOutput";

function MyMessage({ text }: { text: string }) {
  return (
    <div>
      <p>{text}</p>
      <VoiceOutput text={text} />
    </div>
  );
}
```

### Add Language Selector

```tsx
import LanguageSelector from "@/components/LanguageSelector";
import { useState } from "react";

function MyPage() {
  const [lang, setLang] = useState("en");

  return (
    <LanguageSelector
      currentLanguage={lang}
      onLanguageChange={setLang}
    />
  );
}
```

## 5️⃣ Try These Examples

### Voice-Enabled Chat
Visit: `/dashboard/chat`
- Click microphone 🎤
- Speak your message
- It auto-sends and responds with voice

### Interactive Demo
Visit: `/patient/voice-demo`
- Test speech-to-text
- Test text-to-speech
- Try different languages

### Simple Voice Chat
Visit: `/patient/voice-simple`
- Minimal voice conversation
- Clean, simple UI

## 🎯 Common Use Cases

### 1. Voice Messages in Chat
```tsx
<VoiceChat
  onTranscript={(text) => sendMessage(text)}
  language={userLanguage}
/>
```

### 2. Read Messages Aloud
```tsx
{messages.map(msg => (
  <div key={msg.id}>
    {msg.text}
    <VoiceOutput text={msg.text} language="en" />
  </div>
))}
```

### 3. Voice Commands
```tsx
<VoiceChat
  onTranscript={(text) => {
    if (text.includes("show appointments")) {
      router.push("/appointments");
    }
  }}
/>
```

## 🌍 Supported Languages

```tsx
const languages = {
  "en": "English 🇺🇸",
  "es": "Spanish 🇪🇸",
  "fr": "French 🇫🇷",
  "de": "German 🇩🇪",
  "zh": "Chinese 🇨🇳",
  "ja": "Japanese 🇯🇵",
  // + 6 more...
};
```

## 💰 Free Tier Limits

| Service | Free Tier | Cost After |
|---------|-----------|------------|
| Deepgram | 45,000 min/year | $0.0125/min |
| ElevenLabs | 10,000 chars/month | $0.18/1000 chars |
| OpenAI Whisper | None | $0.006/min |

**Good for ~1,500 voice chats/month 
[truncated — 771 more characters]
```

### POKE_KITCHEN_SETUP.md

```markdown
# CareSync - Poke Kitchen Configuration Guide

## Your MCP Server Details
- **Tunnel URL**: `https://tunnel.poke.com/9c6c02a5-e81d-4885-9664-7408066911c6/mcp`
- **Recipe Name**: CareSync
- **Server Name**: caresync-health

---

## Kitchen Setup Instructions

### 1. Create Your Recipe
1. Go to [poke.com/kitchen](https://poke.com/kitchen)
2. Click **"Create recipe"**
3. Follow these settings:

---

### 2. BASICS Configuration

**Recipe Name**: `CareSync - Your AI Health Secretary`

**Description** (optional but recommended):
```
Never miss a health warning. Your AI monitors your health 24/7 and automatically coordinates care when something's wrong.
```

---

### 3. ONBOARDING Configuration

#### **Input Context** (`inputContext`):
```
I'm your AI health secretary. I'll help you:
• Monitor your health metrics 24/7
• Detect concerning patterns before they become emergencies
• Automatically schedule appointments when needed
• Explain what's happening with your health in clear terms

To get started, I need to understand your health monitoring needs.

What brings you to CareSync?
1️⃣ I have a chronic condition I'm managing
2️⃣ I want to monitor my overall health
3️⃣ I'm tracking recovery from illness/injury
4️⃣ Just exploring (demo mode)

Reply with a number or describe your situation.
```

#### **Prefilled First Message** (`prefilledFirstText`):
```
Show me my health summary for the last 30 days
```

**Alternative options to suggest:**
- "Show me my health summary for the last 30 days"
- "Are there any concerning patterns in my recent data?"
- "Run a health check on my wearable data"
- "Schedule a checkup with my doctor"

---

### 4. INTEGRATIONS Configuration

Select these integrations:

✅ **Your MCP Server**: `caresync-health`
- This is your tunnel URL: `https://tunnel.poke.com/9c6c02a5-e81d-4885-9664-7408066911c6/mcp`
- Mark as: "Share with users (no setup required)" ← IMPORTANT for easy onboarding

Optional (if you want to expand):
- **Google Calendar** - for appointment scheduling
- **Apple Health** - for health data (if Poke supports)
- **Google Fit** - for health data (if Poke supports)

---

### 5. AUTOMATIONS Configuration

Add these scheduled automations:

#### **Automation 1: Daily Health Check**
- **Schedule**: Every day at 10:00 PM
- **Cron**: `0 22 * * *`
- **Action Description**:
```
Run anomaly detection on today's health metrics. If anomaly score > 85, immediately analyze and triage. If moderate concern (score 55-85), log for review. Text me if action is needed.
```

#### **Automation 2: Weekly Health Summary**
- **Schedule**: Every Sunday at 9:00 AM
- **Cron**: `0 9 * * 0`
- **Action Description**:
```
Generate and send a comprehensive 7-day health summary including sleep trends, activity patterns, and any concerning changes. Include AI-generated patient-friendly explanation.
```

#### **Automation 3: Appointment Reminder Check**
- **Schedule**: Every day at 8:00 AM
- **Cron**: `0 8 * * *`
- **Action Description**:
```
Check for any s
[truncated — 2183 more characters]
```

### package.json

```
{
  "name": "people-api",
  "private": true,
  "scripts": {
    "dev": "pnpm --filter web dev",
    "build": "pnpm --filter web build",
    "seed": "pnpm --filter web seed",
    "db:push": "pnpm --filter web db:push",
    "mcp:dev": "npx tsx mcp-server/server.ts",
    "mcp:tunnel": "npx poke tunnel http://localhost:8787/mcp --name CareSync"
  },
  "pnpm": {
    "onlyBuiltDependencies": [
      "@clerk/shared",
      "@prisma/client",
      "@prisma/engines",
      "prisma",
      "esbuild",
      "protobufjs"
    ]
  }
}

```

### Doctor_Agent_for_TreeHacks/requirements.txt

```
# Core
fastapi>=0.115.0
uvicorn[standard]>=0.30.0
python-dotenv>=1.0.0
pydantic>=2.7.0
httpx>=0.27.0

# LLM + Agent Orchestration
anthropic>=0.40.0
langchain>=0.3.0
langchain-core>=0.3.0
langgraph>=0.2.0

# Google Calendar
google-api-python-client>=2.140.0
google-auth-httplib2>=0.2.0
google-auth-oauthlib>=1.2.0

# Email (optional, for forms)
sendgrid>=6.11.0

# Modal deployment (optional, for production)
modal>=0.64.0

```

### mcp-server/package.json

```
{
  "name": "caresync-mcp-server",
  "version": "1.0.0",
  "description": "MCP server for CareSync healthcare agent platform — exposes health tools via Poke",
  "type": "module",
  "scripts": {
    "dev": "npx tsx server.ts",
    "build": "tsc",
    "start": "node dist/server.js"
  },
  "dependencies": {
    "@modelcontextprotocol/sdk": "^1.26.0",
    "poke": "^0.2.0",
    "zod": "^3.22.4"
  },
  "devDependencies": {
    "@types/node": "^20.11.5",
    "tsx": "^4.7.0",
    "typescript": "^5.3.3"
  }
}

```

### packages/shared/package.json

```
{
  "name": "@people/shared",
  "version": "0.0.1",
  "private": true,
  "main": "./src/index.ts",
  "types": "./src/index.ts",
  "scripts": {},
  "dependencies": {
    "zod": "^3.22.4"
  },
  "devDependencies": {
    "typescript": "^5.3.3"
  }
}

```

### apps/api/pyproject.toml

```
[project]
name = "treehacks-api"
version = "0.1.0"
description = "FastAPI backend for TreeHacks"
requires-python = ">=3.11"
dependencies = [
    "fastapi[standard]>=0.115.0",
    "uvicorn[standard]>=0.30.0",
    "supabase>=2.0.0",
    "pynacl>=1.5.0",
    "pydantic-settings>=2.0.0",
    "httpx>=0.27.0",
    "anthropic>=0.40.0",
    "openai>=1.50.0",
    "pyjwt[crypto]>=2.9.0",
]

```

### apps/web/package.json

```
{
  "name": "web",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "prisma generate && next build",
    "start": "next start",
    "seed": "npx tsx lib/seed.ts",
    "db:push": "prisma db push"
  },
  "dependencies": {
    "@anthropic-ai/claude-agent-sdk": "^0.2.42",
    "@clerk/nextjs": "^6.37.4",
    "@heygen/streaming-avatar": "^2.1.0",
    "@people/shared": "workspace:*",
    "@prisma/client": "^5.9.1",
    "@radix-ui/react-avatar": "^1.1.11",
    "@radix-ui/react-dialog": "^1.1.15",
    "@radix-ui/react-label": "^2.1.8",
    "@radix-ui/react-progress": "^1.1.8",
    "@radix-ui/react-radio-group": "^1.3.8",
    "@radix-ui/react-separator": "^1.1.8",
    "@radix-ui/react-slot": "^1.2.4",
    "@radix-ui/react-switch": "^1.2.6",
    "@radix-ui/react-tabs": "^1.1.13",
    "@supabase/supabase-js": "^2.95.3",
    "@types/qrcode": "^1.5.6",
    "class-variance-authority": "^0.7.1",
    "clsx": "^2.1.1",
    "framer-motion": "^12.34.0",
    "lucide-react": "^0.564.0",
    "next": "^14.1.0",
    "qrcode": "^1.5.4",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-markdown": "^10.1.0",
    "tailwind-merge": "^3.4.0",
    "tailwindcss-animate": "^1.0.7",
    "tweetnacl": "^1.0.3",
    "tweetnacl-util": "^0.15.1",
    "uuid": "^9.0.0",
    "zod": "^3.22.4"
  },
  "devDependencies": {
    "@types/node": "^20.11.5",
    "@types/react": "^18.2.48",
    "@types/react-dom": "^18.2.18",
    "@types/uuid": "^9.0.7",
    "autoprefixer": "^10.4.24",
    "postcss": "^8.5.6",
    "prisma": "^5.9.1",
    "tailwindcss": "^3.4.19",
    "tsx": "^4.7.0",
    "typescript": "^5.3.3"
  }
}

```

### mcp-server/server.ts

```typescript
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
import { createServer } from "http";
import { z } from "zod";

const NEXTJS_URL = process.env.NEXTJS_URL || "http://localhost:3000";

// ── Tool definitions ────────────────────────────────────────────

function registerTools(server: McpServer) {
  // 1. Health Summary
  server.tool(
    "get_health_summary",
    "Retrieve a 30-day health summary for a patient. Includes sleep, activity, medication adherence, symptom trends, and an AI-generated patient-friendly explanation.",
    {
      patient_handle: z.string().default("pari").describe("The patient's handle (e.g. 'pari')"),
      provider: z.enum(["openai", "claude"]).default("claude").describe("LLM provider for generating the summary"),
    },
    async ({ patient_handle, provider }) => {
      try {
        const url = `${NEXTJS_URL}/api/demo/health?patient=${encodeURIComponent(patient_handle)}&provider=${provider}`;
        const res = await fetch(url);
        const data = await res.json();

        if (data.error) {
          return { content: [{ type: "text", text: `Error: ${data.error}` }], isError: true };
        }

        const s = data.healthSummary;
        const lines = [
          `Health Summary for ${patient_handle}`,
          `Period: ${s.rangeDays} days`,
          ``,
          `Sleep: avg ${s.sleep.avg}h/night (trend: ${s.sleep.trend})`,
          `Activity: avg ${s.activity.avgSteps} steps/day (trend: ${s.activity.trend})`,
          `Medication Adherence: ${s.medication.adherencePct}% (${s.medication.missedDays} missed days)`,
          `Symptoms: avg score ${s.symptoms.avgScore}/10`,
        ];
        if (s.notes?.length > 0) {
          lines.push(``, `Notes:`, ...s.notes.map((n: string) => `  - ${n}`));
        }
        if (s.patientFriendlyText) {
          lines.push(``, `AI Summary: ${s.patientFriendlyText}`);
        }
        lines.push(``, `Trace ID: ${data.traceId}`);

        return { content: [{ type: "text", text: lines.join("\n") }] };
      } catch (err) {
        return {
          content: [{ type: "text", text: `CareSync app not reachable at ${NEXTJS_URL}. Make sure the Next.js app is running.` }],
          isError: true,
        };
      }
    }
  );

  // 2. Analyze Anomaly
  server.tool(
    "analyze_anomaly",
    "Run the full anomaly detection pipeline on sample wearable data. Analyzes severity, triages the patient, and may schedule an appointment automatically. Choose 'severe' for a high anomaly score (92/100) or 'moderate' for a medium score (55/100).",
    {
      severity: z.enum(["severe", "moderate"]).default("severe").describe("Anomaly severity level for the demo"),
      provider: z.enum(["openai", "claude"]).default("claude").describe("LLM provider for analysis"),
    },
    async ({ severity, provider }) => {
      try {
        const url = `${NEXTJS_URL}/api/demo/anomaly?severity=${severity}&provider=${provider}`;
        const res = await fetch(url);
        const data = await res.json();

        if (data.error) {
          return { content: [{ type: "text", text: `Error: ${data.error}. Detail: ${data.detail || ""}` }], isError: true };
        }

        const lines = [
          `Anomaly Analysis (${severity})`,
          `Provider: ${data.provider}`,
          ``,
          `Decision:`,
          JSON.stringify(data.decision, null, 2),
        ];
        if (data.triage_outcome) {
          lines.push(``, `Triage Outcome:`, JSON.stringify(data.triage_outcome, null, 2));
        }
        lines.push(``, `Trace ID: ${data.traceId}`);

        return { content: [{ type: "text", text: lines.join("\n") }] };
      } catch (err) {
        return {
          content: [{ type: "text", text: `CareSync app not reachable at ${NEXTJS_URL}. Make sure the Next.js app is running.` }],
          isError: true,
        };
      }
    }
  );

  // 3. Run Health Trigger (Secretary Agent)
  server.tool(
    "run_health_trigger",
    "Send health trigger data through the Secretary Agent. The agent uses AI function-calling to automatically analyze anomalies, triage patients, retrieve health summaries, and schedule appointments as needed. This is the most powerful tool — it runs the full intelligent pipeline.",
    {
      trigger_type: z.enum(["health_anomaly", "health_summary", "schedule", "custom"]).default("health_anomaly").describe("Type of health trigger"),
      provider: z.enum(["openai", "claude"]).default("claude").describe("LLM provider"),
      description: z.string().optional().describe("Human-readable description of the trigger"),
      user_handle: z.string().default("pari").describe("The patient handle"),
      anomaly_score: z.number().default(85).describe("Anomaly score 0-100 (higher = more anomalous)"),
      sleep_hours: z.number().default(4.5).describe("Patient's sleep hours"),
      resting_hr_bpm: z.number().default(82).describe("Resting heart rate in BPM"),
      steps: z.number().default(2500).describe("Daily step count"),
      flags: z.string().default("SLEEP_DROP,RHR_SPIKE,STEPS_DROP").describe("Comma-separated anomaly flags"),
    },
    async ({ trigger_type, provider, description, user_handle, anomaly_score, sleep_hours, resting_hr_bpm, steps, flags }) => {
      try {
        const triggerData = {
          user_handle,
          date: new Date().toISOString().split("T")[0],
          baseline_window_days: 28,
          metrics: { sleep_hours, resting_hr_bpm, steps },
          baseline: { sleep_mean: 7.1, sleep_std: 0.6, rhr_mean: 62, rhr_std: 3, steps_mean: 7500, steps_std: 1200 },
          flags: flags.split(",").map((f) => f.trim()),
          anomaly_score,
        };

        const res = await fetch(`${NEXTJS_URL}/api/trigger`, {
          method: "POST",
          headers: { "Content-Type": "application/json" },
          body: JSON.stringify({
            trigger_type,
            provid
[truncated — 5805 more characters]
```

### Doctor_Agent_for_TreeHacks/api/main.py

```python
"""
Doctor Agent — FastAPI Application Entry Point
"""

import logging
import sys
from contextlib import asynccontextmanager

from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware

from api.routes import router
import config

# ─── Logging Setup ────────────────────────────────────────────────────────────

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
    handlers=[logging.StreamHandler(sys.stdout)],
)
logger = logging.getLogger(__name__)


# ─── Lifespan (startup/shutdown) ──────────────────────────────────────────────

@asynccontextmanager
async def lifespan(app: FastAPI):
    logger.info("Doctor Agent starting up...")
    logger.info(f"   Doctor: {config.DOCTOR_NAME}")
    logger.info(f"   Model: {config.CLAUDE_MODEL}")

    if not config.ANTHROPIC_API_KEY:
        logger.error("❌ ANTHROPIC_API_KEY is not set! Set it in your .env file.")

    yield

    logger.info("Doctor Agent shutting down.")


# ─── App ──────────────────────────────────────────────────────────────────────

app = FastAPI(
    title="Doctor Agent API",
    description=(
        "An agentic health alert processing system for medical practices. "
        "Receives health alerts from patient devices, triages them using Claude, "
        "and autonomously schedules appointments via agent-to-agent communication."
    ),
    version="1.0.0",
    lifespan=lifespan,
)

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],   # tighten this in production
    allow_methods=["*"],
    allow_headers=["*"],
)

app.include_router(router)


# ─── Local dev entrypoint ─────────────────────────────────────────────────────

if __name__ == "__main__":
    import uvicorn
    uvicorn.run("api.main:app", host="0.0.0.0", port=8000, reload=True)

```

[232 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]