# Project export: Haven This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 12.0 - Tagline: Hospital command center powered by context-aware agentic AI that alerts nurses, unifies EHR data, and answers patient questions, improving understanding and reducing response delays and clinical error - Devpost: https://devpost.com/software/haven-536r1q - GitHub: https://github.com/scrappydevs/haven - Video: https://www.youtube.com/embed/sxdgpbTRsMs?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Result: winner (Fetch AI: Best Use of Fetch AI; Y Combinator: Build an Iconic YC Company - 1st Place; LiveKit: Best Start-up Idea) - Team: 5 GitHub contributor(s) — deeredge (113 commits), davidchung29 (65 commits), Aarush Agarwal (63 commits), JCNTH (46 commits), Claude (13 commits) ## Devpost submission (written by the team) ### Inspiration David's great-uncle was in the hospital this year. He noticed that his great uncle often waited over an hour to find a nurse to answer his questions due to understaffing. When someone finally arrived, they had to bounce across multiple systems to refamiliarize themselves. It felt like the information was there, just not there when we needed. Hospitals have more data than ever, yet nurses spend up to 50% of their shift on documentation instead of patient care. What if the moment a patient calls, an intelligent system helps manage nurse distribution throughout the hospital and flags concerning patterns and prepares exactly what the nurse needs to attend to their patient right away? That's why we built Haven. ### What it does Haven is a multi-agent hospital command center that transforms how nurses access critical patient information. Instead of hunting through fragmented systems, Haven provides three integrated intelligence layers: 1. HavenAI Voice Assistant via LiveKit Patients and families speak naturally to Haven using LiveKit and OpenAI's realtime voice API. Haven asks clarifying follow-up questions, pulls validated EHR data, and delivers nurse-ready summaries and action items—turning a 20-minute wait into an instant, documented interaction. 2. Autonomous Monitoring Dashboard via fetch.ai agents Fetch.ai agents continuously monitor patient vitals, detect concerning patterns, and coordinate with alert-response agents to flag issues before they escalate. Our computer vision pipeline uses facial photoplethysmography (FPPG) to non-invasively track heart rate and stress indicators, feeding real-time data to the monitoring network. 3. Live 3D Hospital Map via Claude agents Powered by an Anthropic Claude chain-tool-calling agent, nurses see a spatial view of the entire floor with real-time alert-based room coloring. They can ask natural language questions like "Tell me about Dheeraj's alerts and questions from the last 6 hours" and instantly understand which patients need attention and why—no system-hopping required. In addition, nurses can generate summary reports from their patients' discussions with Haven AI, so they do not need to turn to fragmented sources to familiarize themselves with the patients' current situation. Together, these agents create a cohesive intelligence network where information flows to the right person at the right time, automatically. ### How we built it Haven is a multi-agent hospital intelligence platform built on three core systems using LiveKit, Fetch.ai, and Claude agents: Voice Interface: HavenAI uses LiveKit for WebRTC streaming, OpenAI Whisper for speech-to-text, and OpenAI Realtime API for fully duplex conversation. The agent streams transcriptions and responses simultaneously, automatically triggering structured prompts to fill missing clinical details and pushing validated data to our backend. Monitoring & Alerts: Fetch.ai agents handle autonomous patient monitoring—a vitals-tracking agent communicates with an alert-response agent to raise or dismiss issues based on real-time thresholds and historical patterns. We also built a computer vision pipeline using OpenCV and facial photoplethysmography (FPPG) to extract heart rate and stress indicators from live video, feeding results directly into the Fetch agent network. Spatial Intelligence: Our live 3D hospital map is powered by an Anthropic Claude chain-tool-calling agent. It interprets natural language commands ("Show me which rooms have active alerts"), autonomously executes multiple tools to query patient data, and dynamically updates room colors and overlays in real-time based on the agent network's events. ### Challenges we ran into Natural voice interaction: Fine-tuning turn-taking, silence detection, and handling dropped connections to make conversations feel human, not robotic. Natural voice interaction: Fine-tuning turn-taking, silence detection, and handling dropped connections to make conversations feel human, not robotic. Multi-stream synchronization: Coordinating concurrent WebSockets from LiveKit, Fetch agents, and Claude while running computer vision without blocking the UI. Multi-stream synchronization: Coordinating concurrent WebSockets from LiveKit, Fetch agents, and Claude while running computer vision without blocking the UI. Context management: Long patient conversations exceeded LLM limits. Built a summarization pipeline to compress transcripts while preserving critical clinical details. Context management: Long patient conversations exceeded LLM limits. Built a summarization pipeline to compress transcripts while preserving critical clinical details. Accomplishments Built a cohesive multi-agent ecosystem where Fetch.ai, LiveKit, and Claude agents autonomously coordinate. Built a cohesive multi-agent ecosystem where Fetch.ai, LiveKit, and Claude agents autonomously coordinate. Implemented Claude's multi-tool-calling to interpret natural language and update the 3D hospital map in real-time. Implemented Claude's multi-tool-calling to interpret natural language and update the 3D hospital map in real-time. Created a facial photoplethysmography (FPPG) pipeline for non-invasive heart rate monitoring integrated with voice and spatial intelligence. Created a facial photoplethysmography (FPPG) pipeline for non-invasive heart rate monitoring integrated with voice and spatial intelligence. ### What we learned Specialized agents outperform monolithic systems—focused tasks are more scalable despite coordination complexity. Specialized agents outperform monolithic systems—focused tasks are more scalable despite coordination complexity. Low-latency architecture requires deep system design when streaming video, analyzing behavior, and raising real-time alerts. Low-latency architecture requires deep system design when streaming video, analyzing behavior, and raising real-time alerts. Next Steps New Agents: Medication Reconciliation: Prevent dangerous drug interactions Discharge Planning: Coordinate patient transitions Resource Allocation: Optimize room and staff assignments Resource Allocation: Optimize room and staff assignments Technical Improvements: Technical Improvements: Reinforcement learning from nurse feedback Computer vision for fall detection and behavioral monitoring Federated learning across hospitals while preserving privacy ## README (from the GitHub repository) # Haven
 

A multi-agent hospital intelligence platform that coordinates autonomous and contextually aware AI agents for comprehensive patient monitoring and clinical decision support.

 

🏆 1st in Y Combinator & Fetch.ai Track and Best Startup Idea at CalHacks 2025

---
Haven AI Dashboard
Haven AI Chat
Haven Seizure Detection
Haven Shoulder Pain Relief
--- ## System Overview Haven operates as a multi-agent hospital command center that coordinates autonomous and contextually aware AI agents for comprehensive patient monitoring and clinical decision support. The platform employs a voice intake agent via **LiveKit and OpenAI** to facilitate natural patient conversations, conducting structured follow-up questioning to collect missing clinical details, accessing validated EHR data, and generating nurse-ready summaries and action items. Multiple **Fetch.ai agents** function as health monitoring agents that continuously track patient vitals and coordinate with specialized agents (including Alert Response Agents) to detect clinical issues and generate appropriate alerts. The system incorporates spatial intelligence capabilities through a **live 3D hospital map** with real-time alert-based room visualization, powered by a **chat agent** with Anthropic Claude chain-tool calling, enabling nurses to manage hospital resources and request clinical summaries through natural language queries. ## System Architecture **Voice Processing Pipeline**: The system implements a robust voice interaction framework using LiveKit for real-time audio streaming, OpenAI Whisper for speech-to-text conversion, and Groq LLM for rapid response generation. Voice activity detection utilizes Silero VAD with configurable silence detection (0.3s) and minimum speech duration (0.1s) parameters. The pipeline handles connection drops, partial transcriptions, and overlapping speech through comprehensive exception handling and recovery mechanisms. **Multi-Stream Synchronization**: Haven manages concurrent WebSocket streams from multiple sources including LiveKit (voice/video), Fetch.ai agents (vital alerts), and Claude toolchain (map updates). The system implements optimized synchronization protocols to maintain real-time data consistency while processing computer vision pipelines in background threads to prevent UI blocking. **Conversation Management**: The platform includes an intelligent summarization pipeline that compresses extended patient conversations while preserving critical clinical information including pain levels, symptom descriptions, and emotional context. This ensures efficient context management across agent communications and LLM interactions. ## Technical Architecture **Multi-Agent Coordination**: Haven employs a distributed agent architecture where specialized agents communicate through message passing and shared state management. Each agent operates independently while maintaining awareness of system-wide events through a centralized event bus. **Real-Time Data Pipeline**: The system processes multiple concurrent data streams including video feeds (30fps), audio streams (16kHz), vital signs (1Hz), and agent communications. All data flows through WebSocket connections with sub-100ms latency requirements. **Computer Vision Pipeline**: Facial photoplethysmography (FPPG) implementation using OpenCV for ROI detection, per-channel standardization for signal preprocessing, and FastICA source separation for heart rate extraction. The pipeline processes forehead regions at 30fps with 8-second rolling windows for frequency domain analysis. **Voice Processing Stack**: LiveKit integration with OpenAI Whisper for real-time transcription, Groq LLM for 75% faster response times, and OpenAI TTS with Nova voice for natural speech synthesis. Voice activity detection uses Silero VAD with 0.3s silence detection and 0.1s speech minimum duration. **Spatial Intelligence**: 3D hospital map rendered with Three.js, updated in real-time through Claude chain-tool calling. Room states are managed through Supabase with WebSocket synchronization for instant updates across all connected clients. ## Tech Stack
Haven Tech Stack
## Future Development **Agent Ecosystem Expansion**: - Medication Reconciliation Agent: Cross-references patient meds with new prescriptions to catch dangerous interactions before they happen - Discharge Planning Agent: Coordinates with social workers, pharmacies, and home health services to streamline patient transitions - Resource Allocation Agent: Dynamically assigns rooms, equipment, and staff based on real-time hospital capacity and patient acuity - Family Communication Agent: Proactively updates loved ones and schedules care conferences, reducing the communication burden on clinical staff **Technical Enhancements**: - Reinforcement Learning from Clinical Feedback: nurses would rate agent suggestions, allowing the system to improve accuracy over time and learn hospital-specific workflows - Multi-modal Patient Monitoring: Integrate computer vision to detect patient movement patterns, fall risk, and behavioral changes that complement vital sign monitoring - Federated Learning Across Hospitals: Enable Haven deployments to learn from each other while maintaining patient privacy, creating a collective intelligence that improves with every installation ## Fetch.ai Agent Network Haven leverages the Fetch.ai uAgents framework and ASI Alliance (asi.one) ecosystem to deploy autonomous agents that coordinate real-time healthcare operations across decentralized infrastructure. Each agent operates independently while communicating through the ASI Alliance network, enabling inter-hospital coordination and access to decentralized AI services from Ocean Protocol and SingularityNET. ### Deployed Agents [Access all Haven agents](https://agentverse.ai/agents/agent1q2w5ktcdjujflcq639lp6kj89zupd28yr4dla0z4qampxjf0txwtqjq3ka0) **haven_nurse** - Primary clinical triage and monitoring agent using Claude AI for real-time patient assessment. Processes vital sign alerts, coordinates emergency responses, escalates critical conditions, and ensures clinical trial compliance. Handles 369 interactions with 2.5 rating on Agentverse. [Access Haven_nurse on Agentverse](https://agentverse.ai/agents/details/agent1q2w5ktcdjujflcq639lp6kj89zupd28yr4dla0z4qampxjf0txwtqjq3ka0/profile) **haven_asi_coordinator** - Central orchestration hub connecting Haven to the broader ASI Alliance ecosystem. Routes complex health queries to Ocean Protocol ML models for predictive analytics, SingularityNET AI services for diagnostic pattern recognition, and Fetch.ai agent networks for resource coordination. Publishes Haven's capabilities to the asi.one marketplace and monitors network health across all three alliance partners. [Access Haven_asi_coordinator on Agentverse](https://agentverse.ai/agents/details/agent1qt0f8spkqljl9w4kjth7mv6p62xpfflfwt2z8rg5dytjszsy9q2lke8gutt/profile) **haven_resource_allocation** - Dynamic hospital resource optimizer managing ICU beds, ventilators, specialist assignments, and equipment allocation based on real-time patient acuity. Coordinates with regional hospitals through ASI Alliance for cross-facility resource sharing during capacity constraints. Monitors utilization metrics and predicts future resource needs using decentralized ML models. Handles 21 interactions on Agentverse. [Access Haven_resource_allocation on Agentverse](https://agentverse.ai/agents/details/agent1qvmkluyfdjw802w7xux9vllslsle3fhzkh5la7csjukg6q706xxk6azk66j/profile) **haven_reconciliation** - Medic [README truncated for size] ## Detected evidence (automated analysis) Indexed codebase: 141 recognized source files, 1350 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - SQL (language) — detected in the code - Supabase (technology) — detected in the code - Swift (language) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - AI coding agent: Claude Code — evidence: commit authorship or trailers - AI coding agent: Cursor — evidence: config files committed to the repository; commit authorship or trailers ## Codebase structure (from repository index) ### Files (120 of 172) ``` .cursor/rules/general.mdc .github/workflows/deployment-status.yml .gitignore .python-version .render-deploy backend/.env.backup backend/.env.example backend/.env.local.template backend/app/agent_system.py backend/app/agents/__init__.py backend/app/agents/haven_agent.py backend/app/agents/intake_agent.py backend/app/agents/listener_agent.py backend/app/ai_tools.py backend/app/alert_monitor.py backend/app/asi_alliance_integration.py backend/app/cache.py backend/app/chat_context.py backend/app/connection_limits.py backend/app/cv_metrics.py backend/app/fetch_handoff_agent.py backend/app/fetch_health_agent.py backend/app/health_agent.py backend/app/health.py backend/app/infisical_config.py backend/app/main.py backend/app/models/handoff_forms.py backend/app/monitoring_control.py backend/app/monitoring_protocols.py backend/app/movement_detector.py backend/app/patient_guardian_agent.py backend/app/pdf_generator.py backend/app/rooms.py backend/app/services/alerts_service.py backend/app/services/email_service.py backend/app/services/pdf_generator.py backend/app/services/wearable_service.py backend/app/simple_movement_detector.py backend/app/supabase_client.py backend/app/voice_call.py backend/app/websocket.py backend/app/worker_pool.py backend/data/.gitkeep backend/data/handoff_forms/.gitignore backend/data/patients.json backend/database/003_wearable_vitals.sql backend/database/004_wearable_devices.sql backend/HANDOFF_AGENT_README.md backend/haven-agents/models/wearable.py backend/main.py backend/migrations/001_create_intake_reports.sql backend/migrations/002_alert_call_trigger.sql backend/migrations/disable_alert_triggers.sql backend/migrations/fix_notify_critical_alert.sql backend/migrations/mock_alerts_data.sql backend/Procfile backend/QUICKSTART_HANDOFF.md backend/requirements.txt backend/runtime.txt backend/scripts/check_secrets.py backend/scripts/fix_enrollment_status.py backend/scripts/fix_env.py backend/scripts/generate_patients.py backend/start_alert_monitor.sh backend/start_backend.sh backend/start_haven_agent.sh backend/start_intake_agent.sh backend/test_agent.py backend/test_health_agent.py backend/test_secrets.py frontend/.eslintrc.json frontend/.gitignore frontend/{ frontend/app/api/intake/start/route.ts frontend/app/dashboard/floorplan/page.tsx frontend/app/dashboard/layout.tsx frontend/app/dashboard/page.tsx frontend/app/globals.css frontend/app/intake/page.tsx frontend/app/layout.tsx frontend/app/page.tsx frontend/app/patient-view/page.tsx frontend/app/stream/page.tsx frontend/components/ActiveProtocols.tsx frontend/components/AgentAlertToast.tsx frontend/components/AIAgentAlerts.tsx frontend/components/AIAgentStatus.tsx frontend/components/AIChat.tsx frontend/components/AIVoiceAnimation.tsx frontend/components/AlertPanel.tsx frontend/components/AlertsModal.tsx frontend/components/AnalysisModeSelector.tsx frontend/components/AppHeader.tsx frontend/components/AppSidebar.tsx frontend/components/ChatEnhancements.tsx frontend/components/DetailPanel.tsx frontend/components/FloorPlanLegend.tsx frontend/components/GlobalActivityFeed.tsx frontend/components/HandoffFormModal.tsx frontend/components/HandoffFormsList.tsx frontend/components/InfoBar.tsx frontend/components/IntakeQueue.tsx frontend/components/IntakeReportModal.tsx frontend/components/ManualAlertsPanel.tsx frontend/components/MonitoringConditionSelector.tsx frontend/components/PatientNurseLookup.tsx frontend/components/PatientSearchModal.tsx frontend/components/PatientTimeline.tsx frontend/components/RoomDetailsPanel.tsx frontend/components/StatsBar.tsx frontend/components/TerminalLog.tsx frontend/components/VideoPlayer.tsx frontend/components/WearablePairingModal.tsx frontend/fix_layout.sh frontend/lib/api-config.ts frontend/lib/api.ts frontend/next-env.d.ts frontend/next.config.js frontend/package.json frontend/postcss.config.js [52 more files omitted for size] ``` ### Dependencies - backend/requirements.txt: anthropic@==0.44.0, fastapi@==0.115.6, gunicorn@==23.0.0, httpx@==0.27.2, livekit@==1.0.17, livekit-agents@==1.1.0, livekit-api@==1.0.7, livekit-plugins-groq@==1.1.0, livekit-plugins-noise-cancellation@==0.2.5, livekit-plugins-openai@==1.1.0, livekit-plugins-silero@==1.1.0, mediapipe@==0.10.21, numpy@==1.26.4, openai@>=1.50.0, opencv-python-headless@==4.10.0.84, pandas@==2.2.3, pillow@==11.1.0, postgrest@==0.19.1, protobuf@>=4.25.3,<5.0, psutil@==7.1.2, python-dotenv@==1.1.1, python-multipart@==0.0.20, realtime@==2.22.2, scipy@==1.16.2, supabase@==2.22.2, uvicorn[standard]@==0.34.0, vonage@==4.7.2, websockets@==15.0.1 - frontend/package.json: @livekit/components-react@^2.9.15, @livekit/components-styles@^1.1.6, @react-three/drei@^10.7.6, @react-three/fiber@^9.4.0, @smplrspace/smplr-loader@^2.42.0, @types/node@^20, @types/react@^19, @types/react-dom@^19, autoprefixer@^10.4.16, eslint@^8, eslint-config-next@^15.0.0, framer-motion@^11.0.0, fuse.js@^7.1.0, livekit-client@^2.15.13, livekit-server-sdk@^2.14.0, lucide-react@^0.400.0, next@^15.0.0, postcss@^8.4.32, react@^19.0.0, react-dom@^19.0.0, react-markdown@^10.1.0, recharts@^2.10.0, remark-gfm@^4.0.1, tailwindcss@^3.4.0, three@^0.180.0, typescript@^5, zustand@^5.0.8 - haven-agents/requirements.txt: anthropic@>=0.44.0, pydantic@>=2.0.0, python-dateutil@>=2.8.0, python-dotenv@>=1.0.0, uagents@>=0.12.0, uagents-ai-engine@>=0.4.0 ### Recent commits (newest first) - refactor - Update README with achievement details - update - Add files via upload - Update README.md - update - Add Fetch.ai ASI Alliance agent network - optimize streaming for smooth preview and expanded view transitions - fix patient search to include all required fields and disable cache for debug - make cache import optional for robust deployment - trigger render redeploy with latest optimizations - fix render startCommand to use app.main:app - display gifs side by side without table for larger view - add demo gifs in 2x2 grid to readme - add refresh button and stream count to assign modal - add caching and optimize database queries for 10x performance boost - merge main: keep optimized voice_call, integrate stream improvements - readme - optimize emergency call speed 100x faster - update requirements ## Key source files (fetched from GitHub, selected and truncated for size) ### backend/QUICKSTART_HANDOFF.md ```markdown # Quick Start: Handoff Form Generator Get the Fetch.ai handoff form generator running in 5 minutes! ## Prerequisites - Python 3.10+ - Supabase account with Haven database - SMTP email account (Gmail recommended for testing) - Anthropic API key (optional, but highly recommended) ## 1. Install Dependencies ```bash cd backend pip install -r requirements.txt ``` ## 2. Create Database Table Run this SQL in your Supabase SQL Editor: ```sql -- Run the handoff_forms schema -- File: database/handoff_forms_schema.sql ``` Or use the provided file: ```bash psql $DATABASE_URL < database/handoff_forms_schema.sql ``` ## 3. Configure Environment Copy `.env.example` to `.env`: ```bash cp .env.example .env ``` **Minimal Configuration (for testing):** ```env # Database SUPABASE_URL=https://your-project.supabase.co SUPABASE_KEY=your-service-role-key # AI (optional but recommended) ANTHROPIC_API_KEY=sk-ant-... # Email (Gmail example) SMTP_SERVER=smtp.gmail.com SMTP_PORT=587 SMTP_USERNAME=your-email@gmail.com SMTP_PASSWORD=your-gmail-app-password SENDER_EMAIL=your-email@gmail.com SENDER_NAME=Haven Health # Nurses to notify NURSE_EMAILS=nurse@example.com,doctor@example.com ``` ### Gmail Setup (Quick) 1. Go to [Google Account → Security](https://myaccount.google.com/security) 2. Enable 2-Factor Authentication 3. Go to [App Passwords](https://myaccount.google.com/apppasswords) 4. Create new app password for "Mail" 5. Copy 16-character password to `SMTP_PASSWORD` ## 4. Test Email Connection ```bash python -c "from app.services.email_service import email_service; print(email_service.test_connection())" ``` Should output: `{'success': True, 'message': 'SMTP connection successful'}` ## 5. Start the Server ```bash uvicorn app.main:app --reload ``` **You should see:** ``` ✅ Fetch.ai Handoff Agent initialized: agent1q... 📧 Default nurse emails: ['nurse@example.com'] 🏥 Haven Backend Services: • Fetch.ai Handoff Agent: ✅ Enabled ``` ## 6. Test Manual Form Generation ### Option A: Using cURL ```bash curl -X POST http://localhost:8000/handoff-agent/generate \ -H "Content-Type: application/json" \ -d '{ "patient_id": "P-DAV-001", "recipient_emails": ["your-test-email@gmail.com"] }' ``` ### Option B: Using the API Docs 1. Open http://localhost:8000/docs 2. Find `POST /handoff-agent/generate` 3. Click "Try it out" 4. Enter: ```json { "patient_id": "P-DAV-001" } ``` 5. Click "Execute" ### Expected Response: ```json { "success": true, "form_id": "uuid-here", "form_number": "HO-20251026143022", "pdf_path": "/tmp/handoff_forms/HO-20251026143022.pdf", "alerts_processed": 1, "email_sent": true, "message": "Successfully generated handoff form HO-20251026143022" } ``` **Check your email!** You should receive: - Subject: `[CRITICAL] Patient Handoff Form HO-... - Patient Name` - PDF attachment with full handoff form ## 7. View Generated Forms ### List all forms: ```bash curl http://localhost:8000/handoff-agent/ [truncated — 3587 more characters] ``` ### backend/HANDOFF_AGENT_README.md ```markdown # Haven Handoff Form Generator - Fetch.ai Agent This document describes the Fetch.ai agent that automatically monitors the `alerts` table, generates professional handoff forms summarizing patient information, and emails them to healthcare staff. ## Overview The **Fetch.ai Handoff Agent** is an autonomous agent that: 1. **Monitors** the `alerts` table in Supabase every 5 minutes 2. **Queries** all active/unhandled alerts for each patient 3. **Generates** AI-powered handoff forms with: - Patient demographics and clinical information - Alert summaries and severity assessments - Clinical context (vitals, medications, history) - Recommended actions and protocols - Complete alert timeline 4. **Creates** professional PDF documents 5. **Stores** forms in the `handoff_forms` database table 6. **Emails** PDFs to configured nurse email addresses 7. **Tracks** email delivery status ## Architecture ``` ┌─────────────────────────────────────────────────────────────────┐ │ Fetch.ai Handoff Agent │ │ (fetch_handoff_agent.py) │ ├─────────────────────────────────────────────────────────────────┤ │ │ │ 1. Periodic Check (every 5 min) │ │ ↓ │ │ 2. Query Alerts Service → Supabase alerts table │ │ ↓ │ │ 3. Group alerts by patient │ │ ↓ │ │ 4. Generate Form Content: │ │ • Fetch patient demographics │ │ • Query recent vitals from alert metadata │ │ • Use Claude AI to generate clinical summary │ │ • Create recommended actions │ │ ↓ │ │ 5. Generate PDF (reportlab) │ │ ↓ │ │ 6. Save to handoff_forms table │ │ ↓ │ │ 7. Email PDF to nurses (SMTP) │ │ ↓ │ │ 8. Update delivery status │ │ │ └─────────────────────────────────────────────────────────────────┘ ``` ## File Structure ``` backend/ ├── app/ │ ├── fetch_handoff_agent.py # Main Fetch.ai agent │ ├── models/ │ │ └── handoff_forms.py # Pydantic data models │ ├── services/ │ │ ├── alerts_service.py # Alert querying service │ │ ├── pdf_generator.py [truncated — 10811 more characters] ``` ### haven-agents/requirements.txt ``` # Haven AI Multi-Agent System Requirements # Fetch.ai uAgents framework uagents>=0.12.0 uagents-ai-engine>=0.4.0 # AI/ML anthropic>=0.44.0 # Data models pydantic>=2.0.0 python-dateutil>=2.8.0 # Utilities python-dotenv>=1.0.0 ``` ### backend/requirements.txt ``` # FastAPI and server fastapi==0.115.6 uvicorn[standard]==0.34.0 python-multipart==0.0.20 # Computer Vision opencv-python-headless==4.10.0.84 mediapipe==0.10.21 numpy==1.26.4 pillow==11.1.0 scipy==1.16.2 # AI/ML anthropic==0.44.0 # Protobuf - required by LiveKit and mediapipe protobuf>=4.25.3,<5.0 # LiveKit for voice AI - using compatible versions livekit==1.0.17 livekit-api==1.0.7 livekit-agents==1.1.0 livekit-plugins-openai==1.1.0 livekit-plugins-silero==1.1.0 livekit-plugins-groq==1.1.0 livekit-plugins-noise-cancellation==0.2.5 openai>=1.50.0 python-dotenv==1.1.1 # Data processing pandas==2.2.3 # Database - using latest compatible versions supabase==2.22.2 realtime==2.22.2 postgrest==0.19.1 websockets==15.0.1 httpx==0.27.2 # PDF Generation (optional - install if needed for discharge reports) # reportlab==4.2.5 # System utilities psutil==7.1.2 # Production gunicorn==23.0.0 # SMS Alerts vonage==4.7.2 # Multi-Agent System (Fetch.ai uAgents) - temporarily disabled due to protobuf conflicts # uagents==0.21.0 ``` ### frontend/package.json ``` { "name": "haven-frontend", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint" }, "dependencies": { "@livekit/components-react": "^2.9.15", "@livekit/components-styles": "^1.1.6", "@react-three/drei": "^10.7.6", "@react-three/fiber": "^9.4.0", "@smplrspace/smplr-loader": "^2.42.0", "framer-motion": "^11.0.0", "fuse.js": "^7.1.0", "livekit-client": "^2.15.13", "livekit-server-sdk": "^2.14.0", "lucide-react": "^0.400.0", "next": "^15.0.0", "react": "^19.0.0", "react-dom": "^19.0.0", "react-markdown": "^10.1.0", "recharts": "^2.10.0", "remark-gfm": "^4.0.1", "three": "^0.180.0", "zustand": "^5.0.8" }, "devDependencies": { "@types/node": "^20", "@types/react": "^19", "@types/react-dom": "^19", "autoprefixer": "^10.4.16", "eslint": "^8", "eslint-config-next": "^15.0.0", "postcss": "^8.4.32", "tailwindcss": "^3.4.0", "typescript": "^5" } } ``` ### backend/main.py ```python """ Haven AI - Backend Application Entry point for Gunicorn deployment Auto-loads secrets from Infisical when run directly """ import sys import os import subprocess from pathlib import Path def check_infisical_cli(): """Check if Infisical CLI is available""" try: subprocess.run(['infisical', '--version'], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True) return True except (subprocess.CalledProcessError, FileNotFoundError): return False def restart_with_infisical(): """Restart the script with Infisical CLI""" print("🔐 Loading secrets from Infisical...") print() # Set environment variable to prevent infinite loop env = os.environ.copy() env['_INFISICAL_WRAPPER'] = '1' # Prepare the command infisical_cmd = [ 'infisical', 'run', '--env=dev', '--', sys.executable, __file__ ] # Execute with Infisical (from repo root where .infisical.json is) backend_dir = Path(__file__).parent os.chdir(backend_dir.parent) # Use subprocess instead of execvp to avoid exec issues try: subprocess.run(infisical_cmd, env=env, check=True) except KeyboardInterrupt: print("\n\n👋 Shutting down...") sys.exit(0) except subprocess.CalledProcessError as e: sys.exit(e.returncode) # Check if we need to restart with Infisical # Only do this if: # 1. Running as main script # 2. Not already wrapped (check for our flag) # 3. No Infisical project ID in env (not running with infisical run) # 4. Infisical CLI is available if (__name__ == "__main__" and not os.getenv("_INFISICAL_WRAPPER") and not os.getenv("INFISICAL_PROJECT_ID") and check_infisical_cli()): restart_with_infisical() sys.exit(0) # Exit after restart # If we get here, either we're running with Infisical or it's not available from app.main import app # This is the entry point for Gunicorn # Run with: gunicorn main:app if __name__ == "__main__": # For local development only import uvicorn # Add backend to path so imports work backend_dir = Path(__file__).parent import sys sys.path.insert(0, str(backend_dir)) # Use reload=False to avoid multiprocessing issues with mediapipe # For auto-reload during development, use: uvicorn app.main:app --reload uvicorn.run("app.main:app", host="0.0.0.0", port=8000, reload=False) ``` ### haven-agents/main.py ```python """ Phase 6: Integration & Polish - Complete Demo Flow Main orchestrator that runs all agents together """ import sys import os import asyncio import signal from pathlib import Path from datetime import datetime from typing import List import multiprocessing as mp # Add current directory to path sys.path.append(str(Path(__file__).parent)) from agents.patient_guardian import create_patient_guardian from agents.nurse_coordinator import NurseCoordinatorAgent from agents.emergency_response import EmergencyResponseAgent from agents.protocol_compliance import ProtocolComplianceAgent from agents.research_insights import ResearchInsightsAgent from utils.mock_data import mock_generator def run_patient_guardian(patient_id: str): """Run a patient guardian agent in a separate process""" guardian = create_patient_guardian(patient_id) try: guardian.run() except KeyboardInterrupt: pass def run_nurse_coordinator(): """Run nurse coordinator agent""" coordinator = NurseCoordinatorAgent() try: coordinator.run() except KeyboardInterrupt: pass def run_emergency_response(): """Run emergency response agent""" emergency = EmergencyResponseAgent() try: emergency.run() except KeyboardInterrupt: pass def run_protocol_compliance(): """Run protocol compliance agent""" compliance = ProtocolComplianceAgent() try: compliance.run() except KeyboardInterrupt: pass def run_research_insights(): """Run research insights agent""" research = ResearchInsightsAgent() try: research.run() except KeyboardInterrupt: pass def print_demo_header(): """Print demo header""" print("\n" + "=" * 80) print("🏥 HAVEN AI - MULTI-AGENT CLINICAL TRIAL MONITORING SYSTEM") print("=" * 80) print("\nBuilt with Fetch.ai uAgents + Anthropic Claude") print("CAR-T Cell Therapy Safety Monitoring Demo\n") print("=" * 80) print("\n📋 DEMO SCRIPT (3-5 minutes):") print("\nMinute 0-1: NORMAL OPERATIONS") print(" • All 3 patients stable") print(" • Routine monitoring every 30 seconds") print(" • Protocol compliance tracking active") print("\nMinute 1-2: CONCERNING ALERT") print(" • P-002 develops elevated vitals") print(" • Guardian Agent detects deviation") print(" • Nurse Coordinator assigns response") print("\nMinute 2-3: CRITICAL EMERGENCY") print(" • P-003 develops Grade 3 CRS") print(" • Emergency Response Agent activates protocol") print(" • Physician paged, medications prepared") print("\nMinute 3-4: PATTERN DETECTION") print(" • Research Insights detects tremor pattern") print(" • Safety signal flagged for investigation") print("\nMinute 4-5: SYSTEM SUMMARY") print(" • All agents coordinated successfully") print(" • Compliance report generated") print(" • Safety monitoring active") print("\n" + "=" * 80) print("\n🚀 STARTING ALL AGENTS...\n") def print_demo_instructions(): """Print instructions for demo scenarios""" print("\n" + "=" * 80) print("📝 DEMO SCENARIO CONTROLS") print("=" * 80) print("\nThe demo will automatically progress through scenarios.") print("You can also manually trigger scenarios by editing utils/mock_data.py:") print("\nAvailable scenarios:") print(" • 'normal' - All patients stable (default)") print(" • 'p002_concerning' - P-002 develops concerning symptoms") print(" • 'p003_critical' - P-003 develops critical CRS") print(" • 'pattern_tremor' - Multiple patients show tremor") print("\nTo change scenario during demo:") print(" 1. Keep this running") print(" 2. In Python console: mock_generator.set_scenario('scenario_name')") print("\n" + "=" * 80) print("\n⌛ Agents will start in 3 seconds...") print(" Press Ctrl+C to stop all agents\n") def main(): """Main orchestrator""" # Print demo information print_demo_header() print_demo_instructions() import time time.sleep(3) print("=" * 80) print("🎬 DEMO STARTING - Minute 0: Normal Operations") print("=" * 80 + "\n") # List of processes processes = [] try: # Start all agents in separate processes # 1. Patient Guardian Agents (3 patients) print("🤖 Starting Patient Guardian Agents...") for patient_id in ["P-001", "P-002", "P-003"]: p = mp.Process(target=run_patient_guardian, args=(patient_id,)) p.start() processes.append(p) time.sleep(1) # Stagger startup # 2. Nurse Coordinator print("👨‍⚕️ Starting Nurse Coordinator Agent...") p = mp.Process(target=run_nurse_coordinator) p.start() processes.append(p) time.sleep(1) # 3. Emergency Response print("🚑 Starting Emergency Response Agent...") p = mp.Process(target=run_emergency_response) p.start() processes.append(p) time.sleep(1) # 4. Protocol Compliance print("📋 Starting Protocol Compliance Agent...") p = mp.Process(target=run_protocol_compliance) p.start() processes.append(p) time.sleep(1) # 5. Research Insights print("🔬 Starting Research Insights Agent...") p = mp.Process(target=run_research_insights) p.start() processes.append(p) print("\n" + "=" * 80) print("✅ ALL AGENTS RUNNING") print("=" * 80) print(f"\nTotal agents: {len(processes)}") print(" • 3 Patient Guardian Agents (P-001, P-002, P-003)") print(" • 1 Nurse Coordinator Agent") print(" • 1 Emergency Response Agent") print(" • 1 Protocol Compliance Agent") print(" • 1 Research Insights Agent") print("\n📊 Monitoring all patients... Wa [truncated — 3192 more characters] ``` ### frontend/app/layout.tsx ```typescript import type { Metadata } from "next"; import { Playfair_Display, Inter } from "next/font/google"; import "./globals.css"; import Script from "next/script"; const playfair = Playfair_Display({ subsets: ["latin"], weight: ["900"], variable: "--font-playfair", display: "swap", }); const inter = Inter({ subsets: ["latin"], weight: ["200", "300", "400", "500"], variable: "--font-inter", display: "swap", }); export const metadata: Metadata = { title: "Haven - Patient Monitoring Platform", description: "Real-time patient monitoring and floor plan management for clinical trials", }; export default function RootLayout({ children, }: { children: React.ReactNode; }) { return ( {children} ); } ``` ### frontend/app/page.tsx ```typescript 'use client'; import { useEffect, useRef, useState } from 'react'; import Script from 'next/script'; import { motion } from 'framer-motion'; declare global { interface Window { smplr: any; } } export default function LandingPage() { const containerRef = useRef(null); const spaceRef = useRef(null); const [smplrLoaded, setSmplrLoaded] = useState(false); const [spaceReady, setSpaceReady] = useState(false); useEffect(() => { if (!containerRef.current || !smplrLoaded || !window.smplr || spaceRef.current) return; const initViewer = async () => { try { console.log('🎬 Initializing landing page viewer...'); // Fetch Smplrspace config const configRes = await fetch(`${process.env.NEXT_PUBLIC_API_URL}/smplrspace/config`); const config = await configRes.json(); const space = new window.smplr.Space({ spaceId: config.spaceId, clientToken: config.clientToken, containerId: 'landing-smplr-container' }); spaceRef.current = space; await space.startViewer({ preview: false, mode: '3d', allowModeChange: false, hideNavigationButtons: true, // Hide toolbar onReady: async () => { console.log('✅ Landing viewer ready'); setSpaceReady(true); // Set camera for beautiful overview space.setCameraPlacement({ alpha: -Math.PI / 6, beta: Math.PI / 4, radius: 35, target: { x: 20, y: 0, z: 0 }, animationDuration: 2 }); // Fetch and color rooms with alerts try { const roomsRes = await fetch(`${process.env.NEXT_PUBLIC_API_URL}/rooms`); const rooms = await roomsRes.json(); // Simulate alerts for visual interest (mix of colors) const criticalRooms = [0]; // Room 1: Critical (red) const warningRooms = [1]; // Room 2: Warning (orange) // Rooms 3-5 will be green (occupied) or gray (empty) // Add colored room polygons const roomPolygons = rooms .filter((r: any) => r.metadata?.polygon && r.room_type === 'patient') .map((room: any, idx: number) => ({ id: room.room_id, coordinates: room.metadata.polygon, levelIndex: 0, alertLevel: criticalRooms.includes(idx) ? 'critical' : warningRooms.includes(idx) ? 'warning' : 'none', occupied: idx >= 2 || !!room.patient_id // Assume rooms 3+ occupied for demo })); if (roomPolygons.length > 0) { space.addDataLayer({ id: 'room-colors', type: 'polygon', data: roomPolygons, color: (data: any) => { if (data.alertLevel === 'critical') return '#ef4444'; // Red if (data.alertLevel === 'warning') return '#f97316'; // Orange if (data.occupied) return '#10b981'; // Green return '#e5e7eb'; // Gray }, alpha: 0.4, height: 0.1 }); } } catch (err) { console.log('Could not fetch rooms for landing page'); } // Smooth continuous rotation using requestAnimationFrame let lastUpdate = Date.now(); const rotationSpeed = 0.0002; // radians per millisecond const animate = () => { if (spaceRef.current) { const now = Date.now(); const delta = now - lastUpdate; lastUpdate = now; const current = spaceRef.current.getCameraPlacement(); spaceRef.current.setCameraPlacement({ ...current, alpha: current.alpha + (rotationSpeed * delta), animationDuration: 0 }); requestAnimationFrame(animate); } }; requestAnimationFrame(animate); }, onError: (error: string) => { console.error('Viewer error:', error); setSpaceReady(true); } }); } catch (error) { console.error('Failed to initialize viewer:', error); setSpaceReady(true); } }; initViewer(); return () => { if (spaceRef.current) { spaceRef.current.remove(); spaceRef.current = null; } }; }, [smplrLoaded]); return (
{/* Smplr.js Script */}