Project Info
Inspiration
SDG 3 (Good Health and Well-being): Focuses on ensuring healthy lives and promoting well-being for all ages. Key targets include reducing maternal and child mortality, ending epidemics of major infectious diseases, and reducing premature mortality from non-communicable diseases. SDG 11 (Sustainable Cities and Communities): Aims to make cities and human settlements inclusive, safe, resilient, and sustainable. This involves ensuring access to adequate, safe, and affordable housing and basic services, improving transportation systems, enhancing urban planning, and reducing the environmental impact of cities. The devastating LA fires and escalating global tensions reminded us that crises don't wait for business hours. When disaster strikesโwhether it's fleeing flames, managing PTSD from conflict zones, or treating injuries in remote areasโpeople need immediate, expert guidance. Current emergency systems are overwhelmed, understaffed, and often unreachable when needed most. We built Riley to ensure no one faces a crisis alone, advancing UN SDGs 3 & 11 by providing universal healthcare access and building resilient communities that can respond to any emergency, anywhere, anytime.
What it does
Riley is an AI-powered crisis response system accessible 24/7 via phone calls. It provides specialized support across three critical areas: Mental Health Crisis: Suicide prevention, PTSD support, panic attacks, and trauma counseling Remote Medical Emergencies: First aid guidance, injury assessment, and medical triage for isolated locations Disaster Evacuation: Fire, flood, earthquake response, and emergency evacuation planning Riley remembers every caller through persistent memory tied to phone numbers, building trusted relationships across multiple interactions. Crisis management boards, insurance companies, hospitals, and emergency institutions can access the platform for better crisis coordination and resource allocation.
How we built it
Voice Infrastructure: VAPI provides low-latency, empathetic phone call handling with natural speech recognition and generation. Memory System: Letta's multi-user architecture creates unique user profiles for each phone number, ensuring persistent context across all interactions. Multi-Agent Architecture: Specialized Letta agents for mental health, medical emergencies, and evacuation work collaboratively while sharing user memory, reducing hallucination and improving response accuracy. Escalation Tools: Agentic tools automatically detect high-priority situations and escalate calls to human operators when necessary. Analytics Dashboard: Real-time crisis categorization and management interface for institutions to track patterns and optimize response strategies.
Challenges we ran into
Memory Persistence: Implementing sticky memory across phone sessions required careful user management in Letta's multi-user system. Agent Coordination: Ensuring multiple specialized agents could share context without conflicting advice or losing conversation flow. Syncing Letta and Vapi was abit of a challenge as well. Emergency Escalation: Building reliable triggers to identify when AI should immediately transfer to human crisis counselors. Voice Latency: Optimizing VAPI integration to maintain empathetic, real-time conversations during high-stress situations. Crisis Detection: Developing algorithms to accurately categorize and prioritize different types of emergencies.
Accomplishments we're proud of
Universal Access: Created a system that works anywhere with phone service, eliminating geographic and infrastructure barriers to crisis support. Persistent Relationships: Successfully implemented memory retention that allows callers to build ongoing relationships with their AI crisis specialist. Multi-Modal Crisis Response: Built the first AI system that handles mental health, medical, and evacuation emergencies in one unified platform. Institutional Integration: Designed scalable analytics that help crisis management organizations optimize their response strategies. Real-Time Escalation: Implemented seamless human handoff for situations requiring immediate professional intervention.
What we learned
Crisis Communication: The importance of empathetic, non-judgmental language in emergency situations cannot be overstated. Memory Architecture: Persistent AI relationships significantly improve trust and response effectiveness in crisis scenarios. Multi-Agent Coordination: Specialized agents working together provide more accurate, contextual advice than single generalist models. Scalability Matters: AI crisis response can provide insights into community risk patterns that improve overall emergency preparedness. Human-AI Collaboration: The most effective crisis response combines AI availability with human expertise for complex situations.
What's next
Global Expansion: Deploy Riley in multiple languages and regions, partnering with international crisis response organizations. Predictive Analytics: Use aggregated crisis data to predict and prevent community-wide emergencies before they escalate. IoT Integration: Connect with smart home devices, medical wearables, and environmental sensors for proactive crisis detection. Training Platform: Develop Riley as a training tool for human crisis counselors and emergency responders. Policy Impact: Work with governments to integrate Riley into national emergency response frameworks, advancing SDGs 3 & 11 at scale. Specialized Verticals: Create industry-specific versions for schools, workplaces, and high-risk environments like oil rigs or remote research stations. Riley represents the future of crisis responseโwhere help is always just a phone call away, building healthier, more resilient communities worldwide.
Emergency Triage Phone App ๐
A 24/7 emergency triaging phone call application that connects injured or endangered individuals with AI agents for situation assessment and guidance. The system persists caller memory across sessions and provides a real-time dashboard for monitoring active calls.
๐๏ธ Architecture Overview
Tech Stack
- Frontend: React + TypeScript + Mantine UI
- Backend: Node.js + Express + TypeScript
- Database: PostgreSQL with emergency call schemas
- Voice API: VAPI integration for phone calls
- AI Agents: Letta (formerly MemGPT) for persistent memory
- Real-time: WebSocket connections for dashboard updates
- Hosting: Vercel (frontend) + Railway/Render (backend)
System Components
โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ
โ Emergency โ โ VAPI Voice โ โ Dashboard โ
โ Caller โโโโโบโ Assistant โโโโโบโ Operators โ
โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ
โ โ
โผ โผ
โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ
โ Letta AI โโโโโบโ Backend API โโโโโบโ Real-time โ
โ Memory Agent โ โ (Node.js) โ โ WebSocket โ
โโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโ
โ
โผ
โโโโโโโโโโโโโโโโโโโโ
โ PostgreSQL โ
โ Database โ
โโโโโโโโโโโโโโโโโโโโ
๐ Quick Start
Prerequisites
- Node.js 18+ and npm
- PostgreSQL 14+
- VAPI account and API key
- Letta account and API key
1. Clone and Setup
# Clone the repository
git clone <repository-url>
cd emergency-triage
# Install backend dependencies
cd backend
npm install
# Install frontend dependencies
cd ../frontend
npm install
2. Environment Configuration
Create .env files for both backend and frontend:
Backend .env:
DATABASE_URL=postgresql://username:password@localhost:5432/emergency_triage
VAPI_API_KEY=your_vapi_api_key
VAPI_PHONE_NUMBER=your_emergency_number
LETTA_API_KEY=your_letta_api_key
JWT_SECRET=your_jwt_secret
PORT=3001
FRONTEND_URL=http://localhost:3000
Frontend .env:
REACT_APP_API_BASE_URL=http://localhost:3001
REACT_APP_WS_URL=http://localhost:3001
3. Database Setup
# Create PostgreSQL database
createdb emergency_triage
# Run setup and test script
npx ts-node setup-and-test.ts
4. Start the Application
# Terminal 1: Start backend
cd backend
npm run dev
# Terminal 2: Start frontend
cd frontend
npm start
5. Access the Dashboard
Open http://localhost:3000 to access the real-time emergency dashboard.
๐ Key Features
Emergency Call Handling
- Instant Response: < 3 second call answering
- AI Triage: Automated situation assessment
- Multi-language: English, Spanish, French support, etc
- Severity Classification: CRITICAL, HIGH, MEDIUM, LOW levels
- Real-time Transcription: Live speech-to-text processing
Persistent Memory System
- Caller Profiles: Automatic identification via phone number
- Medical History: Persistent health condition tracking
- Emergency History: Previous call outcomes and patterns
- Location Data: GPS coordinates and address storage
- Risk Assessment: Dynamic risk profile calculation
Real-time Dashboard
- Live Call Monitoring: Active call status and transcriptions
- Emergency Alerts: Critical situation notifications
- Caller Context: Historical data and risk profiles
- Escalation Controls: Human operator handoff
- Analytics: Call volume, response times, outcomes
Emergency Escalation
- Auto-escalation: Based on severity and caller history
- Human Operators: Specialized emergency response staff
- Emergency Services: Direct 911/emergency services dispatch
- Supervisor Alerts: Complex situation management
๐ง API Documentation
Core Endpoints
# Calls Management
POST /api/calls/initiate # Start emergency call
GET /api/calls/active # Get active calls
GET /api/calls/history # Get call history
POST /api/calls/escalate # Escalate to human operator
POST /api/calls/webhook/vapi # VAPI webhook handler
# User & Memory Management
GET /api/users/:id # Get user profile
POST /api/users # Create user profile
GET /api/users/:id/memory # Get caller memory
PUT /api/users/:id/memory # Update caller memory
# Dashboard & Analytics
GET /api/dashboard/stats # Dashboard statistics
GET /api/dashboard/active-calls # Active calls for dashboard
GET /api/dashboard/recent-calls # Recent call history
WebSocket Events
// Client-side event listeners
socket.on("newCall", (call) => {
/* Handle new emergency call */
});
socket.on("callUpdate", (event) => {
/* Handle call status update */
});
socket.on("callEscalated", (data) => {
/* Handle escalation */
});
socket.on("emergencyAlert", (alert) => {
/* Handle critical alert */
});
๐๏ธ Database Schema
Core Tables
-- User profiles and caller information
users (id, phone_number, risk_profile, preferred_language, created_at)
-- Emergency call records
calls (id, user_id, emergency_type, severity_level, status, transcription, outcome, created_at)
-- Persistent caller memory
memory_records (id, user_id, memory_type, content, created_at, updated_at)
-- Escalation tracking
escalation_log (id, call_id, escalation_type, reason, created_at)
-- Emergency services dispatch
emergency_dispatch_log (id, call_id, service_type, dispatch_time, status)
๐ค AI Agent Configuration
Letta Memory Agent
The system uses Letta AI agents for persistent caller memory:
// Caller memory structure
interface CallerMemory {
userId: string;
phoneNumber: string;
medicalHistory: string[];
previousEmergencies: EmergencyRecord[];
location: LocationData;
emergencyContacts: Contact[];
riskProfile: "LOW" | "MEDIUM" | "HIGH" | "CRITICAL";
lastCallSummary: string;
}
Emergency Assessment Protocol
- Immediate Assessment: Severity classification within 30 seconds
- Contextual Analysis: Historical pattern recognition
- Escalation Logic: Automated decision tree for human handoff
- Memory Updates: Real-time learning from each interaction
๐ VAPI Integration
Voice Assistant Configuration
// Emergency triage assistant settings
{
"name": "Emergency Triage Assistant",
"model": "gpt-4",
"voice": "professional-calm",
"systemMessage": "Emergency triage AI with medical protocols...",
"functions": ["escalate_to_human", "dispatch_emergency_services"],
"recordingEnabled": true,
"maxDurationSeconds": 1800
}
Call Flow Process
- Incoming Call: VAPI answers with emergency greeting
- Situation Assessment: Structured triage questions
- Memory Retrieval: Access caller's historical context
- AI Analysis: Severity determination and guidance
- Escalation Decision: Human/emergency services if needed
- Memory Update: Store call outcomes and learnings
๐จ Emergency Response Protocols
Severity Levels
- CRITICAL: Life-threatening (heart attack, stroke, severe bleeding)
- HIGH: Serious but stable (broken bones, difficulty breathing)
- MEDIUM: Concerning but manageable (moderate injury, infection)
- LOW: Minor issues (small cuts, mild symptoms)
Auto-Escalation Triggers
- Critical severity assessment
- High-risk caller with concerning symptoms
- Extended call duration (>10 minutes)
- Caller becomes unresponsive
- Specific keywords ("can't breathe", "unconscious")
๐ Monitoring & Analytics
Dashboard Metrics
- Active Calls: Real-time emergency call count
- Response Times: Average time to answer and assess
- Escalation Rate: Percentage requiring human intervention
- Outcome Tracking: Call resolution and follow-up
- Risk Distribution: Caller risk profile analytics
System Alerts
- High call volume detection
- Extended response times
- Service availability issues
- Critical emergency patterns
๐ Security & Compliance
Data Protection
- HIPAA-compliant medical information handling
- Encrypted database storage
- Secure API authentication
- Audit logging for all actions
Emergency Access
- Override capabilities for supervisors
- Emergency service direct dispatch
- Call recording and transcription storage
- Regulatory compliance reporting
๐ Deployment
Production Setup
# Frontend (Vercel)
vercel --prod
# Backend (Railway)
railway login
railway deploy
# Database (Railway PostgreSQL)
railway add postgresql
Environment Variables
Ensure all production environment variables are configured:
- Database connection strings
- API keys for VAPI and Letta
- JWT secrets and security tokens
- Emergency service integration credentials
๐งช Testing
Run the comprehensive test suite:
# Setup database and run all tests
npx ts-node setup-and-test.ts
# Individual test components
npm run test:emergency-flow
npm run test:dashboard
npm run test:escalation
๐ Performance Optimization
Response Time Targets
- Call Answer: < 3 seconds
- Initial Assessment: < 30 seconds
- Escalation: < 60 seconds for critical cases
- Memory Retrieval: < 1 second
Scalability Features
- Database connection pooling
- WebSocket connection management
- Caching for frequent queries
- Load balancing for high call volumes
๐ฎ Future Enhancements
- Video call capability for visual assessment
- Integration with local emergency services
- Advanced AI diagnosis and treatment recommendations
- Mobile app for emergency contacts
- Geographic routing to local resources
- Multi-language support expansion
- Wearable device integration
- Predictive emergency analysis
๐ Support & Contact
For technical support or questions about the Emergency Triage System:
- Documentation: See inline code comments and API docs
- Issues: Report bugs via the issue tracker
- Emergency Protocol Questions: Consult medical professionals
- System Status: Monitor dashboard health indicators
โ ๏ธ IMPORTANT: This system is designed to assist with emergency triage but should never replace professional medical advice or emergency services. Always call 911 or local emergency services for life-threatening situations.
Analysis
View
Metric
- 1
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- ExpressIn code
- JavaScriptIn code
- PostgreSQLIn code
- TypeScriptIn code
- ReactClaimed
4 of 5 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
87 KB
Source files
12
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
WeeeHung/riley
19 files ยท 219 KB ยท @ 1367b13
Structure
API & routing
1 file ยท 5%Request entry points: routes, handlers and controllers.
Application logic
9 files ยท 47%Domain rules, services and shared utilities.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here โ open the file browser to check anything the diagram implies.
Languages
- JavaScript81%
- Markdown13%
- TypeScript6%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
backend/package.json
npm ยท 20- @types/axios
- @types/bcryptjs
- @types/cors
- @types/express
- @types/jsonwebtoken
- @types/node
- @types/pg
- @types/uuid
- axios
- bcryptjs
- cors
- dotenv
- express
- jsonwebtoken
- nodemon
- pg
- socket.io
- ts-node
- +2 more
backend/voice-agent/package.json
npm ยท 7- axios
- cors
- dotenv
- express
- node-cron
- uuid
- +1 more
Declared in the repositoryโs manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This projectโs features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.