Memoria - Alzheimer's Digital Companion

An intelligent, proactive digital companion for Alzheimer's patients with perfect memory retrieval, medical safety checks, and natural conversational AI through an avatar interface.

🎯 Project Overview

Memoria combines cutting-edge AI technologies to create a compassionate companion that:

• Remembers Everything: Semantic memory storage with Elasticsearch + JINA v3 embeddings
• Conversational Avatar: Natural interaction via HeyGen Streaming Avatar + LiveKit
• Intelligent Reasoning: Multi-turn conversations powered by LangGraph + GPT-4
• Medical Safety: Basic allergy detection and health disclaimer system

🏗 Architecture

┌─────────────────────────────────────────────────────────────┐
│                   Frontend (Next.js)                         │
│  • HeyGen Streaming Avatar (WebRTC via LiveKit)            │
│  • Real-time voice/video interaction                        │
└─────────────────────┬───────────────────────────────────────┘
                      │
┌─────────────────────┴───────────────────────────────────────┐
│                Backend API (FastAPI)                         │
│  • LangGraph orchestrator (multi-turn reasoning)            │
│  • WebSocket for streaming responses                        │
└─────────┬─────────────────┬───────────────────────────────┘
          │                 │
    ┌─────▼────┐      ┌────▼─────┐
    │Elasticsearch│      │  OpenAI  │
    │  + JINA v3  │      │  GPT-4   │
    │  (Memory)   │      │(Reasoning)│
    └─────────────┘      └──────────┘

🚀 Quick Start

Prerequisites

1. API Keys Required (see .env.example):
   • OpenAI API key ($10 credit recommended)
   • Elastic Cloud (14-day free trial)
   • HeyGen Streaming Avatar ($30/month or trial)
   • LiveKit Cloud (50 GB/month free)

Installation

# 1. Clone the repository
git clone <your-repo-url>
cd memoria

# 2. Set up backend
cd server
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

# 3. Set up frontend
cd ../web
npm install

# 4. Configure environment
cp .env.example .env
# Edit .env with your API keys

# 5. Verify API connectivity
cd ../server
python doctor.py
# Should show: ✓ OpenAI, ✓ Elasticsearch, ✓ HeyGen, ✓ LiveKit

Running Locally

# Terminal 1: Start backend
cd server
uvicorn main:app --reload
# Backend runs on http://localhost:8000

# Terminal 2: Start frontend
cd web
npm run dev
# Frontend runs on http://localhost:3000

Seed Demo Data

cd scripts
python seed_demo_data.py
# Pre-loads demo memories and user profile

📦 Project Structure

memoria/
├── server/                # Python FastAPI backend
│   ├── main.py           # FastAPI app entry point
│   ├── doctor.py         # API health check
│   ├── orchestrator.py   # LangGraph orchestrator
│   ├── services/         # Memory, embeddings services
│   ├── graph/            # LangGraph nodes & state machine
│   ├── models/           # Pydantic models
│   ├── tools/            # LangChain tools
│   └── api/              # Chat & WebSocket endpoints
├── web/                   # Next.js frontend
│   ├── pages/            # Next.js pages
│   ├── components/       # React components (Avatar, VoiceInput)
│   ├── hooks/            # Custom hooks (useAvatar, useChat)
│   └── lib/              # API client, WebSocket
├── scripts/               # Utility scripts
├── docs/                  # Documentation
└── .env.example          # Environment template

🧪 Testing

# Backend tests
cd server
pytest

# Specific test
pytest server/tests/test_memory.py -v

# Frontend type check
cd web
npm run type-check

🎬 Demo Scenarios

1. Memory Retrieval

• You: "What's my daughter's name?"
• Avatar: "Your daughter is Sarah. She lives in Seattle."

2. Learning New Information

• You: "My favorite color is blue"
• Avatar: "Got it! I've noted that your favorite color is blue."

3. Medical Safety

• You: "I have a headache. What should I take?"
• Avatar: "I see you're allergic to aspirin. Please consult your doctor for safe alternatives."

🚢 Deployment

Frontend (Vercel)

cd web
vercel --prod
# Configure environment variables in Vercel dashboard

Backend (Railway/Render)

1. Connect GitHub repo to Railway/Render
2. Set environment variables (OpenAI, Elastic, etc.)
3. Deploy with auto-scaling enabled

See docs/DEPLOYMENT.md for detailed instructions.

📚 Documentation

• ARCHITECTURE.md - System design and components
• API.md - API endpoints and WebSocket protocol
• DEPLOYMENT.md - Production deployment guide
• DEMO_SCRIPT.md - 5-minute demo walkthrough

🛠 Technology Stack

Category	Technology
Frontend	Next.js 15, React 19, TypeScript, TailwindCSS
Backend	FastAPI, Python 3.11+
Memory	Elasticsearch Serverless, JINA v3 embeddings
Avatar	HeyGen Streaming SDK v2, LiveKit WebRTC
AI	LangChain, LangGraph, OpenAI GPT-4o-mini
Deployment	Vercel (frontend), Railway/Render (backend)

🔮 Future Roadmap

• Phase 2: Proactive messaging via SMS/iMessage (Poke integration)
• Phase 4: Web automation for email monitoring and appointment booking
• Phase 6: Monetization with Fetch.ai uAgents and micro-payments
• Production: HIPAA compliance, caregiver dashboard, mobile app

🤝 Contributing

This is a demo project. For production use, additional security, HIPAA compliance, and medical validation are required.

⚠️ Disclaimer

Memoria is a demo application for educational and research purposes. It is NOT a medical device and should not be used for actual medical advice. Always consult healthcare professionals for medical decisions.

📄 License

MIT License - See LICENSE file for details

---

Built with ❤️ for those affected by Alzheimer's disease