# Project export: Memoria This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: Always on Agentic AI companion with long term embedded memories meant for Alzheimer's and dementia patients - Devpost: https://devpost.com/software/memoria-zn70lb - GitHub: https://github.com/jason-zhxn/memoria - Video: https://www.youtube.com/embed/AGWE78snfWM?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 0 GitHub contributor(s) — ## Devpost submission (written by the team) ### Inspiration The summer before my senior year of high school, I took care of my grandmother before she passed away. She had late stage dementia, so she didn't remember who I was, or much of anything for the fact of the matter. Seeing how that frustrated and scared her led me to this idea. There's so tens of millions of seniors developing alzheimer's and dementia around the world, and as all the money has gone to medication to slow the progress of the illness, not enough thought has been put into how to help them live everyday life. ### What it does Memoria is an agentic AI companion for Alzheimer's patients featuring: Realistic Avatar Interaction - A lifelike avatar that is always on and that patients can talk to naturally, powered by HeyGen and WebRTC streaming Realistic Avatar Interaction - A lifelike avatar that is always on and that patients can talk to naturally, powered by HeyGen and WebRTC streaming Perfect Semantic Memory - Every conversation is analyzed by multiple AI agents. These agents can extract meaningful facts, store them, retrieve similar facts, contact emergency contacts, search for medical advice, etc. Has functionally infinite memory, filters and stores facts worth storing in a database and uses elastic search to access relevant context during live conversations has in built timers that allow Memoria to proactively reach out with reminders (if you tell Memoria you usually take meds at 7pm in normal conversation, it will start reminding you so you don't forget) Perfect Semantic Memory - Every conversation is analyzed by multiple AI agents. These agents can extract meaningful facts, store them, retrieve similar facts, contact emergency contacts, search for medical advice, etc. Has functionally infinite memory, filters and stores facts worth storing in a database and uses elastic search to access relevant context during live conversations has in built timers that allow Memoria to proactively reach out with reminders (if you tell Memoria you usually take meds at 7pm in normal conversation, it will start reminding you so you don't forget) Proactive Safety System - Automatically warns about medication interactions and allergy risks Proactive Safety System - Automatically warns about medication interactions and allergy risks Voice-First Design - No typing or complex apps to navigate—just talk Voice-First Design - No typing or complex apps to navigate—just talk ### How we built it Lots of AI help lol :) Multi-Agent Architecture: Classifier Agent - Determines if information is worth remembering Extraction Agent - Pulls out key facts (names, relationships, medications) Medical Safety Agent - Cross-references allergies and drug interactions Response Agent - Generates warm, patient-centered replies Tech Stack: Frontend: Next.js 15, React 19, TypeScript, TailwindCSS Backend: FastAPI, Python, LangGraph for agentic orchestration AI: GPT-4o-mini, OpenAI TTS, semantic embeddings Memory: Elasticsearch with vector search Avatar: HeyGen LiveAvatar SDK, LiveKit WebRTC ### Challenges we ran into Integrating real-time avatar lip-syncing with custom TTS audio Building a semantic memory system that retrieves relevant context without keyword matching Designing agent coordination so safety checks happen automatically without slowing responses ### Accomplishments we're proud of The avatar feels genuinely present—patients respond to faces better than text Our safety agent caught a simulated allergy interaction in testing Memory retrieval works semantically ("Where does my daughter live?" matches "Sarah moved to Seattle") Our safety agent caught a simulated allergy interaction in testing Memory retrieval works semantically ("Where does my daughter live?" matches "Sarah moved to Seattle") ### What we learned Agentic AI isn't just about calling tools—it's about orchestrating multiple specialized systems Voice interfaces need careful latency optimization Designing for vulnerable users requires extra care around tone and safety ### What's next Proactive check-ins via SMS when patients haven't interacted Caregiver dashboard with conversation summaries and alerts Integration with smart home devices for ambient reminders HIPAA compliance for healthcare deployment ## README (from the GitHub repository) # Memoria - Alzheimer's Digital Companion An intelligent, proactive digital companion for Alzheimer's patients with perfect memory retrieval, medical safety checks, and natural conversational AI through an avatar interface. ## 🎯 Project Overview Memoria combines cutting-edge AI technologies to create a compassionate companion that: - **Remembers Everything**: Semantic memory storage with Elasticsearch + JINA v3 embeddings - **Conversational Avatar**: Natural interaction via HeyGen Streaming Avatar + LiveKit - **Intelligent Reasoning**: Multi-turn conversations powered by LangGraph + GPT-4 - **Medical Safety**: Basic allergy detection and health disclaimer system ## 🏗 Architecture ``` ┌─────────────────────────────────────────────────────────────┐ │ Frontend (Next.js) │ │ • HeyGen Streaming Avatar (WebRTC via LiveKit) │ │ • Real-time voice/video interaction │ └─────────────────────┬───────────────────────────────────────┘ │ ┌─────────────────────┴───────────────────────────────────────┐ │ Backend API (FastAPI) │ │ • LangGraph orchestrator (multi-turn reasoning) │ │ • WebSocket for streaming responses │ └─────────┬─────────────────┬───────────────────────────────┘ │ │ ┌─────▼────┐ ┌────▼─────┐ │Elasticsearch│ │ OpenAI │ │ + JINA v3 │ │ GPT-4 │ │ (Memory) │ │(Reasoning)│ └─────────────┘ └──────────┘ ``` ## 🚀 Quick Start ### Prerequisites 1. **API Keys Required** (see `.env.example`): - OpenAI API key ($10 credit recommended) - Elastic Cloud (14-day free trial) - HeyGen Streaming Avatar ($30/month or trial) - LiveKit Cloud (50 GB/month free) ### Installation ```bash # 1. Clone the repository git clone cd memoria # 2. Set up backend cd server python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate pip install -r requirements.txt # 3. Set up frontend cd ../web npm install # 4. Configure environment cp .env.example .env # Edit .env with your API keys # 5. Verify API connectivity cd ../server python doctor.py # Should show: ✓ OpenAI, ✓ Elasticsearch, ✓ HeyGen, ✓ LiveKit ``` ### Running Locally ```bash # Terminal 1: Start backend cd server uvicorn main:app --reload # Backend runs on http://localhost:8000 # Terminal 2: Start frontend cd web npm run dev # Frontend runs on http://localhost:3000 ``` ### Seed Demo Data ```bash cd scripts python seed_demo_data.py # Pre-loads demo memories and user profile ``` ## 📦 Project Structure ``` memoria/ ├── server/ # Python FastAPI backend │ ├── main.py # FastAPI app entry point │ ├── doctor.py # API health check │ ├── orchestrator.py # LangGraph orchestrator │ ├── services/ # Memory, embeddings services │ ├── graph/ # LangGraph nodes & state machine │ ├── models/ # Pydantic models │ ├── tools/ # LangChain tools │ └── api/ # Chat & WebSocket endpoints ├── web/ # Next.js frontend │ ├── pages/ # Next.js pages │ ├── components/ # React components (Avatar, VoiceInput) │ ├── hooks/ # Custom hooks (useAvatar, useChat) │ └── lib/ # API client, WebSocket ├── scripts/ # Utility scripts ├── docs/ # Documentation └── .env.example # Environment template ``` ## 🧪 Testing ```bash # Backend tests cd server pytest # Specific test pytest server/tests/test_memory.py -v # Frontend type check cd web npm run type-check ``` ## 🎬 Demo Scenarios ### 1. Memory Retrieval - **You**: "What's my daughter's name?" - **Avatar**: "Your daughter is Sarah. She lives in Seattle." ### 2. Learning New Information - **You**: "My favorite color is blue" - **Avatar**: "Got it! I've noted that your favorite color is blue." ### 3. Medical Safety - **You**: "I have a headache. What should I take?" - **Avatar**: "I see you're allergic to aspirin. Please consult your doctor for safe alternatives." ## 🚢 Deployment ### Frontend (Vercel) ```bash cd web vercel --prod # Configure environment variables in Vercel dashboard ``` ### Backend (Railway/Render) 1. Connect GitHub repo to Railway/Render 2. Set environment variables (OpenAI, Elastic, etc.) 3. Deploy with auto-scaling enabled See [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md) for detailed instructions. ## 📚 Documentation - [ARCHITECTURE.md](docs/ARCHITECTURE.md) - System design and components - [API.md](docs/API.md) - API endpoints and WebSocket protocol - [DEPLOYMENT.md](docs/DEPLOYMENT.md) - Production deployment guide - [DEMO_SCRIPT.md](docs/DEMO_SCRIPT.md) - 5-minute demo walkthrough ## 🛠 Technology Stack | Category | Technology | |----------|-----------| | Frontend | Next.js 15, React 19, TypeScript, TailwindCSS | | Backend | FastAPI, Python 3.11+ | | Memory | Elasticsearch Serverless, JINA v3 embeddings | | Avatar | HeyGen Streaming SDK v2, LiveKit WebRTC | | AI | LangChain, LangGraph, OpenAI GPT-4o-mini | | Deployment | Vercel (frontend), Railway/Render (backend) | ## 🔮 Future Roadmap - **Phase 2**: Proactive messaging via SMS/iMessage (Poke integration) - **Phase 4**: Web automation for email monitoring and appointment booking - **Phase 6**: Monetization with Fetch.ai uAgents and micro-payments - **Production**: HIPAA compliance, caregiver dashboard, mobile app ## 🤝 Contributing This is a demo project. For production use, additional security, HIPAA compliance, and medical validation are required. ## ⚠️ Disclaimer Memoria is a demo application for educational and research purposes. It is NOT a medical device and should not be used for actual medical advice. Always consult healthcare professionals for medical decisions. ## 📄 License MIT License - See LICENSE file for details --- **Built with ❤️ for those affected by Alzheimer's disease** ## Detected evidence (automated analysis) Indexed codebase: 36 recognized source files, 177 KB. - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - JavaScript (language) — detected in the code - LangChain (technology) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code ## Codebase structure (from repository index) ### Files (47 of 47) ``` .env.example .gitignore DEMO_SCRIPT.md docs/API.md docs/ARCHITECTURE.md docs/ELASTICSEARCH_SETUP.md README.md scripts/reset_elasticsearch.py scripts/seed_demo_data.py server/api/index.py server/api/memory.py server/Dockerfile server/doctor.py server/main.py server/models/memory.py server/requirements.txt server/services/chat_service.py server/services/embeddings.py server/services/memory_service.py server/tests/test_memory.py server/tools/memory_tool.py server/vercel.json vercel.json web/.env.local.example web/components/Avatar.tsx web/components/AvatarEmbed.tsx web/components/ChatHistory.tsx web/components/VoiceInput.tsx web/hooks/useAvatar.ts web/hooks/useChat.ts web/hooks/useVoiceRecognition.ts web/lib/api.ts web/next.config.js web/package.json web/pages/_app.tsx web/pages/api/tts.ts web/pages/companion.tsx web/pages/index.tsx web/pages/voice.tsx web/postcss.config.js web/styles/globals.css web/tailwind.config.js web/test-heygen.js web/test-liveavatar.js web/tsconfig.json web/tsconfig.tsbuildinfo web/vercel.json ``` ### Dependencies - server/requirements.txt: aiohttp@==3.11.7, elasticsearch@==8.16.0, fastapi@==0.115.0, httpx@==0.28.1, langchain@==0.3.7, langchain-openai@==0.2.9, langgraph@==0.2.45, openai@==1.55.3, pydantic@==2.10.3, pydantic-settings@==2.6.1, pytest@==8.3.4, pytest-asyncio@==0.24.0, python-dotenv@==1.0.1, python-multipart@==0.0.20, uvicorn[standard]@==0.32.0, websockets@==14.1 - web/package.json: @heygen/liveavatar-web-sdk@^0.0.10, @heygen/streaming-avatar@^2.0.0, @tailwindcss/postcss@^4.0.0, @types/node@^22.10.2, @types/react@^19.0.2, @types/react-dom@^19.0.2, axios@^1.7.9, clsx@^2.1.1, eslint@^9.17.0, eslint-config-next@^15.1.3, livekit-client@^2.5.0, lucide-react@^0.468.0, next@^15.1.3, react@^19.0.0, react-dom@^19.0.0, tailwindcss@^4.0.0, typescript@^5.7.2 ### Recent commits (newest first) - lint fix - lint fix - lint fix - lint fix - lint fix - lint fix - lint fix - lint fix - lint fix - lint fix - fix avatar attempt - fix avatar attempt - fix avatar attempt - fix avatar attempt - fix avatar attempt - fix avatar attempt - fix avatar attempt - fix avatar attempt - fix avatar attempt - fix avatar attempt ## Key source files (fetched from GitHub, selected and truncated for size) ### DEMO_SCRIPT.md ```markdown # Memoria - TreeHacks Demo Script ## Opening Hook (30 seconds) > "Every 65 seconds, someone in America develops Alzheimer's. That's over 6 million people who struggle daily to remember their loved ones' names, their medications, or even what they had for breakfast. > **This is personal for me.** The summer before my senior year of high school, I took care of my grandmother before she passed away. I watched her struggle to remember my name, forget to take her medications, and feel frustrated when she couldn't recall simple things about her own life. > That experience stuck with me. And it's why we built **Memoria** - an AI companion that never forgets, so your loved ones don't have to." --- ## The Problem (30 seconds) > "Current solutions for Alzheimer's patients are either: > - **Passive** - photo albums, journals that patients forget to check > - **Clinical** - cold, institutional apps that feel like medical devices > - **Dangerous** - no safety checks for medication interactions or allergies > We asked: what if your loved one had a warm, patient companion available 24/7 who remembers everything about them and keeps them safe?" --- ## The Solution: Memoria (1 minute) > "Memoria is an **agentic AI companion** with three core innovations:" ### 1. Natural Avatar Interaction > "We use a realistic avatar powered by HeyGen and WebRTC streaming. Patients can just *talk* - no typing, no apps to navigate. The avatar responds with natural speech and lip-syncing, creating a human-like connection." ### 2. Perfect Semantic Memory > "Every conversation is processed by our **multi-agent system**. When a patient says 'My daughter Sarah lives in Seattle,' multiple AI agents activate simultaneously:" **[GESTURE TO ARCHITECTURE DIAGRAM IF YOU HAVE ONE]** > - **The Classifier Agent** determines this is personal information worth remembering > - **The Extraction Agent** pulls out the key facts: daughter, name=Sarah, location=Seattle > - **The Medical Safety Agent** checks if anything mentioned could interact with their medications or allergies > - **The Storage Agent** creates semantic embeddings and stores it in our vector database > "Later, when they ask 'Where does my daughter live?' - we retrieve it instantly through semantic search, not keyword matching." ### 3. Proactive Safety System > "This is what makes Memoria more than a chatbot. Our **Medical Guardian Agent** continuously monitors conversations for potential dangers:" > - Patient mentions wanting to eat shrimp? Agent checks their allergy profile and warns them. > - Patient asks about taking Advil? Agent cross-references their medication list for interactions. > - Patient seems confused or distressed? Agent can alert emergency contacts. > "We're not just storing memories - we're actively protecting patients." --- ## Live Demo Flow (2-3 minutes) ### Setup > "Let me show you Memoria in action. I'm logged in as a demo patient who has some pre-existing memories and a medical profile." # [truncated — 4643 more characters] ``` ### docs/ELASTICSEARCH_SETUP.md ```markdown # Elasticsearch & Embeddings Setup Guide ## Quick Start (OpenAI Embeddings - Recommended) The easiest way to get started is using OpenAI embeddings (you already have the API key): 1. Make sure your `.env` has: ```bash USE_OPENAI_EMBEDDINGS=true OPENAI_API_KEY=sk-... ELASTIC_CLOUD_ID=... ELASTIC_API_KEY=... ``` 2. That's it! The system will automatically use OpenAI's `text-embedding-3-small` model (1536 dimensions). ## Advanced: JINA v3 Setup (Optional) If you want to use JINA v3 embeddings via Elastic Inference Service instead: ### Step 1: Create JINA Inference Endpoint 1. Go to your Elastic Cloud dashboard: https://cloud.elastic.co/deployments 2. Click on your deployment 3. Navigate to **"Machine Learning"** → **"Trained Models"** 4. Click **"Create inference endpoint"** 5. Configure: - **Name**: `memoria-embeddings` - **Task type**: Text Embedding - **Model**: Select JINA v3 (or upload if not available) - **Dimensions**: 1024 ### Step 2: Update Environment Variables ```bash USE_OPENAI_EMBEDDINGS=false ELASTIC_INFERENCE_ENDPOINT=memoria-embeddings ``` ### Step 3: Verify Setup ```bash cd server python -c "from services.embeddings import embedding_service; import asyncio; asyncio.run(embedding_service.embed_text('test'))" ``` ## Comparison: OpenAI vs JINA | Feature | OpenAI | JINA v3 | |---------|--------|---------| | Setup | ✅ Easy (API key) | ⚠️ Complex (Elastic setup) | | Cost | ~$0.02/1M tokens | Depends on Elastic tier | | Dimensions | 1536 | 1024 | | Quality | Excellent | Excellent | | Speed | ~50ms | ~30ms (after warm-up) | **Recommendation**: Use OpenAI embeddings for the demo. Switch to JINA if you need: - Lower latency (self-hosted) - More control over embedding model - Cost savings at scale ## Troubleshooting ### Error: "Could not find trained model" - JINA endpoint not set up correctly - Solution: Set `USE_OPENAI_EMBEDDINGS=true` in `.env` ### Error: "Zero magnitude vector" - Embedding generation failed and returned zeros - Solution: Check your embedding service logs and ensure API keys are valid ### Error: "OpenAI API error" - Invalid API key or rate limit - Solution: Verify `OPENAI_API_KEY` in `.env` ## Switching Between OpenAI and JINA To switch embedding providers: 1. Update `.env`: ```bash USE_OPENAI_EMBEDDINGS=false # or true ``` 2. **Important**: Delete the old index (embeddings have different dimensions): ```bash curl -X DELETE "http://localhost:9200/memoria_memories" ``` 3. Restart backend and re-seed data: ```bash uvicorn main:app --reload python scripts/seed_demo_data.py ``` ## Performance Tuning ### OpenAI - Use batch embedding for multiple texts (already implemented) - Consider caching frequently accessed embeddings ### JINA - Increase Elastic ML node resources for better throughput - Use dedicated ML nodes for production ``` ### server/requirements.txt ``` # FastAPI Web Framework fastapi==0.115.0 uvicorn[standard]==0.32.0 # LangChain & LangGraph langchain==0.3.7 langchain-openai==0.2.9 langgraph==0.2.45 # Elasticsearch for Memory elasticsearch==8.16.0 # OpenAI Integration openai==1.55.3 # Data Validation pydantic==2.10.3 pydantic-settings==2.6.1 # Environment & Configuration python-dotenv==1.0.1 # HTTP Client aiohttp==3.11.7 httpx==0.28.1 # WebSocket Support websockets==14.1 # Testing pytest==8.3.4 pytest-asyncio==0.24.0 # CORS Support python-multipart==0.0.20 ``` ### server/Dockerfile ``` # Memoria Backend - Docker Configuration FROM python:3.11-slim # Set working directory WORKDIR /app # Install system dependencies RUN apt-get update && apt-get install -y \ gcc \ && rm -rf /var/lib/apt/lists/* # Copy requirements COPY requirements.txt . # Install Python dependencies RUN pip install --no-cache-dir -r requirements.txt # Copy application code COPY . . # Expose port EXPOSE 8000 # Health check HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \ CMD python -c "import requests; requests.get('http://localhost:8000/health')" # Run the application CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"] ``` ### web/package.json ``` { "name": "memoria-web", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint", "type-check": "tsc --noEmit" }, "dependencies": { "@heygen/liveavatar-web-sdk": "^0.0.10", "@heygen/streaming-avatar": "^2.0.0", "axios": "^1.7.9", "clsx": "^2.1.1", "livekit-client": "^2.5.0", "lucide-react": "^0.468.0", "next": "^15.1.3", "react": "^19.0.0", "react-dom": "^19.0.0" }, "devDependencies": { "@tailwindcss/postcss": "^4.0.0", "@types/node": "^22.10.2", "@types/react": "^19.0.2", "@types/react-dom": "^19.0.2", "eslint": "^9.17.0", "eslint-config-next": "^15.1.3", "tailwindcss": "^4.0.0", "typescript": "^5.7.2" } } ``` ### server/main.py ```python """ Memoria Backend API - FastAPI Application Entry Point """ from fastapi import FastAPI from fastapi.middleware.cors import CORSMiddleware from pydantic import BaseModel from typing import Optional, Dict, List from collections import defaultdict from datetime import datetime, timedelta import os from dotenv import load_dotenv # Load environment variables load_dotenv() # In-memory conversation history store # Key: user_id, Value: list of {role, content, timestamp} conversation_store: Dict[str, List[dict]] = defaultdict(list) MAX_HISTORY_LENGTH = 20 # Keep last 20 messages per user HISTORY_TTL_HOURS = 24 # Clear history older than 24 hours def get_conversation_history(user_id: str) -> List[dict]: """Get recent conversation history for a user, filtering out old messages""" cutoff = datetime.utcnow() - timedelta(hours=HISTORY_TTL_HOURS) # Filter out old messages history = [ msg for msg in conversation_store[user_id] if msg.get("timestamp", datetime.utcnow()) > cutoff ] # Update store with filtered history conversation_store[user_id] = history # Return last N messages (without timestamp for API) return [{"role": msg["role"], "content": msg["content"]} for msg in history[-MAX_HISTORY_LENGTH:]] def add_to_conversation(user_id: str, role: str, content: str): """Add a message to conversation history""" conversation_store[user_id].append({ "role": role, "content": content, "timestamp": datetime.utcnow() }) # Trim if too long if len(conversation_store[user_id]) > MAX_HISTORY_LENGTH * 2: conversation_store[user_id] = conversation_store[user_id][-MAX_HISTORY_LENGTH:] # Initialize FastAPI app app = FastAPI( title="Memoria API", description="Alzheimer's Digital Companion Backend", version="0.1.0", ) # CORS configuration for frontend origins = [ "http://localhost:3000", "http://localhost:3001", "https://*.vercel.app", ] app.add_middleware( CORSMiddleware, allow_origins=origins, allow_credentials=True, allow_methods=["*"], allow_headers=["*"], ) # Pydantic models for request/response class HealthResponse(BaseModel): status: str services: dict class ChatRequest(BaseModel): user_id: str message: str class ChatResponse(BaseModel): response: str memories_retrieved: Optional[list] = [] # Health check endpoint @app.get("/health", response_model=HealthResponse) async def health_check(): """ Health check endpoint - verifies API is running """ return { "status": "healthy", "services": { "api": "running", "memory": "pending", # Will be updated when Elasticsearch is integrated "reasoning": "pending", # Will be updated when LangGraph is integrated } } # Root endpoint @app.get("/") async def root(): """ Root endpoint - API information """ return { "name": "Memoria API", "version": "0.1.0", "description": "Alzheimer's Digital Companion - Backend API", "docs": "/docs", "health": "/health", } # Import memory API routes from api.memory import router as memory_router # Include memory routes app.include_router(memory_router) # Chat endpoint with GPT-4 and memory integration @app.post("/api/chat", response_model=ChatResponse) async def chat(request: ChatRequest): """ Chat endpoint - handles user messages with GPT-4 and memory context """ from services.memory_service import memory_service from services.chat_service import chat_service # Get conversation history for context history = get_conversation_history(request.user_id) print(f"💬 Conversation history: {len(history)} messages") # Search for relevant memories memories = [] try: memories = await memory_service.search_memories( query=request.message, user_id=request.user_id, top_k=3 ) except Exception as e: print(f"Memory search failed: {e}") # Generate response using GPT-4 with memory and conversation context try: response = await chat_service.generate_response( message=request.message, memories=memories, user_id=request.user_id, conversation_history=history ) except Exception as e: print(f"Chat generation failed: {e}") # Fallback response response = "I'm having a little trouble right now. Could you please try again?" # Store this exchange in conversation history add_to_conversation(request.user_id, "user", request.message) add_to_conversation(request.user_id, "assistant", response) # Extract and store new memories from user message try: print(f"🧠 Starting memory extraction for: {request.message[:50]}...") extracted = await chat_service.extract_memories( message=request.message, user_id=request.user_id ) print(f"🧠 Extraction returned: {extracted}") for fact in extracted: content = fact.get("content") category = fact.get("category", "general") reminder_time = fact.get("time") # Time for reminders if content: await memory_service.add_memory( content=content, category=category, user_id=request.user_id, reminder_time=reminder_time, metadata={"source": "auto_extracted", "original_message": request.message[:100]} ) print(f"✓ Stored memory: {content[:50]}... (time: {reminder_time})") except Exception as e: print(f"Memory extraction failed: {type(e).__name__}: {e}") import traceback traceback.print_exc() return { "response": response, "memories_retrieved": [m.model_dump() for m in memories] if memories els [truncated — 1456 more characters] ``` ### server/api/index.py ```python # Vercel serverless handler for FastAPI import sys import os # Add parent directory to path so we can import from there sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) from main import app # Vercel looks for 'app' or 'handler' handler = app ``` ### web/pages/index.tsx ```typescript import Head from 'next/head'; import Link from 'next/link'; export default function Home() { return ( <> Memoria - Alzheimer's Digital Companion

Memoria

Your Personal Memory Companion

An intelligent digital companion designed to help with memory retrieval, medical safety, and daily tasks through natural conversation.

🧠

Perfect Memory

Semantic search powered by Elasticsearch + JINA

💬

Natural Conversation

Interactive avatar with HeyGen + LiveKit

🛡️

Medical Safety

Allergy detection and health disclaimers

Start Conversation

Note: Memoria is a demo application for educational purposes.

Not intended for actual medical advice. Always consult healthcare professionals.

); } ``` ### web/postcss.config.js ```javascript module.exports = { plugins: { '@tailwindcss/postcss': {}, }, }; ``` ### web/tailwind.config.js ```javascript /** @type {import('tailwindcss').Config} */ export default { content: [ './pages/**/*.{js,ts,jsx,tsx,mdx}', './components/**/*.{js,ts,jsx,tsx,mdx}', './app/**/*.{js,ts,jsx,tsx,mdx}', './hooks/**/*.{js,ts,jsx,tsx}', './lib/**/*.{js,ts,jsx,tsx}', ], theme: { extend: { colors: { primary: { 50: '#f0f9ff', 100: '#e0f2fe', 500: '#0ea5e9', 600: '#0284c7', 700: '#0369a1', }, }, animation: { 'pulse-slow': 'pulse 3s cubic-bezier(0.4, 0, 0.6, 1) infinite', }, }, }, plugins: [], }; ``` [28 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]