# Project export: MemoryVault - AI Memory Companion for Alzheimer's Patients

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: CruzHacks 2025
- Tagline: Empowering Alzheimer's patients to relive and reconnect with their precious memories through AI-powered conversations and visual experiences.
- Devpost: https://devpost.com/software/memoryvault-ai-memory-companion-for-alzheimer-s-patients-63i1ag
- GitHub: https://github.com/Monishg2004/MemoryVault.git
- Video: https://www.youtube.com/embed/3EWIQxljLpQ?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 0 GitHub contributor(s) — 

## Devpost submission (written by the team)

### Overview

An AI-powered platform helping Alzheimer's patients reconnect with their past through natural conversations and AI-generated visuals. 🏆 Elevator Pitch MemoryVault is an innovative AI platform designed to help Alzheimer's patients preserve and relive their cherished memories. Through advanced AI, it enables natural conversations about past experiences while generating visuals that enhance recollection. Families can actively contribute, creating a collaborative space for memory preservation. By combining interactive dialogue with AI-generated imagery, MemoryVault strengthens emotional connections and improves well-being. 💡 The Problem It Solves Alzheimer’s disease gradually erases memories, leaving patients feeling lost and disconnected. Traditional solutions like photo albums and reminders are passive and lack interaction. MemoryVault solves this by: ✅ Enabling conversational memory retrieval ✅ Creating AI-generated visual representations ✅ Providing an immersive and interactive experience ✅ Encouraging family participation in memory preservation ✅ Supporting emotional well-being through engagement 🌟

### Inspiration

The heartbreaking impact of Alzheimer's inspired us to go beyond traditional memory aids. We envisioned a platform that not only stores memories but brings them to life—helping patients maintain their identity and connections with loved ones. ⚙️ How We Built It 🔹 Technical Architecture 🔹 Backend (Flask + Python): RESTful API for memory management SentenceTransformer for vector embeddings Pinecone vector database for storage Google Gemini Pro for natural language processing FLUX API for image generation 🔹 Memory Processing Pipeline: Text chunking & optimization Vector embedding creation Metadata management Efficient storage & retrieval 🔹 Core Features: 🗣️ Conversational memory retrieval 🎨 AI-powered visual generation 🏡 Family memory contribution 🔍 Interactive memory exploration 🚧 Challenges We Faced 1️⃣ Vector Database Implementation – Optimizing memory storage & retrieval 📚 2️⃣ AI Integration – Coordinating multiple AI models for natural flow 🤖 3️⃣ User Experience – Designing an intuitive interface for elderly users 👵🏻👴🏻 🏅 Accomplishments ✅ Successfully integrated multiple AI technologies into one platform ✅ Built an efficient vector-based memory storage & retrieval system ✅ Implemented context-aware natural conversation capabilities ✅ Developed a scalable architecture for multiple users ✅ Created a user-friendly interface accessible to elderly users 🎓 What We Learned 📌 Vector database optimization & memory retrieval 📌 Large Language Model (LLM) integration 📌 AI-powered image generation techniques 📌 Scalable Flask application development 📌 Cross-platform integration strategies 🚀 What's Next for MemoryVault 🌟 Enhanced Features 🎙️ Voice interaction capabilities 📂 Advanced memory organization tools 🕶️ Virtual reality (VR) integration 📱 Mobile application development 🌍 Platform Expansion 🏥 Healthcare provider partnerships 🎓 Research institution collaboration 🌎 Multi-language support 🔒 Enhanced security features ❤️ Community Development 👨‍👩‍👦 Family account management 📖 Memory sharing capabilities 🤝 Support group integration 👩‍⚕️ Professional caregiver tools 🛠 Built With: Flask • Python • Pinecone • Google Gemini Pro • FLUX API 💙 MemoryVault – Because every memory matters.

## README (from the GitHub repository)

# 🧐 MemoryVault: Compassionate AI Memory Companion

**An AI-powered assistant for Alzheimer's and Dementia patients**
*Built with Generative AI, Face Recognition, RAG, and Emotional Intelligence*

---

## 📌 Overview

**MemoryVault** is a powerful, emotionally aware AI system designed to help patients with **Alzheimer’s and Dementia** recall memories, recognize loved ones, and reduce feelings of **loneliness and confusion**. The system combines **Generative AI**, **Facial Recognition**, **Text-to-Speech/Speech-to-Text**, **Pinecone Vector DB**, and **RAG (Retrieval Augmented Generation)** to bring a personalized, compassionate memory companion to life.

---

## 🌟 Key Features

| Feature                     | Description                                                           |
| --------------------------- | --------------------------------------------------------------------- |
| 👤 Facial Recognition       | Identifies the person speaking using camera input                     |
| 🧠 Memory Recall (RAG)      | Fetches relevant personal memories using vector search                |
| 💬 Gemini AI Chatbot        | Provides emotionally intelligent, memory-based responses              |
| 🗣️ STT + TTS               | Converts voice-to-text (Speech Recognition) and back to audio replies |
| 🖼️ Flux API (Image Gen)    | Generates memory-related visuals from chat context                    |
| 📜 Chat Summary & Narrative | Summarizes the conversation into a personal life story                |
| 🔐 Memory Vault             | Securely stores and manages multimedia memory data                    |

---

## ⚙️ Tech Stack

| Component     | Technologies Used                                        |
| ------------- | -------------------------------------------------------- |
| Frontend      | React, Tailwind CSS                                      |
| Backend       | **FastAPI** (face + voice), **Flask** (memory chatbot)   |
| Vector DB     | **Pinecone**                                             |
| Embeddings    | **SentenceTransformers (mpnet)**                         |
| Image Gen     | HuggingFace **Flux** API                                 |
| Chat AI       | **Gemini 2.0 Flash** (Google Generative AI)              |
| Voice         | **gTTS**, **SpeechRecognition**                          |
| Face Matching | **face\_recognition**, **OpenCV**                        |
| Storage       | JSON (chat), Pickle (face data), Local FS (images/audio) |

---

## 🧠 System Architecture

### 🔹 1. **Face Recognition Module (FastAPI)**

* Uses `face_recognition` lib to detect and encode known faces
* Stores encodings using `pickle` in `EncodeFile.p`
* When a user interacts (via webcam or uploaded photo), the system:

  * Preprocesses image → locates face → encodes → compares with stored faces
  * If matched → returns name and confidence
  * If unknown → stores in `UnknownImages/`

### 🔹 2. **Gemini Chatbot & Text-to-Speech**

* User input (text or speech) is converted and passed to Gemini
* A **custom prompt** instructs Gemini to behave like an *empathetic memory companion*
* Gemini responds with a **brief, caring message**
* The reply is **converted to speech** using `gTTS` and sent as an MP3 audio file

### 🔹 3. **Chat History Management**

* Conversations are stored in `chat_history.json`
* Each message has `role`, `content`, `timestamp`, and `UUID`
* A `/get-narrative` endpoint crafts a **summary life story** from the chat using Gemini

### 🔹 4. **RAG MemoryVault Module (Flask + Pinecone)**

* Users add **personal memories** using `/postMemory`

  * Text is split into **overlapping chunks**
  * Each chunk is **vectorized using SentenceTransformer**
  * Metadata (e.g., source, topic) is attached
  * Vectors are **stored in Pinecone**
* When user asks a question:

  * Query is embedded and matched against Pinecone
  * Top 5 matching memory chunks are used as **context for Gemini**
  * Gemini generates a **context-aware response** (no reference to AI or search)

### 🔹 5. **Image Generation (FLUX API)**

* When asked for visualizations:

  * First memory chapter is extracted
  * A prompt is sent to FLUX to generate a **nostalgic, warm image**
  * The image is stored locally and returned with the story

---

## 🚀 Working Flow

### 🧹 Module 1: RecallMe (FastAPI)

```mermaid
sequenceDiagram
User --> React UI: Starts Conversation
React UI --> FastAPI: Uploads Face + Audio
FastAPI --> face_recognition: Matches Face
FastAPI --> gTTS: Converts Gemini Reply to Audio
FastAPI --> Gemini API: Generates Response using Face Context + Chat History
FastAPI --> UI: Sends Audio URL + Text Reply + Chat History
```

### 🧹 Module 2: MemoryVault RAG (Flask)

```mermaid
sequenceDiagram
User --> React UI: Adds Memory
React UI --> Flask API: POST /postMemory
Flask --> Pinecone: Store Vector + Metadata

User --> React UI: Asks Memory-Based Question
React UI --> Flask: GET /query?query=...
Flask --> Pinecone: Find Relevant Memory Chunks
Flask --> Gemini: Prompt with Memory Context
Flask --> UI: Memory-Based Response
```


## 🔮 Future Enhancements

| Feature                | Description                                       |
| ---------------------- | ------------------------------------------------- |
| Emotion Detection      | Detect facial emotion and adjust tone accordingly |
| Caregiver Dashboard    | Live chat monitor and memory update portal        |
| Mobile App Integration | Android + iOS app for accessibility               |
| Cloud Sync             | Secure memory cloud with multi-user support       |
| AR/VR Walkthrough      | “Walk Through Your Memories” visual experience    |
| Multi-Language Support | Translate voice + chat to native language         |

---

## 🧑‍⚕️ Real Impact

> Helps elderly individuals with memory loss:

* Reconnect with their identity
* Feel emotionally supported
* Communicate with empathy
* Recall specific moments with visuals and warmth

---

## ✅ Run Locally

### 🔹 RecallMe (FastAPI)

```bash
cd RecallMe
pip install -r requirements.txt
python main.py
```

### 🔹 MemoryVault RAG (Flask)

```bash
cd MemoryVault-RAG
pip install -r requirements.txt
python app.py
```

> Ensure your `.env` has:

```env
PINECONE_API_KEY=your_key_here
GOOGLE_API_KEY=your_key_here
```

---


## Detected evidence (automated analysis)

Indexed codebase: 39 recognized source files, 210 KB.
- CSS (language) — detected in the code
- Flask (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- LangChain (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code

## Codebase structure (from repository index)

### Files (44 of 44)

```
Backends/app.py
Backends/appp.py
Backends/recallme/app.py
Backends/recallme/EncodeFile.p
Backends/requirements.txt
Frontend/package.json
Frontend/public/index.html
Frontend/public/manifest.json
Frontend/public/robots.txt
Frontend/src/App.css
Frontend/src/App.js
Frontend/src/App.test.js
Frontend/src/components/Home.css
Frontend/src/components/Home.js
Frontend/src/components/MemoryInput.css
Frontend/src/components/MemoryInput.jsx
Frontend/src/components/QueryPage.css
Frontend/src/components/QueryPage.jsx
Frontend/src/index.css
Frontend/src/index.js
Frontend/src/recall/App.css
Frontend/src/recall/App.js
Frontend/src/recall/App.test.js
Frontend/src/recall/Audiophoto.css
Frontend/src/recall/Audiophoto.js
Frontend/src/recall/FaceRecognition.js
Frontend/src/recall/Family.css
Frontend/src/recall/Family.js
Frontend/src/recall/FamilyGallery.js
Frontend/src/recall/Identify.js
Frontend/src/recall/index.css
Frontend/src/recall/index.js
Frontend/src/recall/Members/Member1.js
Frontend/src/recall/Members/Member2.js
Frontend/src/recall/Members/User.js
Frontend/src/recall/Navbar.css
Frontend/src/recall/Navbar.js
Frontend/src/recall/Photos.js
Frontend/src/recall/reportWebVitals.js
Frontend/src/recall/UnknownFaces.js
Frontend/src/recall/UserProfile.css
Frontend/src/reportWebVitals.js
Frontend/src/setupTests.js
README.md
```

### Dependencies

- Backends/requirements.txt: base64io, Flask, Flask-Cors, groclake, langchain, logging, requests, uuid
- Frontend/package.json: @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^13.5.0, axios@^1.7.7, framer-motion@^12.4.10, lucide-react@^0.474.0, react@^18.3.1, react-dom@^18.3.1, react-frontend@file:, react-icons@^5.5.0, react-router-dom@^6.24.1, react-scripts@5.0.1, react-webcam@^7.2.0, web-vitals@^2.1.4

### Recent commits (newest first)

- Create README.md
- Add files via upload
- Delete README.md
- Delete Image directory
- Delete Frontends directory
- Delete Backends directory
- Add files via upload
- Add files via upload

## Key source files (fetched from GitHub, selected and truncated for size)

### Backends/requirements.txt

```
Flask
Flask-Cors
requests
groclake
langchain
uuid
logging
base64io

```

### Frontend/package.json

```
{
  "proxy": "https://cat-fact.herokuapp.com/",
  "name": "react-frontend",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@testing-library/jest-dom": "^5.17.0",
    "@testing-library/react": "^13.4.0",
    "@testing-library/user-event": "^13.5.0",
    "axios": "^1.7.7",
    "framer-motion": "^12.4.10",
    "lucide-react": "^0.474.0",
    "react": "^18.3.1",
    "react-dom": "^18.3.1",
    "react-frontend": "file:",
    "react-icons": "^5.5.0",
    "react-router-dom": "^6.24.1",
    "react-scripts": "5.0.1",
    "react-webcam": "^7.2.0",
    "web-vitals": "^2.1.4"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  }
}

```

### Backends/app.py

```python

import os
import logging
import requests
from sentence_transformers import SentenceTransformer
from pinecone import Pinecone, Index
from langchain.docstore.document import Document
from langchain.text_splitter import CharacterTextSplitter
import uuid
from dotenv import load_dotenv
from flask import Flask, request, jsonify
from flask_cors import CORS
import google.generativeai as genai

# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

load_dotenv()

app = Flask(__name__)
CORS(app)

# Configure Google Gemini API
genai.configure(api_key=os.environ["GOOGLE_API_KEY"])

# Initialize Pinecone
try:
    pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
    logger.info("Pinecone initialized successfully")
except Exception as e:
    logger.error(f"Failed to initialize Pinecone: {str(e)}")
    pc = None

class SentenceTransformerEmbeddings:
    def __init__(self, model_name):
        self.model = SentenceTransformer(model_name)

    def embed_documents(self, texts):
        return self.model.encode(texts).tolist()

    def embed_query(self, text):
        return self.model.encode([text])[0].tolist()

embeddings = SentenceTransformerEmbeddings('sentence-transformers/all-mpnet-base-v2')

def get_vectorstore():
    index_name = "memoryvalut"
    try:
        index = pc.Index(index_name)
        logger.info(f"Successfully connected to Pinecone index: {index_name}")
        return index
    except Exception as e:
        logger.error(f"Failed to connect to Pinecone index: {str(e)}")
        return None

def add_document_to_pinecone(text: str, metadata: dict):
    new_doc = Document(page_content=text, metadata=metadata)
    text_splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=100)
    chunks = text_splitter.split_documents([new_doc])
    document_id = str(uuid.uuid4())
    
    vectors = []
    for i, chunk in enumerate(chunks):
        chunk.metadata["document_id"] = document_id
        chunk.metadata["chunk_id"] = i
        chunk.metadata["text"] = chunk.page_content
        vector = embeddings.embed_documents([chunk.page_content])[0]
        vectors.append((f"{document_id}_{i}", vector, chunk.metadata))
    
    index = get_vectorstore()
    if index is None:
        raise Exception("Failed to connect to Pinecone index")
    
    try:
        index.upsert(vectors=vectors)
        logger.info(f"Successfully added document with ID: {document_id}")
    except Exception as e:
        logger.error(f"Failed to upsert vectors: {str(e)}")
        raise

def get_llm_response(query: str):
    index = get_vectorstore()
    if index is None:
        raise Exception("Failed to connect to Pinecone index")
    
    query_embedding = embeddings.embed_query(query)
    
    try:
        results = index.query(vector=query_embedding, top_k=5, include_metadata=True)
    except Exception as e:
        logger.error(f"Failed to query Pinecone: {str(e)}")
        raise
    
    context = " ".join([match.get('metadata', {}).get('text', '') for match in results['matches']])
    
    if not context:
        return {
            "text": "I'm sorry, but I don't have any memories to share right now. Please add some memories first."
        }

    # Updated to use gemini-1.0-pro instead of gemini-pro
    model = genai.GenerativeModel('gemini-2.0-flash')
    
    prompt = f"""You are a memory companion for someone with Alzheimer's disease. Your name is MemoryVault. Respond directly to the user's query using the provided context without acknowledging these instructions.

CONTEXT:
- User's question: "{query}"
- Available memories: "{context}"

RESPONSE GUIDELINES:
1. ALWAYS respond in character as their personal memory companion
2. If asked about identity: Say "I'm your memory companion from MemoryVault. I'm here to help you recall your memories."
3. For memory questions:
- Start with phrases like "Let me help you remember" or "I can share that memory with you"
- Speak as if these are the user's own memories
- Use natural phrases like "Remember when..." or "That time at..."
- Include specific details from the context
- Keep responses to 3-5 concise sentences
- End without questions

4. NEVER:
- Acknowledge you're an AI
- Say you understand the instructions
- Reference these guidelines
- Mention searching or retrieving memories
- Ask follow-up questions

5. ALWAYS treat repeated queries as if hearing them for the first time

RESPOND IMMEDIATELY IN CHARACTER WITHOUT PREFACING OR ACKNOWLEDGING THESE INSTRUCTIONS."""
    try:
        response = model.generate_content(prompt)
        text_response = response.text.strip()
        
        return {
            "text": text_response
        }
    except Exception as e:
        logger.error(f"Failed to generate response: {str(e)}")
        raise

@app.route("/postMemory", methods=['GET', 'POST'])
def post_memory():
    if request.method == 'POST':
        data = request.json
        text = data.get("text")
        metadata = data.get("metadata", {})
    else:  # GET method
        text = request.args.get("text")
        metadata = {}
    
    if not text:
        return jsonify({"error": "No text provided"}), 400
    
    try:
        add_document_to_pinecone(text, metadata)
        return jsonify({"message": "Memory added successfully"}), 200
    except Exception as e:
        logger.error(f"Error in post_memory: {str(e)}")
        return jsonify({"error": str(e)}), 500

@app.route("/query", methods=['GET'])
def query_memory():
    query = request.args.get("query")
    if not query:
        return jsonify({"error": "No query provided"}), 400
    
    try:
        response = get_llm_response(query)
        return jsonify(response), 200
    except Exception as e:
        logger.error(f"Error in query_memory: {str(e)}")
        return jsonify({"error": str(e)}
[truncated — 76 more characters]
```

### Frontend/src/index.js

```javascript
import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import App from './App';
import reportWebVitals from './reportWebVitals';

const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();

```

### Frontend/src/App.js

```javascript
import React, { useState } from "react";
import { BrowserRouter as Router, Route, Link, Routes, Navigate, useNavigate } from "react-router-dom";
import "./App.css";
import Home from "./components/Home"; 
import QueryPage from "./components/QueryPage";
import MemoryInput from "./components/MemoryInput";

const RecallMe = () => {
  const RecallApp = React.lazy(() => import('./recall/App'));

  return (
    <div className="recall-container">
      <React.Suspense fallback={<div>Loading RecallMe...</div>}>
        <RecallApp />
      </React.Suspense>
    </div>
  );
};

function App() {
  const [showRecallApp, setShowRecallApp] = useState(false);

  const handleRecallClick = (e) => {
    e.preventDefault();
    setShowRecallApp(true);
  };

  const handleMemoryVaultClick = () => {
    setShowRecallApp(false);
  };

  if (showRecallApp) {
    return (
      <div className="App">
        <header className="App-header">
          <nav className="nav-menu">
            <div className="nav-links-left">
              <button 
                onClick={handleMemoryVaultClick} 
                className="nav-link"
                id="backToMemoryVault"
              >
                Back to MemoryVault
              </button>
            </div>
            <div className="logo-title-container">
              <img
                src="/logo.png"
                alt="MemoryVault Logo"
                className="nav-logo"
                onClick={handleMemoryVaultClick} 
                style={{ cursor: "pointer" }}
              />
              <span 
                className="nav-title" 
                id="recallMeTitle"
                onClick={handleMemoryVaultClick}
                style={{ cursor: "pointer" }}
              >
                RecallMe
              </span>
            </div>
            <div className="nav-links-right">
              <div className="nav-link-placeholder"></div>
            </div>
          </nav>
        </header>

        <RecallMe />
      </div>
    );
  }

  return (
    <Router>
      <div className="App">
        <header className="App-header">
          <nav className="nav-menu">
            <div className="nav-links-left">
              <Link to="/query-memories" className="nav-link" id="queryMemories">
                Query Memories
              </Link>
              <Link to="/add-memory" className="nav-link" id="addMemory">
                Add Memory
              </Link>
            </div>
            <div className="logo-title-container">
              <Link to="/" style={{ textDecoration: "none", display: "flex", alignItems: "center" }}>
                <img
                  src="/logo.png"
                  alt="MemoryVault Logo"
                  className="nav-logo"
                  style={{ cursor: "pointer" }}
                />
                <span className="nav-title" id="memoryVault" style={{ cursor: "pointer" }}>
                  MemoryVault
                </span>
              </Link>
            </div>
            <div className="nav-links-right">
              <a 
                href="#" 
                onClick={handleRecallClick} 
                className="nav-link" 
                id="recallMe"
              >
                RecallMe
              </a>
            </div>
          </nav>
        </header>

        <Routes>
          <Route path="/" element={<Home />} />
          <Route path="/query-memories" element={<QueryPage />} />
          <Route path="/add-memory" element={<MemoryInput />} />
          <Route path="/query" element={<Navigate to="/query-memories" replace />} />
        </Routes>
      </div>
    </Router>
  );
}

export default App;

```

### Backends/recallme/app.py

```python

import cv2
import pickle
import face_recognition
import numpy as np
import os
import uuid
import shutil
from fastapi import FastAPI, File, UploadFile, Form, Request, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse, FileResponse
from fastapi.staticfiles import StaticFiles
from typing import Optional
import uvicorn
from pydantic import BaseModel

# Create the FastAPI app
app = FastAPI(title="Face Recognition Backend API", description="Face Recognition API for MemoryVault")

# Configure CORS
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Directories for storing images
MEMBER_IMAGES_DIR = "MembersData"
UNKNOWN_IMAGES_DIR = "UnknownImages"
IMAGES_DIR = "images"

# Ensure directories exist
os.makedirs(MEMBER_IMAGES_DIR, exist_ok=True)
os.makedirs(UNKNOWN_IMAGES_DIR, exist_ok=True)
os.makedirs(IMAGES_DIR, exist_ok=True)

# Global variables for face encoding
encodeListKnown = []
studentIds = []

# Load the encoding file
def load_encodings():
    global encodeListKnown, studentIds
    print("Loading Encoded File ...")
    try:
        with open("EncodeFile.p", "rb") as file:
            encodeListKnownWithIds = pickle.load(file)
        
        # Ensure encodings are numpy arrays
        encodeListKnown = [np.array(encoding) for encoding in encodeListKnownWithIds[0]]
        studentIds = encodeListKnownWithIds[1]
        print("Encode File Loaded Successfully")
    except Exception as e:
        print(f"Error loading encoding file: {e}")
        encodeListKnown = []
        studentIds = []

# Function to generate encodings from images
def find_encodings(img_list):
    encode_list = []
    for img in img_list:
        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
        try:
            # Get all face encodings
            face_encodings = face_recognition.face_encodings(img)
            if face_encodings:
                # Take the first face encoding
                encode = face_encodings[0]
                encode_list.append(encode)
            else:
                print(f"No face found in an image")
        except Exception as e:
            print(f"Error processing image: {e}")
    return encode_list

# Function to generate encodings and save to file
def generate_encodings():
    global encodeListKnown, studentIds
    
    folder_path = IMAGES_DIR
    path_list = os.listdir(folder_path)
    img_list = []
    student_ids = []

    for path in path_list:
        if path.lower().endswith(('.png', '.jpg', '.jpeg')):
            img_list.append(cv2.imread(os.path.join(folder_path, path)))
            student_ids.append(os.path.splitext(path)[0])

    print("Encoding Started...")
    encode_list_known = find_encodings(img_list)
    encode_list_known_with_ids = [encode_list_known, student_ids]
    print("Encoding Complete")

    file = open("EncodeFile.p", "wb")
    pickle.dump(encode_list_known_with_ids, file)
    file.close()
    print("File Saved")
    
    # Update global variables
    encodeListKnown = encode_list_known
    studentIds = student_ids
    
    return {"status": "success", "encoded_faces": len(encode_list_known)}

# Load encodings on startup
load_encodings()

# Function to check if image is a duplicate
def is_duplicate_image(new_img_path):
    # This is a basic implementation to avoid exact duplicates
    # You might want to enhance this with image similarity comparison if needed
    if not os.path.exists(UNKNOWN_IMAGES_DIR):
        return False
        
    # Get list of existing unknown images
    unknown_images = [f for f in os.listdir(UNKNOWN_IMAGES_DIR) 
                    if f.lower().endswith(('.png', '.jpg', '.jpeg'))]
    
    # If no unknown images exist, it's not a duplicate
    if not unknown_images:
        return False
    
    # Read the new image
    new_img = cv2.imread(new_img_path)
    
    # Simple file size check to quickly filter out obvious non-duplicates
    new_size = os.path.getsize(new_img_path)
    
    for img_name in unknown_images:
        img_path = os.path.join(UNKNOWN_IMAGES_DIR, img_name)
        
        # Quick size check first
        if abs(os.path.getsize(img_path) - new_size) > 1024:  # If size differs by more than 1KB
            continue
            
        # Only compare images that are similar in size
        existing_img = cv2.imread(img_path)
        
        # Simple hash-based comparison
        if existing_img.shape == new_img.shape:
            difference = cv2.norm(existing_img, new_img, cv2.NORM_L2)
            if difference < 100:  # Threshold for considering images as duplicates
                return True
                
    return False

# Face Recognition API Endpoints
@app.post("/detect-face/")
async def detect_face(file: UploadFile = File(...)):
    # Check if any encodings are available
    if not encodeListKnown:
        return {"recognized_person": "No encodings available"}

    # Read image bytes
    image_bytes = await file.read()
    nparr = np.frombuffer(image_bytes, np.uint8)
    img = cv2.imdecode(nparr, cv2.IMREAD_COLOR)

    # Resize & Convert
    imgS = cv2.resize(img, (0, 0), None, 0.25, 0.25)
    imgS = cv2.cvtColor(imgS, cv2.COLOR_BGR2RGB)

    # Recognize faces
    faceCurFrame = face_recognition.face_locations(imgS)
    encodeCurFrame = face_recognition.face_encodings(imgS, faceCurFrame)

    result = "Unknown"
    for encodeFace in encodeCurFrame:
        # Use a try-except block to handle potential errors
        try:
            # Compare faces with a lower tolerance to reduce false negatives
            matches = face_recognition.compare_faces(encodeListKnown, encodeFace, tolerance=0.6)
            
            # If any match is found
            if True in matches
[truncated — 4944 more characters]
```

### Frontend/src/recall/index.js

```javascript
//index.js
import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import App from './App';
import reportWebVitals from './reportWebVitals';

const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();

```

### Frontend/src/recall/App.js

```javascript

import React from 'react';
import { BrowserRouter as Router, Routes, Route } from 'react-router-dom';
import Navbar from './Navbar';
import Family from './Family';
import Identify from './FaceRecognition';
import Photo from './Photos';
import Member1 from './Members/Member1';
import Member2 from './Members/Member2';
import User from './Members/User';
import Audiophoto from './Audiophoto';
import FamilyGallery from './FamilyGallery';
import UnknownFaces from './UnknownFaces';
import './index.css'

import Photos from './Photos';
import FaceRecognition from './FaceRecognition';

function App() {
  return (
    <Router>
      <div style={{ display: 'flex' }}>
        <Navbar />
        <div style={{ flex: 1, padding: '20px' }}>
          <Routes>
            <Route path="/family" element={<Family />} />
            <Route path="/identify" element={<Identify />} />
            <Route path="/photos" element={<Photo />} />
            <Route path="/102" element={<Member1 />} />
            <Route path="/members" element={<FamilyGallery />} />
            <Route path="/101" element={<Member2 />} />
            <Route path="/100" element={<User />} />
            <Route path="/" element={<Photo />} />
            <Route path="/photo/:photoId" element={<Audiophoto />} />
            <Route path="/unknown-faces" element={<UnknownFaces />} />

            <Route path="/identify" element={<FaceRecognition />} />
            <Route path="/photos" element={<Photos />} />
          </Routes>
        </div>
      </div>
    </Router>
  );
}

export default App;
```

### Backends/appp.py

```python
#app.py project 1
import os
import logging
import base64
import requests
from sentence_transformers import SentenceTransformer
from pinecone import Pinecone, Index
from langchain.docstore.document import Document
from langchain.text_splitter import CharacterTextSplitter
import uuid
from dotenv import load_dotenv
from flask import Flask, request, jsonify
from flask_cors import CORS
import google.generativeai as genai

# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

load_dotenv()

app = Flask(__name__)
CORS(app)

# Configure Google Gemini API
genai.configure(api_key=os.environ["GOOGLE_API_KEY"])

# Initialize Hugging Face access token and FLUX API URL
HUGGINGFACE_API_KEY = ''
FLUX_API_URL = ""

# Initialize Pinecone
try:
    pc = Pinecone(api_key=os.environ["PINECONE_API_KEY"])
    logger.info("Pinecone initialized successfully")
except Exception as e:
    logger.error(f"Failed to initialize Pinecone: {str(e)}")
    pc = None

class SentenceTransformerEmbeddings:
    def __init__(self, model_name):
        self.model = SentenceTransformer(model_name)

    def embed_documents(self, texts):
        return self.model.encode(texts).tolist()

    def embed_query(self, text):
        return self.model.encode([text])[0].tolist()

embeddings = SentenceTransformerEmbeddings('sentence-transformers/all-mpnet-base-v2')

def generate_image_from_text(text: str) -> str:
    """Generate an image from text description using FLUX API."""
    try:
        headers = {"Authorization": f"Bearer {HUGGINGFACE_API_KEY}"}
        
        # Create a detailed prompt for image generation
        payload = {
            "inputs": f"A detailed, vivid visualization of this memory: {text}",
            "parameters": {
                "height": 1024,
                "width": 1024,
                "guidance_scale": 3.5,
                "num_inference_steps": 50
            }
        }
        
        response = requests.post(FLUX_API_URL, headers=headers, json=payload)
        
        if response.status_code == 200:
            # Properly handle binary image data
            image_bytes = response.content
            base64_image = base64.b64encode(image_bytes).decode('utf-8')
            return f"data:image/jpeg;base64,{base64_image}"
        else:
            logger.error(f"Error from FLUX API: {response.text}")
            return None
            
    except Exception as e:
        logger.error(f"Error generating image: {e}")
        return None

def get_vectorstore():
    index_name = "memoryvalut"
    try:
        index = pc.Index(index_name)
        logger.info(f"Successfully connected to Pinecone index: {index_name}")
        return index
    except Exception as e:
        logger.error(f"Failed to connect to Pinecone index: {str(e)}")
        return None

def add_document_to_pinecone(text: str, metadata: dict):
    new_doc = Document(page_content=text, metadata=metadata)
    text_splitter = CharacterTextSplitter(chunk_size=500, chunk_overlap=100)
    chunks = text_splitter.split_documents([new_doc])
    document_id = str(uuid.uuid4())
    
    vectors = []
    for i, chunk in enumerate(chunks):
        chunk.metadata["document_id"] = document_id
        chunk.metadata["chunk_id"] = i
        chunk.metadata["text"] = chunk.page_content
        vector = embeddings.embed_documents([chunk.page_content])[0]
        vectors.append((f"{document_id}_{i}", vector, chunk.metadata))
    
    index = get_vectorstore()
    if index is None:
        raise Exception("Failed to connect to Pinecone index")
    
    try:
        index.upsert(vectors=vectors)
        logger.info(f"Successfully added document with ID: {document_id}")
    except Exception as e:
        logger.error(f"Failed to upsert vectors: {str(e)}")
        raise

def get_llm_response(query: str):
    index = get_vectorstore()
    if index is None:
        raise Exception("Failed to connect to Pinecone index")
    
    query_embedding = embeddings.embed_query(query)
    
    try:
        results = index.query(vector=query_embedding, top_k=5, include_metadata=True)
    except Exception as e:
        logger.error(f"Failed to query Pinecone: {str(e)}")
        raise
    
    context = " ".join([match.get('metadata', {}).get('text', '') for match in results['matches']])
    
    if not context:
        return {
            "text": "I'm sorry, but I don't have any memories to share right now. Please add some memories first.",
            "image": None
        }

    model = genai.GenerativeModel('gemini-2.0-flash')
    
    # prompt = f"""
    # You are an AI meant to help Alzheimer's patients remember their memories.
    # The user is asking: "{query}"

    # They might ask for more details about a memory that they remember a little of.
    # Be kind and considerate.
    
    # Retrieve the safety ratings content too and provide to user
    
    # Here is the memory to recall: "{context}"

    # USE AS MUCH DETAIL AS POSSIBLE. You want them to feel like they are living there again.
    
    # Respond in the second person.
    # Make it vivid and paraphrase.

    # REMEMBER THE INFORMATION THAT THE USER TELLS YOU TO.

    # Do NOT:
    # - Mention anything about being an AI.
    # - Mention anything about context.
    # - Make up ANY FALSE INFORMATION.

    # If you can't find any relevant memories, tell them to go to the add memory page and have them or a family member add a memory.
    # """
    prompt = f"""You are a compassionate memory companion for someone with memory challenges. Your primary goal is to help them reconnect with their memories while also maintaining a natural conversation that responds directly to their questions.

UNDERSTANDING CONTEXT:
- The person's query: "{query}"
- Available memories: "{context}"

RESPONSE GUIDELINES:
1. DI
[truncated — 3278 more characters]
```

### Frontend/src/setupTests.js

```javascript
// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';

```

[30 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]