# Project export: Mentora This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 12.0 - Tagline: Voice-Interactive AI Mentor with a Spatial Memory Canvas.Mentora listens, speaks, and sketches in real time—using multimodal RAG and MCPs to explain, visualize, and remember like a human teacher. - Devpost: https://devpost.com/software/mentora-9u4g3r - GitHub: https://github.com/MarkShi17/Mentora - Video: https://www.youtube.com/embed/Bkw-gmL6Qd8?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 5 GitHub contributor(s) — mark (66 commits), aaryanpatil2007 (41 commits), Eric (31 commits), krishiv (19 commits), Claude (1 commits) ## Devpost submission (written by the team) ### Overview What is Mentora Mentora is a voice-interactive AI tutor designed to replicate the experience of learning from a human. It combines a context-aware language model with an infinite canvas workspace and real-time Text-to-Speech to provide explanations that are both auditory and visual. Mentora listens to spoken questions, transcribes them, generates step-by-step solutions (guided or direct), and produces synchronized visual aids such as LaTeX, graphs, code blocks, diagrams, and videos. Everything is coordinated so narration, visuals, and object references stay in sync, allowing learners to follow along naturally, just like in a live tutoring session. ### Inspiration Voice agents often lack visual material to build full understanding. When we learn from a person, _they don’t just talk. _ They draw, gesture, highlight, and point to things as they explain. We wanted to recreate that dynamic: the feeling of someone explaining with both words and visuals, like a human conversation that unfolds naturally on a whiteboard or canvas. ### How we built it Mentora has a central streaming orchestrator that coordinates model output, tool execution, and audio generation, while the object generator and layout engine render visual content on the canvas. Session data is managed in-memory for fast iteration, and modular API endpoints connect the system’s components to deliver synchronized voice and visuals. We used Next.js, TypeScript, and Claude Sonnet 4.5 as the teaching agent, supported by OpenAI Whisper for transcription and OpenAI TTS-1 for speech synthesis. To give the system depth, we integrated many, many Model Context Protocols (MCPs), allowing Mentora to interface with specialized “brains” for different domains math reasoning, coding, biology, or diagram creation. Each MCP module handles its own type of thinking but stays unified through the orchestrator, so Mentora can switch contexts intelligently mid-conversation. This setup enables guided reasoning, thinking aloud, explaining step-by-step, and referencing prior context like a human tutor would. The result feels less like talking to a tool, and more like learning from a friend who sketches, speaks, and reasons with you in real time. Accomplishments that we're proud of Built a multimodal RAG system using ChromaDB that retrieves contextual text, equations, and visuals based on the user’s question, highlighted objects, and conversation history, allowing the tutor to reference prior discussions and provide explanations with rich continuity. Designed a dynamic canvas engine that renders LaTeX, graphs, code, and diagrams on demand, intelligently aligning visuals with narration and feeding new objects back into ChromaDB for future context. Engineered a sophisticated streaming orchestrator that coordinates Claude’s grounded response generation, TTS, and live canvas updates, ensuring smooth, interactive tutoring sessions while automatically ingesting new conversation turns and canvas content into ChromaDB. What we learned RAG Systems and Context Management: Using ChromaDB for retrieval-augmented generation taught us the importance of structuring conversation history, highlighted objects, and visual references so the AI can provide grounded, accurate responses and expand on user selected components. Real-Time Orchestration Challenges: Streaming responses while synchronizing TTS, canvas updates, and AI output highlighted the complexity of building low-latency, live interactive systems. User-Centered Design: Supporting guided (Socratic) and direct modes showed us how flexibility in teaching style improves engagement and understanding. Scalable Architecture Principles: Implementing modular components like the context builder, canvas engine, and streaming orchestrator emphasized maintainability, testability, and future expansion. ### Challenges we ran into Wifi was slow ### What's next In the future, we plan to integrate Claude with more MCP tools so it can better generate diagrams and animations. We also plan for Mentora to highlight the specific parts of the current context and canvas objects it references, clearly showing which prior information it is using in its explanations. ## README (from the GitHub repository) # Mentora - Voice-Interactive AI Mentor with a Spatial Memory Canvas Complete full-stack Agentic tutoring platform using multimodal RAG and MCPs to explain, visualize, and remember like a human mentor. --- ## Architecture **Full-Stack Application:** - **Frontend**: React + Next.js + D3.js (Port 3001) - **Backend**: Next.js API Routes + Claude AI (Port 3000) - **Docker**: Multi-service containerized deployment --- ## Features - **Voice-Interactive Teaching Agent**: Powered by Claude Sonnet 4.5 - **Canvas Object Management**: Create and manage LaTeX equations, graphs, code blocks, diagrams, and text - **Session Management**: Track teaching sessions with full conversation history - **Context-Aware**: References highlighted objects and maintains spatial awareness - **TTS & Transcription**: OpenAI Whisper for speech-to-text and TTS-1 for text-to-speech - **Socratic Teaching**: Guides students with questions rather than direct answers (configurable) - **Multi-Modal RAG** (NEW): ChromaDB-powered knowledge retrieval from past sessions and canvas objects --- ## Tech Stack - **Framework**: Next.js 14 (App Router) - **Language**: TypeScript (strict mode) - **LLM**: Claude Sonnet 4.5 (Anthropic) - **Transcription**: OpenAI Whisper API - **TTS**: OpenAI TTS-1 - **Storage**: In-memory (Map) + ChromaDB for RAG - **Vector DB**: ChromaDB for multi-modal embeddings - **Docker**: Multi-stage builds for dev and production --- ## Prerequisites - Node.js 20+ - Docker and Docker Compose (optional) - OpenAI API key - Anthropic API key --- ## Quick Start ### 1. Clone and Install ```bash cd Mentora npm install ```` ### 2. Configure Environment ```bash cp .env.example .env ``` Edit `.env` and add your API keys: ```env # Required API Keys OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-... # Basic Configuration NODE_ENV=development LOG_LEVEL=info # Optional: Enable RAG (Retrieval-Augmented Generation) ENABLE_RAG=true CHROMADB_URL=http://chromadb:8000 RAG_AUTO_INGEST=true ``` --- ### 3. Run Full Stack Application **Option A: Full Stack with Docker (Recommended)** ```bash docker-compose up ``` * Frontend: [http://localhost:3001](http://localhost:3001) * Backend API: [http://localhost:3000](http://localhost:3000) **Option B: Backend Only (for API development)** ```bash npm run dev ``` * Backend API runs at [http://localhost:3000](http://localhost:3000) **Option C: Frontend Separately (for UI development)** ```bash cd apps/web npm install npm run dev ``` * Frontend runs at [http://localhost:3001](http://localhost:3001) --- ### 4. Test Health Check ```bash curl http://localhost:3000/api/health ``` Expected response: ```json { "status": "ok", "timestamp": 1234567890, "version": "0.1.0" } ``` --- ## MCP Integration Guide **Model Context Protocol (MCP)** integration for Mentora - enabling specialized brains with tool access. **Last Updated:** 2025-10-25 --- ### Overview Mentora includes a complete MCP client layer connecting to multiple MCP servers, enabling: * **Sequential Thinking**: Structured step-by-step problem solving * **Manim Animations**: Mathematical visualizations (Docker-based) * **Python Execution**: Diagram and visualization generation (Docker-based) * **Biology Diagram Generator**: Curated biology schematics (via Python MCP) * **GitHub Integration**: Code search and repository access * **Figma Integration**: Design file and component access --- ### Architecture ``` ┌─────────────────────────────────────────────────────────┐ │ Mentora Backend │ │ │ │ ┌────────────────────────────────────────────────────┐ │ │ │ MCP Connection Manager │ │ │ │ • Manages all MCP server connections │ │ │ │ • Handles reconnection and health checks │ │ │ │ • Routes tool calls to appropriate servers │ │ │ └────────────────────────────────────────────────────┘ │ │ │ │ │ ┌───────────────┼──────────────┬────────────┐ │ │ ▼ ▼ ▼ ▼ │ │ ┌───────────┐ ┌────────────┐ ┌─────────┐ ┌────────┐│ │ │Sequential │ │ Manim │ │ Python │ │GitHub ││ │ │ Thinking │ │ MCP │ │ MCP │ │ MCP ││ │ │(stdio/npx)│ │ (HTTP) │ │ (HTTP) │ │(stdio) ││ │ └───────────┘ └────────────┘ └─────────┘ └────────┘│ └─────────────────────────────────────────────────────────┘ ``` --- ### Components Implemented 1. **Core MCP Client (`lib/mcp/client.ts`)**: Manages single MCP server connection, tool discovery, and execution 2. **Connection Manager (`lib/mcp/manager.ts`)**: Orchestrates all MCP connections, automatic reconnections, unified status reporting 3. **Configuration (`lib/mcp/config.ts`)**: Server registry with transport types and enable flags 4. **Type Definitions (`types/mcp.ts`)**: TypeScript types for server configs, tools, requests/responses 5. **Initialization (`lib/mcp/init.ts`)**: Lazy initialization, singleton pattern, graceful failure handling --- ### API Endpoints **GET** `/api/mcp/status` – Returns status of all MCP servers **POST** `/api/mcp/call` – Calls a tool on an MCP server --- ### Setup Instructions * **Sequential Thinking**: Works out of the box via npx * **GitHub Integration**: Requires Personal Access Token in `.env` * **Figma Integration**: Requires Personal Access Token in `.env` * **Manim MCP**: Docker container pending * **Python MCP**: Docker container pending --- ### Next Steps * Docker containers for Manim and Python MCPs * Specialized Brain implementations * Multimodal memory integration (ChromaDB) * Agent orchestrator to route requests --- ## ChromaDB RAG Integration **Retrieval-Augmented Generation (RAG)** enhances AI responses by retrieving relevant context from past conversations and canvas objects. --- ### Architecture ``` ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Next.js │ HTTP │ Python RAG │ HTTP │ ChromaDB │ │ Backend │ ──────> │ Service │ ──────> │ (Vector DB)│ │ (Port 3000)│ │ (Port 8006) │ │ (Port 8005) │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ OpenAI API ▼ ┌─────────────┐ │ OpenAI │ │ Embeddings │ └─────────────┘ ``` --- ### Quick Start 1. Set API keys (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`) in `.env` 2. Run with RAG: ```bash docker-compose --profile rag up -d ``` 3. Frontend: [http://localhost:3001](http://localhost:3001) Backend API: [http://localhost:3000](http://localhost:3000) --- ### Testing RAG * `/health` – Check service health * `/stats` – Collection statistics * `/ingest` – Add documents * `/search` – Retrieve similar documents Check logs if issues arise: ```bash docker logs mentora-rag-service --tail 50 -f ``` --- ### Configuration * `.env` settings: `RAG_TOP_K`, `RAG_MIN_RELEVANCE_SCORE`, `ENABLE_RAG`, `CHROMADB_URL`, etc. * Adjust search precision and context amount with `RAG_TOP_K` and `RAG_MIN_RELEVANCE_SCORE` --- ### Data Persistence * Stored in Docker volume: `mentora_chromadb-data` * Backup/restore via `docker run` + `tar` --- ### Security Notes * Dev: No authentication, ports exposed to localhost only * Prod: Enable ChromaDB auth, API keys, internal networks, encrypted secrets --- ### Version Compatibility * ChromaDB Server/Client: `0.5.23` * OpenAI Python SDK: `1.57.2` * FastAPI: `0.115.5` --- ## Teaching Agent API Endpoints * **Sessions**: Create, list, retrieve details * **QA**: Ask questions and receive responses * **Canvas Management**: Create, update, highlight objects * **Voice Interaction**: Stream audio to/from TTS and Whisper ## Detected evidence (automated analysis) Indexed codebase: 158 recognized source files, 1106 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - JavaScript (language) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - Docker (technology) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers ## Codebase structure (from repository index) ### Files (120 of 186) ``` .claude/settings.local.json .dockerignore .DS_Store .env.bak .env.example .gitattributes .gitignore app/api/brain/status/route.ts app/api/health/route.ts app/api/mcp/call/route.ts app/api/mcp/status/route.ts app/api/qa-stream/route.ts app/api/qa/route.ts app/api/rag/clear/route.ts app/api/rag/ingest/route.ts app/api/rag/search/route.ts app/api/rag/stats/route.ts app/api/sessions/[id]/route.ts app/api/sessions/route.ts app/api/transcript/route.ts app/api/tts/route.ts app/layout.tsx app/page.tsx apps/web/.dockerignore apps/web/.env.example apps/web/.eslintrc.json apps/web/app/globals.css apps/web/app/layout.tsx apps/web/app/page.tsx apps/web/components/active-session-header.tsx apps/web/components/brain-badge.tsx apps/web/components/canvas-stage.tsx apps/web/components/canvas-toolbar.tsx apps/web/components/captions-overlay.tsx apps/web/components/code-block.tsx apps/web/components/connection-layer.tsx apps/web/components/continuous-ai.tsx apps/web/components/floating-header.tsx apps/web/components/image-preview.tsx apps/web/components/image-upload-button.tsx apps/web/components/mcp-status.tsx apps/web/components/mock-data.ts apps/web/components/object-context-menu.tsx apps/web/components/object-layer.tsx apps/web/components/object-loading-state.tsx apps/web/components/pin-layer.tsx apps/web/components/pin-tray.tsx apps/web/components/prompt-bar.tsx apps/web/components/providers.tsx apps/web/components/session-initializer.tsx apps/web/components/settings-dialog.tsx apps/web/components/sidebar-history.tsx apps/web/components/sources-drawer.tsx apps/web/components/timeline-panel.tsx apps/web/components/ui/button.tsx apps/web/components/ui/input.tsx apps/web/components/ui/textarea.tsx apps/web/components/voice-controls.tsx apps/web/components/voice-toggle.tsx apps/web/Dockerfile apps/web/hooks/use-audio-queue.ts apps/web/hooks/use-clipboard-paste.ts apps/web/hooks/use-continuous-ai.ts apps/web/hooks/use-demo-voice-handler.ts apps/web/hooks/use-openai-tts.ts apps/web/hooks/use-sequential-connections.ts apps/web/hooks/use-speech-recognition.ts apps/web/hooks/use-speech-synthesis.ts apps/web/hooks/use-streaming-qa.ts apps/web/lib/cn.ts apps/web/lib/connection-utils.ts apps/web/lib/image-upload.ts apps/web/lib/mock-data.ts apps/web/lib/session-store.ts apps/web/lib/utils.ts apps/web/next.config.js apps/web/next.config.mjs apps/web/package.json apps/web/playwright.config.ts apps/web/postcss.config.js apps/web/tailwind.config.ts apps/web/tests/ui.spec.ts apps/web/tsconfig.json apps/web/types/index.ts BACKEND_TESTER_README.md backend-tester.js BIOLOGY_MCP_RESEARCH.md BIORENDER_IMPLEMENTATION_GUIDE.md BIORENDER_OAUTH_NOTES.md BRAIN_SYSTEM_IMPLEMENTATION.md BUILD_FIX_SUMMARY.md CACHED_RESPONSE_SYSTEM.md CHROMADB.md CLAUDE.md CURRENT_ARCHITECTURE.md DEBUG_VOICE_TRIGGERS.md DEMO_IMPROVEMENTS.md DEMO_SESSION.md DOCKER_GUIDE.md docker-compose.prod.yml docker-compose.yml docker/biorender-mcp/Dockerfile docker/biorender-mcp/requirements.txt docker/biorender-mcp/server.py docker/chatmol-mcp/Dockerfile docker/chatmol-mcp/requirements.txt docker/chatmol-mcp/server.py docker/manim-mcp/Dockerfile docker/manim-mcp/server.py docker/python-mcp/Dockerfile docker/python-mcp/requirements.txt docker/python-mcp/server.py docker/rag-service/Dockerfile docker/rag-service/requirements.txt docker/rag-service/server.py Dockerfile FILE_TREE.txt fix-docker-deps.sh FRONTEND_DOCKER_DEV.md lib/agent/brainRegistry.ts [66 more files omitted for size] ``` ### Dependencies - apps/web/package.json: @playwright/test@^1.43.1, @radix-ui/react-dialog@^1.0.5, @radix-ui/react-slot@^1.0.2, @tanstack/react-query@^5.28.8, @types/d3@^7.4.3, @types/katex@^0.16.7, @types/node@^20.11.10, @types/react@^18.2.47, @types/react-dom@^18.2.18, @types/react-syntax-highlighter@^15.5.13, autoprefixer@^10.4.16, class-variance-authority@^0.7.0, clsx@^2.0.0, d3@^7.9.0, eslint@^8.57.1, eslint-config-next@^14.2.33, framer-motion@^10.18.0, immer@^10.0.3, katex@^0.16.25, lucide-react@^0.338.0, next@^14.2.33, postcss@^8.4.32, react@18.2.0, react-dom@18.2.0, react-markdown@^10.1.0, react-syntax-highlighter@^15.6.6, rehype-katex@^7.0.1, remark-gfm@^4.0.1, remark-math@^6.0.0, tailwind-merge@^2.2.1, tailwindcss@^3.4.1, typescript@^5.3.3, zustand@^4.4.7 - docker/biorender-mcp/requirements.txt: aiohttp@==3.9.1 - docker/chatmol-mcp/requirements.txt: aiohttp@==3.9.3, biopython@>=1.83 - docker/python-mcp/requirements.txt: aiohttp@==3.9.3, matplotlib@==3.8.3, numpy@==1.26.4, pandas@==2.2.0, pillow@==10.2.0, plotly@==5.18.0, scipy@==1.12.0, seaborn@==0.13.2 - docker/rag-service/requirements.txt: chromadb@==0.5.23, fastapi@==0.115.5, openai@==1.57.2, pydantic@==2.10.3, python-multipart@==0.0.18, uvicorn[standard]@==0.32.1 - package.json: @anthropic-ai/sdk@^0.32.1, @chroma-core/openai@^0.1.7, @modelcontextprotocol/sdk@^1.20.2, @types/node@^22, @types/react@^18, chromadb@^3.0.1, next@14.2.15, openai@^4.67.0, react@^18.3.1, react-dom@^18.3.1, typescript@^5.6.0 ### Recent commits (newest first) - Update README.md - Update README.md - Restored download.mp4 video file for Manim component - Added third command 'how were you made' with Inspiration component, implemented auto-progression demo system, switched to CDN video URL for reliability - Fixed video access with CDN URL and added localStorage TTS caching for consistent dev experience - Updated Architecture with detailed pipeline, enhanced Key Features with live tutoring, cleaned up second answer - Updated demo layout: vertical branching from Key Features, removed Try It Now component, updated model to Sonnet 4.5 - Fix React hooks initialization error in continuous-ai.tsx - Clean up .DS_Store - Merge branch 'main' of https://github.com/MarkShi17/Mentora - dik - Merge branch 'main' of https://github.com/MarkShi17/Mentora - rag impelmeented - almost thee - Merge branch 'main' of https://github.com/MarkShi17/Mentora - idek - no video clipping - Merge branch 'main' of https://github.com/MarkShi17/Mentora - demo mode improvements: realistic delays, single message updates, linear tree layout - Merge branch 'main' of https://github.com/MarkShi17/Mentora ## Key source files (fetched from GitHub, selected and truncated for size) ### BIOLOGY_MCP_RESEARCH.md ```markdown # Biology Brain MCP Tool Research _Date: 2025-10-29_ ## Summary To strengthen the Biology brain, we explored domain-specific Model Context Protocol (MCP) tools that can provide high quality biological visuals and structured reasoning. The objective is to complement Claude’s textual explanations with accurate, ready-to-use diagrams that reflect key life science processes. ## Candidate Tooling | Tool Idea | Description | Pros | Cons / Notes | |-----------|-------------|------|--------------| | **Biology Diagram Generator (matplotlib)** | Procedurally draw canonical cell structures, organelles, and processes using matplotlib primitives (circles, arrows, gradients). | Runs fully offline, deterministic output, easy to extend, aligns with existing Python MCP. | Requires handcrafted templates; best for high-level schematics rather than photorealistic imagery. | | **Pathway Graph Renderer (networkx + matplotlib)** | Build metabolic / signaling pathways from pre-defined templates, render with networkx. | Great for flow-based diagrams; integrates well with graph libraries. | Needs curated datasets; pathway coverage limited unless extended. | | **Protein Ribbon Snapshot (Biopython / Py3Dmol)** | Generate quick ribbon plots from PDB IDs. | Provides structural biology visuals. | Requires larger dependencies and potentially GPU if expanded; adds download latency. | | **AlphaFold / ESMFold MCP** | Call out to structure prediction services. | High value for novel proteins. | Heavy compute / external API reliance; unsuitable without stable hosting. | | **Pathway Illustration Services (e.g., KEGG, BioRender)** | Fetch curated diagrams. | Professional-quality visuals. | Licensing and network access requirements prevent turnkey integration. | ## Selected Approach We prioritized a lightweight, offline-capable solution and extended the existing Python MCP server with a new `render_biology_diagram` tool. This tool ships with handcrafted templates for: - **Cell structure overview** – membranes, nucleus, mitochondria, ribosomes. - **DNA transcription** – illustrates DNA template, RNA polymerase, and mRNA strand. - **Photosynthesis process** – highlights chloroplast components and light/dark reactions. The approach keeps dependencies minimal while giving mentors reliable, annotation-ready schematics that can be refined over time with additional templates or data-driven overlays. ## Future Enhancements - Add networkx-backed pathway diagrams (glycolysis, Krebs cycle). - Support user-specified labels / highlight regions via arguments. - Incorporate protein structure renders through optional Py3Dmol integration. - Offer animated variants via the Manim MCP once biology-centric scenes are available. ``` ### STREAMING_IMPROVEMENTS.md ```markdown # Streaming Improvements Summary ## Changes Made ### 1. Event-Based Streaming Format - Changed from JSON to event-based markers for better streaming - Text streams token-by-token immediately with `[NARRATION]` markers - Objects generate progressively with `[OBJECT_START]`, `[OBJECT_CONTENT]`, `[OBJECT_END]` markers - Much faster initial response time ### 2. Voice Agent Fixed - Fixed TTS audio generation issue (was checking for non-existent `success` property) - Audio chunks now properly generated and sent for each sentence - Voice synthesis working correctly with OpenAI TTS ### 3. Visual Indicators Added - Chat history now shows "..." while streaming is ongoing - Shows 🔊 emoji when audio is playing - Automatically removes indicator when streaming completes - Provides clear feedback about generation state ### 4. Progressive Object Generation - Added `GenerationState` type to CanvasObject ('generating', 'complete', 'error') - Created `ObjectLoadingState` component with animated loading indicator - Objects show loading state while content is being generated - Smooth replacement when object content is ready ### 5. Implementation Details #### Backend Changes: - Modified `streamingOrchestrator.ts` to use event-based format - Fixed audio result check (removed `audioResult.success` check) - Added proper logging for audio generation - System prompt updated to use streaming format #### Frontend Changes: - Updated `prompt-bar.tsx` to show streaming indicators - Modified `object-layer.tsx` to display loading states - Enhanced `use-streaming-qa.ts` hook to handle progressive updates - Added proper generation state handling #### Type Updates: - Added `GenerationState` type to canvas types - Added `placeholder` and `label` fields to CanvasObject - Updated object generator to include labels ## Testing The streaming improvements provide: - ✅ Immediate text response (starts streaming right away) - ✅ Voice synthesis working (audio chunks generated for each sentence) - ✅ Visual feedback during generation (shows "..." or 🔊) - ✅ Progressive object loading (placeholders → complete objects) - ✅ Better user experience with clear generation states ## Performance Improvements - **Response Time**: Text starts appearing immediately instead of waiting for full JSON - **Voice Latency**: Audio generates sentence-by-sentence for faster playback - **Visual Feedback**: Users always know when content is generating - **Object Loading**: Progressive loading prevents UI blocking ## Usage The improvements are automatic and work with the existing streaming QA endpoint: - Text appears word-by-word in chat history - Voice plays as soon as each sentence is ready - Objects show loading animation while generating - Chat history indicates ongoing generation with "..." ``` ### package.json ``` { "name": "mentora-backend", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "docker:dev": "docker-compose up", "docker:build": "docker-compose build", "type-check": "tsc --noEmit" }, "dependencies": { "@anthropic-ai/sdk": "^0.32.1", "@modelcontextprotocol/sdk": "^1.20.2", "next": "14.2.15", "openai": "^4.67.0", "react": "^18.3.1", "react-dom": "^18.3.1" }, "optionalDependencies": { "chromadb": "^3.0.1", "@chroma-core/openai": "^0.1.7" }, "devDependencies": { "@types/node": "^22", "@types/react": "^18", "typescript": "^5.6.0" } } ``` ### Dockerfile ``` FROM node:20-slim AS base # Install Chromium and dependencies for Puppeteer (needed for Mermaid diagram generation) RUN apt-get update && apt-get install -y \ chromium \ chromium-sandbox \ fonts-liberation \ libasound2 \ libatk-bridge2.0-0 \ libatk1.0-0 \ libatspi2.0-0 \ libcups2 \ libdbus-1-3 \ libdrm2 \ libgbm1 \ libgtk-3-0 \ libnspr4 \ libnss3 \ libwayland-client0 \ libxcomposite1 \ libxdamage1 \ libxfixes3 \ libxkbcommon0 \ libxrandr2 \ xdg-utils \ && rm -rf /var/lib/apt/lists/* # Set Puppeteer to skip downloading Chrome (use system Chromium) ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium # Install dependencies only when needed FROM base AS deps WORKDIR /app # Copy package files COPY package.json package-lock.json* ./ # Install all dependencies including optional ones for ChromaDB # Use --force to bypass peer dependency issues RUN npm ci --force # Development stage FROM base AS dev WORKDIR /app COPY --from=deps /app/node_modules ./node_modules COPY . . EXPOSE 3000 ENV PORT 3000 ENV HOSTNAME "0.0.0.0" CMD ["npm", "run", "dev"] # Build stage FROM base AS builder WORKDIR /app COPY --from=deps /app/node_modules ./node_modules COPY . . # Disable Next.js telemetry ENV NEXT_TELEMETRY_DISABLED 1 # Build the application RUN npm run build # Production stage FROM base AS runner WORKDIR /app ENV NODE_ENV production ENV NEXT_TELEMETRY_DISABLED 1 # Create non-root user RUN addgroup --system --gid 1001 nodejs RUN adduser --system --uid 1001 nextjs # Copy necessary files COPY --from=builder /app/public ./public COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./ COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static USER nextjs EXPOSE 3000 ENV PORT 3000 ENV HOSTNAME "0.0.0.0" CMD ["node", "server.js"] ``` ### docker-compose.yml ```yaml version: '3.8' services: # Backend API Service backend: build: context: . dockerfile: Dockerfile target: dev container_name: mentora-backend ports: - "3000:3000" dns: - 8.8.8.8 # Google DNS - 1.1.1.1 # Cloudflare DNS environment: - OPENAI_API_KEY=${OPENAI_API_KEY} - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY} - GITHUB_TOKEN=${GITHUB_TOKEN} - FIGMA_TOKEN=${FIGMA_TOKEN} - ENABLE_MANIM=${ENABLE_MANIM:-false} - ENABLE_PYTHON=${ENABLE_PYTHON:-false} - ENABLE_MERMAID=${ENABLE_MERMAID:-false} - ENABLE_CHATMOL=${ENABLE_CHATMOL:-false} - ENABLE_BIORENDER=${ENABLE_BIORENDER:-false} - MANIM_MCP_URL=${MANIM_MCP_URL:-http://manim-mcp:8000} - PYTHON_MCP_URL=${PYTHON_MCP_URL:-http://python-mcp:8000} - CHATMOL_MCP_URL=${CHATMOL_MCP_URL:-http://chatmol-mcp:8000} - BIORENDER_MCP_URL=${BIORENDER_MCP_URL:-http://biorender-mcp:8000} - BIORENDER_API_KEY=${BIORENDER_API_KEY} - PYMOL_PATH=${PYMOL_PATH:-/usr/bin/pymol} - NODE_ENV=development - LOG_LEVEL=info - PORT=3000 - ENABLE_RAG=${ENABLE_RAG:-false} - RAG_SERVICE_URL=${RAG_SERVICE_URL:-http://rag-service:8006} - CHROMADB_URL=${CHROMADB_URL:-http://chromadb:8000} - CHROMADB_COLLECTION=${CHROMADB_COLLECTION:-mentora_knowledge} - CHROMA_AUTH_TOKEN=${CHROMA_AUTH_TOKEN:-test-token} - EMBEDDING_MODEL=${EMBEDDING_MODEL:-text-embedding-3-small} - RAG_TOP_K=${RAG_TOP_K:-5} - RAG_MIN_RELEVANCE_SCORE=${RAG_MIN_RELEVANCE_SCORE:-0.7} - RAG_AUTO_INGEST=${RAG_AUTO_INGEST:-true} volumes: - .:/app - /app/node_modules - /app/.next networks: - mentora-network restart: unless-stopped healthcheck: test: ["CMD", "curl", "-f", "http://localhost:3000/api/health"] interval: 30s timeout: 10s retries: 3 start_period: 40s # Frontend Web Application frontend: build: context: ./apps/web dockerfile: Dockerfile target: dev container_name: mentora-frontend ports: - "3001:3001" environment: - NEXT_PUBLIC_API_URL=http://localhost:3000 - NODE_ENV=development - PORT=3001 volumes: - ./apps/web:/app - /app/node_modules - /app/.next networks: - mentora-network depends_on: - backend restart: unless-stopped # Python MCP Service (optional, controlled by ENABLE_PYTHON) python-mcp: build: context: ./docker/python-mcp dockerfile: Dockerfile container_name: mentora-python-mcp ports: - "8001:8000" environment: - PYTHONUNBUFFERED=1 - MCP_MODE=http volumes: - python-mcp-media:/app/media networks: - mentora-network restart: unless-stopped healthcheck: test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"] interval: 30s timeout: 10s retries: 3 start_period: 20s profiles: - mcp - python-mcp # Manim MCP Service (optional, controlled by ENABLE_MANIM) manim-mcp: build: context: ./docker/manim-mcp dockerfile: Dockerfile container_name: mentora-manim-mcp ports: - "8002:8000" environment: - MANIM_QUALITY=medium - MCP_MODE=http volumes: - manim-mcp-media:/app/media networks: - mentora-network restart: unless-stopped healthcheck: test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"] interval: 30s timeout: 10s retries: 3 start_period: 30s profiles: - mcp - manim-mcp # ChatMol MCP Service (optional, controlled by ENABLE_CHATMOL) chatmol-mcp: build: context: ./docker/chatmol-mcp dockerfile: Dockerfile container_name: mentora-chatmol-mcp ports: - "8003:8000" environment: - PYMOL_PATH=/usr/bin/pymol - MCP_MODE=http volumes: - chatmol-media:/app/media networks: - mentora-network restart: unless-stopped healthcheck: test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"] interval: 30s timeout: 10s retries: 3 start_period: 40s profiles: - mcp - chatmol-mcp # BioRender MCP Service (optional, controlled by ENABLE_BIORENDER) biorender-mcp: build: context: ./docker/biorender-mcp dockerfile: Dockerfile container_name: mentora-biorender-mcp ports: - "8004:8000" environment: - BIORENDER_API_KEY=${BIORENDER_API_KEY} - MCP_MODE=http networks: - mentora-network restart: unless-stopped healthcheck: test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"] interval: 30s timeout: 10s retries: 3 start_period: 20s profiles: - mcp - biorender-mcp # ChromaDB Vector Database for RAG (optional, controlled by ENABLE_RAG) # Note: No persistent volume - data clears on container restart chromadb: image: chromadb/chroma:0.5.23 container_name: mentora-chromadb ports: - "8005:8000" environment: - ALLOW_RESET=true - ANONYMIZED_TELEMETRY=false - IS_PERSISTENT=FALSE networks: - mentora-network restart: unless-stopped profiles: - rag - full # Python RAG Service - Multi-modal RAG with ChromaDB rag-service: build: context: ./docker/rag-service dockerfile: Dockerfile container_name: mentora-rag-service ports: - "8006:8006" environment: - CHROMADB_HOST=chromadb - CHROMADB_PORT=8000 - CHROMADB_COLLECTION=${CHROMADB_COLLECTION:-mentora_knowledge} - CHROMA_AUTH_TOKEN=${CHROMA_AUTH_TOKEN:-test-token} - OPENAI_API_KEY=$ [truncated — 595 more characters] ``` ### docker/biorender-mcp/requirements.txt ``` aiohttp==3.9.1 ``` ### docker/chatmol-mcp/requirements.txt ``` aiohttp==3.9.3 biopython>=1.83 ``` ### docker/rag-service/requirements.txt ``` fastapi==0.115.5 uvicorn[standard]==0.32.1 chromadb==0.5.23 pydantic==2.10.3 python-multipart==0.0.18 openai==1.57.2 ``` ### docker/python-mcp/requirements.txt ``` numpy==1.26.4 pandas==2.2.0 matplotlib==3.8.3 seaborn==0.13.2 plotly==5.18.0 scipy==1.12.0 pillow==10.2.0 aiohttp==3.9.3 ``` ### docker/biorender-mcp/Dockerfile ``` FROM python:3.11-slim # Install system dependencies RUN apt-get update && apt-get install -y \ curl \ && rm -rf /var/lib/apt/lists/* # Set working directory WORKDIR /app # Copy requirements and server COPY requirements.txt ./ COPY server.py ./ # Install Python dependencies with increased timeout and retries RUN pip install --no-cache-dir --timeout=300 --retries=5 -r requirements.txt # Expose port EXPOSE 8000 # Run the server CMD ["python", "server.py"] ``` [158 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]