# Project export: Empirica This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 12.0 - Tagline: Turn research papers into living knowledge graphs. Agentic AI finds papers, builds networks, generates hypotheses. Chat with your research using RAG. - Devpost: https://devpost.com/software/empirica - GitHub: https://github.com/jalenfran/empirica - Video: https://www.youtube.com/embed/xpLYIeUccAw?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Result: winner (Regeneron: Honorable Mention) - Team: 2 GitHub contributor(s) — Jalen Francis (15 commits), Farhan Sadeek (5 commits) ## Devpost submission (written by the team) ### Inspiration Biomedical researchers spend 15% of an average week up to 50% or more during intense periods of their time on literature review. When working on biomedical research, we realized finding connections across 50+ papers was nearly impossible manually. We needed a way to see the forest, not just the trees. What if AI could not only read papers but actively discover new ones, build knowledge networks, and generate research hypotheses automatically? That's why we built Empirica. ### What it does Empirica transforms biomedical research with three core capabilities: 1. Agentic Research Workflows: Turn on "Agent Mode" and watch Empirica autonomously search PubMed, discover papers via Google Scholar, download PDFs, extract entities, and build knowledge graphs—all while you focus on analysis. 2. Interactive Knowledge Graphs: Visualize genes, diseases, drugs, and their relationships in stunning 2D/3D force-directed graphs. Click nodes to explore connections, filter by entity type, and navigate through your research visually. 3. AI-Powered Intelligence: RAG-Enhanced Chat: Ask questions and get answers with exact paper + page citations Discovery Lab: AI generates research hypotheses by analyzing graph structure and document content, showing confidence scores and supporting evidence Every insight is traceable, persistent, and citation-backed. ### How we built it Backend (Python): FastAPI for async REST API with OAuth 2.0 authentication scispaCy for biomedical named entity recognition (80+ entity types) PyMuPDF for PDF text extraction NetworkX for graph construction and analysis sentence-transformers for semantic embeddings Custom RAG system with entity-aware chunking SQLite for persistence (projects, chat history, hypotheses) Frontend (TypeScript): React 18 + Vite for blazing-fast development react-force-graph for 2D/3D WebGL visualization Three.js for advanced 3D rendering with particles and curved edges Tailwind CSS + Lucide icons for modern UI Axios for API communication AI Integration: Claude 3.5 Sonnet for natural language processing Custom prompt engineering for hypothesis generation RAG architecture combining semantic search with graph context External APIs: PubMed E-utilities for paper search Google Scholar scraping for PDF discovery PMC and DOI resolution for full-text access ### Challenges we ran into 1. Graph Performance: Visualizing 500+ nodes with real-time physics simulation tanked performance. We optimized by implementing WebGL rendering, reducing particle counts, and adding smart chunking for large graphs. 2. RAG Context Windows: Initially, RAG retrieved too much context, hitting Claude's token limits. We implemented entity-aware chunking that prioritizes relevant sections and semantic ranking to surface the best 7 chunks. 3. Real-time Progress Updates: Background jobs ran asynchronously, making it hard to show live progress. We implemented a polling system with granular status updates and persistent job tracking. 4. OAuth Token Expiration: During long research sessions (10+ minutes), Google OAuth tokens would expire mid-process, crashing the workflow. We implemented optional authentication for status endpoints and graceful error handling. ### Accomplishments we're proud of ✅ End-to-end autonomous research pipeline - From query to knowledge graph in minutes, fully automated Production-quality RAG system - Semantic search + entity-aware retrieval with precise citations (paper + page) Stunning visualizations - 3D graphs with colored edges, animated particles, proper lighting, and smooth interactions rival commercial tools Full persistence layer - Chat history and hypotheses survive across sessions, making research continuity seamless Citation transparency - Every AI-generated answer links back to exact source documents and pages Graph-aware hypothesis generation - Discovery Lab analyzes both document content AND network structure to find insights humans might miss Real-time updates - Live progress tracking during agentic research (papers found, analyzed, entities extracted) Clean architecture - Modular, typed codebase with clear separation between services, ready for scale ### What we learned Technical: scispaCy's biomedical NER capabilities are incredible but require careful prompt engineering to maximize extraction quality WebGL force-directed graphs need careful optimization—curved edges, particles, and lighting all impact frame rates RAG isn't just "throw everything at the LLM"—entity-aware chunking and semantic ranking dramatically improve answer quality Background job management in FastAPI requires thoughtful state management and polling strategies Three.js lighting models make a huge difference in 3D visualization readability Research Domain: Biomedical research has VERY specific entity types (genes, proteins, diseases, drugs, pathways) PubMed's API is powerful but Google Scholar often has better PDF availability Relationship extraction is hard—co-occurrence is a decent baseline but pattern matching catches many more connections Researchers care deeply about citations and provenance—"trust but verify" is critical Product: Visual feedback is everything—users want to SEE progress, not just wait Persistence matters more than we thought—researchers return to projects over days/weeks The "magic moment" is when Discovery Lab surfaces a hypothesis the user hadn't considered Autonomous workflows need to show their work—users want to understand what the AI did ### What's next Expand Research Domains: Multi-domain NER models - Integrate specialized models for chemistry, physics, computer science, and social sciences beyond biomedicine Domain-adaptive entity extraction - Let users choose research domain (clinical, genomics, drug discovery) for optimized NER performance Custom entity training - Allow researchers to fine-tune models on their specific subdomain Enhanced Collaboration: Real-time multi-user editing - Multiple researchers annotating and discussing the same graph simultaneously via WebSockets Export to academic formats - BibTeX, RIS, EndNote, and auto-generated literature review sections with proper citations Author network analysis - Visualize collaboration patterns and identify key researchers in your field Smarter AI: Fine-tuned biomedical LLM - Train domain-specific model for better entity extraction and more accurate hypothesis generation Literature review generator - Auto-compose structured academic review sections from your knowledge graphs Experiment suggestions - Recommend methodologies and protocols based on successful papers in your graph Scale & Performance: PostgreSQL migration - Handle concurrent access and larger datasets efficiently Graph versioning - Track how your knowledge evolves over time with diff visualization Handle 10,000+ paper graphs - Streaming processing and smart pagination for massive research projects Our vision: Make AI-accelerated research accessible to every scientist, turning months of literature review into hours of insight discovery. ## README (from the GitHub repository) # 🧬 Empirica **Transform biomedical PDFs into interactive knowledge graphs with AI-powered insights** Empirica is a sophisticated research intelligence tool that ingests biomedical PDF documents and automatically generates interactive, force-directed knowledge graphs. Each PDF gets its own graph, which can be dynamically combined for multi-document analysis. Enhanced with RAG (Retrieval-Augmented Generation) and LLM-powered insights via Lava Payments + Anthropic Claude. ![Version](https://img.shields.io/badge/version-2.0-blue.svg) ![Python](https://img.shields.io/badge/python-3.9+-green.svg) ![React](https://img.shields.io/badge/react-18.2-blue.svg) ![RAG](https://img.shields.io/badge/RAG-enabled-purple.svg) ## ✨ Features ### 🎯 Core Capabilities - **Per-PDF Graph System** - Each PDF gets its own knowledge graph - **Dynamic Graph Merging** - Combine multiple PDF graphs by selection/deselection - **Named Entity Recognition** - scispaCy-powered identification of biomedical entities - **Relationship Extraction** - Pattern-based and co-occurrence analysis - **Interactive Visualization** - 2D/3D force-directed graphs with smooth physics - **Advanced Analytics** - Community detection, centrality analysis, graph statistics ### 🤖 AI-Powered Features - **RAG System** - Document chunking, semantic indexing, and context retrieval - **Hypothesis Generation** - LLM-powered research insights from your documents - **Conversational AI** - Chat with your knowledge graph using natural language - **Evidence-Based Insights** - All AI responses grounded in your source documents - **Lava Payments Integration** - Usage-based billing for AI API calls ### 🚀 Advanced Features - **Smart PDF Management** - Add/remove PDFs from existing projects - **PDF Selection** - Toggle PDFs to dynamically update the merged graph - **Real-time Processing** - Live progress tracking with background job processing - **Import/Export System** - Full project state including RAG indices - **Multi-user Support** - OAuth authentication with Google - **Persistent Storage** - SQLite database with per-PDF graph storage ### 🎨 Beautiful UI - Modern gradient design with dark theme - Responsive layout for all screen sizes - Smooth animations and transitions - Interactive tooltips with evidence sentences - Entity color-coding by type ## 🏗️ Architecture ``` ┌─────────────────┐ ┌──────────────────┐ │ React + Vite │ HTTP │ FastAPI │ │ TypeScript │ ────▶ │ Python 3.9+ │ │ Tailwind CSS │ │ │ └─────────────────┘ └──────────────────┘ │ │ │ ▼ │ ┌──────────────────┐ │ │ NLP Pipeline │ │ │ - PyMuPDF │ │ │ - scispaCy │ │ │ - NetworkX │ │ └──────────────────┘ │ │ ▼ ▼ ┌─────────────────┐ ┌──────────────────┐ │ Force Graph │ │ Graph Builder │ │ - 2D/3D Views │ ◀──── │ - Communities │ │ - Interactions │ JSON │ - Centrality │ └─────────────────┘ └──────────────────┘ ``` ## 🚀 Quick Start ### Prerequisites - **Python 3.9+** - **Node.js 18+** - **npm or yarn** ### 1. Clone Repository ```bash git clone cd calhacks ``` ### 2. Backend Setup ```bash cd backend # Create virtual environment python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate # Install dependencies pip install -r requirements.txt # Download scispaCy model (this may take a few minutes) pip install https://s3-us-west-2.amazonaws.com/ai2-s2-scispacy/releases/v0.5.4/en_ner_bionlp13cg_md-0.5.4.tar.gz # Configure Lava Payments (required for AI features) cat > .env << EOF LAVA_SECRET_KEY=aks_live_... LAVA_CONNECTION_SECRET=cons_live_... LAVA_PRODUCT_SECRET=ps_live_... ENABLE_LAVA=true EOF # Run backend server uvicorn app.main:app --reload ``` Backend runs on `http://localhost:8000` ### 3. Frontend Setup ```bash cd frontend # Install dependencies npm install # Run development server npm run dev ``` Frontend runs on `http://localhost:5173` ### 4. Open Browser Navigate to `http://localhost:5173` and start uploading PDFs! ## 📖 Usage ### Basic Workflow 1. **Upload PDFs** - Drag & drop biomedical PDF files 2. **Wait for Processing** - Monitor real-time progress (NER, relationship extraction, graph building, RAG indexing) 3. **Explore Graph** - Pan, zoom, click nodes to investigate 4. **Generate Hypotheses** - Click "Generate Hypotheses" for AI-powered insights 5. **Chat with Graph** - Ask questions about your documents in natural language 6. **Manage PDFs** - Add/remove PDFs or toggle selection to update the graph 7. **Export/Import** - Save full project state including RAG indices ### Advanced Features #### Per-PDF Graphs - Each PDF gets its own knowledge graph stored separately - Select/deselect PDFs to dynamically merge graphs - Add new PDFs to existing projects - Remove PDFs and their associated graph data #### RAG-Enhanced AI - **Document Chunking**: Smart sentence-aware chunking with entity tracking - **Semantic Search**: Find relevant content using embeddings - **Graph-Aware Retrieval**: Combine semantic similarity with graph connectivity - **Evidence-Based**: All AI responses cite source documents #### Graph Filtering - **Entity Types**: Toggle specific biomedical entities - **PDF Selection**: Show/hide graphs from specific PDFs - **Min Degree**: Show only highly connected nodes - **Search**: Find entities by name #### View Modes - **2D View**: High performance, ideal for large graphs - **3D View**: Impressive visualization, better for presentations - **Labels Toggle**: Show/hide node labels ## 🛠️ Tech Stack ### Backend - **FastAPI** - Modern async API framework - **PyMuPDF** - High-performance PDF processing - **scispaCy** - Biomedical NER (en_ner_bionlp13cg_md model) - **NetworkX** - Graph algorithms and analysis - **sentence-transformers** - RAG embeddings - **python-louvain** - Community detection - **SQLAlchemy** - Database ORM - **Anthropic Claude** - LLM (via Lava Payments) - **Lava Payments** - Usage-based AI billing ### Frontend - **React 18** - UI library with hooks - **TypeScript** - Type safety - **Vite** - Lightning-fast build tool - **Tailwind CSS** - Utility-first styling - **react-force-graph** - WebGL-powered graph rendering - **Recharts** - Analytics visualizations - **Zustand** - Lightweight state management - **Lucide React** - Modern icon library ## 📊 API Endpoints ### Projects - `POST /api/projects` - Create project & upload PDFs - `GET /api/projects` - List user's projects - `GET /api/projects/{id}` - Get project details - `GET /api/projects/{id}/pdfs` - List project PDFs - `GET /api/projects/{id}/graph` - Get merged graph from selected PDFs - `DELETE /api/projects/{id}` - Delete project ### PDFs - `POST /api/projects/{id}/pdfs` - Add PDFs to existing project - `DELETE /api/projects/{project_id}/pdfs/{pdf_id}` - Remove PDF from project - `POST /api/projects/{id}/pdfs/selection` - Update PDF selection status ### AI Features (RAG-Enhanced) - `POST /api/hypotheses` - Generate research insights - `POST /api/chat` - Chat with knowledge graph ### Import/Export - `POST /api/export` - Export project with RAG index - `POST /api/import` - Import project and restore RAG index ### External Data - `GET /api/pubmed/search` - Search PubMed - `GET /api/clinicaltrials/search` - Search ClinicalTrials.gov ### Processing - `GET /api/processing/{job_id}` - Check processing status ## 🎯 Use Cases ### Research Applications - **Literature Review** - Discover connections across multiple papers - **Hypothesis Generation** - Find unexpected relationships - **Drug Discovery** - Identify drug-disease-gene interactions - **Biomarker Discovery** - Explore [README truncated for size] ## Detected evidence (automated analysis) Indexed codebase: 56 recognized source files, 509 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - AWS (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (64 of 64) ``` .gitignore ARCHITECTURE.md backend/.env.backup backend/.gitignore backend/app/__init__.py backend/app/config.py backend/app/main.py backend/app/models/__init__.py backend/app/models/database.py backend/app/models/schemas.py backend/app/services/__init__.py backend/app/services/agentic_ai_service.py backend/app/services/auth_service.py backend/app/services/content_insight_agent.py backend/app/services/ctgov_service.py backend/app/services/document_chunker.py backend/app/services/google_scholar_service.py backend/app/services/graph_agent.py backend/app/services/graph_builder.py backend/app/services/llm_service.py backend/app/services/ner_service.py backend/app/services/pdf_processor.py backend/app/services/pubmed_service.py backend/app/services/rag_service.py backend/app/services/relationship_extractor.py backend/README.md backend/requirements.txt backend/reset_db.sh frontend/.gitignore frontend/index.html frontend/OAUTH_SETUP.md frontend/package.json frontend/postcss.config.js frontend/README.md frontend/src/App.tsx frontend/src/components/AgenticAI.tsx frontend/src/components/Analytics.tsx frontend/src/components/ChatPanel.tsx frontend/src/components/ExportMenu.tsx frontend/src/components/ForceGraph2DView.tsx frontend/src/components/ForceGraph3DView.tsx frontend/src/components/HypothesisBar.tsx frontend/src/components/LoginComponent.tsx frontend/src/components/NodeDetails.tsx frontend/src/components/PDFSelector.tsx frontend/src/components/ProcessingOverlay.tsx frontend/src/components/ProjectSelection.tsx frontend/src/components/Sidebar.tsx frontend/src/components/UploadPanel.tsx frontend/src/components/UserProfile.tsx frontend/src/contexts/AuthContext.tsx frontend/src/index.css frontend/src/main.tsx frontend/src/services/api.ts frontend/src/store/useStore.ts frontend/src/types/index.ts frontend/tailwind.config.js frontend/tsconfig.json frontend/tsconfig.node.json frontend/vite.config.ts RAG_SYSTEM.md README.md run.sh setup.sh ``` ### Dependencies - backend/requirements.txt: aiofiles@==23.2.1, alembic@==1.13.0, anthropic@==0.39.0, beautifulsoup4@==4.12.3, fastapi@==0.115.0, httpx@==0.25.2, lxml@==5.1.0, networkx@==3.2.1, numpy@==1.26.2, pandas@==2.1.4, passlib[bcrypt]@==1.7.4, psycopg2-binary@==2.9.9, pydantic@==2.9.2, pydantic-settings@==2.6.1, PyMuPDF@==1.24.13, python-dotenv@==1.0.0, python-jose[cryptography]@==3.3.0, python-louvain@==0.16, python-multipart@==0.0.12, requests@==2.31.0, scipy@==1.11.4, scispacy@==0.5.5, spacy@==3.7.4, sqlalchemy@==2.0.23, uvicorn[standard]@==0.32.0 - frontend/package.json: @react-oauth/google@^0.12.2, @types/react@^18.2.43, @types/react-dom@^18.2.17, @types/three@^0.180.0, @typescript-eslint/eslint-plugin@^6.14.0, @typescript-eslint/parser@^6.14.0, @vitejs/plugin-react@^4.2.1, autoprefixer@^10.4.16, axios@^1.6.2, clsx@^2.0.0, eslint@^8.55.0, eslint-plugin-react-hooks@^4.6.0, eslint-plugin-react-refresh@^0.4.5, jwt-decode@^4.0.0, lucide-react@^0.294.0, postcss@^8.4.32, react@^18.2.0, react-dom@^18.2.0, react-dropzone@^14.2.3, react-force-graph-2d@^1.25.4, react-force-graph-3d@^1.24.2, react-hot-toast@^2.4.1, recharts@^2.10.3, tailwindcss@^3.3.6, three@^0.180.0, three-spritetext@^1.10.0, typescript@^5.2.2, vite@^5.0.8, zustand@^4.4.7 ### Recent commits (newest first) - further improvements in rag and chat bot - fixed llm service key error - better persistence between chat and discoveries. improved rag modeling and retrieving - fixed ner model error message - improvements to front end - Remove debugging code from AppContent and simplify LoginComponent by eliminating unused logout functionality and unnecessary UI elements for authenticated users. - Refactor AppContent component to improve code formatting and temporarily force display of LoginComponent for debugging purposes. - Add LAVA API configuration and enhance LoginComponent UI - updated with google scholar and other minor improvemtnes - updated everything to Empirica - added agentic ai - implemented RAG and better LLM - Update project branding in index.html and LoginComponent.tsx to reflect new name and description for Empirica - Update Google OAuth client ID in OAUTH_SETUP.md - google oauth is good to go - implemented google oauth - new project structure and improved dynamic papering - add lava LLM integration - improved setup with LAVA LLM integration - Fix edge rendering in imported projects ## Key source files (fetched from GitHub, selected and truncated for size) ### RAG_SYSTEM.md ```markdown # RAG (Retrieval-Augmented Generation) System ## Overview The RAG system enhances LLM responses by combining: 1. **Vector Embeddings**: Semantic search through PDF content 2. **Knowledge Graph**: Entity relationships and connections 3. **Hybrid Retrieval**: Entity-based + semantic + graph-enhanced search 4. **Context Assembly**: Rich prompts with relevant chunks + relationships ## Architecture ``` ┌─────────────────────────────────────────────────────────────┐ │ RAG System Pipeline │ └─────────────────────────────────────────────────────────────┘ 1. INDEXING PHASE (During PDF Processing) ┌──────────────┐ │ PDF Upload │ └──────┬───────┘ │ ▼ ┌──────────────┐ │ Text Extract │ (PDFProcessor) └──────┬───────┘ │ ▼ ┌──────────────┐ │ Chunking │ (DocumentChunker) │ - Semantic │ │ - Overlap │ │ - Entities │ └──────┬───────┘ │ ▼ ┌──────────────┐ │ NER │ (NERService) │ - Extract │ │ - Classify │ └──────┬───────┘ │ ▼ ┌──────────────┐ │ RAG Index │ (RAGService) │ - Store │ │ - Link │ │ - Embed │ └──────────────┘ 2. RETRIEVAL PHASE (During Query) ┌──────────────┐ │ User Query │ └──────┬───────┘ │ ▼ ┌──────────────┐ │ Extract │ │ Entities │ └──────┬───────┘ │ ▼ ┌──────────────────────────────────┐ │ Hybrid Retrieval │ │ ┌────────────────────────────┐ │ │ │ 1. Entity-Based Search │ │ │ │ - Find chunks with │ │ │ │ mentioned entities │ │ │ └────────────────────────────┘ │ │ ┌────────────────────────────┐ │ │ │ 2. Graph Expansion │ │ │ │ - Get neighbors │ │ │ │ - Find relationships │ │ │ └────────────────────────────┘ │ │ ┌────────────────────────────┐ │ │ │ 3. Semantic Search │ │ │ │ - Vector similarity │ │ │ │ - (Future: embeddings) │ │ │ └────────────────────────────┘ │ └──────────────┬───────────────────┘ │ ▼ ┌──────────────────────────────────┐ │ Context Assembly │ │ - Relevant text chunks │ │ - Entity relationships │ │ - Graph context summary │ │ - Metadata (pages, docs) │ └──────────────┬───────────────────┘ │ ▼ ┌──────────────────────────────────┐ │ Build RAG Prompt │ │ - Format context │ │ - Add instructions │ │ - Include citations │ └──────────────┬───────────────────┘ │ ▼ ┌──────────────────────────────────┐ │ LLM Generation │ │ - Process prompt │ │ - Generate response │ [truncated — 7280 more characters] ``` ### ARCHITECTURE.md ```markdown # Empirica Architecture **Clean, modular biomedical research assistant with RAG-enhanced knowledge graphs** ## 🏗️ System Overview Empirica is a full-stack application that processes biomedical research papers, extracts knowledge graphs, and provides AI-powered insights through a RAG (Retrieval-Augmented Generation) system. ### Core Features - ✅ **Per-PDF Graph System**: Each PDF gets its own knowledge graph - ✅ **Dynamic Graph Merging**: Combine multiple PDF graphs by selection - ✅ **RAG Integration**: Document chunking, indexing, and semantic retrieval - ✅ **LLM-Powered Insights**: Hypothesis generation and conversational AI via Lava + Anthropic Claude - ✅ **Import/Export**: Full project state including RAG indices - ✅ **Multi-user Support**: OAuth authentication with Google --- ## 📁 Project Structure ``` calhacks/ ├── backend/ # FastAPI Python backend │ ├── app/ │ │ ├── config.py # Configuration (Lava keys, etc.) │ │ ├── main.py # API endpoints │ │ ├── models/ │ │ │ ├── database.py # SQLAlchemy models (per-PDF graphs) │ │ │ └── schemas.py # Pydantic request/response schemas │ │ └── services/ │ │ ├── pdf_processor.py # PDF text extraction │ │ ├── ner_service.py # Named Entity Recognition │ │ ├── relationship_extractor.py # Entity relationship extraction │ │ ├── graph_builder.py # NetworkX graph construction │ │ ├── document_chunker.py # Smart PDF chunking for RAG │ │ ├── rag_service.py # RAG indexing & retrieval │ │ ├── content_insight_agent.py # Insight generation │ │ ├── graph_agent.py # Conversational AI │ │ ├── llm_service.py # LLM abstraction layer │ │ ├── lava_service.py # Lava Payments integration │ │ ├── pubmed_service.py # PubMed API integration │ │ └── ctgov_service.py # ClinicalTrials.gov API │ ├── uploads/ # PDF storage & RAG indices │ ├── synapse_mapper.db # SQLite database │ ├── requirements.txt # Python dependencies │ └── .env # Environment variables (Lava keys) │ ├── frontend/ # React + TypeScript + Vite │ ├── src/ │ │ ├── App.tsx # Main app component │ │ ├── components/ │ │ │ ├── Sidebar.tsx # Main navigation & controls │ │ │ ├── UploadPanel.tsx # PDF upload interface │ │ │ ├── ProjectSelection.tsx # Project management │ │ │ ├── PDFSelector.tsx # PDF selection & management │ │ │ ├── ForceGraph2DView.tsx # 2D graph visualization │ │ │ ├── ForceGraph3DView.tsx # 3D graph visualization │ │ │ ├── NodeDetails.tsx # Entity details panel │ │ │ ├── Analytics.tsx # Graph analytics │ │ │ ├── ChatPanel.tsx # AI chat interface │ │ │ └── ExportMenu.tsx [truncated — 11863 more characters] ``` ### backend/requirements.txt ``` # Core Framework fastapi==0.115.0 uvicorn[standard]==0.32.0 python-multipart==0.0.12 pydantic==2.9.2 pydantic-settings==2.6.1 # PDF Processing PyMuPDF==1.24.13 # NLP & Entity Recognition spacy==3.7.4 scispacy==0.5.5 # Graph Processing networkx==3.2.1 python-louvain==0.16 # LLM Integration (Anthropic only) anthropic==0.39.0 # Database sqlalchemy==2.0.23 alembic==1.13.0 psycopg2-binary==2.9.9 # Utilities python-dotenv==1.0.0 aiofiles==23.2.1 httpx==0.25.2 requests==2.31.0 beautifulsoup4==4.12.3 lxml==5.1.0 # Data Processing numpy==1.26.2 pandas==2.1.4 scipy==1.11.4 # CORS & Security (CORS is built into FastAPI) python-jose[cryptography]==3.3.0 passlib[bcrypt]==1.7.4 ``` ### frontend/package.json ``` { "name": "empirica-frontend", "private": true, "version": "1.0.0", "type": "module", "scripts": { "dev": "vite", "build": "tsc && vite build", "lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0", "preview": "vite preview" }, "dependencies": { "@react-oauth/google": "^0.12.2", "axios": "^1.6.2", "clsx": "^2.0.0", "jwt-decode": "^4.0.0", "lucide-react": "^0.294.0", "react": "^18.2.0", "react-dom": "^18.2.0", "react-dropzone": "^14.2.3", "react-force-graph-2d": "^1.25.4", "react-force-graph-3d": "^1.24.2", "react-hot-toast": "^2.4.1", "recharts": "^2.10.3", "three": "^0.180.0", "three-spritetext": "^1.10.0", "zustand": "^4.4.7" }, "devDependencies": { "@types/react": "^18.2.43", "@types/react-dom": "^18.2.17", "@types/three": "^0.180.0", "@typescript-eslint/eslint-plugin": "^6.14.0", "@typescript-eslint/parser": "^6.14.0", "@vitejs/plugin-react": "^4.2.1", "autoprefixer": "^10.4.16", "eslint": "^8.55.0", "eslint-plugin-react-hooks": "^4.6.0", "eslint-plugin-react-refresh": "^0.4.5", "postcss": "^8.4.32", "tailwindcss": "^3.3.6", "typescript": "^5.2.2", "vite": "^5.0.8" } } ``` ### frontend/src/main.tsx ```typescript import React from 'react' import ReactDOM from 'react-dom/client' import App from './App.tsx' import './index.css' ReactDOM.createRoot(document.getElementById('root')!).render( , ) ``` ### frontend/src/App.tsx ```typescript import { useEffect } from 'react'; import { Toaster } from 'react-hot-toast'; import { GoogleOAuthProvider } from '@react-oauth/google'; import { useStore } from './store/useStore'; import { useAuth, AuthProvider } from './contexts/AuthContext'; import { apiService } from './services/api'; import { ForceGraph2DView } from './components/ForceGraph2DView'; import { ForceGraph3DView } from './components/ForceGraph3DView'; import { UploadPanel } from './components/UploadPanel'; import { ProcessingOverlay } from './components/ProcessingOverlay'; import { Sidebar } from './components/Sidebar'; import { ChatPanel } from './components/ChatPanel'; import { NodeDetails } from './components/NodeDetails'; import { LoginComponent } from './components/LoginComponent'; import { ProjectSelection } from './components/ProjectSelection'; import { HypothesisBar } from './components/HypothesisBar'; import { Analytics } from './components/Analytics'; function AppContent() { const { graphData, viewMode, filterOptions, setFilteredGraphData, currentProject, showProjectSelection, showUploadPanel, isLoadingProject, setCurrentProject, setShowProjectSelection, setShowUploadPanel, setIsLoadingProject, setGraphData, setPdfs } = useStore(); const { isAuthenticated, isLoading } = useAuth(); // Apply filters when filter options or graph data change useEffect(() => { if (!graphData) { console.log('App: No graphData to filter'); return; } console.log('App: Filtering graph data', { originalNodes: graphData.nodes?.length, originalEdges: graphData.edges?.length, filterOptions }); let filtered = { ...graphData, nodes: [...graphData.nodes], edges: [...graphData.edges] }; // Filter by entity types if (filterOptions.entityTypes.length > 0) { filtered.nodes = filtered.nodes.filter((node) => filterOptions.entityTypes.includes(node.group) ); const nodeIds = new Set(filtered.nodes.map((n) => n.id)); filtered.edges = filtered.edges.filter((edge) => { const sourceId = typeof edge.source === 'string' ? edge.source : (edge.source as any)?.id; const targetId = typeof edge.target === 'string' ? edge.target : (edge.target as any)?.id; return nodeIds.has(sourceId) && nodeIds.has(targetId); }); } // Filter by minimum degree if (filterOptions.minDegree > 1) { const degrees = new Map(); filtered.edges.forEach((edge) => { const sourceId = typeof edge.source === 'string' ? edge.source : (edge.source as any)?.id; const targetId = typeof edge.target === 'string' ? edge.target : (edge.target as any)?.id; degrees.set(sourceId, (degrees.get(sourceId) || 0) + 1); degrees.set(targetId, (degrees.get(targetId) || 0) + 1); }); filtered.nodes = filtered.nodes.filter( (node) => (degrees.get(node.id) || 0) >= filterOptions.minDegree ); const nodeIds = new Set(filtered.nodes.map((n) => n.id)); filtered.edges = filtered.edges.filter((edge) => { const sourceId = typeof edge.source === 'string' ? edge.source : (edge.source as any)?.id; const targetId = typeof edge.target === 'string' ? edge.target : (edge.target as any)?.id; return nodeIds.has(sourceId) && nodeIds.has(targetId); }); } // Filter by search query if (filterOptions.searchQuery) { const query = filterOptions.searchQuery.toLowerCase(); filtered.nodes = filtered.nodes.filter((node) => node.id.toLowerCase().includes(query) ); const nodeIds = new Set(filtered.nodes.map((n) => n.id)); filtered.edges = filtered.edges.filter((edge) => { const sourceId = typeof edge.source === 'string' ? edge.source : (edge.source as any)?.id; const targetId = typeof edge.target === 'string' ? edge.target : (edge.target as any)?.id; return nodeIds.has(sourceId) && nodeIds.has(targetId); }); } console.log('App: Filtered result', { filteredNodes: filtered.nodes.length, filteredEdges: filtered.edges.length, sampleFilteredEdge: filtered.edges[0] }); setFilteredGraphData(filtered); }, [graphData, filterOptions, setFilteredGraphData]); if (isLoading) { return (
Loading...
); } if (!isAuthenticated) { return ; } // Show loading state while project is being loaded if (isLoadingProject) { return (

Loading project...

); } // Show upload panel if explicitly requested if (showUploadPanel) { return (
); } // Show [truncated — 3406 more characters] ``` ### frontend/src/types/index.ts ```typescript export type EntityType = | 'GENE_OR_GENE_PRODUCT' | 'CHEMICAL' | 'DISEASE' | 'ORGANISM' | 'CELL_TYPE' | 'TISSUE' | 'ORGAN' | 'ENTITY' | 'UNKNOWN'; export interface Node { id: string; group: EntityType; value: number; metadata: { count?: number; degree?: number; [key: string]: any; }; } export interface Edge { source: string; target: string; value: number; title: string; metadata: { all_evidence?: string[]; relationship_type?: string; [key: string]: any; }; } export interface GraphData { nodes: Node[]; edges: Edge[]; metadata: { total_nodes?: number; total_edges?: number; density?: number; analytics?: GraphAnalytics; [key: string]: any; }; } export interface GraphAnalytics { total_nodes: number; total_edges: number; density: number; avg_degree: number; communities: string[][]; centrality_scores: Record; entity_counts: Record; } export interface PDFMetadata { document_id: string; filename: string; uploaded_at: string; processed: boolean; selected: boolean; node_count: number; edge_count: number; entity_counts: Record; } export interface ProjectInfo { project_id: string; name: string; description: string; created_at: string; updated_at: string; pdf_count: number; pdfs: PDFMetadata[]; } export interface ProcessingStatus { job_id: string; status: 'pending' | 'processing' | 'completed' | 'failed'; progress: number; message: string; result?: GraphData; } export interface FilterOptions { minDegree: number; entityTypes: EntityType[]; topN?: number; searchQuery: string; } export interface ViewMode { dimension: '2d' | '3d'; showLabels: boolean; showEdgeLabels: boolean; } export const ENTITY_COLORS: Record = { GENE_OR_GENE_PRODUCT: '#3b82f6', // Blue CHEMICAL: '#10b981', // Green DISEASE: '#ef4444', // Red ORGANISM: '#f59e0b', // Orange CELL_TYPE: '#8b5cf6', // Purple TISSUE: '#ec4899', // Pink ORGAN: '#14b8a6', // Teal ENTITY: '#60a5fa', // Light Blue (generic) UNKNOWN: '#6b7280', // Gray }; export const ENTITY_LABELS: Record = { GENE_OR_GENE_PRODUCT: 'Gene/Protein', CHEMICAL: 'Chemical', DISEASE: 'Disease', ORGANISM: 'Organism', CELL_TYPE: 'Cell Type', TISSUE: 'Tissue', ORGAN: 'Organ', ENTITY: 'Entity', UNKNOWN: 'Unknown', }; ``` ### run.sh ```shell #!/bin/bash # Empirica Run Script # Starts both backend and frontend in tmux set -e echo "🧬 Starting Empirica..." echo "" # Check if tmux is installed if ! command -v tmux &> /dev/null; then echo "❌ tmux is not installed. Please install it or run backend and frontend manually." echo "" echo "Manual start:" echo "Terminal 1: cd backend && source venv/bin/activate && python -m app.main" echo "Terminal 2: cd frontend && npm run dev" exit 1 fi # Create new tmux session SESSION="empirica" # Kill existing session if it exists tmux has-session -t $SESSION 2>/dev/null && tmux kill-session -t $SESSION # Create new session with backend tmux new-session -d -s $SESSION -n backend "cd backend && source venv/bin/activate && python -m app.main" # Create new window for frontend tmux new-window -t $SESSION -n frontend "cd frontend && npm run dev" # Attach to session echo "✅ Started Empirica in tmux session '$SESSION'" echo "" echo "To view:" echo " tmux attach -t $SESSION" echo "" echo "To detach: Ctrl+B then D" echo "To switch windows: Ctrl+B then N (next) or P (previous)" echo "To stop: tmux kill-session -t $SESSION" echo "" echo "Opening in 3 seconds..." sleep 3 tmux attach -t $SESSION ``` ### setup.sh ```shell #!/bin/bash # Empirica Setup Script # Automated setup for backend and frontend set -e echo "🧬 Empirica Setup" echo "=======================" echo "" # Check Python version echo "Checking Python version..." python_version=$(python3 --version 2>&1 | awk '{print $2}') echo "✓ Found Python $python_version" # Check Node version echo "Checking Node.js version..." node_version=$(node --version) echo "✓ Found Node.js $node_version" echo "" echo "📦 Setting up Backend..." cd backend # Create virtual environment if [ ! -d "venv" ]; then echo "Creating virtual environment..." python3 -m venv venv fi # Activate virtual environment source venv/bin/activate # Install dependencies echo "Installing Python dependencies..." pip install --upgrade pip pip install -r requirements.txt # Download scispaCy model echo "Downloading scispaCy model (this may take a few minutes)..." echo "Note: This downloads ~1.5GB model from S3..." pip install https://s3-us-west-2.amazonaws.com/ai2-s2-scispacy/releases/v0.5.4/en_ner_bionlp13cg_md-0.5.4.tar.gz # Create .env if it doesn't exist if [ ! -f ".env" ]; then echo "Creating .env file..." cat > .env << EOF # API Configuration API_HOST=0.0.0.0 API_PORT=8000 API_RELOAD=True # Database DATABASE_URL=sqlite:///./synapse_mapper.db # LLM API Keys (Optional) OPENAI_API_KEY= ANTHROPIC_API_KEY= # LAVA API Configuration LAVA_SECRET_KEY=aks_live_m3wV8b44f9_BaljpU1ujJEj3Q5lReJRQBjcuS50ogLPu7OstaqAy1SP LAVA_CONNECTION_SECRET=cons_live_kLJ-brEFgIll0qeBvKXSqGN-k6rgWcss9mmAyNxSH-rmilCjuOcTKZ LAVA_PRODUCT_SECRET=ps_live_HXcIGt5s3lG8BnYYqhqXlFV5BhWHLQI0r7U6Vac5E37m69GA5znZHXLS ENABLE_LAVA=true # Processing Configuration MAX_UPLOAD_SIZE_MB=100 MAX_CONCURRENT_PROCESSING=4 ENABLE_LLM_EXTRACTION=false # CORS CORS_ORIGINS=["http://localhost:5173", "http://localhost:3000"] EOF fi echo "✓ Backend setup complete!" cd .. echo "" echo "🎨 Setting up Frontend..." cd frontend # Install dependencies echo "Installing Node dependencies..." npm install # Create .env if it doesn't exist if [ ! -f ".env" ]; then echo "Creating .env file..." cat > .env << EOF VITE_API_URL=http://localhost:8000 VITE_GOOGLE_CLIENT_ID=865286496790-ftmfmsu2uq8t1d4vf11fh44cp5v5u94i.apps.googleusercontent.com EOF fi echo "✓ Frontend setup complete!" cd .. echo "" echo "✅ Setup Complete!" echo "" echo "To start the application:" echo "" echo "Terminal 1 (Backend):" echo " cd backend" echo " source venv/bin/activate" echo " python -m app.main" echo "" echo "Terminal 2 (Frontend):" echo " cd frontend" echo " npm run dev" echo "" echo "Then open http://localhost:5173 in your browser!" echo "" ``` ### frontend/postcss.config.js ```javascript export default { plugins: { tailwindcss: {}, autoprefixer: {}, }, } ``` [44 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]