Project Info
Inspiration
Clinical documentation is one of the largest hidden burdens in healthcare. Nurses often spend hours typing notes and may record notes that lack key context, contributing to the growing problem of unstructured data in healthcare. Concurrently, most AI scribes and copilots send raw patient conversations to cloud models — creating serious privacy, compliance, and data ownership concerns. We asked a simple question: What if we could build a clinical copilot that improves documentation quality while making privacy the default, not a feature? Hexi solves this: it reduces the burden of documentation, identifies missing clinical information in real time, and ensures that sensitive patient data never leaves the browser unprotected.
What it does
Hexi is a real-time, privacy-first clinical conversation copilot. It: Transcribes patient conversations fully in the browser. Runs a custom 350M parameter LoRA redaction model locally in the browser to detect and censor personal health information. Sends only redacted text to state-of-the-art cloud reasoning models, such as Claude Sonnet. Identifies missing clinical information (HPI gaps, medications, allergies, history, etc.). Guides structured follow-up questions, leading to the construction of a complete patient profile. Knows when to stop asking questions once enough information has been gathered. Unlike typical AI scribes that simply summarize transcripts, Hexi actively detects knowledge gaps and helps clinicians complete structured documentation efficiently. Most importantly, no personal health information is ever sent to the cloud.
How we built it
Our architecture is privacy-first by design: For our frontend, we use React + TypeScript for type safety. In-browser model execution using ONNX/WebGPU. A custom LoRA fine-tuned 350M parameter model trained using transfer learning from Claude Sonnet 4. Real-time streaming transcription that happens completely on device. Redacted text is sent to a state-of-the-art cloud reasoning engine (Claude Sonnet configured as a multi-turn reasoning agent). The redaction model we used is a state-of-the-art "Liquid Foundation Model" by Liquid AI. These models are small yet offer excellent performance and work well in edge deployments. We also implemented a multi-turn agent framework that: Tracks clinical slot completion. Measures marginal information gain. Prevents infinite follow-up loops. This allows Hexi to stop when sufficient information has been gathered instead of asking endless general questions.
Challenges we ran into
Running a 350M parameter model efficiently inside the browser. Combining regex-based redaction with model-based entity detection. Preventing the agent from asking overly general or repetitive questions. Implementing a formal stopping condition for “enough information.” Handling malformed JSON outputs from the reasoning model. Maintaining a smooth real-time UX while models load locally.
Accomplishments we're proud of
Successfully running a 350M parameter redaction model entirely in-browser. Training an accurate model using transfer learning from Claude Sonnet, resulting in a scalable approach. Achieving full privacy-by-default architecture (only redacted text leaves the device). Building a working multi-turn clinical reasoning agent. Implementing a measurable information sufficiency stopping mechanism. In a short timeframe, we built not just a demo, but a functional system with real architectural integrity.
What we learned
Edge AI in the browser is more viable than most people think, but it requires careful performance tuning. Multi-turn agents need explicit structure; without constraints, they drift into generic questioning. Measuring “enough information” is just as important as generating answers. Privacy-first design simplifies downstream trust and compliance conversations. Clear UX is absolutely essential, especially in rushed healthcare environments.
What's next
Clarify how we would integrate with patient-led medical record storage systems such as Healthnix. Explore integrations with EHR systems to prevent the double-entry problem, where nurses first enter information in our software and then in the EHR. Refine the information sufficiency scoring system to prevent redundant follow-ups and better characterize what is "enough." Optimize model loading speed on mobile browsers. Conduct clinical usability testing with nurses to assess the real-world impact of our product. Our long-term vision is to make Hexi the default clinical copilot for nurses — one that enhances care quality while protecting patient privacy at its core.
Hexi — AI Nurse Copilot 🏥
Hexi is an AI-powered nurse copilot that helps healthcare professionals gather complete patient information through intelligent questioning. Built at TreeHacks 2026.
✨ Features
- 🌐 3D Knowledge Globe — Interactive visualization showing what's known (green) vs. unknown (red) about a patient
- 📄 Document Analysis — Upload PDFs/text files and automatically extract known patient information
- 🎤 Voice Input — Real-time speech-to-text for hands-free documentation
- 💬 Smart Questioning — AI suggests the most important questions to ask based on clinical context
- 📊 NCLEX-Aligned — Question patterns based on nursing case study frameworks
- 📋 PDF Export — Generate professional reports from conversations
🏗️ Architecture
TreeHacks-2026/
├── frontend/ # React + Vite frontend
│ ├── src/
│ │ ├── components/
│ │ │ ├── KnowledgeGlobe.jsx # 3D globe visualization
│ │ │ ├── HexiCore.jsx # Animated hexagon core
│ │ │ ├── InputArea.jsx # Chat/voice input
│ │ │ ├── TranscriptionStream.jsx
│ │ │ └── ...
│ │ ├── hooks/
│ │ │ └── useVoiceSession.js # Voice recording hook
│ │ ├── api.js # API client
│ │ └── App.jsx # Main application
│ └── md/ # Design system docs
├── saging-api/ # FastAPI backend
│ └── app/
│ ├── main.py # API endpoints
│ ├── biomcp_client.py # Clinical patterns & scenarios
│ ├── llm.py # LLM integration
│ └── prompts.py # System prompts
├── mimic-mcp/ # MIMIC database MCP server
└── backend/ # Additional backend services
🚀 Quick Start
Prerequisites
- Node.js 18+
- Python 3.10+
- OpenAI API key (for LLM features)
1. Clone & Setup
git clone https://github.com/IvanRatushnyy/TreeHacks-2026.git
cd TreeHacks-2026
2. Start the Backend
cd saging-api
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
# Set your API keys
export OPENAI_API_KEY="your-key-here"
export ANTHROPIC_API_KEY="your-key-here"
# Run the server
./run.sh
# Or manually:
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
3. Start the Frontend
cd frontend
npm install
npm run dev
Open http://localhost:5173 in your browser.
📡 API Endpoints
| Endpoint | Method | Description |
|---|---|---|
/health | GET | Health check |
/api/knowledge-gaps | POST | Detect knowledge gaps from transcript/document |
/api/chat/complete | POST | Complete a chat session with summary |
/api/clinical/validate | POST | Validate clinical snippet and suggest follow-ups |
/api/documents/analyze | POST | Analyze uploaded document for gaps |
/api/workflow/next-question | POST | Get AI-suggested next question |
/api/tts | POST | Text-to-speech (OpenAI voices) |
Knowledge Gaps API
curl -X POST http://localhost:8000/api/knowledge-gaps \
-H "Content-Type: application/json" \
-d '{
"transcript": "Patient is a 16-year-old with type 1 diabetes who fainted at school",
"symptoms": ["syncope", "diabetes"],
"file_content": "Optional uploaded document text"
}'
Response includes:
gaps[]— Array of questions withfilledstatus (green/red on globe)filled_count— Number of known itemsnext_question— Suggested next question to ask
🧠 Clinical Scenarios
Hexi supports intelligent questioning for various clinical presentations:
| Scenario | Keywords Detected |
|---|---|
| Diabetic Hypoglycemia | diabetes, hypoglycemia, syncope, gym |
| Chest Pain | chest pain, cardiac, angina |
| Preeclampsia | pregnant, headache, hypertension |
| Pediatric Anaphylaxis | child, allergic, hives, epipen |
| Syncope | faint, passed out, collapsed |
🎨 Design System
The UI follows a medical-professional aesthetic:
- Primary Blue:
#134074— Trust, professionalism - Accent Red:
#DC2626— Unknown/gaps (action needed) - Accent Green:
#22C55E— Known/filled information - Font: Inter (body), DM Sans (display)
See frontend/md/ for full design documentation.
🔧 Development
Frontend Dev Server
cd frontend && npm run dev
Backend with Hot Reload
cd saging-api && uvicorn app.main:app --reload --port 8000
Run Both
# Terminal 1
cd saging-api && ./run.sh
# Terminal 2
cd frontend && npm run dev
📦 Tech Stack
Frontend:
- React 18 + Vite
- Canvas API (3D globe rendering)
- Web Speech API (voice input)
- jsPDF + html2canvas (PDF export)
Backend:
- FastAPI (Python)
- Anthropic Claude / OpenAI GPT
- Pydantic for validation
🏆 TreeHacks 2026
Built with ❤️ at Stanford TreeHacks 2026.
Team:
- Ivan Ratushnyy
📄 License
MIT License — see LICENSE for details.
Analysis
View
Metric
- 10
- 9
- 2
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- AnthropicIn code
- CSSIn code
- FastAPIIn code
- FlaskIn code
- HTMLIn code
- JavaScriptIn code
- OpenAIIn code
- PythonIn code
- ReactIn code
- Tailwind CSSIn code
- TypeScriptIn code
11 of 11 appear in the indexed code.
AI coding agents
- CursorCommits
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
341 KB
Source files
67
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
IvanRatushnyy/TreeHacks-2026
108 files · 38.7 MB · @ ca4b933
Structure
Interface
23 files · 21%Screens, components and styles rendered to the user.
Application logic
36 files · 33%Domain rules, services and shared utilities.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- JavaScript54%
- Python27%
- Markdown16%
- CSS2%
- TypeScript1%
- Shell1%
- Other (1)0%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
backend/pii-model/lfm2-browser-demo/package.json
npm · 15- @huggingface/transformers
- react
- react-dom
- +12 more
frontend/package.json
npm · 13- react
- react-dom
- +11 more
saging-api/requirements.txt
pypi · 10- anthropic
- boto3
- fastapi
- httpx
- openai
- pydantic-settings
- pypdf
- python-dotenv
- python-multipart
- uvicorn[standard]
backend/requirements.txt
pypi · 3- Flask
- Flask-CORS
- pymupdf
backend/pii-model/package.json
npm · 1- @huggingface/transformers
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.