Helios

💡

Inspiration

Ever been in a meeting where important decisions and action items get lost in the conversation? We built Helios to automatically transform spoken discussions into visual, interactive knowledge graphs - making it easy to see relationships, track decisions, and never miss important details. 🚀

What it does

Helios listens to your conversations through voice AI and automatically generates a live, interactive knowledge graph that: Extracts key entities (people, systems, databases, services) in real-time Identifies relationships between concepts as you speak Tracks decisions and action items automatically Organizes information hierarchically - important nodes at the top, details below 🛠️

How we built it

Frontend: React + TypeScript + TailwindCSS for the UI React Flow for interactive graph visualization Vapi SDK for real-time voice transcription and AI conversation Socket.IO client for real-time updates Backend: Node.js + Express server Socket.IO for WebSocket communication OpenRouter API with Google's Gemini Pro for intelligent graph generation Vapi API for voice processing and transcription Custom graph service with intelligent positioning algorithms 🔊 VAPI Integration Real-Time Voice-to-Knowledge Graph Pipeline 1. Live Transcription & Conversation Tracking Captured real-time speech-to-text during team discussions Tracked full conversation history with speaker attribution Maintained conversation context throughout entire session 2. AI Assistant Integration Two-way dialogue: Users speak naturally, AI responds with clarifying questions Conversational flow management via Vapi's assistant API Dynamic conversation state handling (started, active, ended events) 3. Streaming Transcript Processing Real-time transcript updates pushed to backend via WebSocket Incremental graph building as conversation progresses Periodic graph refinement every 10 seconds during active recording 4. Multi-Modal Input Support Voice recording through Vapi Web SDK Audio file upload for transcription (supporting async workflows) Session-based recording management 5. Advanced Conversation Analysis Leveraged Vapi's conversation metadata for timestamps Speaker identification for attribution in a knowledge graph Confidence scoring from transcription quality 6. Seamless UX Integration Start/stop recording controls tied to Vapi lifecycle Visual status indicators (listening, processing, finalizing) Toast notifications synced with Vapi conversation events Clean error handling for connection issues Result: Users speak naturally about their ideas, and our AI instantly transforms messy conversations into structured, visual knowledge graphs - all powered by Vapi's robust voice infrastructure. Key Technical Achievement: We went beyond basic transcription by using Vapi as the real-time data source for an intelligent graph-building system, demonstrating Vapi's capability to power complex AI workflows, not just simple voice capture. Key Technical Features: Smart node placement algorithm that organizes by importance (top-to-bottom layout) Real-time position synchronization when users drag nodes Rate limiting and throttling to handle API constraints Collision detection to prevent overlapping nodes WebSocket-based live updates across all connected clients 🏆 Accomplishments we're proud of Real-time AI analysis - Processing conversations and updating graphs live with minimal latency Intelligent layout system - Nodes automatically organize by importance with perfect spacing Seamless voice integration - Natural conversation flows directly into visual structures Production-ready architecture - Rate limiting, error handling, and graceful fallbacks 📚

What we learned

Robust schemas are imperative to the full-stack development process as it creates solid foundations to build off of Bring ideas. If you can think it, the tools are there to make it Stick to the MVP. Make something first, then make it better. Not the other way around 🔮

What's next

Persistent storage - Save and reload conversation graphs Advanced AI models - Experiment with different LLMs for better entity extraction Graph analytics - Insights on conversation patterns and key topics Meeting integrations - Connect with Zoom, Teams, or Google Meet Template library - Pre-built graph structures for common meeting types 🎯 Built for CalHacks A project showcasing real-time AI, voice technology, and collaborative visualization - turning conversations into actionable insights. 📊 Tech Stack Summary