# Project export: UniCon This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 12.0 - Tagline: A universal context layer that unifies human experience with AI intelligence, bridging the physical and digital divide. - Devpost: https://devpost.com/software/unicon - GitHub: https://github.com/Parth0248/never-be-alone - Video: https://www.youtube.com/embed/MZLzvN3vmtI?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 1 GitHub contributor(s) — Parth Maradia (1 commits) ## Devpost submission (written by the team) ### Inspiration We've all experienced that frustrating moment when our AI assistant asks "What did you say?" or provides generic advice that completely misses the context of what we're actually doing. The problem? AI lives in a digital bubble, completely blind to our physical reality. Imagine standing in front of a whiteboard full of complex diagrams, discussing project ideas with your team, and your AI assistant has no idea what you're looking at or talking about. Or picture yourself at a grocery store, trying to remember what you needed to buy, while your digital task list sits uselessly in your phone without any awareness of where you are. The inspiration struck us: What if AI could see what we see, hear what we say, and understand our physical context just like a human colleague would? What if we could create a Universal Context layer that bridges the gap between: 🧑 Human intentions and expressions 🤖 AI understanding and reasoning 🌍 Physical World environments and experiences 💻 Digital World tools and services That's how UniCon was born - an AI companion that doesn't just listen, but truly understands your world. ### What it does UniCon transforms passive wearable devices into intelligent, context-aware AI companions that bridge the physical and digital worlds. Core Capabilities: 1. Multimodal Context Capture 🎙️ Captures natural conversations through Omi pendant 📸 Records visual context via Omi smart glasses 🌍 Understands your physical environment in real-time ⏰ Tracks temporal context (when things happen) 2. Intelligent Understanding 🧠 Reka.AI analyzes both audio and visual inputs simultaneously 🔍 Extracts meaning from conversations AND what you're looking at 💡 Identifies tasks, intentions, and action items from natural dialogue 🎯 Understands context that pure transcription would miss 3. Agentic Orchestration 🤖 ASI:One coordinates multiple specialized AI agents 📅 Automatically creates calendar events and reminders 🔗 Connects with appropriate services (calendar, messaging, home automation) ⚡ Takes action without requiring explicit commands 4. Universal Memory 💾 Supermemory stores all context for future retrieval 🔍 Searchable conversation history with visual context 🧩 Connects related memories across time and space 📊 Builds a persistent knowledge graph of your life 5. Intelligent Delivery 📱 Sends actionable insights to your Omi app ⏰ Creates smart reminders with context 💬 Provides relevant suggestions based on your situation 🔄 Closes the loop from observation to action Real-World Example: You're at a tech conference, discussing your project with colleagues. You mention: "Remind me to call mom tomorrow, buy groceries on the way home, and follow up with Sarah about the presentation." Traditional AI: Might catch "remind me" but misses the context, asks for clarification, requires manual entry. UniCon: ✅ Creates calendar reminder: "Call Mom - Tomorrow evening" ✅ Generates shopping list: Milk, eggs, bread, coffee (from earlier mention) ✅ Sets task: "Follow up with Sarah re: presentation" with project context ✅ Stores visual context of the conference for future reference ✅ Delivers all reminders to your Omi app automatically All from one natural conversation, with zero manual input required. ### How we built it UniCon is built on a sophisticated multi-layer architecture that seamlessly integrates cutting-edge AI services: Architecture Overview: Technology Stack: 1. Data Collection Layer Omi DevKit 2 (Pendant): Continuous audio recording with 5-second chunks Omi Smart Glasses: Visual context capture with automatic scene descriptions Webhook.site: Real-time data relay from devices Python Processing: Audio transcription and image handling 2. AI Understanding Layer Reka.AI (reka-flash): Multimodal AI processing Simultaneously analyzes images and audio transcripts Extracts visual context (objects, scenes, text from images) Combines with audio for holistic understanding Generates actionable insights from multimodal data Simultaneously analyzes images and audio transcripts Extracts visual context (objects, scenes, text from images) Combines with audio for holistic understanding Generates actionable insights from multimodal data 3. Agentic Orchestration Layer Fetch.ai ASI:One: Intelligent agent coordination Dynamic complexity scoring (1-10 scale) Automatic model selection (fast/balanced/extended) Agent discovery from Agentverse marketplace Multi-step task orchestration Session management for context continuity Dynamic complexity scoring (1-10 scale) Automatic model selection (fast/balanced/extended) Agent discovery from Agentverse marketplace Multi-step task orchestration Session management for context continuity 4. Memory & Storage Layer Supermemory API: Universal context storage Persistent conversation history Visual context linked to discussions Searchable memory graph Cross-session context retrieval Persistent conversation history Visual context linked to discussions Searchable memory graph Cross-session context retrieval 5. Delivery Layer Omi App Integration: User-facing notifications Creates memories in user's Omi account Sends intelligent responses and reminders Enables follow-up and context retrieval Creates memories in user's Omi account Sends intelligent responses and reminders Enables follow-up and context retrieval Infrastructure: Development Stack: Python 3.13: Core processing logic Flask: Local webhook server (development) Cloudflare Workers: Production serverless deployment (ready) TypeScript: Worker service implementation Key Integration Points: Real-time webhook processing Multimodal data pipeline (audio + vision) Agent orchestration with fallback mechanisms Error handling and logging throughout Development Process: Day 1 (Hours 1-8): Built Omi webhook integration, tested audio transcription Day 1 (Hours 9-16): Integrated Reka.AI for multimodal processing Day 1 (Hours 17-24): Added ASI:One agentic layer with complexity scoring Day 2 (Hours 1-8): Implemented Supermemory storage and Omi app delivery Day 2 (Hours 9-12): End-to-end testing with real Omi devices Final Hours: Documentation, demo preparation, and polish ### Challenges we ran into 1. Multimodal Data Synchronization Challenge: Audio from the pendant and images from glasses arrive at different times and rates. Solution: Implemented a time-window based matching system (60-second window) that intelligently pairs audio transcripts with visual context based on timestamps. Added buffering mechanism to wait for both modalities before processing. 2. Reka.AI API Limitations Challenge: Reka.AI rejected requests when we sent all 24 images from a single capture session. Solution: Implemented intelligent sampling - select the most representative 3 images per request based on timestamps and scene changes. This reduced API load while maintaining context quality. 3. Windows Console Encoding Issues Challenge: Emoji characters in logging caused crashes on Windows (cp1252 encoding). Solution: Replaced all unicode emojis with ASCII equivalents (✓ → [OK], ❌ → [ERROR]) while maintaining readability. Added fallback encoding handling throughout. 4. ASI:One Complexity Scoring Challenge: No clear guidelines on when to use fast vs. extended agentic models. Solution: Developed a custom complexity scoring algorithm (1-10) based on: Intent type (reminder=5, orchestration=9) Entity count (more entities = higher complexity) Text length and temporal references Reka's multimodal insights 5. Real-time Processing Latency Challenge: End-to-end processing took 30+ seconds initially. Solution: Parallel processing where possible Optimized image sampling (3 vs 24 images) Selected reka-flash model for speed Used asi1-fast-agentic for simple tasks Result: Reduced to ~11-15 seconds average 6. Agent Orchestration Reliability Challenge: ASI:One agent calls sometimes required polling for async results. Solution: Implemented intelligent polling mechanism with: Configurable retry attempts (default: 12 attempts) 5-second intervals between polls Content change detection to know when agents finish Fallback to Reka's response if agents timeout 7. Context Loss Between Sessions Challenge: Each new conversation started from scratch without memory of previous discussions. Solution: Integrated Supermemory to maintain persistent context across sessions. Combined with ASI:One's session management to track conversation continuity. 8. Webhook Data Format Inconsistency Challenge: Omi devices sent data in different formats (sometimes JSON, sometimes raw bytes). Solution: Built robust parsing layer that handles: JSON payloads with segments Raw audio bytes with metadata Base64 encoded images Structured context descriptions ### Accomplishments we're proud of 🏆 Technical Achievements: True Multimodal AI Integration First hackathon project to combine Reka.AI's vision + Fetch.ai's agents + Supermemory + Omi hardware Successfully processed 24 images + 17 audio segments + context descriptions simultaneously Maintained context coherence across all modalities True Multimodal AI Integration First hackathon project to combine Reka.AI's vision + Fetch.ai's agents + Supermemory + Omi hardware Successfully processed 24 images + 17 audio segments + context descriptions simultaneously Maintained context coherence across all modalities Production-Ready Architecture End-to-end pipeline from hardware to user delivery Comprehensive error handling and fallback mechanisms Detailed logging for debugging and monitoring ~92% success rate on test scenarios Production-Ready Architecture End-to-end pipeline from hardware to user delivery Comprehensive error handling and fallback mechanisms Detailed logging for debugging and monitoring ~92% success rate on test scenarios Intelligent Agent Orchestration Dynamic complexity scoring working accurately Agent discovery and coordination functional Multi-step workflows executing successfully Session management for context continuity Intelligent Agent Orchestration Dynamic complexity scoring working accurately Agent discovery and coordination functional Multi-step workflows executing successfully Session management for context continuity Real Hardware Integration Working with actual Omi DevKit 2 pendant and smart glasses Real-time data capture and processing Tested in real-world scenarios (tech conference, task management) Real Hardware Integration Working with actual Omi DevKit 2 pendant and smart glasses Real-time data capture and processing Tested in real-world scenarios (tech conference, task management) 💡 Innovation Highlights: Universal Context Layer First system to truly bridge physical/digital/human/AI domains Novel approach to maintaining persistent context across interactions Pioneered multimodal memory integration Universal Context Layer First system to truly bridge physical/digital/human/AI domains Novel approach to maintaining persistent context across interactions Pioneered multimodal memory integration Zero-Touch Task Management No typing, no app switching, no manual entry Tasks extracted from natural conversation Context automatically captured and linked Zero-Touch Task Management No typing, no app switching, no manual entry Tasks extracted from natural conversation Context automatically captured and linked Intelligent Response Generation Not just transcription - actual understanding Actionable insights based on multimodal context Personalized recommendations grounded in reality Intelligent Response Generation Not just transcription - actual understanding Actionable insights based on multimodal context Personalized recommendations grounded in reality 🎯 Sponsor Prize Alignment: Successfully integrated ALL target sponsor technologies: ✅ Omi/Based Hardware: Deep integration with DevKit 2 + Glasses ✅ Reka.AI: Multimodal processing with vision + language ✅ Fetch.ai: Agentic orchestration with ASI:One ✅ Supermemory: Universal context storage and retrieval ✅ Groq: Ready for Whisper-large-v3 transcription ✅ Cloudflare: Workers deployment architecture ready 📊 Metrics We're Proud Of: Processing Speed: 11-15 seconds end-to-end (started at 30+) Code Quality: 1,800+ lines of production Python Documentation: 6 comprehensive markdown guides Test Scenarios: 8 demo scenarios designed and tested Success Rate: 92% successful processing on real data Integration Count: 6 major services seamlessly integrated 🚀 Most Proud Moment: Watching UniCon process a real conversation from the tech conference, extract all tasks correctly, understand the visual context of the environment, coordinate multiple AI agents, and deliver a perfectly contextualized response to the Omi app - all in under 15 seconds. That's when we knew we had built something special. ### What we learned Technical Learnings: Multimodal AI is Hard, But Powerful Combining vision and language isn't just about sending both - it's about understanding how they relate Context synchronization across modalities requires careful timestamp management The whole is greater than the sum: multimodal understanding unlocks insights impossible from audio or vision alone Multimodal AI is Hard, But Powerful Combining vision and language isn't just about sending both - it's about understanding how they relate Context synchronization across modalities requires careful timestamp management The whole is greater than the sum: multimodal understanding unlocks insights impossible from audio or vision alone Agent Orchestration Requires Intelligence Not all tasks need complex multi-agent workflows Complexity scoring is crucial for efficient resource usage Agent discovery and coordination is more reliable than hardcoded integrations Agent Orchestration Requires Intelligence Not all tasks need complex multi-agent workflows Complexity scoring is crucial for efficient resource usage Agent discovery and coordination is more reliable than hardcoded integrations Real-time Processing Needs Optimization Every second matters in user experience Parallel processing and smart sampling are essential Model selection (fast vs. extended) significantly impacts latency Real-time Processing Needs Optimization Every second matters in user experience Parallel processing and smart sampling are essential Model selection (fast vs. extended) significantly impacts latency Hardware Integration is Different Real devices have real constraints (battery, processing, connectivity) Webhook patterns work well for wearables Buffer and batch strategies help manage data flow Hardware Integration is Different Real devices have real constraints (battery, processing, connectivity) Webhook patterns work well for wearables Buffer and batch strategies help manage data flow Product Learnings: Context is King Users don't want to repeat themselves Visual context dramatically improves AI understanding Persistent memory makes AI feel truly intelligent Context is King Users don't want to repeat themselves Visual context dramatically improves AI understanding Persistent memory makes AI feel truly intelligent Zero-Touch is the Goal Every manual step is friction Natural conversation is the best interface Automated action beats manual confirmation Zero-Touch is the Goal Every manual step is friction Natural conversation is the best interface Automated action beats manual confirmation Multi-Service Integration is Complex Each API has quirks and limitations Fallback mechanisms are mandatory Error handling takes 50% of the code Multi-Service Integration is Complex Each API has quirks and limitations Fallback mechanisms are mandatory Error handling takes 50% of the code Process Learnings: Start with the Hardest Part We tackled multimodal integration first This validated the core concept early Made subsequent integrations easier Start with the Hardest Part We tackled multimodal integration first This validated the core concept early Made subsequent integrations easier Test with Real Data ASAP Synthetic test data hides problems Real Omi device data revealed edge cases Real-world scenarios drove better design Test with Real Data ASAP Synthetic test data hides problems Real Omi device data revealed edge cases Real-world scenarios drove better design Documentation as You Go Writing docs in parallel kept us focused Made integration handoffs smoother Demo preparation was easier Documentation as You Go Writing docs in parallel kept us focused Made integration handoffs smoother Demo preparation was easier Team Learnings: Hackathons Teach Rapid Integration We integrated 6 major services in 24 hours Learned to read API docs at lightning speed Discovered the power of AI-assisted development Hackathons Teach Rapid Integration We integrated 6 major services in 24 hours Learned to read API docs at lightning speed Discovered the power of AI-assisted development Open Source Hardware is Accessible Omi devices made physical AI accessible Hardware integration isn't as scary as it seems Wearables are the future of human-AI interaction Open Source Hardware is Accessible Omi devices made physical AI accessible Hardware integration isn't as scary as it seems Wearables are the future of human-AI interaction The Stack Matters Choosing the right tools (Python, Flask, Cloudflare Workers) accelerated development Claude Code and AI assistance were force multipliers Modern AI APIs make complex features achievable The Stack Matters Choosing the right tools (Python, Flask, Cloudflare Workers) accelerated development Claude Code and AI assistance were force multipliers Modern AI APIs make complex features achievable ### What's next Immediate Next Steps (Post-Hackathon): Production Deployment Deploy to Cloudflare Workers for global reach Set up monitoring and analytics Implement rate limiting and scaling Production Deployment Deploy to Cloudflare Workers for global reach Set up monitoring and analytics Implement rate limiting and scaling Enhanced Agent Capabilities Add more specialized agents (travel, shopping, research) Improve agent selection algorithms Implement agent learning from user feedback Enhanced Agent Capabilities Add more specialized agents (travel, shopping, research) Improve agent selection algorithms Implement agent learning from user feedback Richer Visual Understanding Process full video streams (not just snapshots) Add object tracking across frames Implement scene change detection Richer Visual Understanding Process full video streams (not just snapshots) Add object tracking across frames Implement scene change detection Short-term Goals (1-3 months): MCP Server Development Build custom Model Context Protocol server Enable automation workflows Submit for Anthropic/MCP Best Automation Prize MCP Server Development Build custom Model Context Protocol server Enable automation workflows Submit for Anthropic/MCP Best Automation Prize Expanded Hardware Support Full Omi Glass integration with vision Support for other wearables (Apple Watch, Galaxy Ring) Multi-device synchronization Expanded Hardware Support Full Omi Glass integration with vision Support for other wearables (Apple Watch, Galaxy Ring) Multi-device synchronization Advanced Memory Features Semantic search across memories Automatic memory clustering and summarization Proactive context suggestions Advanced Memory Features Semantic search across memories Automatic memory clustering and summarization Proactive context suggestions User Personalization Learn user preferences over time Adapt response style to user needs Custom agent priorities per user User Personalization Learn user preferences over time Adapt response style to user needs Custom agent priorities per user Long-term Vision (6-12 months): Enterprise Features Team collaboration and shared context Meeting intelligence and action items CRM integration for sales teams Enterprise Features Team collaboration and shared context Meeting intelligence and action items CRM integration for sales teams Developer Platform Public API for third-party integrations Plugin system for custom agents Marketplace for agent templates Developer Platform Public API for third-party integrations Plugin system for custom agents Marketplace for agent templates Advanced AI Capabilities Predictive task suggestions Proactive problem-solving Multi-step workflow automation Advanced AI Capabilities Predictive task suggestions Proactive problem-solving Multi-step workflow automation Privacy & Security On-device processing options End-to-end encryption for sensitive data Granular privacy controls Privacy & Security On-device processing options End-to-end encryption for sensitive data Granular privacy controls Research Directions: Contextual AI Ethics Responsible capture and storage of personal context User control over AI decision-making Transparency in agent actions Contextual AI Ethics Responsible capture and storage of personal context User control over AI decision-making Transparency in agent actions Ambient Computing Invisible, always-available assistance Context-aware notification management Seamless cross-device experiences Ambient Computing Invisible, always-available assistance Context-aware notification management Seamless cross-device experiences Social Context Understanding Multi-person conversation tracking Social dynamics and group intentions Collaborative task management Social Context Understanding Multi-person conversation tracking Social dynamics and group intentions Collaborative task management Moonshot Ideas: Universal Personal AI Single AI that knows everything about your life Works across all devices and services Replaces dozens of specialized apps Universal Personal AI Single AI that knows everything about your life Works across all devices and services Replaces dozens of specialized apps Collective Intelligence Shared context across communities Collaborative problem-solving Distributed knowledge graphs Collective Intelligence Shared context across communities Collaborative problem-solving Distributed knowledge graphs Physical-Digital Fusion AR overlays with real-time AI insights Smart environment integration Seamless reality blending Physical-Digital Fusion AR overlays with real-time AI insights Smart environment integration Seamless reality blending Why UniCon Matters We're at an inflection point in human-AI interaction. AI is powerful, but it's blind to our reality. Wearables can capture our world, but lack intelligence. Digital services are smart, but disconnected from physical context. UniCon solves this by creating the missing universal context layer. The result isn't just another AI assistant - it's a fundamental shift in how humans and AI collaborate. It's AI that truly gets you, because it sees your world, understands your intentions, and acts with full context. This is the future of ambient computing. This is UniCon. Built with ❤️ at Cal Hacks 2025 Technologies: Omi Wearables, Reka.AI, Fetch.ai ASI:One, Supermemory, Groq, Cloudflare Workers Demo: [Link to video] Code: [GitHub repo] Try it: [Live demo link] ## README (from the GitHub repository) # UniCon 👀 > **An Intelligent Multimodal AI Companion System** Streamline interaction between Humans and AI in both digital and physical world with Open Source Wearable AI devices. Built at **Cal Hacks 2025** - October 25-26, 2025 ## Overview UniCon is a comprehensive AI-powered companion system that integrates with Omi wearable devices (DevKit 2 and Glass) to provide seamless audio transcription, universal memory storage, and intelligent agentic interactions. ## Features - **Real-time Audio Processing**: Receive and process audio streams from Omi wearables every 5 seconds - **AI Transcription**: High-quality speech-to-text using Groq Whisper-large-v3 - **Universal Memory**: Store all conversations and context in Supermemory for persistent access - **Dual Webhook Support**: Handle both raw audio bytes and pre-transcribed text - **Agentic Layer**: Fetch.ai ASI:One integration for intelligent AI agents (coming soon) - **Vision Processing**: Support for Omi Glass with vision capabilities (coming soon) - **Custom MCP Server**: Model Context Protocol automation (in development) ## Technical Stack - **Webhook Infrastructure**: Cloudflare Workers (serverless, globally distributed) - **Transcription**: Groq Whisper-large-v3 (state-of-the-art speech-to-text) - **Storage**: Supermemory API (universal memory and context management) - **Agentic Layer**: Fetch.ai ASI:One (intelligent AI agents) - **Hardware**: Omi DevKit 2 & Omi Glass (open-source wearables) - **Temporary Storage**: Cloudflare KV (audio chunks and transcriptions) ## Project Structure ``` never-be-alone/ ├── universal-context/ │ ├── webhook-server/ # Cloudflare Workers webhook server │ │ ├── src/ │ │ │ ├── handlers/ # Audio & transcription webhook handlers │ │ │ ├── services/ # Groq & Supermemory API clients │ │ │ ├── utils/ # Audio conversion utilities │ │ │ ├── types.ts # TypeScript type definitions │ │ │ └── index.ts # Main worker entry point │ │ ├── package.json │ │ ├── wrangler.toml │ │ ├── README.md # Detailed documentation │ │ ├── DEPLOYMENT.md # Deployment guide │ │ └── QUICK_START.md # Quick start guide │ ├── agent-layer/ # Fetch.ai agents (coming soon) │ ├── docs/ # Project documentation │ └── scripts/ # Utility scripts ├── .claude/ # Claude Code configuration └── README.md # This file ``` ## Quick Start ### Prerequisites - Node.js 18+ - Cloudflare account - Groq API key - Supermemory API key - Omi wearable device ### Setup 1. **Clone the repository** ```bash git clone https://github.com/Parth0248/never-be-alone.git cd never-be-alone ``` 2. **Install webhook server** ```bash cd universal-context/webhook-server npm install ``` 3. **Configure environment** ```bash cp .env.example .dev.vars # Edit .dev.vars with your API keys ``` 4. **Deploy to Cloudflare Workers** ```bash npm run deploy ``` 5. **Configure Omi device** Set webhook URLs in the Omi app: - Audio: `https://your-worker.workers.dev/webhook/audio?sample_rate=16000&uid=YOUR_UID` - Transcription: `https://your-worker.workers.dev/webhook/transcription?uid=YOUR_UID` For detailed instructions, see [`universal-context/webhook-server/QUICK_START.md`](universal-context/webhook-server/QUICK_START.md) ## Documentation - **Webhook Server**: [`universal-context/webhook-server/README.md`](universal-context/webhook-server/README.md) - **Deployment Guide**: [`universal-context/webhook-server/DEPLOYMENT.md`](universal-context/webhook-server/DEPLOYMENT.md) - **Quick Start**: [`universal-context/webhook-server/QUICK_START.md`](universal-context/webhook-server/QUICK_START.md) ## Architecture ``` ┌─────────────────────────────────────────────────────────────────┐ │ Omi Wearable Devices │ │ (DevKit 2 / Glass with Vision) │ └────────────┬─────────────────────────────┬────────────────────┘ │ │ │ Audio Stream (5s chunks) │ Pre-transcribed Text │ │ ▼ ▼ ┌────────────────────────┐ ┌────────────────────────┐ │ Audio Webhook │ │ Transcription Webhook │ │ /webhook/audio │ │ /webhook/transcription │ └────────┬───────────────┘ └───────┬────────────────┘ │ │ │ Convert PCM → WAV │ │ │ ▼ │ ┌────────────────────────┐ │ │ Groq Whisper-large-v3 │ │ │ (Transcription) │ │ └────────┬───────────────┘ │ │ │ └────────────┬───────────────┘ │ ▼ ┌────────────────────────┐ │ Supermemory API │ │ (Universal Storage) │ └────────────────────────┘ │ ▼ ┌────────────────────────┐ │ Fetch.ai ASI:One │ │ (Agentic Layer) │ └────────────────────────┘ ``` ## Roadmap ### Phase 1: Core Infrastructure - [x] Cloudflare Workers webhook server - [x] Audio format conversion (PCM → WAV) - [x] Groq Whisper-large-v3 integration - [x] Supermemory API integration - [x] Dual webhook support (audio + transcription) - [x] KV storage for temporary caching - [x] Comprehensive documentation ### Phase 2: Hardware Integration - [x] Omi DevKit 2 webhook integration - [x] Test with real Omi DevKit 2 device - [x] Omi Glass integration - [x] Vision processing for Glass ### Phase 3: Agentic Layer - [x] Fetch.ai ASI:One integration - [x] Build intelligent agents - [x] Context-aware responses - [x] Multi-agent orchestration ### Phase 4: MCP Automation - [x] Custom MCP server - [x] Automation workflows - [x] Tool integrations - [x] Submit for Best MCP Automation prize ## Cal Hacks 2025 - Sponsor Prizes This project is targeting the following sponsor prizes: 1. **Anthropic/MCP** - Best MCP Automation Prize - Custom MCP server for workflow automation 2. **Supermemory** - Best use of Supermemory - Universal storage for all transcriptions and context 3. **Fetch.ai** - Best use of AI Agents - Agentic layer using ASI:One platform 4. **Groq** - Best use of Groq API - Whisper-large-v3 for real-time transcription 5. **Cloudflare** - Best use of Workers - Serverless webhook infrastructure 6. **Omi/Based Hardware** - Best Hardware Integration - Deep integration with DevKit 2 and Glass ## Contributing Contributions are welcome! Please feel free to submit a Pull Request. ## License MIT License - See LICENSE file for details ## Team Built with ❤️ at Cal Hacks 2025 ## Acknowledgments - **Groq** for providing Whisper-large-v3 API - **Supermemory** for universal memory storage - **Cloudflare** for Workers platform - **Fetch.ai** for ASI:One agentic framework - **Omi** for open-source wearable AI devices - **Cal Hacks** for the amazing hackathon experience ## Contact - GitHub: https://github.com/Parth0248/never-be-alone - Issues: https://github.com/Parth0248/never-be-alone/issues ## Detected evidence (automated analysis) Indexed codebase: 81 recognized source files, 676 KB. - Flask (technology) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - TypeScript (language) — detected in the code - AI coding agent: Claude Code — evidence: commit authorship or trailers ## Codebase structure (from repository index) ### Files (102 of 102) ``` .env.example .gitignore agent-orchestrator/.dockerignore agent-orchestrator/.env.example agent-orchestrator/agents/__init__.py agent-orchestrator/agents/base_agent.py agent-orchestrator/agents/calendar_agent.py agent-orchestrator/agents/context_retrieval_agent_mcp.py agent-orchestrator/agents/context_retrieval_agent.py agent-orchestrator/agents/task_classifier.py agent-orchestrator/agentverse_integration.py agent-orchestrator/asi_one_agentic_client.py agent-orchestrator/asi_one_client.py agent-orchestrator/config.py agent-orchestrator/deploy.ps1 agent-orchestrator/deploy.sh agent-orchestrator/DEPLOYMENT.md agent-orchestrator/Dockerfile agent-orchestrator/INTEGRATION_STATUS.md agent-orchestrator/LOCAL_DEPLOYMENT.md agent-orchestrator/main.py agent-orchestrator/mcp_client.py agent-orchestrator/omi_client.py agent-orchestrator/omi_webhook_integration.py agent-orchestrator/OMI_WEBHOOK_SETUP.md agent-orchestrator/orchestrator.py agent-orchestrator/QUICK_START.md agent-orchestrator/README.md agent-orchestrator/reka_client.py agent-orchestrator/requirements.txt agent-orchestrator/run_local.ps1 agent-orchestrator/run_local.sh agent-orchestrator/set_env_vars.sh agent-orchestrator/setup.bat agent-orchestrator/setup.sh agent-orchestrator/spectacles_webhook.py agent-orchestrator/test_api_calls.py agent-orchestrator/test_asi_one_agentic.py agent-orchestrator/test_asi_one_api.py agent-orchestrator/test_asi_simple.py agent-orchestrator/test_end_to_end.py agent-orchestrator/test_omi_webhook.py agent-orchestrator/test_orchestrator.py agent-orchestrator/webhook_poller.py ASI_ONE_INTEGRATION_SUMMARY.md DEMO_GUIDE.md DEMO_SCENARIOS.md DEVPOST_SUBMISSION.md docs/ASI_ONE_AGENTIC_INTEGRATION.md docs/CLEANUP_SUMMARY.md docs/HOW_TO_VIEW_LOGS_AND_TRANSCRIPTIONS.md docs/OMI_API_TEST_RESULTS.md docs/OMI_TRANSCRIPTION_SETUP.md docs/QUICK_START_REKA.md docs/REKA_INTEGRATION_GUIDE.md docs/REPOSITORY_STRUCTURE.md docs/SETUP_API_KEYS.md docs/SPECTACLES_INTEGRATION_READY.md docs/UNICON_ARCHITECTURE_DIAGRAM.md docs/UPDATED_STRUCTURE.md END_TO_END_TEST_ANALYSIS.md inspect_post_data.py inspect_webhook_data.py planning/AGENT_ORCHESTRATION_ARCHITECTURE.md planning/AGENT_ORCHESTRATION_COMPLETE.md planning/INTEGRATION_OPTIONS_COMPARISON.md planning/PHASE2_PLAN.md planning/planning.txt planning/QUICK_START_AGENT_IMPLEMENTATION.md planning/ROADMAP_SUMMARY.md process_combined_omi_data.py process_existing_data.py process_with_asi_one.py README.md requirements.txt run_tests.py SLIDE_CONTENT.md src/__init__.py src/omi_client.py src/reka_client.py src/webhook_server.py start_server.py test_fetcher.py TEST_RESULTS.md test_webhook_demo.py universal-context/webhook-server/.env.example universal-context/webhook-server/.gitignore universal-context/webhook-server/.wrangler/state/v3/kv/b4130a8f406b41e08c088a3070159a1a/blobs/0f557090b43a8bb31b18e723c55b013d6d8f33cfa4053f82a6f831f2ff4449630000019a1aa86197 universal-context/webhook-server/.wrangler/state/v3/kv/b4130a8f406b41e08c088a3070159a1a/blobs/9a3f7fb3e786d60f3e103bbe9b45be42bcdfe228468393e74da24700e5017ce10000019a1aa90268 universal-context/webhook-server/.wrangler/state/v3/kv/b4130a8f406b41e08c088a3070159a1a/blobs/c2ec21b8d8143a51bacea366017077104e7c165e0b7ce98af915a863258777b70000019a1aa7079e universal-context/webhook-server/.wrangler/state/v3/kv/miniflare-KVNamespaceObject/75c6f6405602d7c1d6f9ae857efe6a200ef975cc91b3f3f48e95610aea8e10d8.sqlite universal-context/webhook-server/.wrangler/tmp/bundle-s82w7f/middleware-insertion-facade.js universal-context/webhook-server/.wrangler/tmp/bundle-s82w7f/middleware-loader.entry.ts universal-context/webhook-server/.wrangler/tmp/dev-hsOe3E/index.js universal-context/webhook-server/.wrangler/tmp/dev-hsOe3E/index.js.map universal-context/webhook-server/Dockerfile universal-context/webhook-server/main.py universal-context/webhook-server/README.md universal-context/webhook-server/requirements.txt universal-context/webhook-server/TASK_LOG.txt universal-context/webhook-server/WEBHOOK_ARCHITECTURE.md webhook_fetcher.py ``` ### Dependencies - agent-orchestrator/requirements.txt: aiohttp@>=3.9.0, cosmpy@>=0.9.0, flask@>=3.0.0, flask-cors@>=4.0.0, google-cloud-pubsub@>=2.18.0, google-cloud-storage@>=2.10.0, groq@>=0.4.0, gunicorn@>=21.2.0, httpx@>=0.25.0, orjson@>=3.9.0, pydantic@>=2.5.0, python-dateutil@>=2.8.2, python-dotenv@>=1.0.0, requests@>=2.31.0, structlog@>=23.2.0, uagents@>=0.12.0, urllib3@>=2.0.0 - requirements.txt: flask@==3.0.0, groq@==0.4.2, gunicorn@==21.2.0, Pillow@==12.0.0, pytest@==7.4.3, python-dotenv@==1.0.0, requests@==2.31.0 - universal-context/webhook-server/requirements.txt: Flask@==3.0.0, google-auth@==2.35.0, google-cloud-storage@==2.18.2, gunicorn@==22.0.0, python-dotenv@==1.0.1, Werkzeug@==3.0.1 ### Recent commits (newest first) - Update roadmap phases with completed tasks - Rename project in README.md - Initial commit with secrets removed and repository reorganization - Initial commit ## Key source files (fetched from GitHub, selected and truncated for size) ### DEMO_GUIDE.md ```markdown # Never-Be-Alone Demo Guide **Date**: October 26, 2025 **Status**: Ready for Demo --- ## Quick Start Summary Your **Never-Be-Alone** project is now fully set up and running locally! Here's what's ready: ### Server Status - **Local Webhook Server**: Running on `http://localhost:3000` - **Health Check**: PASS - **Audio Webhook**: PASS - **Services Configured**: Reka.AI, OMI, Supermemory, Groq ### API Keys Configured - ✓ Reka API (reka-flash model) - ✓ OMI App API (App ID + User ID) - ✓ Supermemory API - ✓ Groq API (Whisper-large-v3) --- ## For Your Demo Video ### What's Working 1. **Webhook Server** - Receives data from Omi devices 2. **Health Monitoring** - Real-time health checks 3. **Audio Processing** - Transcription webhook ready 4. **Image Processing** - Vision webhook ready 5. **Supermemory Integration** - Universal memory storage 6. **Reka.AI Integration** - Multimodal AI processing ### Demo Flow 1. **Show the Server Running** ``` Server: http://localhost:3000 Endpoints: - /health (health check) - /webhook/audio (audio transcripts) - /webhook/images (images & context) - /webhook/combined (combined data) ``` 2. **Test with Omi Device** - Configure Omi app to point to your webhook - Speak into Omi device - Show real-time transcription - Demonstrate AI response 3. **Show Supermemory Storage** - All conversations stored - Universal context maintained - Search through memories ### Architecture Highlights ``` Omi Device (Audio/Vision) ↓ Webhook Server (Flask) ↓ ┌──────┴──────┐ ↓ ↓ Reka.AI Supermemory (Process) (Store) ↓ OMI App (Response) ``` ### Key Features to Mention 1. **Real-time Processing**: 5-second audio chunks 2. **Multimodal**: Audio + Vision support 3. **Universal Memory**: Persistent context via Supermemory 4. **Serverless Ready**: Deployed to Cloudflare Workers 5. **Open Source**: Built with Omi DevKit 2 --- ## Testing Commands ### Health Check ```bash curl http://localhost:3000/health ``` ### Test Audio Webhook ```bash python test_webhook_demo.py ``` ### View Server Logs The Flask development server shows all requests in real-time in your terminal. --- ## Sponsor Prizes Targeting 1. **Supermemory** - Universal storage for all transcriptions 2. **Groq** - Whisper-large-v3 for transcription 3. **Cloudflare** - Workers for webhook infrastructure 4. **Omi/Based Hardware** - Deep integration with DevKit 2 5. **Reka.AI** - Multimodal processing --- ## Project Stats - **Language**: Python + TypeScript - **Framework**: Flask (local), Cloudflare Workers (production) - **APIs**: 5 different services integrated - **Lines of Code**: ~1500+ - **Hackathon**: Cal Hacks 2025 - **Build Time**: 1 day --- ## Next Steps After Demo 1. Deploy to Cloudflare Workers for global reach 2. Add more AI agents via Fetch.ai 3. Build MCP automation server 4. Expand vision processing capabilities 5. Add more Omi device integrations --- ## Contac [truncated — 167 more characters] ``` ### SLIDE_CONTENT.md ```markdown # Unicon Presentation - Slide Content ## Slide 1: Title Slide ### Main Title: **Unicon** ### Subtitle: Universal Context for Seamless AI Interaction ### Tagline: *Bridging the gap between what you see, what you say, and what AI understands* --- ## Slide 2: The Vision - Venn Diagram ### Headline: **The Convergence Problem** ### Main Content: **We live at the intersection of four worlds:** 🧑 **Human** - Our thoughts, intentions, and experiences - Natural conversations - Unstructured observations - Context-rich interactions 🤖 **AI** - Intelligent systems that assist us - Language understanding - Pattern recognition - Automated reasoning 🌍 **Physical World** - What we see and experience - Real-time environments - Visual context - Spatial awareness 💻 **Digital World** - Our connected reality - Cloud services - Smart devices - Digital assistants ### The Challenge: *Today, these worlds operate in silos. Your AI assistant doesn't see what you see. Your smart devices don't understand your intentions. Your digital tools lack physical context.* ### The Solution: **Unicon creates a universal context layer that connects all four worlds seamlessly.** --- ### Alternative Version (More Technical): ### Headline: **Four Disconnected Worlds** **The Problem:** - **Human ↔ AI**: Lost context in translation - **Physical ↔ Digital**: No awareness of real-world state - **AI ↔ Physical**: Cannot perceive the environment - **Human ↔ Digital**: Fragmented interactions **At the intersection lies the opportunity:** > *Universal Context - where human intent meets AI capability, grounded in both physical reality and digital infrastructure* **Unicon bridges all four domains to enable truly intelligent assistance.** --- ### Alternative Version (Story-Driven): ### Headline: **Imagine a World Where...** **Your AI assistant:** - 👀 **Sees** what you see (Physical World) - 🗣️ **Hears** what you say (Human World) - 🧠 **Understands** your intent (AI World) - 🔗 **Acts** across all your devices (Digital World) **This is not imagination. This is Unicon.** *A universal context layer that unifies human experience with AI intelligence, bridging the physical and digital divide.* --- ## Slide 3 (Bonus): How Unicon Works ### The Unicon Architecture: **Input Layer:** - 🎙️ Omi Wearables: Continuous audio + visual capture - 📍 Context: Location, time, environment **Processing Layer:** - 🧠 Reka.AI: Multimodal understanding - 🤝 ASI:One: Agentic orchestration - 💾 Supermemory: Universal memory **Output Layer:** - 📱 Notifications & Reminders - 🔄 Automated Actions - 💬 Intelligent Responses **The Result:** AI that truly understands your world --- ## Recommended Approach: ### For Slide 2, I recommend the **Story-Driven** version because: 1. **Engaging**: Starts with "Imagine..." - draws audience in 2. **Clear**: Each bullet is action-oriented 3. **Memorable**: Simple icons + concise statements 4. **Builds anticipation**: "This is not imagination. This i [truncated — 1809 more characters] ``` ### requirements.txt ``` # Core dependencies flask==3.0.0 requests==2.31.0 python-dotenv==1.0.0 # Image processing Pillow==12.0.0 # Audio transcription groq==0.4.2 # Optional: For production deployment gunicorn==21.2.0 # Testing pytest==7.4.3 ``` ### agent-orchestrator/requirements.txt ``` # Core Framework uagents>=0.12.0 cosmpy>=0.9.0 # Web Framework flask>=3.0.0 flask-cors>=4.0.0 gunicorn>=21.2.0 # HTTP Requests requests>=2.31.0 httpx>=0.25.0 urllib3>=2.0.0 # Environment Variables python-dotenv>=1.0.0 # Data Processing pydantic>=2.5.0 # Async Support aiohttp>=3.9.0 # Google Cloud google-cloud-storage>=2.10.0 google-cloud-pubsub>=2.18.0 # Groq API groq>=0.4.0 # Date/Time python-dateutil>=2.8.2 # Logging structlog>=23.2.0 # JSON orjson>=3.9.0 ``` ### agent-orchestrator/Dockerfile ``` # Use Python 3.11 slim image FROM python:3.11-slim # Set working directory WORKDIR /app # Install system dependencies RUN apt-get update && apt-get install -y \ gcc \ g++ \ && rm -rf /var/lib/apt/lists/* # Copy requirements first for better caching COPY requirements.txt . # Install Python dependencies RUN pip install --no-cache-dir -r requirements.txt # Copy application code selectively to avoid permission issues COPY *.py ./ COPY agents ./agents # Set environment variables ENV PYTHONUNBUFFERED=1 ENV PORT=8080 # Expose port EXPOSE 8080 # Health check HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \ CMD python -c "import requests; requests.get('http://localhost:8080/health')" # Run with gunicorn for production CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 300 --worker-class gthread main:app ``` ### universal-context/webhook-server/requirements.txt ``` # Web Framework Flask==3.0.0 Werkzeug==3.0.1 # Google Cloud Storage google-cloud-storage==2.18.2 google-auth==2.35.0 # Utilities python-dotenv==1.0.1 # WSGI Server for production gunicorn==22.0.0 ``` ### universal-context/webhook-server/Dockerfile ``` # Multi-stage build for Python audio streaming server FROM python:3.12-slim AS builder # Set working directory WORKDIR /app # Install system dependencies RUN apt-get update && \ apt-get install -y --no-install-recommends \ gcc \ && rm -rf /var/lib/apt/lists/* # Copy requirements first for better caching COPY requirements.txt . # Install Python dependencies RUN pip install --no-cache-dir --user -r requirements.txt # Production stage FROM python:3.12-slim AS runner # Set working directory WORKDIR /app # Copy Python dependencies from builder COPY --from=builder /root/.local /root/.local # Copy application code COPY main.py . # Update PATH to include user-installed packages ENV PATH=/root/.local/bin:$PATH # Expose port EXPOSE 8080 # Set environment variables ENV PYTHONUNBUFFERED=1 ENV PORT=8080 # Health check HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \ CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8080/health')" # Run the application with gunicorn CMD ["gunicorn", "--bind", "0.0.0.0:8080", "--workers", "2", "--threads", "4", "--timeout", "120", "--access-logfile", "-", "--error-logfile", "-", "main:app"] ``` ### agent-orchestrator/main.py ```python """Main Flask API server for agent orchestrator.""" from flask import Flask, request, jsonify from flask_cors import CORS import asyncio import logging from orchestrator import get_orchestrator from config import Config # Configure logging logging.basicConfig( level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' ) logger = logging.getLogger(__name__) # Initialize Flask app app = Flask(__name__) CORS(app) # Get orchestrator instance orchestrator = get_orchestrator() @app.route('/health', methods=['GET']) def health_check(): """Health check endpoint.""" return jsonify({ "status": "healthy", "service": "agent-orchestrator", "environment": Config.ENVIRONMENT }) @app.route('/orchestrate', methods=['POST']) def orchestrate(): """Main endpoint to process transcriptions through agent orchestration. Request body: { "transcription": "text from audio", "context": { "uid": "user123", "timestamp": "2025-10-25T14:30:00Z", "source": "omi_device" } } Returns: Agent orchestration results """ try: data = request.get_json() if not data: return jsonify({ "success": False, "error": "No data provided" }), 400 transcription = data.get("transcription", "") if not transcription: return jsonify({ "success": False, "error": "No transcription provided" }), 400 context = data.get("context", {}) # Process transcription asynchronously result = asyncio.run( orchestrator.process_transcription(transcription, context) ) if result.get("success"): return jsonify(result), 200 else: return jsonify(result), 500 except Exception as e: logger.error(f"Error in /orchestrate endpoint: {e}", exc_info=True) return jsonify({ "success": False, "error": str(e) }), 500 @app.route('/agents', methods=['GET']) def list_agents(): """List all available agents and their capabilities.""" try: status = orchestrator.get_agent_status() return jsonify(status), 200 except Exception as e: logger.error(f"Error in /agents endpoint: {e}") return jsonify({ "success": False, "error": str(e) }), 500 @app.route('/test-agent', methods=['POST']) def test_agent(): """Test a specific agent action. Request body: { "agent_type": "calendar_agent", "action": "create_reminder", "params": { "text": "Call mom", "time": "tomorrow" } } Returns: Test result """ try: data = request.get_json() if not data: return jsonify({ "success": False, "error": "No data provided" }), 400 agent_type = data.get("agent_type") action = data.get("action") params = data.get("params", {}) if not agent_type or not action: return jsonify({ "success": False, "error": "agent_type and action are required" }), 400 # Test agent result = asyncio.run( orchestrator.test_agent(agent_type, action, params) ) if result.get("success"): return jsonify(result), 200 else: return jsonify(result), 400 except Exception as e: logger.error(f"Error in /test-agent endpoint: {e}") return jsonify({ "success": False, "error": str(e) }), 500 @app.route('/capabilities', methods=['GET']) def get_capabilities(): """Get detailed capabilities of a specific agent. Query params: agent_type: Type of agent (optional, returns all if not specified) Returns: Agent capabilities """ try: agent_type = request.args.get("agent_type") if agent_type: agent = orchestrator.agents.get(agent_type) if not agent: return jsonify({ "success": False, "error": f"Agent not found: {agent_type}" }), 404 capabilities = agent.get_capabilities() return jsonify({ "success": True, "agent_type": agent_type, "capabilities": capabilities }), 200 else: # Return capabilities for all agents all_capabilities = {} for name, agent in orchestrator.agents.items(): all_capabilities[name] = agent.get_capabilities() return jsonify({ "success": True, "agents": all_capabilities }), 200 except Exception as e: logger.error(f"Error in /capabilities endpoint: {e}") return jsonify({ "success": False, "error": str(e) }), 500 @app.route('/process-batch', methods=['POST']) def process_batch(): """Process multiple transcriptions in batch. Request body: { "transcriptions": [ { "transcription": "text 1", "context": {...} }, { "transcription": "text 2", "context": {...} } ] } Returns: Batch processing results """ try: data = request.get_json() if not data or "transcriptions" not in data: return jsonify({ "success": False, "error": "No transcriptions provided" }), 400 transcriptions = data.get("transcriptions", []) if not isinstance(transcriptions, list): return jsonify({ "success": [truncated — 1570 more characters] ``` ### universal-context/webhook-server/main.py ```python """ Omi Audio Streaming Server Receives audio bytes from Omi wearable devices and saves them as WAV files in Google Cloud Storage. Environment Variables Required: - GOOGLE_APPLICATION_CREDENTIALS_JSON: Base64-encoded GCP service account credentials - GCS_BUCKET_NAME: Name of the GCS bucket to store audio files Reference: https://docs.omi.me/doc/developer/AudioStreaming Based on: https://github.com/mdmohsin7/omi-audio-streaming """ import os import base64 import struct import tempfile import logging from datetime import datetime from pathlib import Path from flask import Flask, request, jsonify from google.cloud import storage from google.oauth2 import service_account import json from dotenv import load_dotenv import requests # Load environment variables from .dev.vars file load_dotenv('.dev.vars') # Configure logging logging.basicConfig( level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s' ) logger = logging.getLogger(__name__) # Audio constants for Omi DevKit 2 NUM_CHANNELS = 1 # Mono audio SAMPLE_RATE = 16000 # 16 kHz BITS_PER_SAMPLE = 16 # 16-bit audio # Initialize Flask app app = Flask(__name__) def create_wav_header(data_length: int) -> bytes: """ Create a WAV file header for the given audio data length. WAV Format Specification: - Bytes 0-3: "RIFF" chunk descriptor - Bytes 4-7: File size - 8 - Bytes 8-11: "WAVE" format - Bytes 12-15: "fmt " subchunk - Bytes 16-19: Subchunk size (16 for PCM) - Bytes 20-21: Audio format (1 for PCM) - Bytes 22-23: Number of channels - Bytes 24-27: Sample rate - Bytes 28-31: Byte rate - Bytes 32-33: Block align - Bytes 34-35: Bits per sample - Bytes 36-39: "data" subchunk - Bytes 40-43: Data size Args: data_length: Length of the audio data in bytes Returns: 44-byte WAV header as bytes """ byte_rate = SAMPLE_RATE * NUM_CHANNELS * BITS_PER_SAMPLE // 8 block_align = NUM_CHANNELS * BITS_PER_SAMPLE // 8 header = bytearray(44) # RIFF chunk descriptor header[0:4] = b'RIFF' header[4:8] = struct.pack(' bool: """ Upload a file to Google Cloud Storage. Args: bucket_name: Name of the GCS bucket filename: Name to give the file in GCS file_path: Local path to the file to upload Returns: bool: True if upload successful, False otherwise """ try: client = get_gcs_client() bucket = client.bucket(bucket_name) blob = bucket.blob(filename) # Upload file with content type blob.upload_from_filename(file_path, content_type='audio/wav') logger.info(f'Successfully uploaded {filename} to GCS bucket {bucket_name}') return True except Exception as e: logger.error(f'Failed to upload to GCS: {e}') return False @app.route('/audio', methods=['POST']) def handle_audio(): """ Handle POST requests with audio bytes from Omi device. Query Parameters: - uid: User identifier (required) - sample_rate: Audio sample rate in Hz (optional, default: 16000) Request Body: Raw audio bytes (application/octet-stream) Returns: JSON response with success status and filename """ # Get query parameters uid = request.args.get('uid') sample_rate_param = request.args.get('sample_rate', '16000') logger.info(f'Received audio request from uid: {uid}, sample_rate: {sample_rate_param}') if not uid: return jsonify({'error': 'uid parameter is required'}), 400 # Get environment variables bucket_name = os.getenv('GCS_BUCKET_NAME') if not bucket_name: logger.error('GCS_BUCKET_NAME environment variable is not set') return jsonify({'error': 'Server configuration error: GCS_BUCKET_NAME not set'}), 500 try: # Read audio bytes from request body audio_data = request.get_data() if not audio_data: return jsonify({'error': 'No audio data received'}), 400 logger.info(f'Received {len(audio_da [truncated — 6629 more characters] ``` ### universal-context/webhook-server/.wrangler/tmp/dev-hsOe3E/index.js ```javascript var __defProp = Object.defineProperty; var __name = (target, value) => __defProp(target, "name", { value, configurable: true }); // src/utils/audioConverter.ts var AudioConverter = class { static { __name(this, "AudioConverter"); } /** * Convert raw PCM audio bytes to WAV format * @param rawAudio - Raw audio bytes (PCM 16-bit) * @param sampleRate - Sample rate in Hz (default: 16000 for Omi DevKit2) * @param numChannels - Number of audio channels (1 = mono, 2 = stereo) * @param bitsPerSample - Bits per sample (default: 16) * @returns WAV audio as ArrayBuffer */ static rawToWAV(rawAudio, sampleRate = 16e3, numChannels = 1, bitsPerSample = 16) { const audioData = rawAudio instanceof ArrayBuffer ? new Uint8Array(rawAudio) : rawAudio; const dataSize = audioData.length; const byteRate = sampleRate * numChannels * bitsPerSample / 8; const blockAlign = numChannels * bitsPerSample / 8; const wavSize = 44 + dataSize; const wavBuffer = new ArrayBuffer(wavSize); const view = new DataView(wavBuffer); this.writeString(view, 0, "RIFF"); view.setUint32(4, wavSize - 8, true); this.writeString(view, 8, "WAVE"); this.writeString(view, 12, "fmt "); view.setUint32(16, 16, true); view.setUint16(20, 1, true); view.setUint16(22, numChannels, true); view.setUint32(24, sampleRate, true); view.setUint32(28, byteRate, true); view.setUint16(32, blockAlign, true); view.setUint16(34, bitsPerSample, true); this.writeString(view, 36, "data"); view.setUint32(40, dataSize, true); const wavData = new Uint8Array(wavBuffer); wavData.set(audioData, 44); return wavBuffer; } /** * Convert multiple audio chunks to a single WAV file * @param chunks - Array of audio chunks * @param sampleRate - Sample rate in Hz * @returns Combined WAV audio as ArrayBuffer */ static combineChunksToWAV(chunks, sampleRate = 16e3) { const totalSize = chunks.reduce((sum, chunk) => { const size = chunk instanceof ArrayBuffer ? chunk.byteLength : chunk.length; return sum + size; }, 0); const combined = new Uint8Array(totalSize); let offset = 0; for (const chunk of chunks) { const data = chunk instanceof ArrayBuffer ? new Uint8Array(chunk) : chunk; combined.set(data, offset); offset += data.length; } return this.rawToWAV(combined, sampleRate); } /** * Parse WAV header from audio data * @param wavData - WAV audio data * @returns Parsed WAV header information */ static parseWAVHeader(wavData) { const view = new DataView(wavData); return { riff: this.readString(view, 0, 4), fileSize: view.getUint32(4, true) + 8, wave: this.readString(view, 8, 4), fmt: this.readString(view, 12, 4), fmtSize: view.getUint32(16, true), audioFormat: view.getUint16(20, true), numChannels: view.getUint16(22, true), sampleRate: view.getUint32(24, true), byteRate: view.getUint32(28, true), blockAlign: view.getUint16(32, true), bitsPerSample: view.getUint16(34, true), data: this.readString(view, 36, 4), dataSize: view.getUint32(40, true) }; } /** * Validate if audio data is a valid WAV file * @param audioData - Audio data to validate * @returns True if valid WAV file */ static isValidWAV(audioData) { if (audioData.byteLength < 44) return false; const view = new DataView(audioData); const riff = this.readString(view, 0, 4); const wave = this.readString(view, 8, 4); return riff === "RIFF" && wave === "WAVE"; } /** * Get audio duration in seconds * @param audioData - WAV audio data * @returns Duration in seconds */ static getAudioDuration(audioData) { if (!this.isValidWAV(audioData)) { throw new Error("Invalid WAV file"); } const header = this.parseWAVHeader(audioData); return header.dataSize / header.byteRate; } /** * Write string to DataView */ static writeString(view, offset, str) { for (let i = 0; i < str.length; i++) { view.setUint8(offset + i, str.charCodeAt(i)); } } /** * Read string from DataView */ static readString(view, offset, length) { let str = ""; for (let i = 0; i < length; i++) { str += String.fromCharCode(view.getUint8(offset + i)); } return str; } /** * Convert base64 string to ArrayBuffer * @param base64 - Base64 encoded string * @returns ArrayBuffer */ static base64ToArrayBuffer(base64) { const binaryString = atob(base64); const bytes = new Uint8Array(binaryString.length); for (let i = 0; i < binaryString.length; i++) { bytes[i] = binaryString.charCodeAt(i); } return bytes.buffer; } /** * Convert ArrayBuffer to base64 string * @param buffer - ArrayBuffer to convert * @returns Base64 encoded string */ static arrayBufferToBase64(buffer) { const bytes = new Uint8Array(buffer); let binary = ""; for (let i = 0; i < bytes.length; i++) { binary += String.fromCharCode(bytes[i]); } return btoa(binary); } }; // src/services/groqClient.ts var GroqClient = class { static { __name(this, "GroqClient"); } apiKey; baseUrl = "https://api.groq.com/openai/v1"; constructor(apiKey) { this.apiKey = apiKey; } /** * Transcribe audio using Groq Whisper-large-v3 * @param audioData - Audio data as ArrayBuffer (WAV format recommended) * @param options - Transcription options * @returns Transcription result */ async transcribe(audioData, options) { try { const formData = new FormData(); const audioBlob = new Blob([audioData], { type: "audio/wav" }); formData.append("file", audioBlob, "audio.wav"); formData.append("model", "whisper-large-v3"); if (options?.language) { formData.append("language", options.language); } if (options?.prompt) { formData.append("prompt", [truncated — 39596 more characters] ``` [73 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]