Project Info
Inspiration
Given the substantial lack of accessibility resources and compliance in modern website UI—over 96% of websites fail to meet accessibility requirements—we wanted to develop an AI-driven copilot agent, Navi, for human browsing, in addition to creating an easier web infrastructure for AI models to interact with. 🌐🤖
What it does
In short, Navi makes web browsing easier for human users, with accessible voice navigation 🎤, intuitive browsing with user intent prediction 🔮, instant feedback to questions 📝, and personalized further reading suggestions 📚. In terms of serving AI models, Navi extracts web pages more efficiently compared to traditional VLMs. ⚡
How we built it
To integrate visual context, we captured screenshots of the user's navigation, which were then processed by visual language models (VLMs) to inform Navi's AI. This enabled Navi to understand and interpret website layouts in real time 🖼️. Simultaneously, we harnessed Context, a robust context fetching engine, to analyze user interactions, predicting browsing intent and tailoring recommendations. In the background, Scrapybara autonomously suggests related websites 🌐, while Groq converts audio commands into prompts for OpenAI's 4-o LLM, ensuring rapid and accurate responses ⚙️. Additionally, we leveraged Mistral for fast, simplified website renderings that highlight key information, making navigation even easier for the user 🏃♂️.
Challenges we ran into
Balancing rapid AI responses with efficient web page extraction was just one layer of our challenge 🏔️. A deeper complexity arose from integrating multiple AI components into a unified, accessible system. Each component—whether it was the Context engine, VLMs, or voice processing with Groq—had its own processing speeds, data formats, and dependencies. Merging these disparate systems required careful orchestration to ensure they communicated seamlessly in real time ⏱️. We designed robust interfaces and error-handling protocols to bridge differences in performance and data structure ⚖️. This meant synchronizing outputs from slow VLMs with faster processing modules, ensuring that delays in one area wouldn’t disrupt the overall user experience.
Accomplishments we're proud of
We're proud to have built Navi as a truly integrated, AI-driven copilot that enhances web accessibility and usability 🌟. Navi not only improves browsing efficiency through context-aware predictions and voice navigation but also serves as a high-performance data extractor for AI models. This project sets a new benchmark for creating accessible digital experiences and bridging the gap between human and AI interaction 🌉.
What we learned
Throughout this project, we gained invaluable insights into prompt orchestration, context fetching, and the nuances of designing for accessibility. We learned how critical it is to balance technical sophistication with user-centric design, ensuring that advanced features translate into real-world usability for diverse audiences 🎓💡.
What's next
Looking ahead, we plan to expand Navi's compatibility with more websites and enhance its voice and intent recognition capabilities 🔧. Our next steps include optimizing the system for even faster and more accurate responses ⏩, refining personalized recommendations, and exploring additional accessibility features to further empower users with disabilities ♿.
🚀 Navi – Your AI-Powered Browsing Copilot
Navigate smarter. Navi streamlines your browsing experience with immediate assistance, voice navigation, and intelligent website recommendations based on predicted user intent.
🏆 Won TreeHack 2025: Best Multimodal App Challenge by Groq
🛠️ What It Does
Navi revolutionizes browsing by offering:
✅ Voice Navigation – Control your browser with hands-free commands
✅ User Intent Prediction – Understands where you’re headed before you do
✅ Instant Q&A Feedback – Get real-time answers without searching manually
✅ Personalized Reading Suggestions – Discover new, relevant content automatically
✅ Optimized AI Web Extraction – Faster and more accurate page processing for AI models
⚙️ How We Built It
Navi integrates multiple AI-powered components: • Visual Context Processing – Screenshots of user navigation are analyzed using Visual Language Models (VLMs) to interpret webpage layouts in real time. • Context Fetching Engine – The Context framework predicts user browsing intent and generates tailored recommendations. • Web Autonomy with Scrapybara – Automatically suggests related websites. • AI-Powered Voice Commands – Groq processes user speech into structured prompts for OpenAI’s GPT-4o, ensuring accurate responses. • Faster Page Rendering – Mistral extracts and simplifies web content for better readability and navigation.
💻 Tech Stack • Frontend: Next.js, TypeScript • AI Models: Groq, OpenAI GPT-4o, Mistral, Perplexity, Gemini • Backend & Contextual Processing: Context, Windsurf • Voice & Web Extraction: ElevenLabs
🤝 Contributing
Want to contribute to Navi? Here’s how you can help:
1. Fork the repository
2. Create a new branch (feature-branch)
3. Commit your changes
4. Push the branch and submit a PR
📜 License
This project is licensed under the MIT License – see the LICENSE file for details.
Analysis
View
Metric
- 5
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- CSSIn code
- ExpressIn code
- FirebaseIn code
- JavaScriptIn code
- Next.jsIn code
- OpenAIIn code
- ReactIn code
- Tailwind CSSIn code
- TypeScriptIn code
- Google GeminiClaimed
- Mistral AIClaimed
9 of 11 appear in the indexed code. 2 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
75 KB
Source files
27
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
danieltyx/navi-browser-agent
48 files · 848 KB · @ 5c0ef8b
Structure
Interface
14 files · 29%Screens, components and styles rendered to the user.
API & routing
6 files · 13%Request entry points: routes, handlers and controllers.
Application logic
5 files · 10%Domain rules, services and shared utilities.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- TypeScript92%
- JavaScript4%
- Markdown3%
- CSS1%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
package.json
npm · 29- @firebase/storage
- @types/dotenv
- axios
- cheerio
- dotenv
- elevenlabs
- express
- firebase
- form-data
- framer-motion
- groq-sdk
- html2canvas
- install
- multer
- next
- openai
- puppeteer
- react
- +11 more
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.