Project Info
Inspiration
Ever had an incredibly busy day and can't remember all the people you've met, or all the tasks you've got planned for the future? We're humans! Our memory is imperfect and that's difficult, but it doesn't need to be: so we built I-IMO! I-IMO is a conversational AI agent designed with the busiest people in mind. I-IMO scans both visual input (webcam) and aural input (OMI devkit2) throughout your day to organize and summarize profiles for everyone you meet, while maintaining task lists and tracking important events so you never forget anything important from your meetings. N.B: We originally wanted to do this app on any smart glasses, but unfortunately due to supply constraints we couldn't get one. In the future, the plan is to adapt our project to run off of in-built webcams in devices like the Omi Glass. What is I-IMO? I-IMO is a personal conversational intelligence assistant that is an always-on second-in-command: Captures conversations through real-time audio transcription Recognizes faces via webcam to identify meeting participants Builds personal profiles automatically from conversation context Extracts action items and creates task lists from discussions Maintains conversation history with semantic search capabilities Never lets you forget important details about the people you meet Generates context to infer insightful predictions/aids for upcoming events in your calendar. Our Tech Stack React + Typescript + Vite, FastAPI + Custom YOLO11n model for face detection, Tailwind + Shad/CN UI, Convex Backend, Express.js Backend for NLP, Groq-hosted gpt-whisper for audio transcribing, Digital Ocean AI Gradient for GPU intense task, ChromaDB for vectorised queries and semantic search, OpenAI Text Embedding. Architecture Challenges We Ran Into connection issues 😡, streaming raw audio data from Omi (with low latency), high accuracy face classification that remains consistent (tagging id and tag name with it), deduplication of misspellings of names and mistranscriptions, nature of audio streaming: 5 second chunks clipping certain transcribed words, syncing vectorised ChromaDB with traditional Convex backend, minimising non-deterministic nature of profile summaries and task predictions, Important Lessons Learnt how to integrate a lot of different technologies and choose/route between the best models for unique workloads, Designed ways to sync between multiple backends and frameworks, especially with fundamental infra differences between vectorised semantic-search db's and traditional backends, Balanced the benefits lightweight, fast inference models with the lower completion accuracy, Developed a lot of ad-hoc teamwork skills while working on the same project with time-pressure (minimising merge-conflicts, documenting pull-requests using tools like CodeRabbit, keeping commit tree clean...) What's Next Deep integration with glasses (Meta Raybans, Omi Glass, Snapchat Spectacles), Potentially link 3rd party MCP providers, Connect with social media platforms for deeper context / automated actions based on the user's day, Time-sensitive summaries, Mobile, watch, and native versions of the dashboard.
I-IMO - Intelligent IRL Meeting Organizer
If you are applovin judges - please check out this section
The readme for applovin's challange is located at applovin folder. You can also click here
Inspiration
I usually a yapper and talk with a lot of people, but at the end of the day, you look back yourself and can't remember all of them or sometimes you just remember the faces but not what their infomation. Maybe I'm a bad person or I'm just having a bad memory ;D. So we built I-IMO!
A conversational AI agent designed with the busiest people in mind. I-IMO scans both visual input (webcam) and oral input (OMI devkit2) throughout your day to organize and summarize profiles for everyone you meet, while maintaining task lists and tracking important events so you never forget anything important from your meetings.
- We want to do this app on any smart glasses, but we could not get one. Snapchat spectacles are all reserved and Omi glasses - turn out they are giving out omi devkit (voice only) device.
What It Does
I-IMO is your personal meeting intelligence assistant that:
- Captures conversations through real-time audio transcription
- Recognizes faces via webcam to identify meeting participants
- Builds personal profiles automatically from conversation context
- Extracts action items and creates task lists from discussions
- Maintains conversation history with semantic search capabilities
- Never lets you forget important details about the people you meet
How We Built It
- React + Typescript + Vite
- FastAPI + Custom YOLO model for face detection
- Tailwind + Shad/CN UI
- Convex Backend
- Express Backend that process voice
- Groq for audio transcribing
- Digital Ocean AI Gradient for GPU intense task
- ChromaDB for VectorDB and semantic search
- OpenAI Text Embedding
Architecture
Challenges we ran into
- connection issues
- high accuracy face classification (tagging id and tag name with it).
- 5 seconds chunk audio cut of certains words transcripting.
What we learned
- We use a lot of technologies and learn how to use it together.
- Implements muitple backend with low latency.
Whats next
- Deep intergration with glasses.
- Link local profile with linkedin / github and more social medias.
- When you wake up, remind you the whole day ahead.
Analysis
View
Metric
- 38
- 32
- 9
- 1
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- CSSIn code
- ExpressIn code
- FastAPIIn code
- HTMLIn code
- JavaScriptIn code
- OpenAIIn code
- PythonIn code
- ReactIn code
- Tailwind CSSIn code
- TypeScriptIn code
10 of 10 appear in the indexed code.
AI coding agents
- CursorConfig
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
616 KB
Source files
97
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
satvikprasad/calhacks25-imo
123 files · 1019 KB · @ d60844c
Structure
Interface
57 files · 46%Screens, components and styles rendered to the user.
Application logic
28 files · 23%Domain rules, services and shared utilities.
+1 more
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- TypeScript51%
- YAML37%
- Python7%
- Markdown4%
- CSS1%
- JavaScript1%
- Other (1)0%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
frontend/package.json
npm · 69- @auth0/auth0-react
- @chroma-core/default-embed
- @heroicons/react
- @hookform/resolvers
- @radix-ui/react-accordion
- @radix-ui/react-alert-dialog
- @radix-ui/react-aspect-ratio
- @radix-ui/react-avatar
- @radix-ui/react-checkbox
- @radix-ui/react-collapsible
- @radix-ui/react-context-menu
- @radix-ui/react-dialog
- @radix-ui/react-dropdown-menu
- @radix-ui/react-hover-card
- @radix-ui/react-label
- @radix-ui/react-menubar
- @radix-ui/react-navigation-menu
- @radix-ui/react-popover
- +51 more
face_recognition/requirements.txt
pypi · 60- albucore
- albumentations
- annotated-doc
- annotated-types
- anyio
- certifi
- charset-normalizer
- click
- coloredlogs
- contourpy
- cycler
- Cython
- easydict
- fastapi
- flatbuffers
- fonttools
- h11
- humanfriendly
- +42 more
package.json
npm · 12- @chroma-core/default-embed
- @chroma-core/openai
- @heroicons/react
- chromadb
- cors
- dotenv
- express
- openai
- +4 more
applovin/requirements.txt
pypi · 3- duckdb
- pandas
- psutil
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
Feature verification
Action item / task extractionVerified
Extracts action items and creates task lists from discussions
Claimed on readmehigh confidencebackend/main.ts:362— /omi/tasks prompts an LLM over stored transcriptions to produce a task list with due datesfrontend/convex/task.ts:5— createTasks mutation persists the generated tasks to the tasks table
Automatic profile building from conversationsVerified
Builds personal profiles automatically from conversation context
Claimed on readmehigh confidencebackend/main.ts:222— /omi/profiles prompts an LLM to summarise stored transcriptions into per-person profilesfrontend/convex/profile.ts:66— upsertProfiles mutation creates or patches a person's record with the generated conversationSummary
Conversational chat assistant (ask questions about your day)Verified
Conversational AI agent you can query about people and events
Claimed on readmehigh confidencebackend/main.ts:306— /omi/prompt answers a free-text prompt using stored transcriptions as contextfrontend/src/pages/pages-sc/index.tsx:99— handleSendMessage sends chat input to the /omi/prompt endpoint and renders the response
Convex backend for structured dataVerified
Convex Backend
Claimed on Devposthigh confidencefrontend/convex/schema.ts:1— Defines persons, face_embs, face_media, and tasks tables used throughout the app
Deduplication of misspelled names / mistranscriptionsVerified
Deduplication of misspellings of names and mistranscriptions
Claimed on Devpostmedium confidencefrontend/convex/http.ts:128— checkForDuplicate compares new person name/face embedding against existing persons using name-substring and cosine-similarity checks before creating a new recordbackend/main.ts:251— The profile-generation prompt explicitly instructs the LLM to infer true names despite mistranscriptions and repeated misspellings
Digital Ocean AI Gradient for GPU-intensive inferenceVerified
Digital Ocean AI Gradient for GPU intense task
Claimed on Devposthigh confidencebackend/main.ts:346— runInference posts to https://inference.do-ai.run/v1/chat/completions using DIGITAL_OCEAN_API_KEY, and is used for the profiles, tasks, prompt and meeting-prep endpoints
Face recognition via webcamVerified
Recognizes faces via webcam to identify meeting participants
Claimed on readmehigh confidencefrontend/src/components/CameraComponent.tsx:68— Webcam screenshots are captured every few seconds and POSTed to the face detection serviceface_recognition/main.py:200— /detect endpoint runs InsightFace on the uploaded image, compares the embedding to existing Convex embeddings, and upserts a new person if unmatched
OMI devkit2 audio ingestionVerified
Streams raw audio data from the OMI devkit2 device to the backend
Claimed on Devposthigh confidencebackend/main.ts:56— pcmToWav converts raw PCM chunks (as sent by OMI) into a WAV file for transcriptionbackend/main.ts:81— app.post(/omi/audio) accepts octet-stream audio uploads keyed by uid and sample_rate
React + TypeScript + Vite + shadcn/Tailwind dashboardVerified
React + Typescript + Vite, Tailwind + Shad/CN UI
Claimed on Devposthigh confidencefrontend/src/pages/pages-sc/index.tsx:1— Dashboard page built with React/TSX and shadcn-style UI components (Card, Button, Avatar, Alert)
Real-time audio transcription (Groq Whisper)Verified
Captures conversations through real-time audio transcription using Groq-hosted whisper
Claimed on Devposthigh confidencebackend/main.ts:130— client.audio.transcriptions.create is called with model whisper-large-v3-turbo against the Groq API base URLbackend/main.ts:81— /omi/audio endpoint receives PCM audio chunks from the OMI device, converts them to WAV, and transcribes them
Semantic search over conversation history (ChromaDB)Verified
Maintains conversation history with semantic search capabilities using ChromaDB and OpenAI embeddings
Claimed on Devposthigh confidencebackend/main.ts:163— A Chroma collection is created with an OpenAIEmbeddingFunction (text-embedding-3-small) and each transcription is upserted into it
Meeting prep / predictive context for upcoming calendar eventsCode-supported
Generates context to infer insightful predictions/aids for upcoming events in your calendar
Claimed on Devpostmedium confidencebackend/main.ts:446— /omi/meeting-prep generates prep bullet points for a given contact/meeting from stored transcriptionsfrontend/src/pages/pages-sc/index.tsx:50— The meetings list consumed by the UI is a hardcoded array, not a real calendar integration, so the 'calendar' data feeding this feature is mocked
Custom YOLO11n face detection modelClaimed only
FastAPI + Custom YOLO11n model for face detection
Claimed on Devposthigh confidence
An AI agent derived these features from the project’s Devpost page and readme, then searched the code for each one. Verified features are backed by cited code; claimed-only features had no supporting code, which is not by itself proof a feature is missing.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.