Project Info
This project did not submit a demo video on Devpost.
Inspiration
More than half of Americans reported feeling lonely this year. Moreso, everyone needs someone to talk to, yet finding a listening ear when you need it most isn’t always easy. It’s difficult to find someone who truly listens, remembers, and cares, no matter the time of day or topic.
What it does
Akira is your true companion, a fully conversational ai powered by a multi-agent memory architecture you can personalize both visually and functionally. Key features include: Memory-Powered: Learns and remembers what matters. Memory-Powered: Learns and remembers what matters. Multi-Modal: fully conversational, analyzes photos, and fetches live web data on demand. Multi-Modal: fully conversational, analyzes photos, and fetches live web data on demand. Emotion-Aware: Adapts itself to how you feel. Emotion-Aware: Adapts itself to how you feel. Friendly UI: Engaging avatar you can style and interact with. Friendly UI: Engaging avatar you can style and interact with.
How we built it
Backend Memory Agent Deployed a two-agent architecture on Letta Cloud: Low-Latency Agent uses a lightweight model to deliver instant responses. Context Agent leverages a more powerful model to retrieve and feed relevant context to the low-latency agent. Memory Agent Deployed a two-agent architecture on Letta Cloud: Low-Latency Agent uses a lightweight model to deliver instant responses. Context Agent leverages a more powerful model to retrieve and feed relevant context to the low-latency agent. Multi-Modal Pipeline Voice: Vapi Cloud’s STT → custom Letta LLM → TTS workflow for seamless spoken conversations. Images: Next.js API routes send uploads to Claude-sonnet-4 for image-to-text processing, then inject captions into the Vapi conversation stream. Web Search: Letta’s tool use fetches live web data on demand and augments the dialogue. Multi-Modal Pipeline Voice: Vapi Cloud’s STT → custom Letta LLM → TTS workflow for seamless spoken conversations. Images: Next.js API routes send uploads to Claude-sonnet-4 for image-to-text processing, then inject captions into the Vapi conversation stream. Web Search: Letta’s tool use fetches live web data on demand and augments the dialogue. Emotion-Aware Dialogue Tuned Letta agents and Vapi configurations to produce natural, empathetic flow—mirroring real human engagement. Emotion-Aware Dialogue Tuned Letta agents and Vapi configurations to produce natural, empathetic flow—mirroring real human engagement. Auth & Data Secure user authentication and database management ensure controlled access, encrypted storage, and efficient handling of conversation and profile data. Auth & Data Secure user authentication and database management ensure controlled access, encrypted storage, and efficient handling of conversation and profile data. Frontend Framework & Styling Built with Next.js, TypeScript, and Tailwind CSS for a fast, responsive, and modern UI. Framework & Styling Built with Next.js, TypeScript, and Tailwind CSS for a fast, responsive, and modern UI. 3D Avatar & Customization Three.js renders an interactive avatar you can style—change cosmetics, expressions, and animations—and engage with through gestures and reactions in real time. 3D Avatar & Customization Three.js renders an interactive avatar you can style—change cosmetics, expressions, and animations—and engage with through gestures and reactions in real time.
Challenges we ran into
Emotion & Speech Tuning Fine-tuning the Letta memory agents and Vapi’s pipeline to produce natural, empathetic speech took extensive iteration. Emotion & Speech Tuning Fine-tuning the Letta memory agents and Vapi’s pipeline to produce natural, empathetic speech took extensive iteration. Integration Complexity Orchestrating Vapi Cloud and Letta Cloud for seamless multimodal functionality (voice, text, image, web search) required careful design and execution across multiple services and frameworks. Integration Complexity Orchestrating Vapi Cloud and Letta Cloud for seamless multimodal functionality (voice, text, image, web search) required careful design and execution across multiple services and frameworks.
Accomplishments we're proud of
Deployed a fully functional multimodal, memory-enabled AI companion in under 24 hours. Seamlessly combined Vapi Cloud’s STT→Letta LLM→TTS pipeline with Letta Cloud’s two-agent memory architecture for real-time, context-rich conversations. Built a friendly, interactive UI and a fully customizable 3D avatar using Three.js, Next.js, TypeScript, and Tailwind CSS—bringing Akira to life.
What we learned
The power of a two-agent memory system for smooth, engaging conversations by balancing low-latency responses with deep contextual recall. Orchestrating Vapi Cloud’s STT→LLM→TTS pipeline alongside Letta Cloud’s memory agents revealed best practices for multi-cloud service integration and error handling. Iteratively tuning emotion and speech parameters underscored how small prosody adjustments can dramatically improve perceived empathy and naturalness. Building multi-modal support (voice, text, image, web search) showed us the importance of designing flexible data flows and fallbacks for each modality. Empowering users with avatar customization highlighted how personalization drives deeper connection and sustained engagement.
What's next
Multi-Language Support Expand beyond English to enable truly global companionship in the user’s native language or dialect. Multi-Language Support Expand beyond English to enable truly global companionship in the user’s native language or dialect. Voice-Call Integration Allow Akira to “call in” and check on you via phone or integrate directly with VoIP for hands-free conversations. Voice-Call Integration Allow Akira to “call in” and check on you via phone or integrate directly with VoIP for hands-free conversations. Long-Term Memory Enhancements Introduce “memory pruning” and “highlight reels” so Akira can surface your most important moments and learn over months or years. Long-Term Memory Enhancements Introduce “memory pruning” and “highlight reels” so Akira can surface your most important moments and learn over months or years. Third-Party Integrations Plug into calendars, music services (Spotify, Apple Music), fitness trackers, and smart-home devices to make Akira an even more useful companion. Third-Party Integrations Plug into calendars, music services (Spotify, Apple Music), fitness trackers, and smart-home devices to make Akira an even more useful companion. More Avatar Configuration Offer finer-grained customization options—hair styles, outfits, expressions, and dynamic animations—so users can craft a unique companion. More Avatar Configuration Offer finer-grained customization options—hair styles, outfits, expressions, and dynamic animations—so users can craft a unique companion. Immersive Three.js World Build a virtual environment where your avatar can explore, interact, and host mini-experiences, turning chats into immersive encounters. Immersive Three.js World Build a virtual environment where your avatar can explore, interact, and host mini-experiences, turning chats into immersive encounters.
This is a Next.js project bootstrapped with create-next-app.
Getting Started
First, run the development server:
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
Open http://localhost:3000 with your browser to see the result.
You can start editing the page by modifying pages/index.tsx. The page auto-updates as you edit the file.
API routes can be accessed on http://localhost:3000/api/hello. This endpoint can be edited in pages/api/hello.ts.
The pages/api directory is mapped to /api/*. Files in this directory are treated as API routes instead of React pages.
This project uses next/font to automatically optimize and load Inter, a custom Google Font.
Learn More
To learn more about Next.js, take a look at the following resources:
- Next.js Documentation - learn about Next.js features and API.
- Learn Next.js - an interactive Next.js tutorial.
You can check out the Next.js GitHub repository - your feedback and contributions are welcome!
Deploy on Vercel
The easiest way to deploy your Next.js app is to use the Vercel Platform from the creators of Next.js.
Check out our Next.js deployment documentation for more details.
Analysis
View
Metric
- 26
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- AnthropicIn code
- CSSIn code
- JavaScriptIn code
- Next.jsIn code
- ReactIn code
- SupabaseIn code
- Tailwind CSSIn code
- TypeScriptIn code
8 of 8 appear in the indexed code.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
91 KB
Source files
52
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
maxakiralee/akira
82 files · 9.0 MB · @ 102d39f
Structure
Interface
21 files · 26%Screens, components and styles rendered to the user.
API & routing
3 files · 4%Request entry points: routes, handlers and controllers.
Application logic
35 files · 43%Domain rules, services and shared utilities.
Data & schema
3 files · 4%Schema definitions, migrations and data access.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- TypeScript74%
- JavaScript24%
- Markdown2%
- CSS0%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
package.json
npm · 29- @anthropic-ai/sdk
- @letta-ai/letta-client
- @radix-ui/react-slot
- @react-three/drei
- @react-three/fiber
- @supabase/auth-helpers-nextjs
- @supabase/supabase-js
- @vapi-ai/web
- class-variance-authority
- clsx
- fuse.js
- lucide-react
- next
- react
- react-dom
- tailwind-merge
- tailwindcss-animate
- three
- +11 more
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.