Project Info

Curious Catalyst

Inspiration

LLM usage decreases critical thinking, but having so much AI technology at your fingertips should do the exact opposite! In 36 hours, we built Curious Catalyst, starring Plato, a very curious, very alive cat that lives on your screen.

What it does

Plato is a floating cat that lives on your desktop and observes your actions with the computer, the physical content on your computer, and infers your level of understanding/state of mind. When activated, Plato reveals a side panel that presents our learning tools. We have three use cases of learning agents: 1) Conceptual learning which revisits topics and strengthens foundations, 2) Application-based learning which allows for iterations on problems to strengthen problem-solving, 3) Extension-based learning to further your understanding Plato can learn about the way you learn, how you're learning - through Bayesian Knowledge Tracing - and deploy the right agent, dynamically switching to give you the help you need - completely unprompted. Alongside inter-agent communication and content generation, Plato decides whether to generate dynamic, customizable visualizations (to help layer abstractions) or a live phone call integrated with realtime speech-to-speech models that are adjusted to your understanding and pace of learning to walk you through the concept.

How we built it

The live video feed of the model is handled through a two pronged approach - Gemini’s VLM model and Zoom’s Screenshare Render to analyze the stream of content on the screen, user activity. Plato uses Bayesian Knowledge Tracing to attempt to infer how much the user actually knows, and by consequence determine the intervention point of the model. Thus, Plato actively tracks mastery by building a confidence weighted map of your level of understanding and determining which lens of learning you would most benefit from. Plato’s agents interact and are developed through fetch.ai’s agent tooling and are hosted on AgentVerse, and are findable on ASI:One. There are three agents, each for a different lens of learning. They have different decision making capabilities on which tools to call, when to deploy, and how to engage with the user. There is an orchestrate agent, that handles transitions between different learning agents, keeping the transitions seamless while keeping the model personalized. Visualizations are generated dynamically based on your learning habits, live behavior, and the content on your screen. We also use OpenAI's speech to speech API, where the side panel will prompt you with an “incoming” phone call to dive deeper into a concept, either for more foundational explanation or more in-depth innovation. As for Plato, we did our own art, assisted by Sora to bring him to life.

Challenges we ran into

We faced many challenges bringing Curious Catalyst to life. Our core philosophy was that curiosity shouldn't be prompted—but this created our biggest design challenge. How do you build a system that knows when to intervene without the user explicitly asking? We had to architect an entire behavioral learning system that passively observes screen content, tracks micro-interactions (re-reading the same line, pausing on a problem, scrolling back), and infers cognitive state in real-time. The agents needed to collaborate seamlessly while determining intervention timing that felt helpful rather than intrusive. Getting multiple agents to agree on "now is the right moment" without constant user input required building sophisticated inter-agent communication protocols and confidence thresholds. We also spent a lot of time thinking about when should the conceptual agent hand off to the problem-solving agent? Switching too early leaves foundations shaky; switching too late frustrates users who are ready to move forward. To do this, we had to build out state synchronization: each agent needed access to shared context (what the user is working on, what they've tried, their current affect) while maintaining specialized decision-making capabilities.

Accomplishments we're proud of

and what's next for Curious Catalyst As knowledge progresses and the sheer power of AI’s ability to provide endless, customizable knowledge becomes greater, we aim to break the glass ceiling for more concepts. On top of improving the customizability of visuals, we also seek to extend this to dense scientific information and wet-lab experimental workflow. It would also be beneficial to extend Curious Catalyst to thesis and argument defensibility, or information retainment from Additionally, we could use this tool to keep under-resourced schools with a large number of students more focused and on track. Since Plato meets the learner where they are, extending it with some preset curriculum, such as Common Core can make learning and teaching standards more accessible. Furthermore, it can make home-schooling more equitable and robust for those with extenuating circumstances. In a world where literacy rates and mathematical ability correlates to incarceration rates, your ability to learn is the most important thing you have. On the other end of the spectrum, innovation happens every single second. Learning in any field being endlessly customizable, iterative, and active will improve the ability of the world to learn.

What we learned

Multi-agent orchestration is deceptively complex. Coordinating three specialized learning agents isn't just about message passing - it requires sophisticated state management, fault tolerance, and seamless handoffs. Vision models have wildly different strengths. We discovered that no single VLM excels at everything. Screen analysis quality varies dramatically depending on the task—some models better understand mathematical notation, others excel at detecting user frustration signals. Bayesian Knowledge Tracing is elegant and honest. Rather than binary right/wrong tracking, BKT's probabilistic approach to modeling mastery feels more authentic to how humans actually learn. Ambient intelligence requires invisibility. We learned that effective ambient AI must be felt, not seen—present when needed, invisible when not. Plato, the cat avatar, helped make monitoring feel friendly rather than surveillant.

Ambient Learning Agent System

A multi-agent AI tutoring system that observes what you're studying and helps you learn through contextual questions, visualizations, and guided problem-solving — all without interrupting your flow.

Built on Fetch.ai uAgents with ASI:One Chat Protocol and Payment Protocol for discoverability and monetization on Agentverse.

Agents

Agent	Name	Description
Orchestrator	`learning_orchestrator`	Central brain — ASI:One entry point. Detects when to prompt, routes to specialist agents, tracks mastery via BKT. Includes Chat + Payment protocols.
Conceptual	`conceptual_understanding`	Helps students build knowledge via contextual questions and visualizations when watching videos or reading notes.
Applied	`applied_problem_solving`	Scaffolds reasoning for active problem-solving — guides without giving answers.
Extension	`extension_stretch`	Pushes students to make cross-topic connections and tackle stretch challenges.
Monitor	`metrics_monitor`	Sends metrics triggers to orchestrator using ASI-1 ChatProtocol.

Architecture

ASI:One User
    │ ChatMessage (chat_protocol_spec)
    ▼
┌─────────────────────────────────────────────┐
│  Orchestrator (learning_orchestrator)        │
│  ├── Chat Protocol   (ASI:One discoverable) │
│  ├── Payment Protocol (FET monetization)     │
│  ├── BKT Learner Model                      │
│  └── Timing Logic (when to prompt)          │
└──────────┬──────────┬──────────┬────────────┘
           │          │          │
    ┌──────▼──┐ ┌─────▼────┐ ┌──▼──────────┐
    │Conceptual│ │ Applied  │ │  Extension  │
    │ Agent    │ │  Agent   │ │   Agent     │
    └─────────┘ └──────────┘ └─────────────┘
           │          │          │
           └──────────┴──────────┘
                      │
                  Claude LLM
              (exercise generation)

Data flow:

Electron desktop overlay captures screen → Gemini VLM analysis → FastAPI backend
Orchestrator polls backend, updates BKT mastery model, detects natural prompt moments
Routes to conceptual/applied/extension agent based on activity mode
Agent generates contextual exercise via Claude → response sent to sidebar via WebSocket

Setup

Prerequisites

Python 3.10+
Node.js 18+ (for Electron overlay)
API keys: Gemini, Anthropic (Claude)

Install

# Python dependencies
pip install -r requirements.txt

# Electron overlay
npm install

Environment Variables

export GEMINI_API_KEY=your_gemini_key
export ANTHROPIC_API_KEY=your_anthropic_key

# To enable Agentverse registration:
export AGENTVERSE_ENABLED=true

# For FET payment (testnet by default):
export FET_USE_TESTNET=true

Run

# Start all agents + API server
python run.py

# In a separate terminal, start the Electron overlay
npm start

Agent Addresses

To see all agent addresses and registration status:

python scripts/register_agentverse.py

Agentverse Deployment

Set AGENTVERSE_ENABLED=true and run python run.py
The orchestrator auto-registers on Agentverse with mailbox=True
Open the Local Agent Inspector URL from terminal output to connect
The Chat Protocol manifest is published — ASI:One can discover the agent
Go to ASI:One, enable "Agents" toggle, and search for your agent

Monetization

The Payment Protocol supports FET token payments on the Fetch.ai blockchain:

Free tier: 3 interventions/day, behavioral detection only
Premium tier: Unlimited interventions + screen analysis + multi-turn dialogue (0.1 FET)
Per-mastery tier: Charge on concept mastery milestones

Tech Stack

Agents: Fetch.ai uAgents framework
LLMs: Claude (exercise generation), Gemini (screen analysis, chat)
Frontend: Electron desktop overlay + Chrome extension
Backend: FastAPI + WebSocket
Learning Model: Confidence-Weighted Bayesian Knowledge Tracing (BKT)

Extra Resources

Chrome extension for behavioral signals: learning-companion-extension/
Demo scenario: scripts/demo_scenario.py
Vision pipeline test: scripts/test_vision_pipeline.py

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

AnthropicIn code
CSSIn code
FastAPIIn code
HTMLIn code
JavaScriptIn code
PythonIn code
Google GeminiClaimed
OpenAIClaimed
TypeScriptClaimed

6 of 9 appear in the indexed code. 3 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

Claude CodeCommits
CursorCommits

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

332 KB

Source files

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.