# Project export: AgentDex
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: UC Berkeley AI Hackathon 2026
- Tagline: A dev's agent queries AgentDex about a topic. AgentDex in parallel researches semantically-adjacent topics, and has them ready as machine-readable interfaces before the agent ever asks.
- Devpost: https://devpost.com/software/agentdex
- GitHub: https://github.com/codebyemily/AgentDex
- Video: https://www.youtube.com/embed/KHnceddxIsQ?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 4 GitHub contributor(s) — emily (7 commits), Claude Sonnet 4.6 (5 commits), trista-chen-29 (5 commits), ra2y (1 commits)
## Devpost submission (written by the team)
### Inspiration
We were inspired by YC's Summer 2026 Requests For Startups, "Software For Agents" blog by Aaron Epstein: Agents need a completely different foundation. Instead of visual interfaces like forms, buttons, and dashboards, they need machine-readable interfaces like APIs, MCPs, and CLIs. Agents also need thorough documentation, to enable them to discover, sign up for, and instantly start using new tools programmatically, without needing a human in the loop. Here is the problem: When an agent does multi-step research, it works serially: think → search topic A → read → think → search topic B → read. Every related sub-topic only gets discovered, browsed, and structured after the agent's reasoning arrives there — even when the adjacency was predictable from the start. Research about atoms is reliably going to touch electrons, protons, and basic chemistry; research about a company's filings is reliably going to touch its competitors and sector. That predictability is currently wasted — nothing acts on it until the agent stumbles into the next topic itself. Solution AgentDex sits beside the dev's agent as a concurrent research assistant, not just a request/response API: The agent queries AgentDex about a topic. AgentDex immediately starts the normal pipeline for that exact topic — browse (Browserbase), structure (Claude + redis vector store), generate an MCP server — and returns it as fast as possible. At the same time, AgentDex computes which other topics sit in close vector proximity to the one just asked about, and kicks off the same pipeline for the top few candidates in parallel, without blocking the agent's current answer. If the agent's next question lands on one of those candidates, the interface is already built — the agent gets a warm, structured answer instead of triggering a cold pipeline run. The pitch in one line: your agent never has to wait for the obvious next question — by the time it asks, AgentDex already bet on it and did the work in the background.
### How we built it
Browserbase does the actual legwork: fetching and rendering real pages for every research worker, whether it's answering the question that was asked or betting on the one that's coming. Claude wears two hats: it turns messy page content into something an agent can actually call, and it makes the one judgment call this whole idea lives or dies on, deciding which similar topics are genuine next-question bets and which are just vector-space lookalikes. Skip that step and you've built an expensive nearest-neighbor toy, not a research assistant. Redis, via RedisVL, holds the growing map of everything AgentDex has ever researched and how it all relates: the proximity graph that makes the bet possible in the first place. MCP turns every researched topic, asked-for or speculative, into a live, callable interface, with a thin REST mirror riding along for anything that still wants a plain HTTP endpoint. Sentry watches both the path you asked for and every parallel bet running quietly in the background. Concurrency means more ways for things to break, so we wanted to see the moment something did. Every speculative bet runs on a leash: a cap on how many candidates get researched, a confidence bar a candidate has to clear before we spend real research budget on it, and a timeout so a slow bet never blocks the answer you actually asked for.
### Challenges we ran into
The connection to Redis Cloud was a problem that made us use Redis locally instead Determining our architecture to utilize the sponsors and solve YC's problem statement
### Accomplishments we're proud of
Watching a question get answered by work that finished before the question was asked: a real, timestamped, no-cache warm hit. Seeing AgentDex make an actual difference in deep research time. Considerations Good fit: domains with a known, fairly stable concept graph — science/technical research (atoms → electrons → chemistry), company research (a firm → its named competitors → its sector), documentation research (an API endpoint → its sibling endpoints). Bad fit: domains where "related" doesn't mean "likely to be asked next" — vector proximity is a similarity measure, not an intent predictor, and conflating the two is the main way this idea fails quietly. Pay a small embedding cost on every query to avoid a large crawl cost on every paraphrased hit.
### What's next
Letting real usage train the relevance filter instead of a static prompt, tracking which bets actually get asked and which get wasted, so the system gets sharper the more it's used instead of staying frozen at hackathon quality. Giving AgentDex memory across sessions, so the bet it makes today is informed by every bet it's made before.
## README (from the GitHub repository)
# AgentDex
A **speculative pre-fetching research pipeline** built on [Fetch.ai uAgents](https://docs.fetch.ai/uAgents/) and [RedisVL](https://github.com/RedisVentures/redisvl). AgentDex predicts which topics a researcher will ask about next and pre-warms them in the background — so the second query returns instantly from cache instead of waiting for a full crawl.
The analogy is CPU branch prediction or browser link prefetching: AgentDex bets on your next question before you ask it.
## Demo
```
dev_agent queries "atoms" → cold run → Wikipedia crawl + Claude classification
speculative worker pre-warms: electrons, protons, periodic table
dev_agent queries "electrons" → *** WARM HIT *** → served instantly from Redis
```
## Architecture
| Agent | File | Role |
|---|---|---|
| `orchestrator` | `agents/orchestrator_agent.py` | Checks Redis cache; dispatches primary + speculative workers; two-stage speculation (vector KNN → Claude filter) |
| `primary_worker` | `agents/primary_worker.py` | Crawls Wikipedia via Browserbase/Playwright; classifies with Claude; populates cache |
| `speculative_worker` | `agents/speculative_worker.py` | Pre-fetches predicted follow-up topics concurrently; discarded on timeout |
| `dev_agent` | `agents/dev_agent.py` | Demo client — sends `atoms` → `electrons` to show cold run then warm hit |
### Speculation pipeline
1. **Exact cache check** — Redis hash lookup by topic key; return immediately if warm
2. **Semantic cache check** — embed query, KNN search; if nearest neighbor distance < `SEMANTIC_HIT_THRESHOLD` (default `0.15`), serve from cache without crawling
3. **Primary worker dispatch** — crawl + classify the requested topic
4. **Speculative expansion** — Redis KNN returns similar topics → Claude filters to genuine "next question" candidates → dispatch speculative workers for each
## Setup
```bash
# Install Python dependencies
pip install -r requirements.txt
# Install Playwright browsers
playwright install chromium
# Copy and fill in environment variables
cp .env.example .env
# Run the pipeline
python main.py
```
## Environment variables
| Variable | Required | Purpose |
|---|---|---|
| `ANTHROPIC_API_KEY` | Yes | Claude API calls (classification + speculation filter) |
| `BROWSERBASE_API_KEY` | Yes | Browserbase session for Wikipedia crawling |
| `BROWSERBASE_PROJECT_ID` | Yes | Browserbase project |
| `REDIS_URL` | Yes | Redis connection — use `redis://localhost:6379` for local Redis Stack |
| `SENTRY_DSN` | No | Error reporting in `speculative_worker` |
| `SPECULATION_BUDGET` | No | Max speculative topics per query (default `3`) |
| `SPECULATIVE_TIMEOUT_SECS` | No | Timeout for each speculative fetch (default `30`) |
| `SEMANTIC_HIT_THRESHOLD` | No | Cosine distance for a semantic cache hit (default `0.15`) |
## Local Redis Stack (required for vector search)
Redis Cloud free tier uses TLS 1.0/1.1 which is incompatible with Python 3.13 / OpenSSL 3.x. Run Redis Stack locally instead:
```bash
docker run -d --name redis-stack -p 6379:6379 redis/redis-stack-server:latest
```
Then set `REDIS_URL=redis://localhost:6379` in `.env`.
## Shared modules
| Module | Purpose |
|---|---|
| `shared/redis_client.py` | RedisVL vector index — `embed`, `set_warm`, `get_warm`, `search_nearest_with_scores`, `all_topics` |
| `shared/pipeline.py` | I/O — `crawl_topic` (Browserbase), `classify_and_structure` (Claude), `filter_speculative_candidates` (Claude), `research_topic` |
| `shared/cache.py` | Re-exports `get_warm`, `set_warm`, `all_topics` from `redis_client` |
| `shared/messages.py` | uAgents message models: `TopicQuery`, `ResearchRequest`, `ResearchResult` |
| `shared/config.py` | Seeds, address slots, and tuning constants |
## MCP server — external agent access
`mcp_server.py` exposes the AgentDex pipeline as an [MCP](https://modelcontextprotocol.io) server. Any MCP-compatible agent can use it without joining the internal uAgents Bureau.
### How it fits in
```
┌─────────────────── Bureau (main.py) ───────────────────┐
│ dev_agent ──TopicQuery──► orchestrator ──► workers │ ← internal, uAgents protocol
└────────────────────────────────────────────────────────┘
External developer agent
└── mcp_server.py (stdio) ──► shared/pipeline.py ← external, MCP protocol
└──► shared/redis_client.py
```
The Bureau and the MCP server share the same Redis backend, so topics pre-warmed by the speculative pipeline are immediately available to external agents via `get_cached_topic` and `search_similar_topics`.
### Tools
| Tool | Parameters | What it does |
|---|---|---|
| `research_topic` | `topic: str` | Crawls Wikipedia, classifies with Claude, stores in Redis cache. Returns JSON with `summary`, `key_facts`, `related_concepts`, `mcp_tools`. |
| `get_cached_topic` | `topic: str` | Direct Redis lookup. Returns cached JSON or `{}` if not warm. |
| `search_similar_topics` | `query: str`, `k: int = 5` | Semantic KNN search. Returns `[{topic, distance}]` sorted by cosine distance. |
| `list_warm_topics` | — | Lists all topics currently in the Redis cache. |
### Claude Desktop / Claude Code
Add to `claude_desktop_config.json` (macOS: `~/Library/Application Support/Claude/`):
```json
{
"mcpServers": {
"agentdex": {
"command": "python",
"args": ["/path/to/AgentDex/mcp_server.py"],
"env": {
"ANTHROPIC_API_KEY": "...",
"BROWSERBASE_API_KEY": "...",
"BROWSERBASE_PROJECT_ID": "...",
"REDIS_URL": "redis://localhost:6379"
}
}
}
}
```
Claude will then call `research_topic`, `search_similar_topics`, etc. as native tools.
### Python agent (MCP client)
```python
from mcp import ClientSession
from mcp.client.stdio import stdio_client, StdioServerParameters
params = StdioServerParameters(
command="python",
args=["/path/to/AgentDex/mcp_server.py"],
env={"ANTHROPIC_API_KEY": "...", "REDIS_URL": "redis://localhost:6379", ...},
)
async with stdio_client(params) as (read, write):
async with ClientSession(read, write) as session:
await session.initialize()
# Cold research — crawls Wikipedia + caches result
result = await session.call_tool("research_topic", {"topic": "black holes"})
# Warm lookup — instant if already cached
cached = await session.call_tool("get_cached_topic", {"topic": "black holes"})
# Semantic search across everything in the cache
similar = await session.call_tool("search_similar_topics", {"query": "event horizon", "k": 5})
```
### Direct Python import (same repo)
If the developer's agent runs in the same Python environment, MCP is optional:
```python
from shared.pipeline import research_topic
from shared.redis_client import get_warm, search_nearest_with_scores
result = await research_topic("quantum entanglement")
similar = await search_nearest_with_scores("spooky action", k=5)
```
## Validation
```bash
python scripts/test_person_a.py
```
Tests the full Redis pipeline: upsert → warm lookup → KNN search → Claude speculation filter → import compatibility. Requires local Redis Stack and `ANTHROPIC_API_KEY`.
## Detected evidence (automated analysis)
Indexed codebase: 22 recognized source files, 92 KB.
- Anthropic (technology) — detected in the code
- FastAPI (technology) — detected in the code
- Python (language) — detected in the code
- Redis (technology) — detected in the code
- AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers
## Codebase structure (from repository index)
### Files (24 of 24)
```
.gitignore
agents/__init__.py
agents/dev_agent.py
agents/orchestrator_agent.py
agents/primary_worker.py
agents/speculative_worker.py
CLAUDE.md
INTEGRATION.md
main.py
mcp_server.py
panel/__init__.py
panel/demo_replay.py
panel/server.py
README.md
requirements.txt
scripts/test_person_a.py
shared/__init__.py
shared/cache.py
shared/config.py
shared/demo_events.py
shared/messages.py
shared/observability.py
shared/pipeline.py
shared/redis_client.py
```
### Dependencies
- requirements.txt: anthropic@>=0.30.0, browserbase@>=0.3.0, fastapi@>=0.100.0, mcp@>=1.0.0, numpy@>=1.24.0, playwright@>=1.44.0, python-dotenv@>=1.0.0, redis@>=5.0.0, redisvl@>=0.3.0, sentence-transformers@>=2.7.0, sentry-sdk@>=2.0.0, uagents@>=0.13.0, uvicorn@>=0.23.0
### Recent commits (newest first)
- Merge fix-workflow into main: restore speculative_worker, full pipeline, all fixes
- Fix missing await on set_warm in speculative handler
- Update README.md
- Removed speculative_worker wiring; adjusted print statements for clarity.
- Wrap per-agent sentry_sdk.init in try/except to handle malformed DSN gracefully
- Wrap per-agent sentry_sdk.init in try/except to handle malformed DSN gracefully
- Suppress verbose INFO logs from httpx/sentence-transformers/redisvl for cleaner demo output
- Suppress verbose INFO logs from httpx/sentence-transformers/redisvl for cleaner demo output
- logs
- Degrade gracefully when Redis is unavailable
- Fix port conflict: Bureau on 8001, panel server on 8000
- Add fastapi and uvicorn to requirements.txt for panel server
- updated mcp server connection instructions
- added mcp server gateway
- Add README and semantic cache hit feature
- browserbase to redis connection
- browserbase to redis connection
- create live panel
- Wire speculative_worker into Bureau and fix routing
- Add Redis vector index + speculative pre-fetch pipeline (Person A)
## Key source files (fetched from GitHub, selected and truncated for size)
### CLAUDE.md
```markdown
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Running the project
```bash
# Install Python dependencies
pip install -r requirements.txt
# Install Node dependencies (Anthropic SDK + dotenv — used by test.js)
npm install
# Copy and populate environment variables
cp .env.example .env
# Run the multi-agent system
python main.py
```
## Required environment variables
| Variable | Purpose |
|---|---|
| `ANTHROPIC_API_KEY` | Claude API calls in `shared/pipeline.py` |
| `BROWSERBASE_API_KEY` / `BROWSERBASE_PROJECT_ID` | Browserbase session for Wikipedia crawling |
| `REDIS_URL` | (Optional) Cache backend — currently unused, cache is in-process |
| `SENTRY_DSN` | (Optional) Error reporting in `speculative_worker` |
Optional tuning via env:
- `SPECULATION_BUDGET` (default `3`) — how many speculative topics to prefetch per query
- `SPECULATIVE_TIMEOUT_SECS` (default `30`) — max time the speculative worker waits per topic
## Architecture
AgentDex is a **speculative pre-fetching research pipeline** built on [Fetch.ai uAgents](https://docs.fetch.ai/uAgents/). Four agents run concurrently inside a single `Bureau`; addresses are wired in `main.py` before the bureau starts so agents can message each other.
```
dev_agent ──TopicQuery──► orchestrator ──ResearchRequest──► primary_worker
│ │
│ ──ResearchRequest──► speculative_worker │
│ │ │
◄──ResearchResult──────────────────────◄──────────────◄
```
### Agent responsibilities
| Agent | File | Role |
|---|---|---|
| `orchestrator` | `agents/orchestrator_agent.py` | Routes queries; checks warm cache; dispatches primary + speculative workers; forwards results back to requester |
| `primary_worker` | `agents/primary_worker.py` | Crawls the requested topic via Browserbase/Playwright, classifies it with Claude, populates cache |
| `speculative_worker` | `agents/speculative_worker.py` | Prefetches likely follow-up topics concurrently; discarded on timeout; concurrent message handling enabled |
| `dev_agent` | `agents/dev_agent.py` | Demo client — sends two queries (`atoms` → `electrons`) to show a cold run then a warm hit |
### Shared modules
- **`shared/pipeline.py`** — all I/O: `crawl_topic` (Browserbase + Playwright in a thread pool), `classify_and_structure` (Claude call to JSON), `get_speculative_candidates` (Claude call for predicted follow-ups), `research_topic` (combines both)
- **`shared/cache.py`** — in-process dict store (`get_warm` / `set_warm` / `all_topics`); keyed by lowercased topic
- **`shared/messages.py`** — uAgents `Model` classes: `TopicQuery`, `ResearchRequest`, `ResearchResult`
- **`shared/config.py`** — seeds for deterministic agent addresses; runtime address slots populated by `main.py`; speculation constants
### Key design details
[truncated — 671 more characters]
```
### INTEGRATION.md
```markdown
# Integration contract — observability/panel (Person C) ↔ A & B
**TL;DR:** The panel, Sentry, and the Redis event bus are fully self-contained
and already work standalone (`python -m panel.demo_replay`). The *only* coupling
to A's and B's code is a handful of `emit_demo_event(...)` calls living inside
your files. The merge already dropped them once. This doc is the contract so it
doesn't happen again.
Nothing here blocks A or B from building independently. These are the lines to
**keep** (or move intact) when you rewrite your internals.
---
## The event bus (don't worry about it)
- `shared/demo_events.py :: emit_demo_event(type, data)` — fire-and-forget.
**Safe to call anywhere**: if Redis is down it's a no-op (no exception), so it
can never crash your pipeline. It also drops a Sentry breadcrumb.
- The panel reads the Redis stream over SSE. It never imports A's or B's code.
- So: call `emit_demo_event` at the right spots and you're done. No other wiring.
---
## Person A — orchestrator + speculation
You own `agents/orchestrator_agent.py` and the speculation/vector layer. Keep
these emits when you swap the in-memory cache for RedisVL and the candidate list
for embeddings + the relevance filter. They fire at *logical* pipeline points,
so they're identical whether speculation is a plain async pool or uAgents.
| Event | Fire when | Required fields | Panel use |
|---|---|---|---|
| `query_received` | a query arrives | `topic`, `session_id` | breadcrumb |
| `warm_hit` | served from the vector cache | `topic`, `session_id`, **`matched_topic`**, **`warmed_at`**, `similarity` | **payoff** |
| `cold_dispatch` | no hit → dispatch primary | `topic`, `session_id` | left column |
| `speculation_planned` | candidates chosen | `parent`, `candidates` | breadcrumb |
| `spec_dispatch` | each bet is fired | `topic`, `parent` | makes worker card |
| `candidate_skipped` | candidate already warm | `topic`, `reason` | breadcrumb |
| `candidate_rejected` | relevance filter drops a candidate | `topic`, `reason` | greyed-out card in pool |
### ⚠️ The one that matters most: `warm_hit` and the semantic match
The panel shows "answer was ready N.Ns before the question" by matching a
`warm_hit` to the earlier `spec_warm` **for the same topic**. Your vector search
is *semantic* — a query ("electron") may hit a cached entry ("electrons"). When
that happens the strings differ and the proof is lost unless you tell the panel
what matched:
- **`matched_topic`** — the cached topic that actually satisfied the query
(the string that was `spec_warm`-ed). The panel matches on this.
- **`warmed_at`** — epoch seconds when that entry was warmed (so the lead time
still shows even across a panel reconnect). RedisVL must store this alongside
the vector; return it on the hit.
- **`similarity`** — optional; if present the panel shows it ("matched
'electrons' (0.91 sim)"), which actively *demos your vector IP*.
Keep the topic label byte-for-byte consistent between `spec_warm
[truncated — 2843 more characters]
```
### requirements.txt
```
uagents>=0.13.0
anthropic>=0.30.0
browserbase>=0.3.0
playwright>=1.44.0
python-dotenv>=1.0.0
sentry-sdk>=2.0.0
redisvl>=0.3.0
sentence-transformers>=2.7.0
redis>=5.0.0
numpy>=1.24.0
mcp>=1.0.0
fastapi>=0.100.0
uvicorn>=0.23.0
```
### main.py
```python
from dotenv import load_dotenv
load_dotenv()
import logging
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("sentence_transformers").setLevel(logging.WARNING)
logging.getLogger("redisvl").setLevel(logging.WARNING)
logging.getLogger("huggingface_hub").setLevel(logging.WARNING)
# Python 3.14 no longer auto-creates an event loop in the main thread, but
# uagents' Agent() grabs one at construction time. Create it before any agent
# is imported/built, or the Bureau can't start.
import asyncio
asyncio.set_event_loop(asyncio.new_event_loop())
from shared.observability import init_sentry
init_sentry() # one init for the whole process — covers primary + speculative paths
from shared.demo_events import reset
reset() # start every run with a clean panel stream
import shared.config as config
from agents.orchestrator_agent import orchestrator
from agents.primary_worker import primary_worker
from agents.dev_agent import dev_agent
from uagents import Bureau
# Wire up addresses before the Bureau starts so agents can send to each other
config.ORCHESTRATOR_ADDRESS = orchestrator.address
config.PRIMARY_WORKER_ADDRESS = primary_worker.address
config.DEV_AGENT_ADDRESS = dev_agent.address
print("─" * 60)
print(f" orchestrator {config.ORCHESTRATOR_ADDRESS}")
print(f" primary_worker {config.PRIMARY_WORKER_ADDRESS}")
print(f" dev_agent {config.DEV_AGENT_ADDRESS}")
print("─" * 60)
bureau = Bureau(port=8001)
bureau.add(orchestrator)
bureau.add(primary_worker)
bureau.add(dev_agent)
bureau.run()
```
### panel/server.py
```python
"""AgentDex live demo panel.
Tails the Redis demo-event stream and pushes every event to the browser over
SSE. The browser reconstructs state and renders two columns:
• Live queries — what the "developer's agent" asks, and how it's served.
• Speculative pool — workers pre-warming likely follow-ups, with the exact
timestamp each topic became ready.
The payoff: when a query is served as a WARM HIT, the panel shows how many
seconds *earlier* the speculative worker had already finished that topic — i.e.
the answer existed before the question was asked.
Run:
uvicorn panel.server:app --reload --port 8000
Then open http://localhost:8000
"""
import asyncio
import json
from dotenv import load_dotenv
from fastapi import FastAPI
from fastapi.responses import HTMLResponse, StreamingResponse
from shared.demo_events import history, read_events, reset
load_dotenv()
app = FastAPI(title="AgentDex Panel")
@app.get("/", response_class=HTMLResponse)
async def index():
return _PAGE
@app.post("/reset")
async def do_reset():
reset()
return {"ok": True}
@app.get("/events")
async def events():
"""SSE: replay history, then stream live events."""
async def gen():
last_id = "0"
# Cold-start: send everything already in the stream.
for entry_id, payload in await asyncio.to_thread(history):
last_id = entry_id
yield f"data: {json.dumps(payload)}\n\n"
# Live tail.
while True:
batch = await asyncio.to_thread(read_events, last_id, 15_000)
if not batch:
yield ": keep-alive\n\n" # comment frame so the connection stays open
continue
for entry_id, payload in batch:
last_id = entry_id
yield f"data: {json.dumps(payload)}\n\n"
return StreamingResponse(
gen(),
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"},
)
_PAGE = """
AgentDex — Live