# Project export: UserSwarm This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: AI personas that browse your app like real users, find UX friction, and grade themselves against human reviewers so you ship better products faster. - Devpost: https://devpost.com/software/userswarm - GitHub: https://github.com/Deepak-karani/userswarm.git - Video: https://www.youtube.com/embed/ZcZiE7iPoSs?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 4 GitHub contributor(s) — Claude Opus 4.8 (1M context) (16 commits), Deepak Karani (12 commits), Kalyan (5 commits), Deepak-karani (1 commits) ## Devpost submission (written by the team) ### Inspiration User testing is expensive, slow, and hard to scale. Recruiting real humans to click through your app takes days and costs hundreds of dollars per round. We asked: what if AI personas could simulate diverse real users — skeptics, multitaskers, accessibility-focused users — and give you actionable UX feedback in minutes? And then, how do you know the AI feedback is any good? That's where the eval loop comes in: we grade our AI testers against real human reviewers to keep them honest. ### What it does UserSwarm generates diverse AI personas, each with distinct traits and goals, then launches them into your live web app via browser automation. Each persona navigates your site, attempts a task, and writes a structured UX report citing real on-screen evidence. Reports are aggregated, evaluated with code + LLM-judge evals, and compared against human reviewer labels to measure human-likeness — do the AI testers fail where real users fail? A built-in improve-and-rerun loop uses eval failures and human annotations to refine the testing prompt, then reruns to measure the delta. ### How we built it Backend: FastAPI orchestrating a DAG — PersonaGenerator → fan-out UXTesters → ReportCritic → Aggregator → Evals Browser automation: Playwright driving headless Chromium; each AI persona gets its own browser session with screenshot capture LLM: Anthropic Claude (Sonnet for per-persona testing, Opus for aggregation/judging) with structured JSON output Evals: Code evals (task success, evidence coverage), LLM-judge evals (actionability, hallucination risk), and agreement evals (human agreement, human likeness) Observability: Arize AX via OpenTelemetry — every LLM call and eval is traced end-to-end Human baseline: Terac marketplace integration to recruit real reviewers; their labels feed the human-agreement eval Frontend: Next.js 14 App Router for run management, report viewing, and base-vs-improved comparison Orchestration: Agentspan (Orkeble execution of reasoning agents, with automatic fallback to in-process Challenges we faced Browser + LLM coordination: Each persona needs its own isolated browser session. RuPlaywright instances in parallelavigation step required carefulconcurrency management and timeout tuning. - Hallucination in UX reports: E elements that didn't exist. Weadded a ReportCritic agent that cross-checks claims against screenshot evidence, plushallucination-risk eval. Rate limits at scale: Fan-out of 3+ testers hitting the Anthropic API simultaneousltoken-per-minute limits. We tune logic with backoff. Eval grounding: Defining what "good" AI UX feedback means is hard. We settled on a multi-signal approach — code che for quality, and human agreementfor ground truth. What we learned The gap between "AI can browse a website" and "AI can give feedback a product team would actually act on" is enormous. Evructured evals, and ahuman-in-the-loop calibration step are what make the difference between a demo and a tool.## Inspiration ### Accomplishments we're proud of End-to-end autonomy: A single API call spins up personas, launches browsers, tests your app, critiques the reports, runs evals, and produces an aggregated analysis — zero human intervention required. Human-likeness eval: We built a metric that measures whether AI testers fail where real humans fail. This isn't just "does the AI find bugs" — it's "does the AI behave like a real user would?" Self-improving loop: The improve-and-rerun pipeline takes human annotations and eval failures, rewrites the testing prompt, and reruns — then shows you a side-by-side delta of what got better. Closed-loop optimization for UX testing. Evidence-grounded reports: Every friction point cites exact on-screen text, button labels, or screenshot steps. The ReportCritic agent enforces this — no vague "the UX could be better" hand-waving survives. Full observability from day one: Every LLM call, every eval, every persona session is traced to Arize AX via OpenTelemetry. We can debug a single persona's reasoning chain across 20+ browser steps. ### What's next Video replays: Record each persona's browser session as a video so product teams can watch the AI user struggle, not just read about it. Multi-device testing: Simulate mobile viewports, slow 3G connections, and tablet layouts — real users aren't all on MacBook Pros. Accessibility personas: Personas that use screen readers, keyboard-only navigation, and high-contrast mode to surface WCAG compliance issues. CI/CD integration: Run UserSwarm on every pull request — catch UX regressions before they ship, just like unit tests catch code regressions. Larger human baselines: Scale up Terac reviewer recruitment to build richer ground-truth datasets, driving the human-likeness eval from a sanity check to a statistically significant benchmark. Custom persona libraries: Let teams define their own user archetypes — "our power user," "a first-time visitor from organic search" — and reuse them across sprints. ## README (from the GitHub repository) # UserSwarm **AI user-testing agents for builders, validated by real human feedback.** A builder enters a product URL + a task. UserSwarm fans out AI **personas** that drive the product with a browser agent, produce strict-JSON **UX reports**, then: - **Orkes / Agentspan** orchestrates the durable multi-agent workflow, - **Terac** collects human preference labels that calibrate the agent, - **Arize** traces every run and evaluates report quality, …and the agent is **improved from those labels and rerun** to prove measurable before/after gains. > AI user agents give instant UX feedback; Terac human labels calibrate whether > that feedback is trustworthy; Arize proves improvement; Orkes/Agentspan > coordinates the workflow. *Not a replacement for real user research — a way to > make synthetic user testing trustworthy.* --- ## Mock-first, real-ready Every sponsor integration has a **real** code path and a **mock** code path behind one interface. Mock mode auto-enables when a credential/server is missing, so the **entire demo runs fully offline with zero env vars**. Add a key to flip that integration live. Persistence uses **Supabase/Postgres** via `DATABASE_URL`, with a **SQLite fallback** when it's unset. | Integration | Live path | Mock path (default) | | ----------- | --------- | ------------------- | | Anthropic LLM | `anthropic` SDK, claude-sonnet-4-6 / claude-opus-4-8 | Canned plausible JSON | | Agentspan | Agentspan SDK against `AGENTSPAN_SERVER_URL` | In-process DAG runner (same nodes, still visible) | | Arize | `arize-otel` + OpenInference spans + evals | Spans/evals logged locally + to console | | Terac | Terac API annotation jobs | Synthetic human labels | --- ## Quick start (offline, no keys needed) ### Backend ```bash cd backend python3 -m venv .venv && source .venv/bin/activate pip install -r requirements.txt playwright install chromium # optional; static fallback works without it uvicorn app.main:app --reload # http://localhost:8000 (GET / shows mock/live modes) ``` ### Frontend ```bash cd frontend npm install cp .env.local.example .env.local # NEXT_PUBLIC_API_URL=http://localhost:8000 npm run dev # http://localhost:3000 ``` Open **http://localhost:3000** and submit a run. --- ## Demo script (≈3 min) 1. **Submit** on `/`: a real public URL (e.g. `https://example.com`), product description, target audience, a task, and success criteria. → redirects to the run page. 2. **`/runs/[id]`** — watch the **Agentspan workflow timeline** advance: `PersonaGenerator → UXTester ×N → Aggregator → Evals`. See 3–5 personas, each tester's **real browser step log** (open_url / get_page_state / click / scroll) with screenshots, friction points, severity, recommendations, **Arize eval scores**, and the **Terac human-agreement** score. 3. **Annotate** → `/annotate/[id]`: act as a Terac human labeller; answer the six questions (usefulness, specificity, hallucination, task understanding, real-user agreement, A/B). Submit → the human-agreement eval updates. 4. **Run Improvement** — the `ImproverAgent` turns labels + failing evals into an improved prompt and reruns the workflow as an `improved` variant. 5. **Compare** → `/runs/[id]/compare`: **base vs improved** table — usefulness, evidence coverage, hallucination risk, human agreement, actionability pass rate, task-success rate, with deltas. To show **live** mode: set `ANTHROPIC_API_KEY` (and optionally `AGENTSPAN_SERVER_URL`, `ARIZE_API_KEY`+`ARIZE_SPACE_ID`, `TERAC_API_KEY`), restart, and rerun — real Claude output and real spans, with mock fallback intact for any still-missing credential. --- ## Going live per sponsor - **Agentspan / Orkes:** `pip install agentspan` → `agentspan server start` (UI at `http://localhost:6767`) → set `AGENTSPAN_SERVER_URL`. The same DAG nodes register/run on Agentspan; the dashboard timeline reflects them. - **Arize:** set `ARIZE_API_KEY` + `ARIZE_SPACE_ID` (install `arize-otel` + `openinference-instrumentation`). Every run, persona-gen, UX agent, browser tool call, aggregation, improved-prompt gen, and rerun is traced; six evals are logged. - **Terac:** set `TERAC_API_KEY`. Real annotation jobs are created from agent reports; the `terac.py` adapter has `# TODO` markers for the exact endpoint/payload — drop them in. The `/annotate` route already writes **real** human labels regardless of mode. --- ## Architecture ``` backend/ FastAPI · SQLAlchemy (Postgres|SQLite) · Anthropic · Playwright app/agents/ PersonaGenerator · UXTester · ReportCritic · Aggregator · Improver app/browser/ Playwright tools + safety guard (no destructive clicks; 8–12 action cap) app/orchestration/ WorkflowRunner (InProcess | Agentspan) + event persistence app/integrations/ arize.py · terac.py (real + mock) app/evals/ code + LLM-judge evals, human agreement, improvement score app/routers/ /runs · /annotate · /runs/{id}/compare frontend/ Next.js 14 (App Router) + TS + Tailwind app/ / · /runs/[id] · /runs/[id]/compare · /annotate/[id] ``` ### Evals `has_task_success` · `has_evidence` (code) · `actionability` · `hallucination_risk` (LLM judge) · `human_agreement` (Terac) · `improvement_score` (base vs improved). ### Browser safety Each agent is capped at 8–12 actions; purchase / payment / delete / send / invite / destructive clicks are blocked; login-required pages with no credentials stop and report; if automation is blocked the agent falls back to static page review. --- ## API | Method | Path | Purpose | | ------ | ---- | ------- | | POST | `/runs` | Start a workflow → `{id}` | | GET | `/runs/{id}` | Full run: status, personas, reports+steps, events, evals, aggregate | | GET | `/runs` | Recent runs | | POST | `/runs/{id}/improve` | Improve from labels + rerun → improved `{id}` | | GET | `/annotate/{id}` | Payload for the Terac annotation UI | | POST | `/annotate/{id}` | Store a human label | | GET | `/runs/{id}/compare` | Base vs improved metrics + deltas | Env vars: see `.env.example`. ## Detected evidence (automated analysis) Indexed codebase: 83 recognized source files, 348 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - JavaScript (language) — detected in the code - Next.js (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - Vercel (technology) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: commit authorship or trailers ## Codebase structure (from repository index) ### Files (98 of 98) ``` .env.example .gitignore backend/app/__init__.py backend/app/agents/__init__.py backend/app/agents/aggregator.py backend/app/agents/improver.py backend/app/agents/persona_generator.py backend/app/agents/report_critic.py backend/app/agents/ux_tester.py backend/app/browser/__init__.py backend/app/browser/playwright_driver.py backend/app/browser/safety.py backend/app/browser/tools.py backend/app/config.py backend/app/db.py backend/app/evals/__init__.py backend/app/evals/agreement.py backend/app/evals/code_evals.py backend/app/evals/llm_evals.py backend/app/evals/runner.py backend/app/integrations/__init__.py backend/app/integrations/arize.py backend/app/integrations/terac.py backend/app/llm/__init__.py backend/app/llm/client.py backend/app/main.py backend/app/models.py backend/app/orchestration/__init__.py backend/app/orchestration/agentspan_agents.py backend/app/orchestration/agentspan_browser_agent.py backend/app/orchestration/events.py backend/app/orchestration/runner.py backend/app/orchestration/workflow.py backend/app/routers/__init__.py backend/app/routers/annotate.py backend/app/routers/compare.py backend/app/routers/runs.py backend/app/schemas.py backend/requirements.txt frontend/.env.local.example frontend/.gitignore frontend/app/annotate/[runId]/page.tsx frontend/app/globals.css frontend/app/layout.tsx frontend/app/page.tsx frontend/app/runs/[runId]/compare/page.tsx frontend/app/runs/[runId]/page.tsx frontend/components/BeforeAfterTable.tsx frontend/components/EvalScores.tsx frontend/components/FrictionList.tsx frontend/components/LiveStatus.tsx frontend/components/PersonaCard.tsx frontend/components/PitchFooter.tsx frontend/components/ScreenshotGrid.tsx frontend/components/SessionReplay.tsx frontend/components/SeverityBadge.tsx frontend/components/StatusTimeline.tsx frontend/components/StepLog.tsx frontend/components/SwarmField.tsx frontend/lib/api.ts frontend/next.config.mjs frontend/package.json frontend/postcss.config.js frontend/tailwind.config.ts frontend/tsconfig.json PITCH.md PLAN.md README.md skill/userswarm/browser.py skill/userswarm/README.md skill/userswarm/requirements.txt skill/userswarm/SKILL.md test-app/.gitignore test-app/.vercelignore test-app/GROUND_TRUTH.md test-app/next.config.mjs test-app/package.json test-app/postcss.config.mjs test-app/README.md test-app/src/app/app/[projectId]/page.tsx test-app/src/app/app/activity/page.tsx test-app/src/app/app/calendar/page.tsx test-app/src/app/app/game/page.tsx test-app/src/app/app/layout.tsx test-app/src/app/app/page.tsx test-app/src/app/app/settings/page.tsx test-app/src/app/app/team/page.tsx test-app/src/app/globals.css test-app/src/app/layout.tsx test-app/src/app/onboarding/page.tsx test-app/src/app/page.tsx test-app/src/app/pricing/page.tsx test-app/src/app/signup/page.tsx test-app/src/components/LandingDemo.tsx test-app/src/components/Wordmark.tsx test-app/src/lib/store.tsx test-app/tailwind.config.ts test-app/tsconfig.json ``` ### Dependencies - backend/requirements.txt: agentspan@>=0.1.10, anthropic@>=0.40, arize-otel@>=0.7, fastapi@>=0.110, httpx@>=0.27, openinference-instrumentation-anthropic@>=0.1, playwright@>=1.44, psycopg2-binary@>=2.9, pydantic@>=2.6, python-dotenv@>=1.0, SQLAlchemy@>=2.0, uvicorn[standard]@>=0.29 - frontend/package.json: @types/node@^20.14.0, @types/react@^18.3.3, @types/react-dom@^18.3.0, autoprefixer@^10.4.19, next@^14.2.33, postcss@^8.4.39, react@^18.3.1, react-dom@^18.3.1, tailwindcss@^3.4.6, typescript@^5.5.3 - skill/userswarm/requirements.txt: playwright@>=1.44 - test-app/package.json: @types/node@20.14.10, @types/react@18.3.3, @types/react-dom@18.3.0, autoprefixer@10.4.19, next@^14.2.35, postcss@8.4.39, react@18.3.1, react-dom@18.3.1, tailwindcss@3.4.6, typescript@5.5.3 ### Recent commits (newest first) - Merge pull request #3 from Deepak-karani/claude/userswarm-mvp - Add fullscreen screenshot lightbox, pitch deck, and overview tweaks - Merge pull request #2 from Deepak-karani/claude/userswarm-mvp - Live-only integrations, UI polish, and browser-tooling hardening - Merge pull request #1 from Deepak-karani/claude/userswarm-mvp - Fix Arize integration to follow official AX setup guide - Integrate real Arize Phoenix evals + OpenInference tracing - Persona picker (1-5) + optional persona types + per-persona one-liner - Run page: split into Overview / Results tabs - F1 + F3 + concise run UI: human-like testers with distinct perspectives - Add test-app (Nimbus) — expanded multi-page project management demo - F2: voice-of-customer quotes + session replay - Redesign web UI (Friction-is-heat) + free-explore mode - Add local UserSwarm skill (Claude Code native) - Goal-driven UXTester loop + speed/timeout fixes + ReportCritic & Improver on Orkes - Make browser automation Playwright-only (remove static fallback) - Run PersonaGenerator + Aggregator on Agentspan (browser stays in-process) - Wire real Agentspan SDK for UX tester agents - Build UserSwarm MVP: Agentspan workflow, Terac labels, Arize evals - Scaffold UserSwarm: README brief, .env.example, .gitignore ## Key source files (fetched from GitHub, selected and truncated for size) ### PLAN.md ```markdown # Plan: Make the swarm replicate real users (revised after CEO review) ## Framing (revised) Lead with **trustworthy, prioritized friction a founder can act on before launch.** Realism is *supporting evidence* that the friction is believable, not the headline. Founders buy "here's the painful moment and the fix," not "our agents fail like humans." ## Decision log (from /autoplan Phase 1 — premise gate) - Resequence: **F2 first**, F1 second (reframed), F3 third. - **F1 is deterministic, not stochastic.** No random give-up. Annotate "a low-patience user would abandon here because X" on a run that still completes the flow. Same insight, reproducible, no coverage loss. (Stochastic imperfection rejected: trust-eroding non-determinism reads worse in a demo than a clean annotation.) - **Keep F3** despite the review's "cut it": the human-likeness score is the Terac + Arize prize story. Honesty rule: label it "mock-calibrated, real-ready" until real Terac labels exist. Real moat = the Terac calibration loop, so wire it real-ready, not faked-deeper. ## Build order ### F2 (first). Painful moment + voice-of-customer quotes + replay - Each friction point carries a first-person quote ("I couldn't tell what 'Go' does"). - Stitch the per-step screenshots we already save into a scrubbable per-persona replay (client-side image sequence; no video encoding). - Files: `backend/app/agents/ux_tester.py` (quote per friction in the report), `backend/app/schemas.py` (optional `quote` on friction), frontend `components/FrictionList.tsx`, new `components/SessionReplay.tsx`, `StepLog.tsx`. - Risk: quotes must be grounded in observed UI (reuse the existing "cite on-page evidence" rule) or they become hallucinated flavor text. ### F1 (second). Deterministic human-failure annotations - Persona traits become *behavioral lenses* the agent reasons through, not dice: patience, skim-vs-read, tech-savviness. Output: `would_abandon_here` markers + `confusion_events` (count), on a run that still attempts the whole flow. - Files: `backend/app/agents/persona_generator.py` (structured traits), `backend/app/agents/ux_tester.py` (`_agentic_test`/`_synthesize_report` lens prompt), `backend/app/schemas.py` (annotation fields). - Risk: keep it diagnostic. Never let a trait *change which actions run* (that reintroduces non-determinism); it only changes what gets *flagged*. ### F3 (third). Human-likeness score, real-ready (mock-labeled for now) - From Terac labels: "do agents fail where humans fail." Surface as a score, trend via Arize. - Files: `backend/app/evals/agreement.py`, `backend/app/evals/runner.py`, `backend/app/integrations/arize.py`, compare/run frontend. - Honesty: UI badge "calibrated on N human labels (mock)" until real labels land. The `terac.py` real path already exists; the moat is making one real label loop work. ## Backlog (deferred, with rationale now) - 5-second first-impression test (strong, bounded, reproducible — promote if F1 slips) [truncated — 727 more characters] ``` ### PITCH.md ```markdown # UserSwarm — Pitch Script (Sponsor-tuned: **Orkes ▸ Arize ▸ Terac**) > Engineered against the **AI Hackathon 2026** rubric (Application · Functionality/Quality · > Creativity · Technical Complexity · **Ethical Considerations** · **Brainstorming & Process**) > **and** weighted toward the sponsor tracks in priority order: **Orkes first, then Arize, then > Terac.** Format: science-fair table, **~3 min pitch + 2 min Q&A**, judged ≥2×. > > **Spine of the story:** *Orkes Conductor is what makes a swarm of AI users actually work* — > it's the durable runtime that coordinates persona-generation, a parallel fan-out of browser > testers, aggregation, and evals as one observable, fault-tolerant workflow. Arize proves the > output is trustworthy; Terac calibrates it against humans. --- ## 0. One-liner (memorize this) **"UserSwarm releases a swarm of distinct AI users on your product and watches where they hit friction in a real browser — all coordinated as one durable **Orkes** workflow, scored live in **Arize**, and calibrated against human labels from **Terac** so the feedback is provably trustworthy."** Orkes-forward variant (lead with this at the Orkes table): > **"We turned 'a swarm of AI users testing your product' from a script into a real orchestrated > system — Orkes Conductor runs the whole multi-agent workflow durably and observably, so it > survives failures, fans out testers in parallel, and you can watch every node execute."** --- ## 1. The 3-minute pitch (timed) Have the app open at `http://localhost:3000` with a finished Nimbus run ready, **Overview tab showing the Workflow Status timeline**. ### [0:00–0:30] Hook + problem > "Every team ships UX they think is fine. Real user testing catches the friction — but it's slow > and expensive. AI can give instant feedback, but two problems: why would you trust it, and how > do you run a whole *swarm* of AI users reliably instead of a flaky script? UserSwarm solves > both — and the reliability half is pure Orkes." ### [0:30–1:00] What it does > "You give it a URL, what the product is, and who it's for. It generates a **swarm of distinct > personas** — different patience, tech-savviness, disposition — and each one drives a **real > headless browser** like an actual user. No test scripts." ### [1:00–1:50] Demo — LEAD WITH ORKES (point at the Workflow Status timeline) > "This is the workflow, and it's a real **Orkes Conductor** orchestration — not a for-loop. > See these engine badges? **Persona generation and aggregation run as durable Orkes agents**; > the browser testers fan out in **parallel**, each in its own isolated session. Orkes > coordinates the whole DAG — persona-gen → parallel testers → aggregate → evals — as one > observable workflow. If a node fails, it surfaces as a real workflow error instead of silently > faking a result. That's what makes 'a swarm' actually dependable." Then the results (Results tab): > "Each AI user took ~30 real browser actions — replayable screenshots. [truncated — 8337 more characters] ``` ### backend/requirements.txt ``` fastapi>=0.110 uvicorn[standard]>=0.29 pydantic>=2.6 SQLAlchemy>=2.0 python-dotenv>=1.0 anthropic>=0.40 playwright>=1.44 httpx>=0.27 psycopg2-binary>=2.9 # Sponsor SDKs — REQUIRED (live-only mode; mock fallbacks removed): arize-otel>=0.7 openinference-instrumentation-anthropic>=0.1 agentspan>=0.1.10 ``` ### test-app/package.json ``` { "name": "nimbus", "version": "1.0.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint" }, "dependencies": { "next": "^14.2.35", "react": "18.3.1", "react-dom": "18.3.1" }, "devDependencies": { "@types/node": "20.14.10", "@types/react": "18.3.3", "@types/react-dom": "18.3.0", "autoprefixer": "10.4.19", "postcss": "8.4.39", "tailwindcss": "3.4.6", "typescript": "5.5.3" } } ``` ### frontend/package.json ``` { "name": "userswarm-frontend", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint" }, "dependencies": { "next": "^14.2.33", "react": "^18.3.1", "react-dom": "^18.3.1" }, "devDependencies": { "@types/node": "^20.14.0", "@types/react": "^18.3.3", "@types/react-dom": "^18.3.0", "autoprefixer": "^10.4.19", "postcss": "^8.4.39", "tailwindcss": "^3.4.6", "typescript": "^5.5.3" } } ``` ### skill/userswarm/requirements.txt ``` playwright>=1.44 ``` ### backend/app/main.py ```python """UserSwarm FastAPI app entrypoint.""" from __future__ import annotations import os from fastapi import FastAPI from fastapi.middleware.cors import CORSMiddleware from fastapi.staticfiles import StaticFiles from .config import settings from .db import init_db from .routers import annotate, compare, runs _SCREENSHOT_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "screenshots")) app = FastAPI(title="UserSwarm") app.add_middleware( CORSMiddleware, allow_origins=["http://localhost:3000"], allow_credentials=True, allow_methods=["*"], allow_headers=["*"], ) @app.on_event("startup") def _startup() -> None: # Live-only: fail fast if any required credential / Orkes URL is missing, # instead of silently booting into a mock path. settings.require_live() init_db() os.makedirs(_SCREENSHOT_DIR, exist_ok=True) app.mount("/static/screenshots", StaticFiles(directory=_SCREENSHOT_DIR), name="screenshots") app.include_router(runs.router) app.include_router(annotate.router) app.include_router(compare.router) @app.get("/") def root() -> dict: return {"app": "UserSwarm", "modes": settings.mode_summary()} ``` ### frontend/app/layout.tsx ```typescript import type { Metadata } from "next"; import Link from "next/link"; import "./globals.css"; import LiveStatus from "@/components/LiveStatus"; export const metadata: Metadata = { title: "UserSwarm — release a swarm of AI users on your product", description: "Release a swarm of AI users on your product and watch where they hit friction. Human labels calibrate trust; Arize proves improvement; Orkes/Agentspan coordinates the swarm.", }; export default function RootLayout({ children, }: { children: React.ReactNode; }) { return (
{/* swarm glyph — cool dots with one running hot */} UserSwarm
{children}
); } ``` ### frontend/app/page.tsx ```typescript "use client"; import { useEffect, useState } from "react"; import { useRouter } from "next/navigation"; import Link from "next/link"; import { createRun, listRuns, RunOut } from "@/lib/api"; import SeverityBadge from "@/components/SeverityBadge"; import SwarmField from "@/components/SwarmField"; const PILLARS = [ ["Agentspan", "coordinates the swarm"], ["Terac", "humans calibrate trust"], ["Arize", "evals prove improvement"], ]; export default function HomePage() { const router = useRouter(); const [form, setForm] = useState({ url: "", description: "", audience: "", persona_types: "", do_not_click: "", }); const [numPersonas, setNumPersonas] = useState(3); const [submitting, setSubmitting] = useState(false); const [error, setError] = useState(null); const [recent, setRecent] = useState([]); useEffect(() => { listRuns() .then((r) => setRecent(Array.isArray(r) ? r.slice(0, 6) : [])) .catch(() => setRecent([])); }, []); function set(k: K, v: string) { setForm((f) => ({ ...f, [k]: v })); } async function onSubmit(e: React.FormEvent) { e.preventDefault(); setSubmitting(true); setError(null); try { const do_not_click_rules = form.do_not_click .split(/[\n,]/) .map((s) => s.trim()) .filter(Boolean); const persona_types = form.persona_types .split(/[\n,]/) .map((s) => s.trim()) .filter(Boolean); const { id } = await createRun({ url: form.url, description: form.description, audience: form.audience, num_personas: numPersonas, persona_types, do_not_click_rules, }); router.push(`/runs/${id}`); } catch (err) { setError(err instanceof Error ? err.message : "Failed to start run"); setSubmitting(false); } } const inputCls = "w-full rounded-md border border-ink-line bg-ink-900/60 px-3 py-2.5 text-sm text-fog placeholder:text-fog-faint outline-none transition focus:border-cool focus:ring-2 focus:ring-cool/25"; const labelCls = "mb-1.5 block font-mono text-[11px] uppercase tracking-[0.15em] text-fog-muted"; return ( <>
{/* thesis */}

AI user testing · human-calibrated · eval-proven

Release a swarm of AI users on your product. Watch where they hit{" "} friction.

Point it at a URL, say what the product is and who it's for. A swarm of distinct personas explores it like real users and reports the friction, in minutes. No test scripts.

Not a replacement for real user research.

{PILLARS.map(([k, v]) => (
{k} {v}
))}
{/* the input is the hero */}

Set the target

free explore
set("url", e.target.value)} />