Project Info

Winner

Finalist; Hacker's Choice Award

Inspector

Inspiration

AI coding agents made developers 100x faster, but bug rates are up 41%. The problem is simple: your agent writes the code, then hands it back to you. You launch the app, click around, find what's broken, describe the bug, and wait for a fix. You are the QA engineer. We wanted to close that loop. Let the agent see its own work, find the bugs, and fix them without a human in the middle.

What it does

Inspector is an MCP server that gives your coding agent (Claude Code, Cursor, or any MCP client) the ability to autonomously test the app it just built. It launches your app in a cloud sandbox, uses computer-use to operate it like a real user (screenshots, element detection, clicks), and maps the UI into regions. Each region gets its own agent that tests in parallel. Findings are independently verified with 2-of-2 replay agreement, de-duplicated, and sent back as a structured fix list. The coding agent patches the code and Inspector re-verifies. It works across web, Electron, Android, and iOS through pure pixel-level interaction. Anything that renders to a screen, Inspector can test. Inspector surfaces the errors found back to the coding agent in the terminal. Additionally, from the dashboard, you can copy a prompt with full context to paste into Claude Code, or send Devin to open a PR automatically.

How we built it

The core is a Python MCP server built on FastMCP that exposes tools like launch_app, observe, act, and verify. The architecture has three layers: the host coding agent (the brain), the Inspector control plane (eyes and hands), and sandboxed execution planes where apps run. Inside the control plane, five roles form the pipeline: a Mapper that clusters detected UI elements into bounded regions using OmniParser (YOLOv8 + Florence-2), a Coordinator that spawns one region-agent per region, the Region-Agents themselves that run hypothesis-driven testing (navigate, capture, trigger, measure), a Verifier that replays findings independently, and a Merger that de-duplicates and outputs the fix list. For execution, web and Electron apps run in E2B cloud sandboxes, Linux microVMs with a virtual desktop where Chrome is launched in app mode and interaction happens via xdotool and screenshots. Android and iOS run on local emulators on the developer's machine. All four surfaces implement one SurfaceAdapter interface, so the entire core never branches on surface type. The dashboard is a Next.js app that displays runs, findings, screenshots, and replay traces. It includes a Bug Ledger that tracks issues across runs with evidence-based status (open, verified, fixing, dismissed) and a Devin integration for automated PR creation.

Challenges we ran into

The biggest challenge was getting the computer to traverse the UI intelligently and reliably resurface bugs. A single agent wandering the entire app would get lost, miss regions, or repeat the same actions without finding anything new. We solved this by adding a planning stage: a short session launches the app, observes the screen, and the agent decomposes the UI into different regions to test. It then dispatches agents with their own isolated sandbox to target each region independently, and finally verifies the bugs through independent replay. This turned unreliable wandering into structured, parallel, thorough coverage.

Accomplishments we're proud of

The multi-agent region decomposition is the design decision we're most proud of. Instead of one agent wandering the entire UI, each region-agent is confined to its bounding box and can't stray. This makes testing parallel, thorough, and deterministic. The SurfaceAdapter abstraction lets us support four completely different platforms (web, Electron, Android, iOS) with zero branching in the core. The MCP tools, session manager, perception pipeline, detection engine, and loop controller are all written once. The verify-after-act pattern, where every action returns the post-action screenshot and a changed flag, gives the agent self-correction without needing a DOM. This is the single most important reliability mechanism in the system.

What we learned

Computer-use is powerful but fragile without grounding. Raw pixel coordinate clicking fails constantly on dense UIs. Grounding-by-ID through Set-of-Mark (numbered element boxes from OmniParser) makes it reliable. The MCP protocol is a great fit for this kind of tool. Long-running operations (app builds, test loops) map cleanly onto the Tasks pattern, and the tool interface keeps Inspector composable while the host agent supplies all the reasoning.

What's next

Making the dashboard production-ready and hosting it as a service. Right now it runs locally. We want to deploy it so teams can share runs, track bugs across projects, and manage the Devin integration from one place.

Inspector

An MCP server that lets your coding agent see, click, and test the app it just built, then hand back reproducible findings so it can fix bugs on its own. Works on web, Electron, Android, and iOS.

Inspector plugs into Claude Code, Cursor, or any MCP coding agent. It spins up a sandbox, launches your app with your own dev command, and gives the agent real eyes and hands: screenshot, find the element, click, verify. One loop, every surface.

What makes it good

One loop for every surface. A single SurfaceAdapter runs the same observe, act, verify loop on web, Electron, Android, and iOS. Pixel-level computer-use reaches canvas, WebGL, and native UI that DOM selectors can't touch.
Clicks that land. Cheap element detection numbers every interactive element (Set-of-Mark). The agent picks an id, Inspector clicks it. No more misfired coordinate guesses.
Hard evidence, not vibes. audit_dom runs axe-core over the live DOM for real pass/fail on accessibility, broken images, and unlabeled inputs, while vision still catches the visual breakage the DOM hides.
Findings point at your code. Every bug ties back to file:line with a repro trail, so the agent goes straight to the fix.
Built to break things. Three rounds of planning and a per-feature attack catalog push edge inputs, double-submits, keyboard nav, bad routes, and mobile overflow. The goal is to break the app, not confirm it works.
Safe and reproducible. Apps run in isolated VMs with full lifecycle. Every run writes a trace that replays as HTML and video, and findings move open to fixed to verified.

How it compares

	Raw computer-use	Browser agents	Playwright-class	Inspector
Web	✅	✅	✅	✅
Electron / native mobile	⚠️	❌	❌	✅ one loop
Canvas / WebGL / pixel-only UI	✅	✅	❌	✅
Deterministic checks (axe, console)	❌	❌	✅	✅
Sees visual breakage	✅	✅	❌	✅
Precise clicks	❌	❌	✅	✅
Findings link to source	❌	❌	❌	✅
Reproducible replay	❌	❌	✅	✅
Adversarial by default	❌	❌	⚠️	✅
Isolated sandbox	❌	⚠️	⚠️	✅

Status

Web is live and proven end to end. Pure-Python core with 13+ MCP tools, audit_dom, adversarial planning, and findings plus replay (100+ tests). Electron is one refactor out; Android and iOS adapters are in progress. All four surfaces are in scope as a personal dev tool (no hosting or payments yet).

Quickstart

pip install -e ".[dev]"     # runtime plus dev tools
cp .env.example .env        # set REPLICATE_API_TOKEN; E2B_API_KEY optional
inspector-mcp doctor        # verify env and keys
inspector-mcp serve         # run the MCP server (stdio)
pytest -q                   # unit tests

Layout

inspector/     # core plus adapters (web, electron, android, ios) plus perception
infra/         # how each VM is provisioned
examples/      # one buggy sample app per surface
docs/          # design docs 01 through 13
scripts/       # run helpers, doctor, probes

Docs

Full design docs live in docs/, covering vision, architecture, the MCP contract, the core loop, detection, data schema, roadmap, and the agentic test loop. Start with 01. Vision & Strategy and 08. Build Plan. See TESTING.md to validate with a real Claude Code agent.

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

AnthropicIn code
CIn code
CSSIn code
DartIn code
HTMLIn code
JavaIn code
JavaScriptIn code
KotlinIn code
Next.jsIn code
PythonIn code
ReactIn code
SwiftIn code
Tailwind CSSIn code
TypeScriptIn code

14 of 14 appear in the indexed code.

AI coding agents

Claude CodeConfig · Commits
CodexConfig

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

981 KB

Source files

258

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.

Feature verification

13 Verified7 Code-supported

100+ unit testsVerified
Pure-Python core with 13+ MCP tools, audit_dom, adversarial planning, and findings plus replay (100+ tests)
Claimed on readmehigh confidence
- tests — 48 test files containing 322 def test_ occurrences, comfortably exceeding the 100+ claim
Bug Ledger tracking issues across runs with evidence-based statusVerified
A Bug Ledger that tracks issues across runs with evidence-based status (open, verified, fixing, dismissed)
Claimed on Devposthigh confidence
- inspector/dashboard/aggregate.py:191 — bug_ledger() function with status enum open|fixing|fixed|verified|dismissed
- inspector/dashboard/render.py:217 — _ledger_table renders the ledger
- tests/test_ledger.py — Dedicated test coverage
Copy prompt with full context to paste into Claude CodeVerified
From the dashboard, you can copy a prompt with full context to paste into Claude Code
Claimed on Devpostmedium confidence
- inspector/replay.py:226 — copyText() uses navigator.clipboard.writeText with 'copy repro'/'copy fix prompt' buttons embedded in generated replay HTML
Devin integration for automated PR creationVerified
Send Devin to open a PR automatically
Claimed on Devposthigh confidence
- inspector/devin.py:141 — fix_with_devin() builds a fix prompt, calls Devin API, extracts PR URL
- inspector/server.py:230 — fix_with_devin wired as an MCP tool
E2B cloud sandbox with Linux microVM virtual desktop, Chrome app mode, xdotoolVerified
Web and Electron apps run in E2B cloud sandboxes, Linux microVMs with a virtual desktop where Chrome is launched in app mode and interaction happens via xdotool and screenshots
Claimed on Devposthigh confidence
- inspector/sandbox.py:8 — E2BSandbox wraps e2b_desktop.Sandbox
- inspector/planes/linux.py:8 — LinuxPlane implementation
- inspector/adapters/web.py:203 — Launches chrome --app={url} --no-sandbox and drives via xdotool
Every run writes a trace that replays as HTML and videoVerified
Every run writes a trace that replays as HTML and video, and findings move open to fixed to verified
Claimed on readmehigh confidence
- inspector/replay.py:374 — write_replay_video() stitches frames into replay.gif and replay.mp4 via ffmpeg
- inspector/replay.py:558 — _build_html builds the interactive HTML replay player
Findings tie back to file:line via source scanningVerified
Every bug ties back to file:line with a repro trail, so the agent goes straight to the fix
Claimed on readmehigh confidence
- inspector/source_scan.py:25 — extract_expected() with per-framework extractors for web/android/ios/rn/flutter
- inspector/source_scan.py:117 — _lineno() computes line numbers, feeding a suspected_area field like 'App.jsx:10'
inspector-mcp CLI with doctor and serve commandsVerified
inspector-mcp doctor verifies env and keys; inspector-mcp serve runs the MCP server
Claimed on readmehigh confidence
- pyproject.toml:49 — Entry point inspector-mcp = inspector.cli:main
- inspector/cli.py:25 — main() dispatches serve (default) vs doctor
- inspector/doctor.py:18 — doctor main() runs env/key/dependency checks
MCP server exposing launch_app, observe, act, verify toolsVerified
Python MCP server built on FastMCP exposing tools like launch_app, observe, act, and verify
Claimed on Devposthigh confidence
- inspector/server.py:48 — FastMCP('Inspector', ...) instance created
- inspector/server.py:376 — launch_app tool defined with @mcp.tool
- inspector/server.py:459 — observe tool defined
- inspector/server.py:482 — act tool defined
- inspector/server.py:780 — verify_fix tool defined
Merger de-duplicates findings into structured fix listVerified
A Merger that de-duplicates and outputs the fix list
Claimed on Devposthigh confidence
- inspector/parallel.py:90 — De-dupes merged findings by normalized summary key
- inspector/cartographer/orchestrator.py:85 — run_regions builds a structured, severity-sorted fixes list with suggested_fix text
- inspector/findings.py:6 — build_finding is the structured Finding builder used throughout
Set-of-Mark grounding for clicks by numbered element idVerified
Grounding-by-ID through Set-of-Mark (numbered element boxes from OmniParser) makes it reliable
Claimed on Devposthigh confidence
- inspector/perception/som.py:8 — render_set_of_mark draws numbered boxes over the screenshot keyed to Element.id
- inspector/detection.py — Maps numeric id back to bbox center for clicks
SurfaceAdapter interface across web, Electron, Android, iOS with no core branchingVerified
All four surfaces implement one SurfaceAdapter interface, so the entire core never branches on surface type
Claimed on Devposthigh confidence
- inspector/adapters/base.py:24 — Abstract SurfaceAdapter contract, docstring states core never branches on surface type
- inspector/adapters/web.py — Concrete web adapter implementing the interface
- inspector/adapters/electron.py — Concrete electron adapter implementing the interface
- inspector/adapters/android.py — Concrete android adapter implementing the interface
- inspector/adapters/ios.py — Concrete ios adapter implementing the interface
Verifier replays findings with 2-of-2 agreementVerified
A Verifier that replays findings independently; findings are independently verified with 2-of-2 replay agreement
Claimed on Devposthigh confidence
- inspector/cartographer/orchestrator.py:76 — Comment and code literally re-run the same protocol (2-of-2 verify) and drop candidates that don't reproduce
- inspector/verify.py:15 — verify_findings refutes judgment-based findings via driver.verify_finding
- inspector/reverify.py — Independent post-fix repro replay logic
Adversarial testing: three rounds of planning with per-feature attack catalogCode-supported
Three rounds of planning and a per-feature attack catalog push edge inputs, double-submits, keyboard nav, bad routes, and mobile overflow
Claimed on readmemedium confidence
- inspector/adversarial.py:7 — EDGE_INPUTS and PATTERNS dict cover forms/modals/navigation/accessibility/robustness/responsive attack categories
- inspector/server.py:1181 — The 'three rounds of planning' is only a prompt string instructing the calling LLM agent to think in rounds, not an enforced round-loop in code
Android and iOS run on local emulatorsCode-supported
Android and iOS run on local emulators on the developer's machine
Claimed on Devpostmedium confidence
- inspector/planes/android.py:62 — LocalEmulatorRuntime boots a real local AVD emulator via subprocess
- inspector/planes/android.py:127 — RedroidRuntime.start() explicitly raises NotImplementedError, a scaffold
- inspector/adapters/ios.py — Drives macOS Simulator/idb as an iOS local runtime; infra/ios-corellium is documented as a non-default scaffold option
audit_dom runs axe-core over live DOM for accessibility/broken images/unlabeled inputsCode-supported
audit_dom runs axe-core over the live DOM for real pass/fail on accessibility, broken images, and unlabeled inputs
Claimed on readmemedium confidence
- inspector/audit.py:6 — Maps axe-core impact levels to severity
- inspector/audit.py:37 — Audit result handling logic
- tests/test_audit_integration.py — Dedicated integration test exists
Coordinator spawning one region-agent per region, tested in parallelCode-supported
A Coordinator that spawns one region-agent per region, the Region-Agents themselves that run hypothesis-driven testing
Claimed on Devpostlow confidence
- inspector/cartographer/orchestrator.py:58 — run_regions loops over regions sequentially applying deterministic lenses, docstring explicitly says 'Sequential on one session'; no per-region agent object or parallel dispatch found
- inspector/parallel.py:73 — Real ThreadPoolExecutor-based parallelism exists but fans out over planner-defined parts/goals, not Cartographer regions
Dashboard displays runs, findings, screenshots, replay tracesCode-supported
The dashboard is a Next.js app that displays runs, findings, screenshots, and replay traces
Claimed on Devpostmedium confidence
- inspector/dashboard/render.py — Python-generated static HTML dashboard renders runs, findings
- inspector/dashboard/build.py — Builds dashboard artifacts
- inspector/replay.py:558 — _build_html builds interactive replay player with scrubber/timeline
- web/app/page.tsx — The Next.js app under web/ is a marketing landing page (Hero, Pricing, HowItWorks), not the runs/findings dashboard itself, so the specific 'Next.js dashboard' claim is only partially supported
Electron support one refactor out, Android/iOS adapters in progress (status honesty)Code-supported
Web is live and proven end to end. Electron is one refactor out; Android and iOS adapters are in progress
Claimed on readmemedium confidence
- inspector/adapters/electron.py — Electron adapter code exists and largely mirrors web adapter's xdotool approach, consistent with 'one refactor out' rather than fully proven
- inspector/planes/android.py:127 — RedroidRuntime is an explicit NotImplementedError scaffold, consistent with 'in progress' framing
UI mapping into regions using OmniParser (YOLOv8 + Florence-2)Code-supported
A Mapper that clusters detected UI elements into bounded regions using OmniParser (YOLOv8 + Florence-2)
Claimed on Devpostmedium confidence
- inspector/perception/detector.py:25 — OmniParserDetector calls Replicate microsoft/omniparser-v2 or self-hosted HTTP for element detection
- inspector/cartographer/mapper.py:1 — Docstring states region clustering is deterministic spatial connected-components, explicitly not using an LLM/OmniParser for the clustering step itself

An AI agent derived these features from the project’s Devpost page and readme, then searched the code for each one. Verified features are backed by cited code; claimed-only features had no supporting code, which is not by itself proof a feature is missing.

Project Info

Inspector

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments we're proud of

What we learned

What's next

Inspector

What makes it good

How it compares

Status

Quickstart

Layout

Docs

Analysis

Technology

AI coding agents

Codebase size

Structure

Interface

Application logic

Languages

Dependencies

Feature verification