# Project export: Conjure This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: See it. Say it. Surf it. - Devpost: https://devpost.com/software/conjure-ghipod - GitHub: https://github.com/Preet37/AIHACKS2026/ - Video: https://www.youtube.com/embed/CErXb4GKzbo?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 4 GitHub contributor(s) — dgne58 (12 commits), tk (4 commits), KaiSong06 (4 commits), Claude Opus 4.8 (1 commits) ## Devpost submission (written by the team) ### Inspiration We wanted to build a browser that adapts to people instead of forcing everyone into the same default experience. Browsers serve billions of users, but most customization still requires extensions, settings menus, or technical knowledge. At the same time, tools like Claude Code, Cursor, Devin, and other agentic coding systems have shown that software can now be generated from intent. Conjure started from that idea: what if changing your browser was as simple as asking for what you want? Instead of browsing around limitations, users could describe the experience they need and have Conjure build it directly into their browser. ### What it does Conjure is a self-building browser agent. You ask for a browser feature, and it generates a Chrome MV3 extension mod for that task. For example, a user can ask Conjure to remove YouTube Shorts, add custom page controls, modify a website UI, or create cross-site workflow helpers. The Chrome extension gathers browser context and gives the user a command interface, while the FastAPI backend routes the request to an agent provider such as Devin, Claude, or Nemotron. Conjure also keeps memory through Redis, so projects, conversations, rules, sandbox results, and agent job streams persist across sessions. Generated mods can be verified in a sandbox before being applied, and the side panel tracks build progress, sandbox results, screenshots, and active mods. ### How we built it Frontend React + TypeScript: Chrome extension UI, side panel, settings, voice overlay, design/run surfaces. Vite + CRXJS: Builds the MV3 Chrome extension. Chrome Extension APIs: sidePanel, tabs, scripting, userScripts, offscreen, storage. Vite + CRXJS: Builds the MV3 Chrome extension. Chrome Extension APIs: sidePanel, tabs, scripting, userScripts, offscreen, storage. Backend FastAPI: HTTP + WebSocket backend for chat, voice, mods, and browser-agent tasks. Python async stack: Handles streaming agent events, tool calls, voice requests, and cloud-browser jobs. LangChain: Shared tool-calling loop across Groq, Claude, and Nemotron. AI / Agent Services Groq: Primary fast LLM provider. Anthropic Claude: Higher-quality fallback/model option. NVIDIA Nemotron / NIM: Alternative hosted or self-hosted model path. Deepgram: Speech-to-text with nova-2, text-to-speech with Aura. Browserbase: Cloud Chrome sessions for browsing, testing, and replay. Stagehand: AI browser automation inside Browserbase. Data / Observability Redis: Persists conversations, messages, memory rules, job state, sandbox cache. Filesystem: Stores generated browser mods under demo_code//mods. Sentry: Captures backend, extension, sandbox, and generated-mod errors. Flow Extension gathers browser context and sends chat/voice input to FastAPI. Backend runs the selected LLM agent through LangChain tools. Agent creates or edits browser mods on disk. Extension fetches active mod bundles and injects them via chrome.userScripts. Browserbase/Stagehand verify or browse pages off-device. Redis stores app state; Sentry tracks failures. ### Challenges we ran into Difficulty sleeping Wifi issues Working between workshops Ideating alone took us 12 hours ### Accomplishments we're proud of We built what we set out to build and more. This AI agent codes extensions which are loaded directly into the user's browser in real-time. Here are some cool extensions we created (these were all one shotted, taking only one prompt): 1) We added a search functionality to a website that had none. We used Deepgram to speak to our browser directly using natural language. 2) We blocked YouTube Shorts to lock in on the devpost submission 3) Thomas gets a lot of spam emails for things he is not interested in. Being the intelligent young man he is, he uses Conjure to create an email filter that quickly deletes all these spam emails from his inbox. ### What we learned Chromium is very difficult to fork :( ### What's next Next, we want to move beyond Chrome extension mods into a custom Chromium fork for deeper browser control. That would let Conjure modify more of the browser experience directly instead of being limited to extension APIs. We also want to improve the Simular/Sai testing loop, expand the mod registry, add richer cross-site workflows, and make Conjure usable by people who have no idea how browser extensions work. The long-term goal is simple: describe the browser you want, and Conjure builds it. ## README (from the GitHub repository) # conjure `conjure` is a self-building browser agent. A local Chrome extension gathers browser context and hosts the chat UI; a FastAPI backend routes work to the configured coding provider, tracks progress, stores conversation/session state, and reports finished agent session or PR links back to the user. See the [left-to-right system diagram](ARCHITECTURE.md) for the implemented stack and [MASTER_DESIGN_DOC.md](MASTER_DESIGN_DOC.md) for the architecture source of truth. ## Expected Layout ```text backend/ FastAPI agent service, Redis store, sandbox/test/heal loop conjure-extension/ Vite + React + CRXJS MV3 Chrome extension tests/ repo-level smoke and integration checks .env.example local configuration template, no real secrets ``` ## Configuration Create a local env file from the template: ```powershell Copy-Item .env.example .env ``` Fill in real values for the selected coding provider, Redis, Browserbase, Simular, and Sentry in `.env`. Do not commit secrets. The extension reads only `VITE_` values at build time, so do not put private API keys behind `VITE_` names. Required config groups: - Agent provider: `CONJURE_AGENT_PROVIDER=groq` (default, fastest), `CONJURE_AGENT_PROVIDER=claude` for Claude, or `CONJURE_AGENT_PROVIDER=nemotron` for NVIDIA Nemotron - Groq: `GROQ_API_KEY` and `GROQ_MODEL` (default qwen/qwen3-32b) - Claude: `ANTHROPIC_API_KEY` and `CONJURE_ANTHROPIC_MODEL` - Nemotron: `NVIDIA_API_KEY`, `NVIDIA_MODEL`, and optional `NVIDIA_API_BASE_URL` for self-hosted NIM - Redis: `REDIS_URL`, `REDIS_NAMESPACE`, sandbox cache TTL - Browserbase: API key, project ID, session settings - Simular: API key and optional endpoint/model override - Sentry: backend DSN, sandbox DSN, environment, trace sample rate - Backend URL: HTTP and WebSocket base URLs - Extension config: Vite-exposed backend URLs, extension environment, public Sentry DSN ## Backend Setup Run these after the backend worker lands `backend/pyproject.toml` and `backend/main.py`: ```powershell python -m venv .venv .\.venv\Scripts\Activate.ps1 python -m pip install --upgrade pip python -m pip install -r .\backend\requirements.txt python -m playwright install chromium npm run dev:backend ``` The backend should read `.env`, connect to Redis, expose the WebSocket contract from the design doc, and route each Conjure conversation to the selected provider. Demo mode simulates provider progress without external credentials. Nemotron uses NVIDIA's LangChain `ChatNVIDIA` integration and the same local backend tool loop as Claude. For hosted API Catalog usage, set `CONJURE_AGENT_PROVIDER=nemotron`, `CONJURE_DEMO_MODE=false`, `NVIDIA_API_KEY`, and optionally override `NVIDIA_MODEL`. For local NIM later, set `NVIDIA_API_BASE_URL=http://localhost:8000/v1`. ## Extension Setup Run these after the extension worker lands `conjure-extension/package.json`: ```powershell npm --prefix conjure-extension install npm run dev:extension ``` For manual Chrome testing, build the extension and load the unpacked output directory from Chrome's Extensions page. Keep Groq, Claude, Nemotron, and other provider keys server-side; the extension should use `VITE_BACKEND_URL` and `VITE_BACKEND_WS_URL` only. ## Dev Commands ```powershell npm run dev:backend # FastAPI on 127.0.0.1:8000 npm run dev:extension # Vite/CRXJS extension dev server npm run build:extension # extension production build npm run test # root smoke checks npm run test:smoke # same smoke checks, explicit name npm run test:backend-smoke ``` `npm run test:backend-smoke` is opt-in. Set `CONJURE_SMOKE_BACKEND_URL` to a running backend URL when the service exists; without it, the test skips. ## Test Commands The current scaffold uses stdlib `unittest` so it works before backend and extension dependencies exist: ```powershell python -m unittest discover -s tests/smoke ``` Pytest is also configured for future workers: ```powershell python -m pytest ``` When backend and extension implementations arrive, add focused tests under `tests/` or `e2e/` without importing generated extension projects from `demo_code/`. ## Integration Notes - Redis should hold projects, conversations, memory rules, sandbox result cache entries, and sandbox job streams. - Browserbase owns disposable Chrome sessions and replay/screenshot capture. - Simular owns autonomous functional, crash, and security passes against the Browserbase session. - Sentry should use separate environments or projects for backend, extension, and sandbox crashes. - Generated extension artifacts belong in `demo_code/` and are ignored by Git. ## Detected evidence (automated analysis) Indexed codebase: 104 recognized source files, 771 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - LangChain (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Redis (technology) — detected in the code - TypeScript (language) — detected in the code - JavaScript (language) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: commit authorship or trailers ## Codebase structure (from repository index) ### Files (113 of 113) ``` .env.example .gitignore .stitch/17399145846593091740/html/conjure_agent_run_trace.html .stitch/17399145846593091740/html/conjure_command_bar_overlay.html .stitch/17399145846593091740/html/conjure_design_language_indigo.html.html .stitch/17399145846593091740/html/conjure_design_mode_inspector.html .stitch/17399145846593091740/html/conjure_home_sidebar.html .stitch/17399145846593091740/html/conjure_idle_splash.html .stitch/17399145846593091740/html/conjure_planning_mode.html .stitch/17399145846593091740/html/conjure_settings.html .stitch/17399145846593091740/manifest.json ARCHITECTURE.md backend/bot.py backend/main.py backend/requirements.txt backend/tests/test_agent_protocol.py backend/tests/test_agent_task_endpoint.py backend/tests/test_agentspan_finder.py backend/tests/test_browser_agent.py backend/tests/test_extension_validator.py backend/tests/test_memory.py backend/tests/test_mods.py backend/tests/test_sandbox.py backend/tests/test_store.py backend/utils/__init__.py backend/utils/agent.py backend/utils/agentspan_finder.py backend/utils/browser_agent.py backend/utils/config.py backend/utils/extension_validator.py backend/utils/memory.py backend/utils/mods.py backend/utils/prompts.py backend/utils/sandbox.py backend/utils/sentry.py backend/utils/store.py backend/utils/tester.py backend/utils/tools.py backend/utils/voice.py CONJURE-BUILD.md CONJURE-DESIGN.md conjure-extension/design.html conjure-extension/index.html conjure-extension/manifest.config.ts conjure-extension/offscreen.html conjure-extension/package.json conjure-extension/run.html conjure-extension/scripts/check-generated-wrapper.mjs conjure-extension/settings.html conjure-extension/src/background.ts conjure-extension/src/content/main.tsx conjure-extension/src/generatedModWrapper.ts conjure-extension/src/offscreen.ts conjure-extension/src/pages/design/main.tsx conjure-extension/src/pages/run/main.tsx conjure-extension/src/pages/settings/main.tsx conjure-extension/src/pages/shared/page.css conjure-extension/src/pages/shared/staticSurface.tsx conjure-extension/src/shared/config.ts conjure-extension/src/shared/fonts.ts conjure-extension/src/shared/keybind.ts conjure-extension/src/shared/messages.ts conjure-extension/src/shared/providerSettings.ts conjure-extension/src/sidepanel/App.tsx conjure-extension/src/sidepanel/components/Button.tsx conjure-extension/src/sidepanel/components/CommandInput.tsx conjure-extension/src/sidepanel/components/index.ts conjure-extension/src/sidepanel/components/MetadataBlock.tsx conjure-extension/src/sidepanel/components/OptionCard.tsx conjure-extension/src/sidepanel/components/Pane.tsx conjure-extension/src/sidepanel/components/primitives.css conjure-extension/src/sidepanel/components/ProgressBar.tsx conjure-extension/src/sidepanel/components/SelectionOverlay.tsx conjure-extension/src/sidepanel/components/StatusBar.tsx conjure-extension/src/sidepanel/components/StatusBlock.tsx conjure-extension/src/sidepanel/components/Toggle.tsx conjure-extension/src/sidepanel/components/Window.tsx conjure-extension/src/sidepanel/lib/format.ts conjure-extension/src/sidepanel/main.tsx conjure-extension/src/sidepanel/styles.css conjure-extension/src/sidepanel/surfaceContext.tsx conjure-extension/src/sidepanel/surfaces/Composer.tsx conjure-extension/src/sidepanel/surfaces/Design.css conjure-extension/src/sidepanel/surfaces/DesignPanel.tsx conjure-extension/src/sidepanel/surfaces/DesignStage.tsx conjure-extension/src/sidepanel/surfaces/FindingsPanel.css conjure-extension/src/sidepanel/surfaces/FindingsPanel.tsx conjure-extension/src/sidepanel/surfaces/HomePanel.css conjure-extension/src/sidepanel/surfaces/HomePanel.tsx conjure-extension/src/sidepanel/surfaces/Invoke.css conjure-extension/src/sidepanel/surfaces/PlanningPanel.tsx conjure-extension/src/sidepanel/surfaces/RightPanel.tsx conjure-extension/src/sidepanel/surfaces/Settings.css conjure-extension/src/sidepanel/surfaces/SettingsPanel.tsx conjure-extension/src/sidepanel/surfaces/Trace.css conjure-extension/src/sidepanel/surfaces/TracePanel.tsx conjure-extension/src/sidepanel/surfaces/TraceStage.tsx conjure-extension/src/sidepanel/tokens.css conjure-extension/src/sidepanel/useBackendHealth.ts conjure-extension/src/sidepanel/useFinder.ts conjure-extension/src/sidepanel/useVoice.ts conjure-extension/src/sidepanel/VoiceOverlay.tsx conjure-extension/tsconfig.json conjure-extension/vite.config.ts docker-compose.yml HOW_TO_RUN.md package.json pytest.ini README.md scripts/verify_sentry.py tests/smoke/test_backend_smoke.py tests/smoke/test_scaffold_contract.py USING_CONJURE.md ``` ### Dependencies - backend/requirements.txt: agentspan@>=0.1, anthropic@>=0.40, browserbase@>=1.0, fastapi@>=0.115, httpx@>=0.27, langchain@>=0.3, langchain-anthropic@>=0.2, langchain-core@>=0.3, langchain-nvidia-ai-endpoints@>=1.0, langchain-openai@>=0.3, playwright@>=1.49, python-dotenv@>=1.0, redis@>=5.0, sentry-sdk@>=2.0, stagehand@>=3.0, uvicorn[standard]@>=0.30 - conjure-extension/package.json: @crxjs/vite-plugin@^2.6.1, @fontsource/jetbrains-mono@^5.2.8, @sentry/browser@^10.56.0, @types/chrome@^0.1.43, @types/react@^19.2.0, @types/react-dom@^19.2.0, @vitejs/plugin-react@^6.0.2, conjure-workspace@file:.., lucide-react@^1.21.0, react@^19.2.7, react-dom@^19.2.7, react-markdown@^10.1.0, typescript@^5.9.3, vite@^8.0.16 ### Recent commits (newest first) - final - bug fixes - voice and agents fixes - voice and agents merge - feat: add fetchai finder + multi-site mods onto voice-agent base - final changes - fix(voice): restore Deepgram STT/TTS lost in frontend merge - Merge remote-tracking branch 'origin/frontend' into voice-agent - Merge remote-tracking branch 'origin/main' into voice-agent - frontend works - fix: target command overlay to webpage tabs - feat: add shortcut diagnostics and stitch motion polish - feat(extension): report generated mod errors to Sentry - style: apply conjure design surfaces - feat: route extension surfaces for mv3 - fix: stop mv3 runtime and csp errors - feat: integrate voice + mods + frontend UI - merge: pull frontend UI into voice-agent - frontend - feat: add Conjure frontend design system ## Key source files (fetched from GitHub, selected and truncated for size) ### ARCHITECTURE.md ```markdown # Conjure architecture Conjure is a self-building browser agent powered by a Redis state spine and Browserbase cloud-browser runtime. The diagram follows the product from left to right. ```mermaid flowchart LR %% Conjure palette: tokens.css classDef human fill:#6C6AF5,stroke:#ADABFF,color:#FFFFFF,stroke-width:2px classDef primary fill:#222290,stroke:#6C6AF5,color:#FFFFFF,stroke-width:2px classDef service fill:#101026,stroke:#6C6AF5,color:#F0F0F5,stroke-width:1.5px classDef secondary fill:#16163A,stroke:#54546E,color:#F0F0F5,stroke-width:1px classDef core fill:#222290,stroke:#ADABFF,color:#FFFFFF,stroke-width:3px classDef voicecore fill:#6C6AF5,stroke:#F0F0F5,color:#FFFFFF,stroke-width:3px classDef external fill:#F0F0F5,stroke:#6C6AF5,color:#08080F,stroke-width:1.5px classDef observe fill:#ADABFF,stroke:#222290,color:#08080F,stroke-width:2px USER["USER
describe what the browser should do"]:::human subgraph CHROME["CONJURE CHROME EXTENSION"] direction TB UI["CHAT + VOICE
React · TypeScript · Vite"]:::primary CONTEXT["LIVE BROWSER CONTEXT
tabs · DOM · console · cookies"]:::service UI <--> CONTEXT end subgraph BACKEND["PYTHON AGENT BACKEND"] direction TB API["FASTAPI
REST + WebSocket streaming"]:::primary AGENT["LANGCHAIN AGENT
plan · build · act · self-correct"]:::service MODS["MOD ENGINE
generate · validate · ship JS/CSS"]:::service API --> AGENT --> MODS end subgraph HEART["CONJURE CORE"] direction TB REDIS[("REDIS
persistent memory + conversations
live job streams + sandbox cache")]:::core BB["BROWSERBASE
cloud browser execution
isolated testing + replay + logs"]:::core REDIS <-->|"state coordinates every run"| BB end subgraph VOICE_LAYER["CONVERSATIONAL VOICE LAYER"] direction TB DEEPGRAM["DEEPGRAM
real-time voice interface"]:::voicecore STT["NOVA-2
speech → intent"]:::service TTS["AURA
response → natural speech"]:::service STT --> DEEPGRAM --> TTS end subgraph INTELLIGENCE["INTELLIGENCE + AUTOMATION"] direction TB MODELS["GROQ · CLAUDE · NEMOTRON
reasoning + code generation"]:::external STAGEHAND["STAGEHAND SDK + PLAYWRIGHT
navigate · act · extract · verify"]:::external end RESULT["TESTED BROWSER MODS
applied instantly in Chrome"]:::human SENTRY["SENTRY
errors + traces + healing signal"]:::observe USER --> UI CONTEXT <-->|"context + streamed progress"| API UI -->|"push-to-talk audio"| STT TTS -->|"spoken acknowledgement + result"| UI DEEPGRAM <-->|"voice requests through FastAPI"| API AGENT <-->|"memory + orchestration"| REDIS AGENT <-->|"tool-calling"| MODELS MODS -->|"builds to test"| BB STAGEHAND <-->|"controls sessions"| BB BB -->|"verified findings + replay"| MODS MODS [truncated — 1818 more characters] ``` ### USING_CONJURE.md ```markdown # Using Conjure (everyday flow) `HOW_TO_RUN.md` covers first-time setup. This file covers what happens once it's running: where generated code goes, how it gets applied, and why your session no longer resets. ## TL;DR 1. Type a request in the Conjure side panel (e.g. "remove YouTube Shorts"). 2. The backend generates a small MV3 extension under `demo_code//`. 3. **Conjure applies it for you** — it injects the generated script + CSS into matching tabs and reloads them. You never open `chrome://extensions` to load or reload anything. 4. Your conversation is saved, so closing/reopening the panel (or restarting the backend) keeps your history. ## One-time setup (required once after this update) Two things changed that need a one-time action: 1. **Reload the Conjure extension** so Chrome picks up the new build and the new `userScripts` permission: - `chrome://extensions` → find **Conjure** → click the circular **reload** icon. (Or remove it and `Load unpacked` → `conjure-extension/dist` again.) 2. **Allow user scripts** for Conjure. The auto-apply feature uses Chrome's `chrome.userScripts` API, which Chrome gates behind a toggle: - Chrome 138+: `chrome://extensions` → Conjure → **Details** → turn on **"Allow user scripts"**. - Chrome 120–137: just enable **Developer mode** (top-right of `chrome://extensions`). If this is off, Conjure will tell you in the panel instead of silently failing: *"turn on 'Allow user scripts' (or enable Developer mode), then send your request again."* ## Mods: list, change, remove Every customization Conjure builds is a **mod** — a self-contained content-script bundle in its own folder under `demo_code//mods//`, tracked in `demo_code//mods/registry.json`. The side panel shows a **Mods** list: - **Change** — edit a mod's starter prompt and rebuild it. A prompt change *always* regenerates the mod (no "is it already there?" check). - **Remove** — deletes the mod's files and unregisters its user script from the browser (its `chrome.userScripts` entry `conjure-mod-` is removed). - Each mod shows a verification badge: `verified` / `failed` / `unverified`, with a link to the Browserbase **sandbox replay** when available. ### Build vs. reuse (test-before-remake) When you ask for something new, Conjure first checks the existing mods: - If a mod already implements your request, it runs that mod through the **Browserbase sandbox** to confirm it still works. If it passes, Conjure does **not** rebuild it — it tells you it already exists and is verified. - If no mod matches, or the sandbox check fails, Conjure builds (or rebuilds) it. - Editing a mod's prompt skips this check and rebuilds immediately. Verification runs the mod's extension in a real Browserbase browser session (`verify_mod` → `sandbox.py`), so it needs `BROWSERBASE_API_KEY` and `BROWSERBASE_PROJECT_ID` in `.env`. Progress streams into the **Sandbox** panel. [truncated — 3160 more characters] ``` ### package.json ``` { "name": "conjure-workspace", "version": "0.0.0", "private": true, "description": "Root developer workflow scripts for the conjure hackathon workspace.", "scripts": { "dev": "node -e \"console.log('Run npm run dev:backend and npm run dev:extension in separate terminals.')\"", "dev:backend": "python3 -m uvicorn backend.main:app --reload --host 127.0.0.1 --port 8000", "dev:extension": "npm --prefix conjure-extension run dev", "build:extension": "npm --prefix conjure-extension run build", "test": "python3 -m unittest discover -s tests/smoke", "test:smoke": "python3 -m unittest discover -s tests/smoke", "test:backend-smoke": "python3 -m unittest discover -s tests/smoke -p test_backend_smoke.py" }, "engines": { "node": ">=20" } } ``` ### docker-compose.yml ```yaml # Local Redis for conjure: datastore + sandbox cache + job streams. # Reproducible on any machine with Docker Desktop: # docker compose up -d # start Redis on localhost:6379 # docker compose ps # check status # docker compose down # stop (data persists in the named volume) # docker compose down -v # stop AND wipe data services: redis: image: redis:7-alpine # pinned so every machine runs the same Redis container_name: conjure-redis ports: - "6379:6379" # matches REDIS_URL=redis://localhost:6379/0 command: ["redis-server", "--appendonly", "yes"] # persist to disk volumes: - conjure-redis-data:/data restart: unless-stopped healthcheck: test: ["CMD", "redis-cli", "ping"] interval: 5s timeout: 3s retries: 5 volumes: conjure-redis-data: ``` ### backend/requirements.txt ``` fastapi>=0.115 uvicorn[standard]>=0.30 python-dotenv>=1.0 httpx>=0.27 redis>=5.0 sentry-sdk>=2.0 langchain>=0.3 langchain-core>=0.3 langchain-openai>=0.3 langchain-anthropic>=0.2 langchain-nvidia-ai-endpoints>=1.0 anthropic>=0.40 playwright>=1.49 browserbase>=1.0 agentspan>=0.1 stagehand>=3.0 ``` ### conjure-extension/package.json ``` { "name": "conjure-extension", "version": "0.1.0", "private": true, "type": "module", "scripts": { "dev": "vite", "build": "vite build", "test:generated-wrapper": "node scripts/check-generated-wrapper.mjs", "typecheck": "tsc --noEmit", "preview": "vite preview" }, "dependencies": { "@fontsource/jetbrains-mono": "^5.2.8", "@sentry/browser": "^10.56.0", "conjure-workspace": "file:..", "lucide-react": "^1.21.0", "react": "^19.2.7", "react-dom": "^19.2.7", "react-markdown": "^10.1.0" }, "devDependencies": { "@crxjs/vite-plugin": "^2.6.1", "@types/chrome": "^0.1.43", "@types/react": "^19.2.0", "@types/react-dom": "^19.2.0", "@vitejs/plugin-react": "^6.0.2", "typescript": "^5.9.3", "vite": "^8.0.16" } } ``` ### backend/main.py ```python from __future__ import annotations import asyncio import contextlib import json import re import uuid from dataclasses import replace from pathlib import Path from typing import Any from urllib.parse import urlparse from fastapi import FastAPI, HTTPException, Request, WebSocket, WebSocketDisconnect from fastapi.middleware.cors import CORSMiddleware from fastapi.responses import Response from pydantic import BaseModel from .utils import browser_agent from .utils import mods as mods_registry from .utils import voice as voice_utils from .utils.agent import ConjureAgent from .utils.browser_agent import BrowserAgentError, BrowserAgentSettings from .utils.config import load_settings from .utils.memory import extract_and_save_rules from .utils.store import create_store from .utils.tools import project_dir_for app = FastAPI(title="conjure backend") # The Chrome extension (side panel + service worker) fetches the generated # bundle over HTTP from a different origin, so allow cross-origin reads. app.add_middleware( CORSMiddleware, allow_origins=["*"], allow_methods=["GET", "POST", "PATCH", "DELETE", "OPTIONS"], allow_headers=["*"], ) @app.get("/health") async def health() -> dict[str, str]: return {"status": "ok", "service": "conjure-backend"} class AgentTaskRequest(BaseModel): """A 'find items' task run by an off-device Browserbase cloud browser (Stagehand). The extension sends the current tab URL plus the user's cookies, so the cloud browser browses as the logged-in user, off-device — it is not the user's browser.""" task: str url: str = "" cookies: list[dict[str, Any]] = [] @app.post("/projects/{project_id}/agent-task") async def run_agent_task(project_id: str, payload: AgentTaskRequest) -> dict[str, Any]: """Run an off-device browse (Browserbase + Stagehand) and return findings.""" task = payload.task.strip() if not task: raise HTTPException(status_code=400, detail="task is required") if not payload.url.strip(): raise HTTPException(status_code=400, detail="url is required") settings = load_settings() browse_settings = BrowserAgentSettings( browserbase_api_key=settings.browserbase_api_key, browserbase_project_id=settings.browserbase_project_id, model=settings.browse_model, max_results=settings.browse_max_results, max_steps=settings.browse_max_steps, region=settings.browserbase_session_region, use_proxies=settings.browse_use_proxies, verified=settings.browse_verified, advanced_stealth=settings.browse_advanced_stealth, current_tab_only=settings.browse_current_tab_only, ) blocker = browser_agent.missing_requirement(browse_settings) if blocker: raise HTTPException(status_code=503, detail=blocker) try: result = await browser_agent.find_items_remote( task=task, settings=browse_settings, start_url=payload.url, cookies=payload.cookies, ) except BrowserAgentError as exc: raise HTTPException(status_code=502, detail=str(exc)) from exc return { "project_id": project_id, "task": task, "url": payload.url, "findings": result.get("findings", []), "session_id": result.get("session_id", ""), "replay_url": result.get("replay_url", ""), } @app.post("/voice/transcribe") async def voice_transcribe(request: Request) -> dict[str, str]: """Accept raw audio from the extension and return a Deepgram nova-2 transcript.""" audio = await request.body() content_type = request.headers.get("content-type", "audio/webm") try: transcript = await voice_utils.transcribe_audio(audio, content_type) except Exception as exc: raise HTTPException(status_code=502, detail=str(exc)) from exc return {"transcript": transcript} @app.get("/voice/status") async def voice_status() -> dict[str, str | bool]: """Report Deepgram readiness without returning credentials.""" return voice_utils.status() @app.post("/voice/speak") async def voice_speak(body: dict[str, str]) -> Response: """Convert assistant reply text to MP3 via Deepgram Aura TTS.""" text = body.get("text", "").strip() if not text: raise HTTPException(status_code=422, detail="text is required") try: audio_bytes = await voice_utils.speak_text(text) except Exception as exc: raise HTTPException(status_code=502, detail=str(exc)) from exc return Response(content=audio_bytes, media_type="audio/mpeg") @app.get("/projects/{project_id}/mods") async def list_project_mods(project_id: str) -> dict[str, Any]: """List every mod (browser customization) built for this project.""" project_dir = project_dir_for(load_settings(), project_id) return {"project_id": project_id, "mods": mods_registry.list_mods(project_dir)} @app.get("/projects/{project_id}/mods/bundle") async def get_project_mod_bundles(project_id: str) -> dict[str, Any]: """Return every active mod's content-script bundle for the extension to apply.""" project_dir = project_dir_for(load_settings(), project_id) bundles = mods_registry.active_bundles(project_dir) return {"project_id": project_id, "ready": bool(bundles), "bundles": bundles} @app.delete("/projects/{project_id}/mods/{mod_id}") async def delete_project_mod(project_id: str, mod_id: str) -> dict[str, Any]: """Remove a mod and its generated files.""" project_dir = project_dir_for(load_settings(), project_id) deleted = mods_registry.delete_mod(project_dir, mod_id) if not deleted: raise HTTPException(status_code=404, detail=f"No mod with id {mod_id}") return {"project_id": project_id, "deleted": mod_id, "mods": mods_registry.list_mods(project_dir)} @app.patch("/projects/{project_id}/mods/{mod_id}") async def update_project_mod(project_id: str, mod_id: str, payload: dict[str, Any]) -> dict[str, Any] [truncated — 15444 more characters] ``` ### conjure-extension/src/sidepanel/main.tsx ```typescript import React from "react"; import { createRoot } from "react-dom/client"; import { loadExtensionFonts } from "../shared/fonts"; import App from "./App"; import "./tokens.css"; import "./styles.css"; import "./components/primitives.css"; loadExtensionFonts(); createRoot(document.getElementById("root") as HTMLElement).render( ); ``` ### conjure-extension/src/sidepanel/components/index.ts ```typescript // Conjure primitives (CONJURE-DESIGN.md §6). One implementation each; // every surface composes from these. export { StatusBlock, type StatusState } from "./StatusBlock"; export { MetadataBlock, type MetaRow } from "./MetadataBlock"; export { Pane } from "./Pane"; export { Window } from "./Window"; export { Toggle } from "./Toggle"; export { Button } from "./Button"; export { ProgressBar } from "./ProgressBar"; export { OptionCard } from "./OptionCard"; export { CommandInput } from "./CommandInput"; export { StatusBar, type Workspace } from "./StatusBar"; export { SelectionOverlay, type OverlayTool } from "./SelectionOverlay"; ``` ### conjure-extension/src/pages/run/main.tsx ```typescript import React, { useMemo, useRef, useState } from "react"; import { createRoot } from "react-dom/client"; import { loadExtensionFonts } from "../../shared/fonts"; import { StatusBar, StatusBlock } from "../../sidepanel/components"; import { SurfaceProvider } from "../../sidepanel/surfaceContext"; import { TracePanel } from "../../sidepanel/surfaces/TracePanel"; import { TraceStage } from "../../sidepanel/surfaces/TraceStage"; import { createStaticSurfaceValue, defaultUiSettings } from "../shared/staticSurface"; import "../../sidepanel/tokens.css"; import "../../sidepanel/styles.css"; import "../../sidepanel/components/primitives.css"; import "../../sidepanel/surfaces/Trace.css"; import "../shared/page.css"; loadExtensionFonts(); function RunPage() { const messagesEndRef = useRef(null); const [uiSettings, setUiSettings] = useState(defaultUiSettings); const [projectId, setProjectId] = useState("local-demo"); const surface = useMemo( () => createStaticSurfaceValue({ messagesEndRef, uiSettings, setUiSettings, projectId, setProjectId }), [projectId, uiSettings] ); return (
undefined} right={ <> run trace } />
); } createRoot(document.getElementById("root") as HTMLElement).render( ); ``` [95 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]