# Project export: OnTab

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2026
- Tagline: Cursor Tab for Everything
- Devpost: https://devpost.com/software/ontab
- GitHub: https://github.com/csaye/ontab
- Video: https://www.youtube.com/embed/76gxlzWBOzY?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner ([Human Capital] Human Capital Fellowship Prize ( $50k equity-free check per team member (up to $200k total)))
- Team: 5 GitHub contributor(s) — Claude Opus 4.6 (47 commits), ammarateya (14 commits), llfuchiall (9 commits), Mihir Arya (5 commits), Cooper Saye (2 commits)

## Devpost submission (written by the team)

### Overview

Introduction We built OnTab: Cursor Tab for everything. It's a Chrome extension that watches how you work, predicts your next browser action (clicking a button, filling a form, navigating to a page), and executes it when you press Tab. Today's computer-operating agents are where coding agents were in 2024: not yet trusted to run autonomously end-to-end, but powerful with a human in the loop. Just like Cursor's thesis for code, we believe users want the speed of automation without giving up control over their device. OnTab gives you both, one tap at a time.

### Inspiration

We noticed that knowledge workers spend hours on the same mechanical browser sequences. Recruiters cycle between LinkedIn profiles and spreadsheets. Medical office staff copy patient details across portals to fill out prior authorization forms. Accountants tab between bank statements, tax software, and client records. These are patterns a model should be able to learn and predict. Crucially, these are also high-trust environments. A wrong autofill on a prior auth can delay a patient's care, a misplaced number in a tax filing has real consequences. Full autonomy isn't appropriate here. But a system that suggests the next step and lets a human confirm it? That's the right level of AI for these workflows today. The insight was that browser actions are a lot like code completions. They're sequential, context-dependent, and repetitive. If Cursor can predict your next line of code, we should be able to predict your next browser action. But the key constraint is trust. People are protective of their browser, especially when handling sensitive data. Tab-to-accept felt like the natural interaction: show the prediction, let the user verify, then execute. The Product Background agent that passively observes your browsing and suggests actions when confident One-step-at-a-time execution: press Tab to advance, Esc to dismiss: full user control Custom per-user LLM that adapts to your individual workflows via reinforcement learning Cross-tab continuity: workflows that span multiple tabs (e.g., LinkedIn → Google Sheets → Gmail) work seamlessly How We Built It See image attached! Two-Layer Prediction Pipeline Our core architectural decision was splitting prediction into two layers: understanding what the user is doing vs. figuring out how to do the next step. Layer 1 — Task Description (fast). When a user acts on a page, we call Claude Haiku 4.5 to generate a short natural-language summary of the user's current task and recent steps. This runs in ~200ms and gives us a semantic understanding of intent based on a sequence of raw DOM events. Layer 2 — Action Planning (precise). We pass that task description along with a snapshot of the current page's interactive elements to Claude Sonnet 4.5 / ChatGPT 5-mini, which generates a concrete sequence of browser actions (clicks, inputs, navigations, keypresses) as structured JSON. This is the step that actually knows where to click and what to type. This separation matters. The task describer generalizes across pages (it doesn't care about specific button IDs), while the action planner is grounded in the actual DOM. Splitting them made each layer simpler and more reliable than a single monolithic model trying to do both. We also maintain an on-device fallback using WebLLM (Qwen2.5-1.5B via WebGPU), which can replace Layer 1 for fully offline, private inference. We pivoted to cloud-first as the default after measuring that Haiku is faster than on-device inference for most users. Action Execution Actions are executed directly via DOM manipulation in the content script (querySelector, click(), dispatchEvent(), setting input values), no external runtime dependencies. The content script captures a snapshot of visible interactive elements (buttons, inputs, links, up to 40 per page) and resolves targets using a layered selector strategy: element ID, className, XPath, text content, and ARIA roles. A queue controller manages multi-step execution with up to 3 retries per step to handle flaky selectors and dynamic pages. Chrome Extension Architecture The extension runs as a Manifest V3 Chrome extension with three main components: A content script that records user actions and executes predicted steps A service worker that routes messages and manages queue state across tabs An offscreen document that runs the inference pipeline in an isolated context, keeping the main browsing experience snappy Post-Training & Reinforcement Learning To fully personalize suggestions, we fine-tune a per-user model. Every time a user accepts or rejects a prediction, we log it: the predicted action, whether it was accepted, and what the user did instead if they rejected it. Once we accumulate 50+ examples, we trigger an async training job on Modal (A10G GPU). The training uses KTO (Kahneman-Tversky Optimization), a reinforcement learning method that learns from both acceptances and rejections. Unlike standard supervised fine-tuning which only learns from correct examples, KTO explicitly downweights rejected predictions, so the model learns what not to suggest just as much as what to suggest. We fine-tune Qwen2.5-1.5B with LoRA adapters (10-50MB), keeping the base model frozen. The trained adapter gets compiled into an MLC-compatible WebAssembly library and uploaded to HuggingFace. On the client side, a model lifecycle manager polls for new adapters and hot-swaps them into the on-device predictor: no restart, no user intervention. Since RL is computationally expensive and can contain 20,000+ rows of evals, we containerize the function in Modal. We leverage Modal Volumes to store the LoRA adapter files, and fetch them during runtime when users log-in. This allows concurrency and scaling. After post-training our Qwen model with 1,000+ data points, tab acceptance rate doubled from 26% to 52%. [see image attached] For cold-start, we synthetically generated 1,000+ browser action entries using Browserbase's Stagehand. We scripted diverse browser tasks (navigating pages, filling forms, clicking through flows), executed them in parallel cloud browsers via the Stagehand API, and parsed the event logs into our RL training schema. This gave us a solid base dataset with minimal compute, generated in ~20 minutes. Challenges Latency vs. accuracy tradeoff. Our first approach used two on-device WebLLM instances: one for task description, one for action generation. Inference was private but slow (300-500ms per layer). We found that users notice anything over ~1 second total. Moving Layer 1 to Haiku and Layer 2 to Sonnet cut perceived latency dramatically while improving prediction quality, at the cost of requiring an API key. DOM fragility. Browser pages are messy. Elements load asynchronously, IDs change between sessions, and class names are often auto-generated. We went through multiple selector strategies before landing on a layered approach that tries ID, className, XPath, text content, and ARIA roles in sequence. Learning from rejections. Standard fine-tuning only learns from positive examples. Early on, our model would keep suggesting the same wrong action because it never learned not to. Switching to KTO was valuable because it treats rejected predictions as explicit negative signals, which made the model noticeably better at avoiding repeated mistakes. Cross-tab state. Browser workflows naturally span tabs (LinkedIn → Sheets → Gmail), but Chrome extensions have limited cross-tab communication. We had to build a shared state layer in the service worker that tracks queue progress across tab boundaries, so a workflow that starts on one page can continue seamlessly when the user switches tabs. What We Learned The biggest lesson was that the right level of AI autonomy depends on the domain. For browser automation, one-step-at-a-time with Tab-to-accept is the sweet spot right now. It builds trust incrementally. Users who start by carefully reviewing every suggestion eventually just Tab through entire workflows without looking. The trust has to be earned, not assumed. We also learned that splitting inference into semantic layers (understanding vs. planning) is a powerful pattern. It made each component independently testable, swappable, and debuggable. When Haiku gives a bad task description, we can see it immediately without digging through action-level logs. Finally: reinforcement learning from real user feedback, even small amounts of it, beats large amounts of synthetic data. Our 50-example personalized models outperform the base model trained on 1,000+ synthetic examples, because real rejection signals encode exactly what matters to that specific user.

## README (from the GitHub repository)

# TreeHacks Chrome Extension

A basic Chrome extension framework (Manifest V3).

## Structure

- **manifest.json** – Extension config (name, version, permissions, popup, background)
- **popup.html / popup.css / popup.js** – UI when you click the extension icon
- **background.js** – Service worker (runs in background, survives when popup is closed)

## Load in Chrome

1. Open `chrome://extensions`
2. Turn on **Developer mode**
3. Click **Load unpacked**
4. Select this folder

## Optional: custom icons

Add PNGs to an `icons/` folder and reference them in `manifest.json`:

- `icons/icon16.png` (16×16)
- `icons/icon48.png` (48×48)
- `icons/icon128.png` (128×128)

Then add under `"action"` and at top level:

```json
"default_icon": { "16": "icons/icon16.png", "48": "icons/icon48.png", "128": "icons/icon128.png" },
"icons": { "16": "icons/icon16.png", "48": "icons/icon48.png", "128": "icons/icon128.png" }
```

## Common next steps

- **Content scripts**: inject JS/CSS into web pages (add `"content_scripts"` in manifest)
- **Permissions**: e.g. `"activeTab"`, `"storage"`, `"tabs"` for more capabilities
- **Options page**: add `"options_page": "options.html"` or `"options_ui"` for settings


## Detected evidence (automated analysis)

Indexed codebase: 58 recognized source files, 6223 KB.
- CSS (language) — detected in the code
- FastAPI (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- TypeScript (language) — detected in the code
- Anthropic (technology) — claimed on Devpost, not found in the code
- Hugging Face (technology) — claimed on Devpost, not found in the code
- OpenAI (technology) — claimed on Devpost, not found in the code
- PyTorch (technology) — claimed on Devpost, not found in the code
- AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers

## Codebase structure (from repository index)

### Files (68 of 68)

```
.gitignore
background.js
background.ts
build.js
content.js
contents/CLAUDE.md
contents/treehacks-ui.tsx
contents/ui-state.ts
demo-webllm.js
demo.js
generate-rl-data/consts.py
generate-rl-data/main.py
generate-rl-data/package.json
generate-rl-data/stagehand_script.tsx
generate-rl-data/steps_output.json
generate-rl-data/tsconfig.json
inference/base-predictor.js
inference/browser-action-adapter.js
inference/cache.js
inference/cloud-planner.js
inference/confidence.js
inference/config-loader.js
inference/flash-predictor.js
inference/model-lifecycle.js
inference/ngram-predictor.js
inference/orchestrator.js
inference/queue-controller.js
inference/training-client.js
inference/training-data-collector.js
inference/webllm-predictor.js
model-config.yaml
model-rl/CLAUDE.md
model-rl/const.py
model-rl/data_processing/__init__.py
model-rl/data_processing/data_processor.py
model-rl/data_processing/example_system_prompt.txt
model-rl/fetch_volumes.py
model-rl/main.py
model-rl/requirements.txt
model-rl/trainer/__init__.py
model-rl/trainer/compile_mlc.py
model-rl/trainer/defaults.py
model-rl/trainer/rl_trainer.py
models/action-schema.js
offscreen-src.js
offscreen.html
offscreen.js
offscreen.js.map
package.json
popup.css
popup.html
popup.js
README.md
scripts/check-plasmo-manifest.mjs
test/action-schema.test.js
test/browser-action-adapter.test.js
test/cache.test.js
test/mock-actions.js
test/model-lifecycle.test.js
test/ngram-predictor.test.js
test/offscreen.test.js
test/orchestrator.test.js
test/queue-controller.test.js
test/training-client.test.js
test/training-data-collector.test.js
test/ui-state.test.ts
TRD.md
tsconfig.json
```

### Dependencies

- generate-rl-data/package.json: @browserbasehq/stagehand@^3.0.8, @types/node@^25.2.3, dotenv@^16.6.1, typescript@^5.9.3
- model-rl/requirements.txt: fastapi, modal, pydantic, uvicorn
- package.json: @browserbasehq/stagehand@^3.0.8, @huggingface/transformers@^3.8.1, @mlc-ai/web-llm@^0.2.80, @types/chrome@^0.1.29, @types/react@^18.3.3, @types/react-dom@^18.3.0, esbuild@^0.27.3, js-yaml@^4.1.1, plasmo@^0.90.5, react@^18.3.1, react-dom@^18.3.1, typescript@^5.6.3, vitest@^3.0.0, zod@^4.3.6

### Recent commits (newest first)

- fix
- Add fallback AI API
- Remove extra brackets
- Add configurable user_context system prompt to cloud planner
- Add flash predictor (Haiku) and remove DOM gate from pipeline
- Parallelize prediction pipeline: WebLLM + DOM prefetch, remove dwell timer
- Remove unused agent task execution flow and fix stale WebLLM backlog
- Add max 3 retries per step and verbose content-script logging
- chore: decrease dwell time
- Dwell-time debounce, request cancellation, and pipeline logging
- Read Anthropic API key from .env via esbuild define
- Two-layer prediction: WebLLM task describer → Claude cloud planner → Tab-through
- comment out ngram
- Repurpose WebLLM as task describer + integrate autonomous browser agent
- fix: missing comma
- Enrich predictor context with page title, DOM snapshot, and domain-aware n-grams
- Fix Tab key capture in contenteditable elements (Gmail compose)
- Wire up NgramPredictor as primary predictor with WebLLM fallback
- Merge pull request #8 from csaye/landing-page
- Merge branch 'main' into landing-page

## Key source files (fetched from GitHub, selected and truncated for size)

### TRD.md

```markdown
Technical Requirements Document (TRD)
Tab Automation for Browser Workflows
LinkedIn Recruiter | Demo
Product Overview
Problem: Knowledge workers spend hours copying data between browser tabs and spreadsheets but don't trust full AI automation.

Solution: Chrome extension that learns workflows by observation, suggests automation via Tab key. Like Cursor Tab but for browser actions.

Wedge: LinkedIn → Google Sheets data entry (name, email, school, etc.) → send email → create Google Calendar

ICP: Recruiting coordinators, sales ops, anyone doing repetitive work on software

Distribution: PLG (r/recruiting, r/sales, ProductHunt)

System Architecture
Core Flow:

Extension records actions → IndexedDB storage

→ Modal trains LoRA adapter (when 50+ examples)

→ Local inference (ONNX) uses adapter

→ Tab shows prediction → User executes

Key Components:

Action Recorder (Chrome extension): Captures clicks, inputs, navigation
Pattern Detector: Heuristics + ML prediction
Training Pipeline (Modal): LoRA fine-tuning on Gemma-2-2B
Inference Engine (Local): ONNX Runtime, <100ms latency
UI Layer: Floating notification (bottom-right, Tab/Esc controls)

Technical Details
Action Recording
Captures: DOM events (clicks, inputs, navigation), context (URL, page state, selectors), metadata (timestamps, success signals)

Storage: IndexedDB, encrypted
Pattern Detection & Prediction
Cold start: Heuristic matching (detect 3+ repeated sequences) ML phase: LoRA adapter trained on Modal after 50+ examples Inference: Local ONNX Runtime, <100ms latency
Training (Modal)
Model: Gemma-2-2B (INT4 quantized, ~1-2GB) Method: LoRA fine-tuning (rank 8-16, ~10-50MB adapter) Trigger: 50+ examples OR user idle Time: 30-60 seconds on A10G GPU Cost: ~$0.14/month per user
Inference (Local)
Runtime: ONNX Runtime Web Latency: <100ms total (encode 10ms, inference 80ms, decode 10ms) Confidence threshold: >0.7 to show suggestion
UI Layer
Type: Floating notification (NOT system notifications) Position: Bottom-right, fixed Controls: Tab (execute), Esc (dismiss) States: Compact → Expanded preview → Executing → Success/Feedback

Privacy & Security
All action logs stored locally (IndexedDB, encrypted)
Training data anonymized before Modal upload
No PII collected or sold (only action patterns)
Inference runs on-device (no network calls)
Enterprise: opt-out available

Tech Stack
Extension: Chrome Manifest V3, Vanilla JS/React, IndexedDB Training: Modal, PyTorch, HuggingFace Transformers, PEFT (LoRA) Inference: ONNX Runtime Web, Gemma-2-2B (INT4) Requirements: 8GB+ RAM, 2-3GB storage, Chrome 90+

Performance Targets
Latency: Notification <100ms, Inference <100ms, Action execution <500ms per step Accuracy: >60% acceptance rate after 100 examples Resources: <5% CPU, 2-3GB RAM, minimal battery impact
Differentiation
vs Cursor Tab: Code → Browser actions vs Pointer/Bardeen: Manual setup → Auto-learning vs Computer Use: Full autonomy → Tab-to-preview control

Moat: Personal models, Tab UX (already trained by C
[truncated — 4273 more characters]
```

### model-rl/CLAUDE.md

```markdown
# Model RL - Async RL Training Service

A REST API service for asynchronous reinforcement learning training. Submit your action data, get a job ID back instantly, and retrieve your trained LoRA adapter from Modal storage when ready.

## Quick Start

**1. Deploy the training function to Modal:**
```bash
modal deploy trainer/rl_trainer.py
```

**2. Start the API servers:**
```bash
python main.py           # Training API on port 8000
python fetch_volumes.py  # Adapter fetch API on port 5001
```

**3. Submit a training job:**
```bash
curl -X POST http://localhost:8000/train \
  -H "Content-Type: application/json" \
  -d '{"user_id": "your_user_id", "data": [...]}'
```

**4. Download your trained adapter:**
```bash
curl http://localhost:5001/adapters/your_user_id -o adapter.zip
```

## API Reference

### POST /train

Submits an async training job. Returns immediately with a job ID.

**Request Body:**
```json
{
  "user_id": "string (required)",
  "data": [ActionEntry]
}
```

**Response:**
```json
{
  "status": "started",
  "user_id": "user_001",
  "call_id": "fc-abc123..."
}
```

The `call_id` can be used to track job status via Modal's dashboard.

### GET /adapters/{user_id}

Downloads the trained LoRA adapter for a user as a zip file. Runs on port **5001** (via `fetch_volumes.py`).

**Path Parameters:**
| Parameter | Type | Description |
|-----------|------|-------------|
| `user_id` | string | The user ID used during training |

**Response:**
- **200 OK**: Returns a zip file containing the adapter files (`adapter_config.json`, `adapter_model.safetensors`, etc.)
- **404 Not Found**: No adapter exists for this user

**Example:**
```bash
# Download adapter to a zip file
curl http://localhost:5001/adapters/user_001 -o user_001_adapter.zip

# Unzip the adapter
unzip user_001_adapter.zip -d ./my_adapter
```

**Response Headers:**
```
Content-Type: application/zip
Content-Disposition: attachment; filename=user_001_adapter.zip
```

## Input Schema

### ActionEntry

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `user_id` | string | Yes | Unique identifier for the user |
| `previous_actions` | string[] | Yes | List of previous actions in the workflow (can be empty `[]`) |
| `current_action` | string | Yes | The action that was predicted/suggested |
| `accepted` | boolean | Yes | `true` if user accepted, `false` if rejected |
| `action_instead` | string \| null | No | If rejected, the action user took instead |

### Full Request Example

```json
{
  "user_id": "user_001",
  "data": [
    {
      "user_id": "user_001",
      "previous_actions": [],
      "current_action": "Open LinkedIn",
      "accepted": true,
      "action_instead": null
    },
    {
      "user_id": "user_001",
      "previous_actions": ["Open LinkedIn"],
      "current_action": "Click search",
      "accepted": true,
      "action_instead": null
    },
    {
      "user_id": "user_001",
      "previous_actions": ["Open LinkedIn", "Click search"],
   
[truncated — 1120 more characters]
```

### package.json

```
{
  "name": "treehacks-2026",
  "version": "1.0.0",
  "private": true,
  "type": "module",
  "scripts": {
    "build": "node build.js",
    "check:manifest": "node scripts/check-plasmo-manifest.mjs",
    "plasmo:dev": "plasmo dev",
    "plasmo:build": "npm run build && plasmo build && npm run check:manifest",
    "test": "vitest run",
    "test:watch": "vitest",
    "demo": "node demo.js",
    "demo:llm": "node demo-webllm.js"
  },
  "manifest": {
    "manifest_version": 3,
    "name": "TreeHacks Extension",
    "version": "1.0.0",
    "description": "A Chrome extension that learns browser workflows and suggests automation",
    "action": {
      "default_title": "TreeHacks Extension"
    },
    "permissions": [
      "storage",
      "offscreen",
      "activeTab",
      "webNavigation",
      "tabs"
    ],
    "host_permissions": [
      "<all_urls>"
    ],
    "content_security_policy": {
      "extension_pages": "script-src 'self' 'wasm-unsafe-eval'; connect-src 'self' https://huggingface.co https://cdn-lfs.huggingface.co https://cdn-lfs-us-1.huggingface.co https://raw.githubusercontent.com https://cdn-lfs-us-1.hf.co https://cas-bridge.xethub.hf.co https://*.modal.run https://api.anthropic.com; object-src 'self'"
    },
    "content_scripts": [
      {
        "matches": [
          "<all_urls>"
        ],
        "js": [
          "../content.js"
        ],
        "run_at": "document_idle"
      }
    ],
    "web_accessible_resources": [
      {
        "resources": [
          "offscreen.html",
          "offscreen.js",
          "offscreen.js.map"
        ],
        "matches": [
          "<all_urls>"
        ]
      }
    ]
  },
  "devDependencies": {
    "@huggingface/transformers": "^3.8.1",
    "@types/chrome": "^0.1.29",
    "@types/react": "^18.3.3",
    "@types/react-dom": "^18.3.0",
    "esbuild": "^0.27.3",
    "plasmo": "^0.90.5",
    "typescript": "^5.6.3",
    "vitest": "^3.0.0"
  },
  "dependencies": {
    "@browserbasehq/stagehand": "^3.0.8",
    "@mlc-ai/web-llm": "^0.2.80",
    "js-yaml": "^4.1.1",
    "zod": "^4.3.6",
    "react": "^18.3.1",
    "react-dom": "^18.3.1"
  }
}

```

### model-rl/requirements.txt

```
fastapi
pydantic
modal
uvicorn

```

### generate-rl-data/package.json

```
{
  "name": "generate-rl-data",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "description": "",
  "dependencies": {
    "@browserbasehq/stagehand": "^3.0.8",
    "dotenv": "^16.6.1"
  },
  "devDependencies": {
    "@types/node": "^25.2.3",
    "typescript": "^5.9.3"
  }
}

```

### model-rl/main.py

```python
from fastapi import FastAPI
import uvicorn
import modal

from const import TrainRequest
from data_processing.data_processor import DataProcessor

app = FastAPI()
train = modal.Function.from_name("rl-post-training", "train")

@app.post("/train")
def train_endpoint(request: TrainRequest):
    print(f"[POST /train] user_id={request.user_id}, entries={len(request.data)}")

    processor = DataProcessor()
    processor.process_data([entry.model_dump() for entry in request.data])

    function_call = train.spawn(request.user_id, processor.kto_data)

    print(f"[POST /train] spawned for user_id={request.user_id}, call_id={function_call.object_id}")
    return {"status": "started", "user_id": request.user_id, "call_id": function_call.object_id}


if __name__ == "__main__":
    print("[main] Starting server on 0.0.0.0:8000")
    uvicorn.run(app, host="0.0.0.0", port=8000)

```

### generate-rl-data/main.py

```python
import subprocess
import json
import os
import urllib.request

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
STAGEHAND_SCRIPT = os.path.join(SCRIPT_DIR, "stagehand_script.tsx")

RESULT_MARKER = "__RESULT__"
OUTPUT_FILE = os.path.join(SCRIPT_DIR, "steps_output.json")
TRAIN_URL = "http://localhost:8000/train"


def run_stagehand(prompt: str, start_url: str) -> list[dict] | None:
    """Run a Stagehand task with a single prompt and return structured action events."""
    task = {"prompt": prompt, "startUrl": start_url}

    result = subprocess.run(
        ["npx", "tsx", STAGEHAND_SCRIPT],
        input=json.dumps(task),
        capture_output=True,
        text=True,
        cwd=SCRIPT_DIR,
    )

    for line in result.stdout.splitlines():
        if line.startswith(RESULT_MARKER):
            data = json.loads(line[len(RESULT_MARKER):])
            return data.get("events", [])
        else:
            print(line)

    if result.returncode != 0:
        print(f"Task failed (exit code {result.returncode})")
        if result.stderr:
            print(result.stderr)

    return None


PROMPTS = [
  {"prompt": "Type 'machine learning' in the Google search box and press Enter", "start_url": "https://www.google.com"},
  {"prompt": "Type 'OpenAI' in the Wikipedia search box and press Enter", "start_url": "https://en.wikipedia.org"},
  {"prompt": "Click the 'Evals' button in the top navigation", "start_url": "https://stagehand.dev"},
  {"prompt": "Type 'latest Python release' in the Google search box and press Enter, then click the python.org result", "start_url": "https://www.google.com"},
  {"prompt": "Type 'JavaScript array methods MDN' in the Google search box and press Enter, then open the MDN page", "start_url": "https://www.google.com"},
  {"prompt": "Type 'lofi hip hop' in the YouTube search bar and press Enter, then open the first video", "start_url": "https://www.youtube.com"},
  {"prompt": "Type 'weather in San Francisco' in the Google search box and press Enter", "start_url": "https://www.google.com"},
  {"prompt": "Type 'Artificial intelligence' in the Wikipedia search box and press Enter, then click 'History' in the table of contents", "start_url": "https://en.wikipedia.org"},
  {"prompt": "Type 'GitHub Copilot' in the Google search box and press Enter, then click the official GitHub result", "start_url": "https://www.google.com"},
  {"prompt": "Type 'wireless mouse' in the Amazon search bar and press Enter, then sort by price low to high", "start_url": "https://www.amazon.com"},
  {"prompt": "Type 'notion templates' in the Google search box and press Enter, then open the first notion.so result", "start_url": "https://www.google.com"},
  {"prompt": "Type 'productivity apps' in the Reddit search bar and press Enter, then open the top post in r/productivity", "start_url": "https://www.reddit.com"},
  {"prompt": "Type 'python list comprehension' in the Stack Overflow search box and press Enter, then open the top question", "start_url": "https://stackoverflow.com"},
  {"prompt": "Type 'fetch API' in the MDN search box and press Enter, then open the main documentation page", "start_url": "https://developer.mozilla.org"},
  {"prompt": "Type 'Tesla stock' in the Google search box and press Enter", "start_url": "https://www.google.com"},
  {"prompt": "Type 'Figma' in the Google search box and press Enter, then click the official figma.com result", "start_url": "https://www.google.com"},
  {"prompt": "Type 'Duolingo login' in the Google search box and press Enter, then open the Duolingo login page", "start_url": "https://www.google.com"},
  {"prompt": "Type 'best pasta recipe allrecipes' in the Google search box and press Enter, then open the first Allrecipes result", "start_url": "https://www.google.com"},
  {"prompt": "Type 'New York Times technology' in the New York Times site search and press Enter, then open the Technology section", "start_url": "https://www.nytimes.com"},
  {"prompt": "Type 'awesome-python' in the GitHub search bar and press Enter, then open the top repository result", "start_url": "https://github.com"}
]


def main():
    all_events = []

    for i, p in enumerate(PROMPTS):
        print(f"\n{'='*60}")
        print(f"[{i + 1}/{len(PROMPTS)}] {p['prompt']}")
        print(f"  URL: {p['start_url']}")
        print(f"{'='*60}")

        events = run_stagehand(prompt=p["prompt"], start_url=p["start_url"])

        if events:
            all_events.extend(events)
            print(f"Collected {len(events)} events")
        else:
            print("No events collected")

    # Write events to file
    with open(OUTPUT_FILE, "w") as f:
        json.dump(all_events, f, indent=2)
    print(f"\n{'='*60}")
    print(f"Total events: {len(all_events)}")
    print(f"Saved to {OUTPUT_FILE}")
    print(f"{'='*60}")

    # Call the RL training endpoint
    if all_events:
        payload = json.dumps({"user_id": "base-model", "data": all_events}).encode()
        req = urllib.request.Request(TRAIN_URL, data=payload, headers={"Content-Type": "application/json"})
        try:
            with urllib.request.urlopen(req) as resp:
                print(f"Training response: {resp.read().decode()}")
        except Exception as e:
            print(f"Failed to call training endpoint: {e}")


if __name__ == "__main__":
    main()

```

### background.ts

```typescript
// Plasmo background entrypoint. Reuse existing MV3 background logic.
import "./background.js"

```

### offscreen.html

```html
<!DOCTYPE html>
<html>
<head><title>Inference Offscreen</title></head>
<body>
  <script type="module" src="offscreen.js"></script>
</body>
</html>

```

### build.js

```javascript
import * as esbuild from 'esbuild';
import { config } from 'dotenv';

config();

await esbuild.build({
  entryPoints: ['offscreen-src.js'],
  bundle: true,
  outfile: 'offscreen.js',
  format: 'esm',
  platform: 'browser',
  target: 'chrome120',
  external: ['fs/promises', 'fs', 'path', 'url', 'js-yaml',
             'node:*', 'sharp'],
  define: {
    'process.env.ANTHROPIC_API_KEY': JSON.stringify(process.env.ANTHROPIC_API_KEY || ''),
    'process.env.OPENAI_API_KEY': JSON.stringify(process.env.OPENAI_API_KEY || ''),
  },
  sourcemap: true,
  logLevel: 'info',
});

```

[46 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]