Pull Guard

A maintainer workspace for understanding a busy pull-request queue before it turns into a review bottleneck.

Evaluate it at the live demo: https://pullguard.divagr.com

Source: https://github.com/divagr18/Pull-Guard

Try it first

For evaluation, use pullguard.divagr.com. It is already connected to a prepared demo repository and shows the complete review flow. Local setup requires a GitHub App, OAuth credentials, a database, model access, and disposable runner infrastructure, so it is better suited to development than a quick product evaluation.

Pull Guard captures each PR at an exact base and head commit, reads the changed code and nearby context, groups genuinely related work, and gives maintainers a clear next step. It can run existing repository checks and, where a focused behaviour warrants it, add a private disposable verification layer. It never merges or changes GitHub on its own.

---

The problem

Coding agents made it cheap to generate plausible-looking pull requests in minutes. A repository that once received a handful of thoughtful PRs a week can now receive dozens of shallow, overlapping, or subtly incorrect changes — many describing the same idea, some hiding large unrelated churn, others passing their own tests while failing outside the narrow path the author checked.

The result is not just a bigger queue; it is a collapse in signal. Pull Guard exists to protect maintainer attention: turn a queue of 500 unknown pull requests into five trustworthy decisions.

How it works

Pull Guard operates around three actions.

1. Understand

• Change fingerprinting — fingerprints the actual transformation (files, symbols, imports, behaviour hints, test/protected/generated paths, dependencies, CI state), not just titles.
• Duplicate & flood detection — collapses overlapping submissions into one review target and detects bursts of shallow, highly similar PRs.
• Patch–claim alignment — flags where the patch does not match what the description promises.
• Contribution classification — distinguishes runtime, test-only, configuration, documentation, and extension work so expensive checks are used where they help.

2. Review and validate

• Source-grounded code review — traces changed symbols, call sites, tests, manifests, and CI configuration before retaining a finding.
• Repository validation — selects relevant existing tests, lint, type, or build checks and records what actually ran.
• Claim extraction — identifies a bounded observable behaviour only when a private proof would add useful information.
• Adversarial test generation — generates hidden tests designed to expose incomplete or misleading implementations (boundary, concurrency, malformed input, backwards-compatibility, and more).
• Self-verification — every generated test is checked against a clean baseline, re-run for reproducibility, validated against a deliberate regression (mutation), checked for isolation, and independently reviewed.
• Evidence grading — tests are graded decisive, supporting, uncertain, or invalid. Only validated evidence can justify blocking a PR.
• Proof matrix — three fresh runs each for the base revision, the PR, and an intentionally broken copy, so a maintainer can see the test separates them.

3. Compile

• Queue-level analysis — detects dependencies, semantic conflicts Git cannot see, and changes made redundant by stronger submissions.
• Merge readiness — an evidence-led ledger (behaviour proof, patch alignment, CI, test coverage, change scope) with a plain-language recommendation.
• Combined-change checks — maintainers can test selected immutable PR versions together before considering a composite change.

Decision buckets

Each pull request lands in one maintainer-facing bucket, always with supporting evidence:

Bucket	Meaning
review_first	Worth human review ahead of the rest of the queue.
safe_to_merge	Evidence supports merging; the maintainer still decides.
request_changes	A validated problem needs fixing first.
close_superseded	An equivalent or stronger change already exists.
low_value_or_policy_violating	Noise, churn, or a contribution-rule violation.
human_judgment	A product decision only a maintainer can make.

Key features

• GitHub App intake — webhook ingestion with signature validation and duplicate-delivery protection; immutable PR versions bound to exact base/head SHAs (force-pushed versions become stale rather than silently changing evidence).
• Maintainer sign-in — GitHub OAuth, encrypted server-side token storage, and repository permission checks.
• Conservative related-work clustering — exact-patch duplicates are authoritative; optional semantic embedding retrieval precedes structural selection.
• Secure execution — rootless, network-disabled verification containers with read-only root, dropped capabilities, no-new-privileges, bounded resources, and redacted output; SHA-verified artifacts.
• Disposable Compute Engine runners (optional) — untrusted code runs on short-lived VMs that receive only scoped signed URLs; no GitHub, model, database, or ambient cloud credentials ever reach repository code.
• Durable audit trail — evidence, runner, model-invocation, and audit records are persisted.

See FEATURES.md for the available and planned capabilities.

Built with Codex and GPT-5.6 Luna

Codex accelerated Pull Guard's build substantially. It was used to explore the early codebase, turn a hackathon prototype into a deployable control plane, and iterate quickly on the product, runner, and maintainer experience. It was also especially effective at the otherwise slow operational work: shaping Cloud Run services and jobs, Cloud Build images, Cloud SQL migrations, Secret Manager configuration, runner artifacts, deployment scripts, and production debugging. That let the project move from local experiments to a live hosted demo without turning deployment work into a separate multi-day project.

In the product itself, GPT-5.6 Luna is the core intelligence layer. Luna performs the source-grounded code exploration, review and critique passes, then helps derive focused validation and private-proof candidates from the immutable PR context. Luna has been a strong fit for this workload: it provides the depth needed to follow unfamiliar code and reason across a diff, while keeping the price-to-performance practical enough to analyse a real queue rather than only hand-picked pull requests. Its output stays advisory: Pull Guard retains source citations, separates model analysis from executed evidence, and leaves GitHub decisions with the maintainer.

Roadmap: agent-facing CLI

The web workspace is the current maintainer surface. Next, Pull Guard will wrap the same immutable evidence and queue APIs in a CLI so coding agents and automation can inspect a repository, ask for the next review decision, retrieve evidence packets, and request a reanalysis without driving a browser. The CLI will preserve the same safety boundary: it can read, prepare, and explain; any GitHub write remains an explicit maintainer-authorized action.

Architecture

                 ┌────────────────────────────┐
 GitHub App ───▶ │  FastAPI control plane      │ ───▶ React (Vite) frontend
 webhooks/OAuth  │  (pullguard/api)            │      (served from frontend_dist)
                 └──────────────┬─────────────┘
                                │ SQLAlchemy
                 ┌──────────────▼─────────────┐
                 │  PostgreSQL + pgvector      │
                 └──────────────┬─────────────┘
                                │ queued work
                 ┌──────────────▼─────────────┐
                 │  Analysis worker            │ ───▶ Model stages (claims, tests,
                 │  (pullguard.orchestrator)   │      review) + disposable runners
                 └─────────────────────────────┘

• Control plane — FastAPI app (pullguard/api) serving the REST API and the built single-page frontend.
• Worker — python -m pullguard.orchestrator.worker; processes queued model and runner stages.
• Database — PostgreSQL with pgvector for semantic embeddings; schema managed by Alembic (alembic/).
• Runners — local Docker executor for development; optional hardened Compute Engine VMs for production (see RUNNER_OPERATIONS.md).

Quick start (local)

Prerequisites: Docker (with Compose) for the full stack, or uv + Node 22 for running services manually.

Option A — full stack with Docker Compose

cp .env.example .env       # fill in the required GitHub values
docker compose up --build

This starts Postgres, runs migrations, and launches the API and worker. The app is served at http://localhost:8000 (the API serves the built frontend).

Option B — run services manually

# Backend
uv sync
uv run alembic upgrade head
uv run uvicorn pullguard.api.main:app --host 0.0.0.0 --port 8000

# Frontend dev server (proxies /api/v1 and /auth to http://127.0.0.1:8000)
cd frontend
npm install
npm run dev          # http://localhost:5173

Set PULL_GUARD_API_ORIGIN to point the Vite dev proxy at a different control plane origin.

Deployment

Production runs on Google Cloud Run (Cloud SQL for Postgres, Artifact Registry for images, Secret Manager for secrets). See DEPLOYMENT.md for the full guide, including one-time GCP setup, the deploy script, database migrations, and the optional disposable Compute Engine runner.

The public instance of this repository is deployed at https://pullguard.divagr.com.

Configuration

Configuration is environment-driven (see .env.example). The most important variables:

Variable	Purpose
PULL_GUARD_DATABASE_URL	PostgreSQL connection string.
PULL_GUARD_GITHUB_APP_ID / ..._PRIVATE_KEY	GitHub App identity for webhooks.
PULL_GUARD_GITHUB_WEBHOOK_SECRET	Validates incoming webhook signatures.
PULL_GUARD_GITHUB_OAUTH_CLIENT_ID / ..._SECRET	Maintainer OAuth sign-in.
PULL_GUARD_SESSION_SECRET	Derives at-rest OAuth-token encryption (≥32 chars).
PULL_GUARD_OPENAI_API_KEY	Model stages (claims, tests, review).
PULL_GUARD_RUNNER_BACKEND	local (default) or compute for disposable VMs.

Development & tests

# Backend tests
uv run pytest                      # or: python -m unittest discover -s tests

# Frontend checks
cd frontend
npm run lint
npm run build

Project layout

pullguard/            FastAPI control plane, worker, and analysis pipeline
  api/                HTTP API + response contracts
  db/                 SQLAlchemy models and session helpers
  orchestrator/       Worker, queue, and outbox
  runner/             Disposable runner integration
  github/             GitHub App, OAuth, and ingestion
alembic/              Database migrations
frontend/             React + Vite single-page app
scripts/              Deployment and operations scripts (see scripts/README.md)
triage.py             Legacy standalone CLI

The deployed product (this repository's pullguard/ control plane and frontend/) supersedes the CLI for live, multi-repository use.

Safety & limits

• Advisory by default — Pull Guard never approves, merges, closes, or comments on a PR automatically. Write-side actions require explicit, typed maintainer approval and are fully logged.
• Untrusted code is sandboxed — generated tests and PR code run in disposable, network-disabled, resource-bounded environments that never receive repository, cloud, or model credentials.
• Missing evidence stays visible — an incomplete check is shown as incomplete, never fabricated as passing.

Contributing and licence

Pull Guard is released under the MIT License. Contributions are welcome; see CONTRIBUTING.md. For sensitive reports, follow SECURITY.md.