# Project export: escrow This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: OpenAI Build Week - Tagline: AI interprets your repo instructions. Deterministic evidence decides if they're still true - Devpost: https://devpost.com/software/escrow-t5g129 - GitHub: https://github.com/PlutonicSauce/escrow - Video: https://www.youtube.com/embed/kH4J-5D5FQ4?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — Seshank Mallireddigari (22 commits), ukkandi (9 commits) ## Devpost submission (written by the team) ### Overview Escrow Coding agents increasingly rely on AGENTS.md and other repository instructions. The problem is that those files can quietly become wrong: a project moves from npm to pnpm, a test script gets renamed, documentation disappears, or a dependency changes. Escrow makes those instructions testable. It reads repository guidance, turns it into structured claims, and checks those claims against the actual codebase. It can verify package managers, package scripts, dependencies, paths, nested instruction scopes, and explicitly approved commands. Our core idea is simple: AI interprets the instruction. Deterministic repository evidence decides whether it is true. What inspired us We kept seeing the same failure mode with coding agents: the repository’s instructions sounded confident, but they were stale. An agent following outdated guidance can use the wrong package manager, run a nonexistent command, or make incorrect assumptions before it even begins real work. We wanted a tool that treats agent instructions like something worth testing, not just documentation that developers hope stays accurate. How it works Escrow finds the effective AGENTS.md and AGENTS.override.md files for a target directory, including nested scopes. GPT-5.6 is used where language understanding is useful: extracting candidate claims from natural-language instructions and proposing restricted documentation repairs. But GPT-5.6 does not decide whether a claim passes or fails. Escrow’s TypeScript validators compare claims with deterministic repository evidence: lockfiles and package.json metadata for package-manager claims nearest package.json scripts for documented commands dependency metadata for tool and framework claims Git-root-bounded path resolution for referenced files temporary Git worktrees for approved command execution scope resolution for nested instruction overrides Escrow produces console, JSON, Markdown, HTML, browser UI, and GitHub Actions reports. It can also preview a restricted repair, but only permits changes to instruction files and rejects source code, tests, lockfiles, build files, and CI changes. ### How we built it Escrow is a TypeScript and Node.js CLI with a local browser interface and a GitHub Actions integration. We used Codex throughout development to plan the architecture, implement features, write and review tests, debug workflow issues, document the project, and integrate the GitHub Action. At runtime, GPT 5.6 extracts structured instruction claims and can propose narrowly constrained repairs. For a no-cost self-hosted GitHub Actions demo, Escrow can also use local Ollama models while preserving the same deterministic validation layer. Challenges we faced The hardest part was drawing a real trust boundary around AI. It would have been easy to ask a model whether an instruction “looks correct.” But that is not enough for a tool that agents and teams may rely on. We redesigned the flow so the model only interprets language; deterministic code makes every final verdict and records the evidence behind it. We also had to make command verification safe. Escrow never runs documented commands in the active repository. It uses temporary Git worktrees, blocks unsafe commands, and disables network-capable commands by default. Finally, smaller local models can extract inconsistent metadata. We improved the pipeline so explicit instructions are normalized against their original source text before validation, preventing model interpretation mistakes from becoming misleading results. ### What we learned We learned that the best use of AI in developer tooling is often not replacing verification, but making unstructured human language usable by reliable systems. Escrow is not “AI that reads documentation.” It is repository instruction integrity infrastructure: AI helps understand what a repository claims, and deterministic evidence establishes whether that claim is true. ## README (from the GitHub repository) # Escrow [![CI](https://github.com/PlutonicSauce/escrow/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/PlutonicSauce/escrow/actions/workflows/ci.yml?query=branch%3Amain) > Executable tests for the instructions coding agents rely on. Escrow verifies whether `AGENTS.md` and `AGENTS.override.md` still match the repository they describe. It uses Codex to extract structured candidate claims, then assigns every status through deterministic TypeScript validators. It can optionally execute documented commands in isolated Git worktrees, produce four evidence-report formats, and preview or apply restricted repairs to instruction files only. ## Why Escrow is different Most instruction-enforcement tools ask whether an agent followed its instructions. Escrow asks whether the instructions themselves still match repository reality. It combines deterministic repository evidence, nested instruction scope, opt-in command verification in isolated Git worktrees, and restricted instruction-only repair without claiming to be the first or only tool in this category. ## Built with Codex and GPT-5.6 during OpenAI Build Week The tracked implementation history begins with commits on July 13, 2026, during the July 13–21 OpenAI Build Week submission period. The repository then records validation tests, architecture and safety iteration, the local UI, demo/onboarding work, GitHub Actions integration, and follow-up reliability fixes. See the [dated implementation history](IMPLEMENTATION.md) and the repository history beginning July 13 (`git log --since=2026-07-13`). This evidence establishes when tracked implementation began; it does not by itself prove that every idea or artifact was created from scratch during the event. ### Built with Codex The implementation record documents Codex-assisted architecture planning, focused TypeScript implementation and iteration, fixture and test generation, security reviews, failure debugging, documentation, onboarding, demo preparation, and local UI integration. Codex accelerated the build loop while the repository's tests and deterministic code remained the acceptance authority. - **TODO — team confirmation:** Confirm whether any prototype, design, or code existed before the first tracked commit on July 13, 2026. - **TODO — team confirmation:** Add a short personal account of the most useful Codex-assisted development moment, if desired for the submission narrative. ### Key human decisions - Codex and GPT-5.6 may interpret natural-language claims and propose minimal instruction repairs; they never assign final verdicts. - Deterministic TypeScript validators assign every passed, failed, warning, blocked, inconclusive, advisory, and overridden status. - Documented commands run only after explicit opt-in and only inside temporary Git worktrees, never in the active checkout. - Repair proposals are restricted to effective `AGENTS.md` and `AGENTS.override.md` files. - Source code, tests, package/build configuration, CI files, lockfiles, and unrelated files cannot be repaired. ### Codex at runtime This is separate from using Codex to build Escrow. At runtime, Escrow invokes Codex in a read-only sandbox for two narrow tasks: schema-constrained claim extraction with source locations, and the smallest truthful instruction-file repair proposal from deterministic failures and evidence. The OpenAI Build Week judge workflow selects `gpt-5.6-luna` because Escrow's primary model task is structured, repeatable extraction and classification. Zod validates model output again; deterministic code hydrates repository evidence, validates claims, calculates totals, enforces command policy, and accepts or rejects repairs. See the [architecture and trust boundaries](docs/architecture.md). ### Judge verification The fastest path is the [Judge Quick Test](#judge-quick-test) below. The [three-minute walkthrough](docs/demo-script.md) shows the intended scan, four deterministic failures, restricted repair preview, revalidation, and static HTML report. Run it from this checkout on macOS or Linux with Node.js 20+, Git, npm, and an installed, authenticated Codex CLI with access to the selected model. ## Supported claim types | Claim type | Deterministic check | | --- | --- | | `path_exists` | A referenced file or directory exists inside the repository. | | `package_manager` | npm, pnpm, or Yarn guidance matches lockfiles and `packageManager` metadata. | | `package_script` | A documented npm, pnpm, or Yarn script exists in the nearest `package.json`. | | `dependency_present` | A supported framework/tool is declared in a dependency section. | | `command_runs` | With `--execute`, the documented command passes inside a temporary Git worktree. | | `advisory` | Non-verifiable guidance is preserved but never counted as passed or failed. | Nested instruction files are resolved from the Git root toward the selected target. A non-empty `AGENTS.override.md` wins over `AGENTS.md` only in the same directory, and valid nested overrides do not become false conflicts. ## Requirements - Node.js 20 or newer - Git - npm for installing and building Escrow - macOS or Linux - Codex CLI installed, authenticated, and available as `codex` Escrow invokes Codex non-interactively with `codex exec`. Verify the prerequisite with: ```bash codex --version codex login status ``` If needed, authenticate interactively with `codex login`, or use the API-key login described in the official [Codex authentication documentation](https://learn.chatgpt.com/docs/auth). See the official [Codex CLI documentation](https://learn.chatgpt.com/docs/codex/cli) and [non-interactive mode documentation](https://learn.chatgpt.com/docs/non-interactive-mode) for installation and automation details. ## Install from a checkout ```bash npm ci npm run build node dist/index.js --help ``` Optionally expose the local binary through npm: ```bash npm link escrow --help ``` No OpenAI SDK or application API key is read directly by Escrow. Codex CLI owns authentication. The CLI may use a saved ChatGPT login or an API-key login, subject to the account and workspace configuration. For a future tagged package, see [judge installation from a GitHub Release](docs/judge-installation.md). This repository does not claim that a GitHub Release currently exists. ## Judge Quick Test Supported on macOS or Linux. From the Escrow checkout, use Node.js 20+, Git, npm, and an installed/authenticated Codex CLI. The model must be available to the authenticated account; override `ESCROW_DEMO_MODEL` when needed. Copy and run this block. The reset command creates a committed, ignored demo repository under `.escrow-demo/`, so the tracked fixture and Escrow checkout remain unchanged: ```bash codex --version codex login status npm ci npm run build npm link npm run demo:reset escrow ui .escrow-demo/sample-monorepo \ --model "${ESCROW_DEMO_MODEL:-gpt-5.6-luna}" --execute ``` Expected output: - The terminal prints a loopback `http://127.0.0.1:` URL. - **Scan instructions** shows exactly four failures: package manager, deleted path, missing script, and outdated Jest guidance. The safe health command passes in an isolated worktree. - Claim and instruction locations are repository-relative. Advisory cards are hidden initially but remain available through **Advisory** or **Show all**. - **Download JSON**, **Download Markdown**, and **Download HTML** use the same report and totals. - **Preview instruction repair** shows an `AGENTS.md`-only diff. Preview leaves `.escrow-demo/sample-monorepo` clean. **Revalidate** displays `No broken instructions were found.` from the verified repair worktree. - An explicit confirmed apply may change only the disposable demo's `AGENTS.md`. Run `npm run demo:reset` to restore the broken state. Codex output can vary. If preview or apply rejects malformed output, confirm `git status --short` is empty and rerun that same command, or set `ESCROW_DEMO_MODEL [README truncated for size] ## Detected evidence (automated analysis) Indexed codebase: 189 recognized source files, 927 KB. - HTML (language) — detected in the code - Next.js (technology) — detected in the code - React (technology) — detected in the code - TypeScript (language) — detected in the code - CSS (language) — claimed on Devpost, not found in the code - Node.js (technology) — claimed on Devpost, not found in the code - Ollama (technology) — claimed on Devpost, not found in the code - OpenAI (technology) — claimed on Devpost, not found in the code - AI coding agent: Codex — evidence: config files committed to the repository ## Codebase structure (from repository index) ### Files (120 of 234) ``` .github/workflows/ci.yml .github/workflows/escrow-ollama.yml .github/workflows/escrow.yml .github/workflows/release.yml .gitignore action.yml AGENTS.md demo/dangerous-command-fixture/AGENTS.md demo/dangerous-command-fixture/README.md demo/README.md demo/sample-monorepo/AGENTS.md demo/sample-monorepo/docs/architecture.md demo/sample-monorepo/package.json demo/sample-monorepo/packages/api/AGENTS.override.md demo/sample-monorepo/packages/api/package.json demo/sample-monorepo/packages/api/src/index.mjs demo/sample-monorepo/pnpm-workspace.yaml demo/sample-monorepo/scripts/healthcheck.mjs demo/sample-monorepo/scripts/unit-test.mjs demo/sample-reports/broken-console.txt demo/sample-reports/broken-report.html demo/sample-reports/broken-report.json demo/sample-reports/broken-report.md docs/architecture.md docs/case-study.md docs/demo-script.md docs/devpost-submission.md docs/github-actions.md docs/judge-installation.md IMPLEMENTATION.md LICENSE package.json PLAN.md README.md schemas/claims.schema.json schemas/repair.schema.json scripts/package-smoke.mjs scripts/render-ci-summary.mjs scripts/reset-demo.mjs scripts/verify-ci-report.mjs site/index.html site/server.mjs SPEC.md src/cli.ts src/commands/check.ts src/commands/fix.ts src/commands/init.ts src/commands/ui.ts src/discovery/buildInstructionChain.ts src/discovery/discoverInstructions.ts src/discovery/findGitRoot.ts src/execution/cleanupWorktree.ts src/execution/commandPolicy.ts src/execution/createWorktree.ts src/execution/executeCommand.ts src/execution/gitAttributes.ts src/execution/gitEnvironment.ts src/execution/processRunner.ts src/extraction/claimSchema.ts src/extraction/codexClient.ts src/extraction/extractClaims.ts src/extraction/extractionPrompt.ts src/extraction/pathClaimIntent.ts src/index.ts src/models/claims.ts src/models/conflicts.ts src/models/instructions.ts src/models/reports.ts src/repair/generateRepair.ts src/repair/repairPrompt.ts src/repair/verifyRepair.ts src/reporting/consoleReporter.ts src/reporting/displayPaths.ts src/reporting/htmlReporter.ts src/reporting/jsonReporter.ts src/reporting/markdownReporter.ts src/reporting/reportFormatting.ts src/utils/errors.ts src/utils/packageCommands.ts src/utils/paths.ts src/validation/commandValidator.ts src/validation/conflictValidator.ts src/validation/dependencyMappings.ts src/validation/dependencyValidator.ts src/validation/packageManagerValidator.ts src/validation/packageScriptValidator.ts src/validation/pathValidator.ts src/validation/validateClaim.ts src/version.ts src/web/assets.ts src/web/claimFilters.ts src/web/openBrowser.ts src/web/server.ts test/fixtures/command-execution/repository/AGENTS.md test/fixtures/command-execution/repository/packages/api/AGENTS.md test/fixtures/command-execution/repository/packages/api/scope.txt test/fixtures/command-execution/repository/tracked.txt test/fixtures/dependencies/all-mappings/AGENTS.md test/fixtures/dependencies/all-mappings/package.json test/fixtures/dependencies/malformed-entry/AGENTS.md test/fixtures/dependencies/malformed-entry/package.json test/fixtures/dependencies/malformed-json/AGENTS.md test/fixtures/dependencies/malformed-json/package.json test/fixtures/dependencies/malformed-section/AGENTS.md test/fixtures/dependencies/malformed-section/package.json test/fixtures/dependencies/missing/AGENTS.md test/fixtures/dependencies/missing/package.json test/fixtures/dependencies/nested/AGENTS.md test/fixtures/dependencies/nested/package.json test/fixtures/dependencies/nested/packages/api/AGENTS.md test/fixtures/dependencies/nested/packages/api/package.json test/fixtures/dependencies/no-package/AGENTS.md test/fixtures/dependencies/playwright-package/AGENTS.md test/fixtures/dependencies/playwright-package/package.json test/fixtures/dependencies/sections/AGENTS.md test/fixtures/dependencies/sections/package.json test/fixtures/discovery/empty-instructions/AGENTS.md test/fixtures/discovery/empty-instructions/AGENTS.override.md test/fixtures/discovery/empty-instructions/packages/api/AGENTS.md test/fixtures/discovery/empty-override-fallback/AGENTS.md [114 more files omitted for size] ``` ### Dependencies - demo/sample-monorepo/package.json: vitest@^4.0.0 - package.json: @types/node@^24.0.0, commander@^14.0.0, typescript@^5.9.0, vitest@^4.0.0, zod@^4.4.3 - test/fixtures/dependencies/all-mappings/package.json: @playwright/test@^1.50.0, eslint@^9.0.0, jest@^30.0.0, next@^15.0.0, prettier@^3.0.0, react@^19.0.0, typescript@^5.9.0, vite@^7.0.0, vitest@^4.0.0, zod@^4.0.0 - test/fixtures/dependencies/nested/package.json: react@^19.0.0 - test/fixtures/dependencies/nested/packages/api/package.json: vitest@^4.0.0, zod@^4.0.0 - test/fixtures/dependencies/playwright-package/package.json: playwright@^1.50.0 - test/fixtures/dependencies/sections/package.json: eslint@1.0.0, jest@1.0.0, typescript@1.0.0, vitest@1.0.0 ### Recent commits (newest first) - Align demo instructions with pnpm workspace - Normalize unambiguous extracted claim types - Canonicalize known dependency claim metadata - Ground package script claims in instruction text - UI change aligned with current developer tools - Fix legacy naming in Escrow documentation - Prepare Escrow for OpenAI Build Week submission - Clarify decisive CI validation evidence - Fail CI when claim extraction is incomplete - Ignore invalid local model line ranges - Tolerate blank local model claims - Target demo repository in Ollama workflow - Add self-hosted Ollama workflow - Add local Ollama provider mode - Upload Escrow hidden report artifacts - Create Codex home in CI action - Fix CI lockfile for Node 20 - Complete generated Escrow workflow - Add Escrow GitHub Actions integration - Fix duplicate Escrow model environment variable ## Key source files (fetched from GitHub, selected and truncated for size) ### AGENTS.md ```markdown # Escrow Engineering Rules ## Project purpose Escrow is an executable verification tool for `AGENTS.md` and `AGENTS.override.md`. It extracts verifiable repository instructions, validates them against the actual repository, optionally executes documented commands in an isolated Git worktree, and produces evidence reports. ## Required reading Before making changes: 1. Read `SPEC.md`. 2. Read `PLAN.md`. 3. Implement only the milestone explicitly requested by the user. 4. Review existing tests before changing architecture. ## Scope rules - Do not implement future milestones early. - Do not expand the product beyond `SPEC.md`. - Do not add React, a database, authentication, cloud hosting, GitHub integration, GitLab integration, or a team dashboard. - Prefer small, direct modules over abstract frameworks. - Avoid unnecessary dependencies. - Keep the MVP focused on validating the truthfulness of coding-agent instructions. ## Validation rules - GPT-5.6 or Codex may extract candidate claims from natural language. - AI must never assign final pass, fail, warning, blocked, or inconclusive statuses. - All verdicts must come from deterministic code. - Every extracted claim must preserve its source file and line numbers. - Every failed result must include repository evidence. - Advisory instructions must never be counted as passed or failed. - Nested instructions are not automatically conflicts. - More specific instruction files may override broader instructions. ## Safety rules - Never execute documented commands in the active working tree. - Use a temporary Git worktree for command execution. - Never run dangerous commands. - Never access credentials or sensitive user directories. - Network access is disabled by default. - Never modify application source code during repair mode. - Repair mode may modify only: - `AGENTS.md` - `AGENTS.override.md` - nested files with those exact names - Never silently apply an AI-generated patch. - Never commit, push, delete branches, force-reset, or clean the user repository. ## Engineering workflow For every milestone: 1. Inspect the current implementation. 2. State the files expected to change. 3. Implement only the requested milestone. 4. Add or update focused tests. 5. Run: - type checking - unit tests - relevant integration tests 6. Fix failures before stopping. 7. Update `PLAN.md` milestone status. 8. Update `IMPLEMENTATION.md` with: - completed work - files changed - commands run - test results - known limitations 9. Summarize completed work and remaining work. ## Code quality - Use TypeScript strict mode. - Use explicit types at module boundaries. - Validate external and AI-generated data with Zod. - Keep subprocess execution isolated behind a small interface. - Keep report generation separate from validation. - Keep claim extraction separate from deterministic validation. - Prefer pure functions for discovery, parsing, and validation. - Include actionable error me [truncated — 387 more characters] ``` ### SPEC.md ```markdown # Escrow Product Specification ## 1. Product summary Escrow is an executable verification tool for repository instructions used by coding agents. Coding agents rely on files such as `AGENTS.md` and `AGENTS.override.md`. Those files may contain setup commands, test commands, package-manager rules, framework claims, and referenced documentation. As repositories evolve, those instructions can become stale. Escrow converts verifiable instructions into structured claims, validates them against the repository, optionally executes documented commands inside an isolated Git worktree, and generates evidence reports. ## 2. Tagline > Executable tests for the instructions coding agents rely on. ## 3. Category Developer Tools ## 4. Core principle AI may identify and explain candidate claims, but AI must not decide whether a claim passed. All final statuses must come from deterministic validation. ## 5. Primary user story As a developer using Codex or another coding agent, I want to verify that my repository instructions still match the actual project so the agent does not follow missing files, outdated commands, incorrect package-manager guidance, or stale framework information. ## 6. MVP scope The MVP supports: - local Git repositories - macOS and Linux - `AGENTS.md` - `AGENTS.override.md` - root and nested instruction files - JavaScript and TypeScript repositories - npm, pnpm, and Yarn - static verification - optional documented-command execution - Codex-assisted claim extraction - Codex-assisted repair suggestions - console, JSON, Markdown, and static HTML reports - a loopback-only local browser interface ## 7. Supported claim types ```ts type ClaimType = | "path_exists" | "package_manager" | "package_script" | "dependency_present" | "command_runs" | "advisory"; ``` The MVP verifies only: 1. referenced paths exist 2. declared package manager matches repository evidence 3. referenced package scripts exist 4. declared tools or frameworks exist in dependencies 5. documented shell commands execute successfully 6. nested instruction scope is interpreted correctly 7. overrides are distinguished from genuine contradictions ## 8. Non-goals Do not implement: - GitHub or GitLab integration - pull-request comments - hosted SaaS - user accounts - team dashboards - repository indexing - long-term memory - general documentation linting - general code review - vulnerability scanning - support for every programming language - automatic source-code repairs - automatic commits or pushes - React A self-contained static HTML report is sufficient. ## 9. Example problem Instruction file: ```md # AGENTS.md - Install dependencies with `npm install`. - Run tests with `npm test`. - Use Jest for unit tests. - Read `docs/API_ARCHITECTURE.md` before changing API routes. ``` Repository evidence: ```text pnpm-lock.yaml package.json contains "test:unit" but no "test" package.json contains Vitest but not Jest docs/API_ARCHITECTURE.md does not exis [truncated — 14697 more characters] ``` ### package.json ``` { "name": "escrow", "version": "0.1.0", "description": "Executable verification for coding-agent repository instructions.", "license": "MIT", "repository": { "type": "git", "url": "git+https://github.com/PlutonicSauce/escrow.git" }, "homepage": "https://github.com/PlutonicSauce/escrow#readme", "bugs": { "url": "https://github.com/PlutonicSauce/escrow/issues" }, "type": "module", "bin": { "escrow": "dist/index.js" }, "files": [ "dist", "schemas", "README.md", "LICENSE" ], "scripts": { "build": "tsc --project tsconfig.json", "test": "vitest run", "test:codex-integration": "vitest run --config vitest.manual.config.ts", "prepack": "npm run build", "package:smoke": "node scripts/package-smoke.mjs", "site": "node site/server.mjs", "typecheck": "tsc --project tsconfig.json --noEmit", "demo:reset": "node scripts/reset-demo.mjs" }, "engines": { "node": ">=20" }, "dependencies": { "commander": "^14.0.0", "zod": "^4.4.3" }, "devDependencies": { "@types/node": "^24.0.0", "typescript": "^5.9.0", "vitest": "^4.0.0" } } ``` ### demo/sample-monorepo/package.json ``` { "name": "escrow-sample-monorepo", "private": true, "packageManager": "pnpm@10.0.0", "scripts": { "test:unit": "node scripts/unit-test.mjs" }, "devDependencies": { "vitest": "^4.0.0" } } ``` ### test/fixtures/package-scripts/malformed-package-json/package.json ``` {"scripts": ``` ### test/fixtures/package-managers/malformed-package-json/package.json ``` {"packageManager": ``` ### test/fixtures/package-managers/malformed-with-lockfile/package.json ``` {"packageManager": ``` ### test/fixtures/dependencies/missing/package.json ``` { "dependencies": {} } ``` ### test/fixtures/package-scripts/malformed-scripts/package.json ``` { "scripts": ["test"] } ``` ### test/fixtures/package-scripts/no-scripts/package.json ``` { "name": "no-scripts" } ``` [197 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]