# Project export: Ramp This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: Ramp finds accessibility issues axe misses, fixes the code with Claude, and opens a merge-ready PR. - Devpost: https://devpost.com/software/ramp-2igw7p - GitHub: https://github.com/yangzhang75/Ramp - Video: https://www.youtube.com/embed/h6nY35pDjRk?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — yangzhang75 (46 commits), kz930 (23 commits) ## Devpost submission (written by the team) ### Inspiration 1.3 billion people live with a disability. Whether they can use a website comes down to whether its code was written correctly — and the reality is that most of the web is broken for them. Over 96% of the top one million homepages have detectable accessibility failures. Existing tools weren't built to close this gap. Industry standards like axe and Lighthouse can tell you whether an attribute exists — but they're blind to whether it's meaningful, and they only ever hand you a report. An image with alt="DSC_1042.JPG" passes axe cleanly, yet to a screen-reader user that filename means nothing. Ramp was built to catch what standard tools can't see — and to actually fix it. ### What it does Ramp is an end-to-end accessibility platform that audits a rendered web page, scores its compliance, and automatically fixes the code and opens a merge-ready pull request — dropping straight into the GitHub workflow teams already use. Where existing tools stop at mechanical checks and static reports, Ramp pairs a real benchmark with an agent harness and an autonomous fix loop that turns evaluation into action. It has three core capabilities: Semantic judgment axe cannot do. Across three realistic pages, axe-core reported 0 issues — everything passed. Ramp found 12 semantic issues axe let through silently: a product image labeled DSC_1042.JPG, a button labeled "button", and a link reading "click here". Only a model that understands language can judge whether an accessible name is meaningful. Detection that depends on rendering. Many accessibility failures only appear once a page is actually rendered. Color contrast, for example, lives in external CSS — a model reading raw HTML can't see it. On a contrast-focused evaluation suite, the same model (GPT-4o-mini) reading only the HTML caught 40% of issues; equipped with Ramp's harness — which renders the page and measures contrast for real — it caught 92%. Across the full 18-page suite, the harness wins on both recall (84.5%) and precision (89.3%). This is the core thesis: accessibility lives in what the user experiences, not just in the source code. An autonomous fix loop. Ramp doesn't stop at flagging issues — it fixes them. It takes a real open-source project, edits the code with Claude Code, re-verifies the result with axe, and opens a real pull request — raising compliance from 92 to 100, with axe violations going from 2 to 0. We don't send a report. We send the fix. How Claude Code powered Ramp Claude Code wasn't just a tool we integrated — it was the force multiplier that made the project possible within a hackathon timeframe. It contributed in three compounding ways. Claude Code is the agent at the heart of our fix loop. When the audit surfaces a finding, Claude Code edits the source, makes a minimal targeted change, and self-verifies — turning a detected issue into real, working code. The image alt text DSC_1042.JPG becomes "Aero Runner shoe"; the unlabeled icon button receives aria-label="Search". The output isn't an alert — it's a merge-ready pull request, indistinguishable from a human-authored accessibility fix. Claude Code built the system alongside us. The harness, control plane, sandbox checkout, fixer, and dashboard came together during the hackathon because Claude Code worked as a collaborator. We developed parallel slices while focusing on architecture and scoring, giving the codebase the surface area of a much larger team's project. Claude Code makes semantic fixes possible. Detecting that alt="image" is meaningless is only half the problem. Deriving the right replacement from the surrounding context — and applying it as a clean diff — is what closes the loop. Without an agent capable of long-horizon, context-aware editing, this system wouldn't exist. ### How we built it Ramp is a TypeScript monorepo that connects model providers, sandboxed execution, accessibility tooling, and GitHub workflows through one control plane. Auditing: Playwright and axe-core, combined with a rendered accessibility tree, screen-reader serialization, and real contrast measurement. Multi-provider model layer: The Vercel AI SDK, allowing models to be evaluated on identical tasks. The fix loop: Sandboxed checkout → Claude Code headless edits → axe re-verification → real pull request through Octokit. Benchmark and scoring: A11y-Bench tasks plus a hand-built HTML-live evaluation suite, scored deterministically. Observability: The complete fix loop is instrumented with Sentry, with every audit, fix, and pull request represented as a traced span. Dashboard: React, Vite, and Tailwind, showing axe-versus-Ramp comparisons and before-and-after compliance scores. ### Challenges we ran into Stitching auditing, sandboxed checkout, Claude Code, and GitHub into one reproducible loop required significant infrastructure work. Our most important insight came from an evaluation failure: scoring a harness against single-PR ground truth can actually penalize a thorough auditor. A real page may contain dozens of accessibility issues, but the answer key contains only the one issue fixed by the original pull request. As a result, a system that finds additional real problems can incorrectly appear imprecise. This taught us that precision is only meaningful on fully annotated pages. It reshaped our evaluation approach and led us to build a fully annotated HTML-live suite where the harness's advantage could be measured fairly. ### Accomplishments we're proud of Built a capability axe fundamentally lacks: judging whether accessible names are meaningful, not merely present — axe: 0 findings, Ramp: 12 findings on the same pages. Demonstrated a measurable render-dependent advantage: 40% to 92% contrast-detection recall using the same model, with only the harness changing. Built a closed loop from detection to a merge-ready pull request on real open-source projects. Covered issues including landmarks, form labels, button names, alternative text, and color contrast. Curated a benchmark of 51 real accessibility tasks, each tied to a reviewable pull request. ### What we learned Tools and prompts are a real lever on capability: the same model behaves very differently when equipped with a domain-specific harness. The pull request is the right interface for accessibility work because review, CI, collaboration, and merging are already solved by the tools developers use. The value isn't in reporting more issues — it's in being precise and actually fixing them. ### What's next We plan to integrate Ramp directly into CI so that newly introduced accessibility issues can be automatically reproduced, fixed, and submitted upstream as pull requests. We also want to move the fix engine to a pure API path so anyone can run it using their own keys, without requiring a local agent installation. Finally, we plan to extend Ramp's source-level analysis capabilities, building on the source-code tasks already included in our benchmark. The goal is to make accessibility something developers complete by reviewing a single pull request. ## README (from the GitHub repository) # Ramp **Accessibility audit → fix → PR. axe detects; Ramp understands and fixes.** Lighthouse and axe-core *find* WCAG violations and stop at a report. Ramp closes the loop: it audits a real rendered page, reasons about the WCAG criterion, **writes the fix, verifies it with axe-core, and opens a merge-ready pull request** — and it catches semantic issues axe is blind to (alt text that just says `"image"`, links that say `"click here"`). > **axe: 0 violations · Ramp: 12 semantic issues axe can't see** (across 3 demo pages). --- ## Three pillars | Pillar | Package | What it does | |---|---|---| | **A11y-Bench** | `packages/bench` | 51 ground-truth tasks mined from real merged a11y PRs; scores **naked LLM vs harness** on recall **and** precision, split by `html-live` / `source-code`. | | **Harness** | `packages/harness` | Drives a headless page through Playwright + axe-core + accessibility tree + screen-reader simulation + contrast/focus inspectors + **semantic review**; an LLM agent reasons over the evidence (`runAudit`). | | **Auto-fix loop** | `packages/control-plane` | sandbox checkout → audit → **Claude Code** fix → **axe verify (before/after score)** → **GitHub PR**. Sentry monitors the loop. | ## Real fix PRs (verified, before → after) | Repo | Fix | Score | PR | |---|---|---|---| | `bad.html` (fixture) | alt + contrast + button names | **60 → 96** | [yangzhang75/Ramp#7](https://github.com/yangzhang75/Ramp/pull/7) | | semantic (fixture) | meaningless alt/link/button names axe passes | semantic 5 → 0 | [yangzhang75/Ramp#10](https://github.com/yangzhang75/Ramp/pull/10) | | `aigov-ops…` (real OSS) | landmarks + skip link + `
` | **92 → 100** | [PR#1](https://github.com/yangzhang75/aigov-ops-open-source-vendor-rfi-rapp-johnston-june-2026/pull/1) | | `caelaria` (real OSS) | unlabeled `