Project Info
Inspiration
One of our teammates is an international student, and deals with immigration paperwork often — RFEs, biometrics notices, status letters full of legal language that's genuinely stressful to parse even when English isn't the barrier. What kept bothering her wasn't just that these documents are confusing. It's that getting help with them almost always means handing over your name, your A-number, your case details, your address — to a person, a translation app, or an AI tool with a privacy policy you're just supposed to trust. For a population that's frequently already anxious about its legal status, asking them to expose more of themselves to get a letter explained felt like exactly the wrong tradeoff. We wanted to build something that didn't ask people to make that tradeoff at all. Not "we promise not to misuse your data" — an actual architecture where your identity structurally cannot reach a third party in the first place, whether that third party is an AI model, our own server logs, or our own error monitoring.
What it does
Passage takes an official immigration document — an RFE, a biometrics notice, an EAD receipt, a Notice to Appear — and translates and explains it in plain language, in the reader's own language, across ten supported languages. Before any of that happens, everything personally identifying in the document — names, A-numbers, SSNs, dates of birth, passport numbers, addresses — is detected and replaced with a placeholder token entirely inside the browser. Nothing touches the network until you explicitly press send, and what gets sent is never the original text — only the tokenized version. You can verify this yourself, live: open your browser's network tab, and the request body to our translation endpoint contains nothing but tokens like ⟦PII:NAME:1⟧. If our automatic detection ever misses something, you're not stuck trusting it blindly — you can select the missed text yourself and redact it manually, and it's treated identically to anything caught automatically. On the way back, after Claude responds, Passage checks that every token it sent is accounted for and that no raw personal data leaked into the response. If that check fails for any reason, nothing gets displayed — not a partial result, nothing reconstructed. It fails closed. From there, you can listen to the explanation read back to you, ask follow-up questions out loud (also redacted before they reach Claude), choose how much detail you want in the explanation, and see a list of other documents commonly relevant to your specific situation, based on what kind of document you uploaded — not a generic checklist. Throughout, Passage is explicit about one boundary it never crosses: it explains what a document is asking for. It never tells you what to write back, file, or do next. That's a deliberate legal line, not a missing feature.
How we built it
Detection runs entirely client-side: a named-entity recognition model (Xenova/bert-base-NER, via Transformers.js) running in-browser with zero network calls after the initial model download, layered with hand-written regex patterns for structured identifiers like A-numbers, SSNs, and dates. Detected spans get tokenized in the browser before anything is transmitted. The backend is a thin proxy — it holds the Anthropic API key and forwards only tokenized text to Claude (Sonnet 4.6) for translation and explanation. It never receives raw personal data either, which matters: our own server logs, our own Sentry error events, and our own observability traces can't leak what was never in that part of the system to begin with. Redis (via Upstash) stores only an ephemeral session marker, not PII, with no persistence. Two optional Redis-backed services — Agent Memory and LangCache — handle multi-turn voice conversation context and repeated-question caching, and both are scoped to store only redacted, tokenized text, never raw values. Sentry monitors our fail-closed validation path and our pre-send leakage scan, scrubbing common PII patterns from its own events as a second line of defense. Arize and Phoenix (dual-export, switchable per launch) track detection recall per document type with custom spans, so we have a real, measured accuracy number instead of a claim — including for our hardest detection case, non-Latin names. Deepgram handles voice input and read-back, transcribing questions which are then redacted client-side before ever reaching Claude, and synthesizing speech only from already-tokenized, PII-free explanation text. We built the application itself using Claude Code and Cursor throughout, and our team worked in parallel across two separate forks before merging the strongest pieces of each into one final codebase.
Challenges we ran into
The most serious one surfaced midway through the build: we found that names were sometimes appearing in completely plain text, fully unredacted, in translated output. Tracing it down, we discovered our entire name-detection capability depended on a single point of failure — the NER model — and that model was silently failing to load on one team member's machine, due to a native binary built for a newer macOS version than the one actually running it. The app fell back to regex-only detection with no warning shown anywhere in the interface. Since our regex patterns had never covered names in the first place — only structured identifiers like A-numbers and dates — every name was leaking, invisibly, the entire time someone tested on that machine. We treated this as the architectural failure it was, not a one-off bug. The fix was a real, independent regex layer for names — label-anchored patterns that work without depending on any model loading successfully — so name detection no longer has a single point of failure. We also found and fixed a quieter bug in the same investigation: a length-based filter meant to apply only to NER-sourced spans was silently dropping legitimate regex-detected name matches too, which had been suppressing detection on certain formats (ALL CAPS names, hyphenated names) independent of the NER issue entirely. Separately, while testing translated output across languages, we caught our own system drifting past a line we'd explicitly drawn. A Spanish translation of a document with a response deadline included a line recommending the reader consult an immigration lawyer — a soft form of advice-giving, well-intentioned, but exactly the kind of thing our system prompt was supposed to prevent. We tightened the prompt to explicitly separate "stating that a deadline or consequence exists" from "recommending any course of action, including seeking legal help," and re-verified across our full test set in both languages we'd built that rule for.
Accomplishments we're proud of
We didn't just build a privacy claim — we built one a judge, or anyone, can verify themselves in about ten seconds with their browser's devtools open, with nothing to take on faith. We're proud that when we found real failures during the build — the silent name-detection gap, the address leakage edge case, the advice-language drift in Spanish — we caught every one of them ourselves, through deliberate adversarial testing, before any of them reached a demo. We kept an honest, dated log of every failure and fix rather than quietly patching and moving on, and we think that record is some of the strongest evidence of how this project was actually built. We're also proud of the layered failure handling itself: detection that's allowed to be imperfect because we built two independent ways to catch its mistakes — a pre-send leakage scan with manual override, and post-response validation that fails closed rather than ever showing a partial or unverified result.
What we learned
That a privacy architecture is only as strong as its quietest failure mode. The scariest bug we found in this entire build wasn't loud — it produced no error, no crash, nothing in the console. It just silently leaked, the whole time, on one specific machine. That taught us to design every detection layer assuming it will eventually fail, and to build explicit, visible checks for that failure rather than trusting any single component to always work. We also learned that a rule like "never give advice" is far easier to write into a system prompt than to actually hold under real-world phrasing and multiple languages — it took deliberately reading raw model output, not just trusting that a check passed, to catch where our own system was drifting past a boundary we thought we'd already enforced.
What's next
Closing the one gap we're upfront about today: voice input currently reaches Deepgram as raw audio before any redaction happens on our end, so we tell users to type identifying numbers rather than say them aloud. A self-hosted speech pipeline would let voice questions get the same client-side redaction guarantee that typed and pasted text already has. We'd also like to expand full Aura-2 native voice coverage so read-back doesn't fall back to an English-accented voice for languages like Chinese, Vietnamese, or Arabic. And longer term, we want to build the citation-grounded "what does this actually mean for my specific situation" flow we deliberately scoped out of this build — it's the natural next step, but it's also exactly the kind of feature that needs to be built carefully, not quickly, given how easily it could cross from explaining a document into giving advice.
Passage
UC Berkeley AI Hackathon 2026 — World Track
Passage helps immigrants understand confusing official letters — like a notice to appear in immigration court — written in English. You paste or upload the letter and it translates and explains it in plain language in your own language (11 supported), and you can ask follow-up questions by voice.
The key idea: before anything is sent to the AI, your personal details (name, address, ID numbers) are stripped out and replaced with placeholders inside your browser, so the sensitive data never leaves your device — and you can watch in the browser's developer tools that only the placeholders get sent. It also double-checks that critical dates and deadlines survive translation correctly, since a missed deadline can sink an immigration case, and it's honest about what it can't guarantee.
Technical (brief): A monorepo with four packages — client (React 19 + TypeScript + Vite 6 SPA), server (Express 4 + TypeScript), shared (@passage/shared: sentry-scrub + explanation-text, single source to prevent drift), and a launcher (launch.mjs / macOS .app). The client runs the whole privacy pipeline in-browser: PDF/image extraction (pdf.js + Tesseract.js), PII detection (regex + Xenova/bert-base-NER via Transformers.js, recall-first at score ≥ 0.35 plus Unicode label-line names), tokenization to ⟦PII:TYPE:n⟧, an explicit send gate, and fail-closed post-Claude validation — all orchestrated by a usePassageFlow phase machine (input → preview → translating → done | blocked). The server is intentionally thin (prompt + API call per feature): Claude Sonnet 4.6 for translation plus a back-translation verify pass for dates/deadlines, Deepgram for voice, Upstash + optional Redis Cloud for session/memory/cache, OpenTelemetry/OpenInference exporting to Phoenix or Arize AX, and Sentry on both ends. The thesis is that the sophistication lives in the privacy architecture and verification harness — Playwright network audits asserting no raw PII in request bodies, per-type recall metrics, fail-closed validation — not in model orchestration: the innovation is what you don't send.
For architecture, data flow, and file reference, see PROJECT_ARCHITECTURE.md.
First-time setup
- Install dependencies (once):
npm run install:all
- Copy env files and fill in keys:
cp server/.env.example server/.env
cp client/.env.local.example client/.env.local # optional — browser Sentry
- Required in
server/.env:
| Variable | Purpose |
|---|---|
ANTHROPIC_API_KEY | Claude translation + voice Q&A |
UPSTASH_REDIS_REST_URL | Upstash — session markers, launcher heartbeat, rate limits |
UPSTASH_REDIS_REST_TOKEN | Upstash REST credentials |
SENTRY_DSN | Server error monitoring |
DEEPGRAM_API_KEY | Voice transcription + TTS |
RECALL_ALERT_THRESHOLD | Optional (default 0.75) — Sentry alert when recall or NAME recall drops below |
For Arize AX Cloud traces with npm run launch -- --cloud, also set ARIZE_SPACE_ID and ARIZE_API_KEY (app.arize.com → Settings).
Optional Redis Cloud (voice memory + FAQ cache — redacted text only):
| Variable | Purpose |
|---|---|
AGENT_MEMORY_URL + AGENT_MEMORY_STORE_ID + AGENT_MEMORY_API_KEY | Multi-turn voice Q&A |
LANGCACHE_URL + LANGCACHE_CACHE_ID + LANGCACHE_API_KEY | Semantic cache for repeated voice questions |
- macOS only — allow double-click launch (once):
./scripts/fix-launch-app.sh
If macOS still warns, right-click Launch Passage.app → Open → Open once.
Run Passage
After setup, start Passage one of these ways (both use the same launcher — server, client, observability picker, and auto-shutdown when you close the browser tab):
| Method | How |
|---|---|
| macOS app | Double-click Launch Passage.app in the repo root |
| Terminal | From the repo root: npm run launch |
Optional flags (terminal only): --cloud for Arize AX Cloud traces, --local for local Phoenix (Docker). On macOS, the app shows a dialog to pick observability instead.
npm run launch # default — observability picker (app) or Phoenix (terminal)
npm run launch -- --cloud # Arize AX Cloud traces
# npm run launch -- --local # Local Phoenix (Docker) instead
Re-run ./scripts/fix-launch-app.sh only if macOS blocks the app again — not needed for every launch.
Browser opens at http://localhost:5173. Pick your translation language on the landing screen first — the whole UI (nav, redaction review, tabs, voice controls, warnings) follows that choice. Close that tab when you are done — the launcher stops server and client automatically.
Logs if something fails: .passage-launch.log
Port conflict? If voice or API calls fail, kill any stale server:
lsof -ti:3001 | xargs kill
npm run launch -- --cloud
Configure secrets (reference)
The server refuses to start without working Redis and Claude credentials. See First-time setup for the required variables.
Observability — pick one at launch
| Mode | Set in .env | Where to get keys |
|---|---|---|
| Local Phoenix (default) | OBSERVABILITY_TARGET=phoenix | No keys needed — Docker only |
| Arize AX Cloud | OBSERVABILITY_TARGET=ax | app.arize.com → Settings → Space ID + API Key |
# Local Phoenix
OBSERVABILITY_TARGET=phoenix
PHOENIX_COLLECTOR_ENDPOINT=http://localhost:6006
PHOENIX_PROJECT_NAME=immigration-redaction-demo
# Arize AX Cloud
OBSERVABILITY_TARGET=ax
ARIZE_SPACE_ID=your-space-id
ARIZE_API_KEY=your-api-key
ARIZE_PROJECT_NAME=immigration-redaction-demo
Both modes use the same OpenTelemetry + OpenInference stack — Claude traces and redaction-check recall spans work identically; only the export destination changes.
Launcher vs .env: Launch Passage.app / launch.mjs asks which backend to use (macOS dialog) or accepts --local / --cloud. That choice is passed to the server for that session and overrides OBSERVABILITY_TARGET in .env. For npm run dev without the launcher, .env controls the target.
Redis Agent (optional — voice memory + FAQ cache)
Create both services at cloud.redis.io when you want multi-turn voice or cache hits. Only redacted/tokenized text is stored.
Client (client/.env.local)
| Variable | Purpose |
|---|---|
VITE_SENTRY_CLIENT_DSN | Public browser Sentry DSN (optional) |
Sponsor integrations
Anthropic (Claude)
Plain: Does the actual translating and plain-language explaining of the letter, and answers spoken follow-ups — but only ever sees a version with personal details swapped out for placeholders.
Technical: claude-sonnet-4-6 via @anthropic-ai/sdk. Powers /api/translate (structured tool output + immigration glossary + prompt caching), a second back-translation pass for date/deadline verification, /api/voice/question, and /api/related-documents. Receives only ⟦PII:TYPE:n⟧ tokens. Backend is deliberately thin — prompt + API call per feature.
Redis
Plain: Keeps the app running smoothly — remembers your session, blocks abuse, shuts down when you close the tab, and optionally remembers your voice conversation and caches repeated questions. Never stores personal info.
Technical: Three surfaces, all PII-free by design. Upstash (required): scoped session markers, per-session rate limits on translate/voice/extract, and launcher heartbeat (replaces an in-memory map). Redis Cloud Agent Memory (optional): tokenized multi-turn voice history with safety asserts before persist. Redis Cloud LangCache (optional): semantic cache for paraphrased voice FAQ, returns hit rate + similarity.
Sentry
Plain: Watches for errors and raises an alarm — including when the system catches itself leaking or mistranslating, or when name-detection accuracy drops.
Technical: @sentry/node (server, required) + @sentry/react (browser, optional). Captures validation mismatches, leakage blocks, date/deadline-drift blocks, recall-drop alerts (incl. a NAME-specific one), and Claude/Deepgram/API failures. Scrubbed via @passage/shared/sentry-scrub — events carry token keys only, never raw PII.
Arize AX (observability)
Plain: A dashboard that measures how well the privacy detection actually works — including how often it misses names — so the weakness is tracked openly instead of hidden.
Technical: OpenTelemetry + OpenInference. Auto-instruments Claude calls via @arizeai/openinference-instrumentation-anthropic. Custom redaction-check spans carry aggregate + per-type recall (redaction.recall.name, etc.) plus doc/session/run IDs; eval-dataset export to Arize AX Datasets. Local alternative is Phoenix on the same OTEL pipeline (Docker), chosen at launch.
Deepgram (voice)
Plain: Lets you ask follow-ups out loud and hear the explanation read back. Honest caveat the team volunteers: spoken audio reaches Deepgram before redaction — the one spot where the privacy boundary is weaker.
Technical: Nova-3 STT in the document's target language (redact=pii,numbers + keyword boosting); Aura-2 TTS speaking explanation-only tokenized text (PII safety assert before synthesis). Short-lived grant via /api/deepgram-token, or server-proxy via /api/voice/transcribe. Structural exposure: raw audio hits Deepgram before client-side transcript redaction.
What Passage does
Input: Paste text, upload a .txt file (stays in-browser), or upload .pdf / image (extracted in-browser via pdf.js + Tesseract.js; server fallback only if client extraction fails). Synthetic demo documents are available for testing.
Output: Plain-language translation + explanation in 11 languages (English, Spanish, French, Chinese, Vietnamese, Korean, Portuguese, Arabic, Hindi, Tagalog, Ukrainian). The translation column stays tokenized for audit; the explanation and voice “what it means” panels reinsert names and dates from in-memory tokenMap for on-screen reading only (never sent to Claude or TTS). Optional listen-back via TTS still uses tokenized text to Deepgram. Voice follow-up questions go to Claude as redacted tokens. Related documents tab lists commonly associated immigration document types (redacted input only).
UI language: Choose translation language on the landing screen before pasting. Site chrome, redaction review, tab labels, voice buttons, and TTS fallback warnings all follow that choice (11 locale packs under client/src/i18n/). Your choice is saved in localStorage so the connection-lost screen stays in the same language.
Scope line: Explains what a section is asking for. Does not tell anyone what to write in response.
What we don't claim
Volunteer these in a judged room — credibility beats polish.
| Exposure | Reality |
|---|---|
| Undetected names | Best-effort detection with recall-first tuning; names without label anchors or outside NER training can still reach Claude |
| Raw audio to Deepgram | Spoken PII hits a third party before redaction; mic disclaimer relies on user compliance |
| Server extract fallback | If client-side pdf.js/Tesseract fails, raw file bytes hit the server (rate-limited) |
| Translation correctness | Date/deadline back-translate catches one failure mode; it does not verify that form fields or legal meaning are correct |
Verify it works
Keep Passage running (via launcher or separate npm run dev in server/ and client/). For Phoenix mode, use the launcher or run docker compose -f docker-compose.phoenix.yml up -d first.
test:phase5 and test:phase6 call live HTTP endpoints — the server must already be listening on port 3001.
cd server
npm run test:phases # redaction, Redis, Claude, validation + Sentry
npm run test:phase5 # observability + recall scoring
npm run test:phase6 # Deepgram voice + TTS safety
npm run test:explanation-text
cd ../client
npm run verify:all # validation, regex, redact, voice, TTS, upload, i18n, names
npm run verify:demo-network # Playwright — token-only API payloads (needs dev server)
npm run verify:detection # Playwright — ?detection-test harness
npm run verify:tokenized-ui # Playwright — translation pane stays tokenized
npm run verify:sentry-browser # Playwright — fail-closed validation beat
node scripts/verify-connection-lost.mjs # Playwright — connection-lost UI (client only, no server)
npm run score:redaction -- run-before-fix
npm run score:redaction -- run-after-fix # from server/ — compare trend in observability UI
npm run sync:synthetic-docs --prefix server # regenerate JSON from client/src/data/synthetic-docs.ts
npm run export:eval-dataset --prefix server # Arize eval dataset JSONL
Filter observability UI by span name redaction-check or attribute redaction.run_id.
Failures logged in 08-error-log.md.
Demo script
See 09-demo-script.md for the timed 5-minute rehearsal (two distinct failure beats: detection gap vs validation mismatch).
API surface
GET /api/health — startup probe (used by launcher)
POST /api/launcher/heartbeat — browser tab alive (launcher auto-shutdown)
GET /api/launcher/session — last heartbeat timestamp
POST /api/launcher/goodbye — explicit tab close
POST /api/redaction-session-token scoped Upstash credentials (no PII)
POST /api/translate redacted text → translated tokens
POST /api/score-redaction recall metrics → observability span
POST /api/extract-document PDF/image → text (server fallback only; rate-limited)
POST /api/related-documents redacted text → process + associated doc types
POST /api/deepgram-token short-lived client token or server-proxy
POST /api/voice/question transcript + redacted context → answer + TTS text
POST /api/voice/speak PII-free text → MP3
POST /api/voice/transcribe raw audio → transcript
Not legal advice
Passage explains immigration paperwork in plain language. It does not provide legal advice, draft responses, or tell anyone how to answer a form.
Analysis
View
Metric
- 27
- 3
- 3
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- AnthropicIn code
- CSSIn code
- ExpressIn code
- HTMLIn code
- ReactIn code
- RedisIn code
- TypeScriptIn code
- DockerClaimed
- JavaScriptClaimed
- Node.jsClaimed
7 of 10 appear in the indexed code. 3 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
531 KB
Source files
109
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
shanghanyan/Passage
155 files · 1.2 MB · @ 849207a
Structure
Interface
25 files · 16%Screens, components and styles rendered to the user.
API & routing
38 files · 25%Request entry points: routes, handlers and controllers.
Application logic
45 files · 29%Domain rules, services and shared utilities.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- TypeScript68%
- CSS11%
- HTML11%
- Markdown10%
- Shell0%
- YAML0%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
server/package.json
npm · 24- @anthropic-ai/sdk
- @arizeai/openinference-instrumentation-anthropic
- @arizeai/openinference-semantic-conventions
- @arizeai/phoenix-otel
- @opentelemetry/exporter-trace-otlp-proto
- @opentelemetry/resources
- @opentelemetry/sdk-trace-base
- @opentelemetry/sdk-trace-node
- @passage/shared
- @sentry/node
- @upstash/redis
- cors
- dotenv
- express
- multer
- pdfjs-dist
- redis
- tesseract.js
- +6 more
client/package.json
npm · 15- @deepgram/sdk
- @huggingface/transformers
- @passage/shared
- @sentry/react
- pdfjs-dist
- react
- react-dom
- tesseract.js
- +7 more
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.