AIntercept

Pipeline: call leg → Twilio Media Streams → Deepgram (speech-to-text) → transcript → Claude (scam verdict) → alert. The audio source is swappable: a local file for development, a real Twilio call for the demo.

Build status: Pass 1 complete + Pass 2a — transcription spine, Twilio call leg, live dashboard, gated Claude scam judge, verdict-driven action, and a Redis-backed semantic gate + per-call / per-household memory.

Submission — UC Berkeley AI Hackathon 2026

AIntercept is our submission to the UC Berkeley AI Hackathon 2026. We're applying to the following tracks:

• Best Use of Claude (projects built with Claude Code) — the codebase was built with Claude Code
• Best Use of Redis (Beyond Caching) — Redis is our memory and retrieval layer, not a cache. We use RediSearch vector search (COSINE KNN over an embedded scam-script corpus) as the semantic suspicion gate, and Redis as agent memory: per-call transcripts/gate-hits/verdicts and per-household scam history that biases the next call from a known caller. The new read-only past-call history view is served straight from these records (call_summary:{id} + a calls:index sorted set).
• Best Creativity & Originality — a calm, family-facing "call guardian" that protects elderly people from phone scams in real time, turning the dashboard red and hanging up the moment a call looks dangerous. Solving a real, human problem (elder fraud) in a way we haven't seen done live on a phone call.
• Best Technical Implementation — a clean, swappable architecture (audio source → transcriber → suspicion gate → judge → action, each behind an interface), a two-stage cheap-filter-then-LLM design that keeps cost and latency down, fail-safe verdict parsing, graceful Redis fallback, and Redis-indexed history that scales the list view without reading every full record.
• Best Use of Deepgram — Deepgram (nova-2) powers the live voice experience: it streams speech-to-text off the Twilio media socket in real time, distinguishing interim vs. final results so the dashboard shows a live transcript while only final lines feed the scam judge.
• Best Use of TokenRouter by PaleBlueDot AI — the Claude judge is reached through TokenRouter, a single OpenAI-compatible gateway (one base URL + key). The judge's prompt, verdict schema, and parser are transport-agnostic, so we route Claude through TokenRouter by default and can swap to the direct Anthropic SDK with one env flag — no other code changes.

How the scam judge works (Steps 4–5)

Three stages, so the expensive reasoning is protected by a cheap filter and the verdict drives a real consequence:

1. Gate — every final transcript line updates a rolling window of recent turns, screened by a SuspicionGate (today a keyword KeywordGate). Benign windows stop here and never reach Claude.

2. Verdict — only flagged windows go to Claude (claude-sonnet-4-6), which returns a structured { riskLevel, reason, category } verdict.

3. Action — the verdict drives the dashboard and the action layer:

   • alert → terminate the call and notify family
   • warn → notify family, leave the call up for a human to judge
   • safe → do nothing

   Actions are idempotent per session (terminate once, notify once). The hang-up (CallController) and family alert (FamilyNotifier) are clean interfaces with logging/stub impls now; real telephony + SMS drop in behind them later.

The gate and action layers all sit behind interfaces so Pass 2 can drop in a Redis semantic gate and live Twilio actions with no other changes.

Judge transport — Claude through Token Router (Pass 2b)

Claude is still the reasoning model, but the verdict call is sent through Token Router, an OpenAI-compatible gateway (one base URL, one key), via the standard chat.completions shape. This is a transport swap only — the prompt, the { riskLevel, reason, category } schema, and the tolerant parser are unchanged, and it sits behind the JudgeTransport seam so the rest of the pipeline is untouched. Configure with TOKENROUTER_BASE_URL, TOKENROUTER_API_KEY, and JUDGE_MODEL (the model id as Token Router expects it — defaults to the free MiniMax-M3; set a Claude id like claude-sonnet-4-6 once you have credit). The active base URL is logged once at startup ([judge] transport: …); the key is never logged. The direct Anthropic SDK path is still available behind JUDGE_TRANSPORT=anthropic, which is the only case that needs ANTHROPIC_API_KEY.

Redis vector gate + memory (Pass 2a)

The keyword gate only catches literal phrases. The RedisVectorGate (same SuspicionGate interface) instead embeds the recent transcript and KNN-matches it against an embedded corpus of scam scripts in Redis — so a reworded scam with no shared keywords still gets flagged. Selecting the gate is one config switch: GATE=redis (default) vs GATE=keyword (fallback, no Redis needed).

• Embeddings sit behind an Embedder interface: EMBEDDER=local runs all-MiniLM-L6-v2 in-process (no API key), EMBEDDER=api uses an OpenAI-compatible endpoint. Vectors are unit-normalized; the index is COSINE.

• Flag threshold is cosine similarity, default REDIS_GATE_THRESHOLD=0.45 (paraphrased scams land ~0.46–0.55 vs ≤0.15 for benign chatter).

• Memory persists context beyond the current chunk:

  Key	Holds
  call:{id} (hash)	startedAt, lastRisk, lastReason, lastCategory, terminated
  call:{id}:lines (list)	rolling transcript
  call:{id}:gateHits (list)	category|score per flagged window
  call_summary:{id} (hash)	end-of-call summary for the history list (see below)
  calls:index (zset)	call ids scored by start time, for newest-first listing
  household:{id} (hash)	alerts, warns, lastCategory, first/last-seen
  household:{id}:cats (hash)	per-category alert counts

  Per-call keys expire after 1h; household keys persist across calls, so a repeat scam pattern is surfaced on the dashboard the moment the next call starts.

Past-call history (read-only)

At call end a compact call_summary:{id} is written and the id is added to the calls:index sorted set — the only writes this feature adds. Everything else is read straight from the keys the pipeline already persisted. Two read-only endpoints serve the history view (they never mutate call data), reusing the same Redis connection as the gate:

Endpoint	Returns
GET /api/calls	recent call summaries, newest first (cap 50)
GET /api/calls/:id	one call's full detail: transcript, gate hits, verdict, outcome

The dashboard's Past calls view lives at /history (linked from the live dashboard topbar): a list of past calls on the left — time, caller/household, detected category, final risk level, and outcome — and the selected call's transcript + verdict on the right. It's a separate page with no websocket, so it can't interfere with the live monitoring stream. When GATE=keyword (no Redis), the list is simply empty.

Start Redis Stack

docker run -d --name aintercept-redis -p 6379:6379 redis/redis-stack:latest

(Redis Stack bundles RediSearch for the vector index. If Redis isn't running and GATE=redis, the app logs a warning and falls back to the keyword gate.)

See semantic beat keyword (the before/after)

Run the SAME keyword-free scam line through each gate:

# Misses it — no literal trigger word:
GATE=keyword npm run dev -- --scam-demo "settle the outstanding balance today using prepaid vouchers from the pharmacy"
# Catches it — semantic match to the gift-card corpus:
GATE=redis   npm run dev -- --scam-demo "settle the outstanding balance today using prepaid vouchers from the pharmacy"

The pipeline panel shows the semantic hit, e.g. Redis: matched "Gift Card" (sim 0.55).

See per-household memory persist across calls

Run the scam demo twice with the same HOUSEHOLD_ID:

npm run dev -- --scam-demo          # first call records an alert in household:{id}
npm run dev -- --scam-demo          # second call surfaces "seen before" at the start

Setup

npm install
cp .env.example .env      # then fill in DEEPGRAM_API_KEY

Run — file source (no phone needed)

Streams a local WAV through the pipeline and onto the dashboard.

npm run dev               # benign: assets/spacewalk.wav — gate stays clear, no Claude
npm run dev -- path/to/your.wav

Then open the dashboard at http://localhost:3000 and watch the transcript scroll live (interim text is grey/italic; finalized lines are solid).

See the escalation path fire (no scam WAV needed) — injects a canned scam script straight into the judge, so the gate flags it and Claude returns a verdict:

npm run dev -- --scam-demo                       # built-in scam script
npm run dev -- --scam-demo "line one" "line two" # your own lines

The benign run needs only DEEPGRAM_API_KEY; the escalation path needs TOKENROUTER_API_KEY (read lazily, only when a window is flagged). Use ANTHROPIC_API_KEY instead only if you set JUDGE_TRANSPORT=anthropic.

Run — real Twilio call

npm run serve             # starts the call server + dashboard on :3000
ngrok http 3000           # expose it; copy the https URL

Point your Twilio number's Voice → "A call comes in" webhook (HTTP POST) at https://<your-ngrok>/twilio/voice, then call the number. The transcript appears on the dashboard at https://<your-ngrok> (and the console). The public websocket URL is derived automatically from the tunnel host.

Configuration

All secrets are environment variables — see .env.example. Only DEEPGRAM_API_KEY is required for Steps 1–3.

Security note

The dashboard has no authentication — it's a single, open, in-memory view for local development and the demo. In production this would be gated behind per-household authentication so only a senior's own family can watch their calls.