# Project export: Loop This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: Stay in the Loop with the people you care about - Devpost: https://devpost.com/software/loop-7tnj5g - GitHub: https://github.com/CalHacksAIHackathon/loop-publish - Video: https://www.youtube.com/embed/TeHZjwV7qvU?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 6 GitHub contributor(s) — Girish Skandha Sudhakar (19 commits), Cursor (15 commits), FrostedPyromaniac (10 commits), Abhiram510 (8 commits), shovan (8 commits), Claude Opus 4.8 (4 commits) ## Devpost submission (written by the team) ### Inspiration We all have elderly loved ones who we know are not very modern or aware of the latest trends in financial scams, which makes them prime targets for scams. In 2025, American seniors alone lost $7.7 billion to scams, with phishing scams being among the most common. We wanted a preventative tool which makes it very difficult to fall for scams without relying much on the end user's tech-skills: it needed to be a seamless experience for them. ### What it does The core functionality of the app is that it monitors any call from a number not on a pre-created whitelist for red flags. Calls that are flagged with scam hallmarks, such as a sense of urgency, requests for money, or opportunities that sound too good to be true, are then routed in different ways depending on the user's settings and privacy preferences. The least intrusive is just a small visual and audio warning for the end user, a somewhat intrusive route is doing the earlier route plus notifying a trusted adult about the call, and the most intrusive option is letting the trusted adult either enter the call, or even terminate it themselves based on the scam transcript. The idea is that this trusted adult and the user discuss and agree upon a policy that works best for their situation and comfort levels. ### How we built it We mostly used agentic workflows to write the code, allowing us to spend a lot of time thinking and describing features. This helped us keep the scope focused but also rich in terms of what features we supported. This allowed us to create a product which does one thing very well. The heart of Loop is Deepgram. We stream both sides of the call into Deepgram's nova-3 model in real time, with language=multi set. This allows us to handle multi-language conversations, which expands the reach of our product beyond English-speaking countries. Deepgram's extensive support for major languages like Hindi, Spanish, German, and many more means we can help people in every region and transcribe in people's native languages instantly. We tuned endpointing to 100 milliseconds so every utterance finalizes the moment it's spoken. That speed is what lets us interrupt a scam as it happens. Deepgram's agentic tools allow us to follow up on scams via automated trusted adult reach-out and email notification, enabling permanent records of every interaction. Those transcripts flow into Redis, where we run vector KNN search against known scam patterns and a caller-fingerprint database allows us to flags repeat offenders across our userbase. ### Challenges we ran into We ran into a lot of integration challenges, especially with Twilio, which we used for call forwarding. One of our main goals was to make the experience as seamless as possible so users only had to download the app once and could keep receiving calls normally. Getting that to work was honestly much harder than expected since Twilio was the only service that really supported what we needed, and working around the limitations of free accounts took quite a bit of trial and error. We also had to deal with the challenges of integrating with existing phone call workflows on mobile devices, which required some creative solutions. Beyond that, a big challenge was keeping the scope realistic for a hackathon while still making sure each tool we used actually added value instead of feeling forced into the project. ### Accomplishments we're proud of We built a working platform that can detect potential scam activity, explain why something looks suspicious, and provide clear next steps for users. We were also able to integrate multiple services into a single experience that felt simple and easy to use. ### What we learned We learned a lot about building reliable AI systems under a tight deadline, especially around handling real-world scam scenarios and making the results understandable for users who may not be super tech-savvy. ### What's next We want to improve detection accuracy, support more scam channels like phone calls and text messages, and add some custom features that help can family members stay more informed when potential scams are detected. ## README (from the GitHub repository) # Loop **Protected call-routing and live scam intervention for families.** > Scammers isolate. Loop reconnects. Loop sits on a call, transcribes both sides in real time, recognizes scam tactics as they unfold, **pauses the call and warns the user the instant they're asked for a code or a payment**, alerts a trusted family member live, and — once the call ends — plays back a plain-spoken voice recap of what happened. Two technologies do the heavy lifting: - **Deepgram** — real-time, multilingual speech-to-text (and the spoken post-call recap). - **Redis** — the vector store for scam-pattern matching, caller fingerprints, and live call state. > For the hackathon, the paid telephony/carrier routing is mocked — but the > intelligence pipeline is real. ```mermaid graph TD classDef client fill:#e1f5fe,stroke:#0288d1,stroke-width:2px; classDef server fill:#ede7f6,stroke:#5e35b1,stroke-width:2px; classDef ext fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px; classDef db fill:#fff3e0,stroke:#f57c00,stroke-width:2px; subgraph Clients ["App — one app, two roles"] UserApp["Protected User"] Dashboard["Family Dashboard"] end subgraph Backend ["Loop Backend (Node + TypeScript)"] WS["WebSocket gateway"] Pipeline["Live pipeline: rules → risk scoring → protective intervention"] Report["Post-call recap (PII-redacted)"] end Deepgram["Deepgram — nova-3 multilingual STT + TTS"] Redis["Redis — vector KNN scam patterns + caller fingerprints + call state"] %% Live ingestion UserApp -->|dual-track audio| WS WS -->|stream audio| Deepgram Deepgram -->|live transcript| WS WS --> Pipeline %% Detection Pipeline -->|KNN vector search| Redis Redis -->|pattern + caller matches| Pipeline %% Intervention Pipeline -->|pause + warning| UserApp Pipeline -->|live alert| Dashboard %% Post-call WS -->|call ends| Report Report -->|generate voice recap| Deepgram Deepgram -->|spoken summary| UserApp Report -->|recap + summary| Dashboard class UserApp,Dashboard client; class WS,Pipeline,Report server; class Deepgram ext; class Redis db; ``` **Routing is mocked. Intelligence is real.** --- ## Monorepo layout ``` loop/ backend/ Node + TypeScript server (the real intelligence pipeline) mobile/ Expo (React Native + TS) iOS app -> TestFlight shared/ Types shared between backend and mobile ``` - The **mobile app** is the protected call bridge. It captures two audio tracks (caller side + the device mic / user side) and streams them to the backend. One app, two roles: **Protected User** and **Family**. - The **backend** holds the API keys and runs the pipeline, powered by **Deepgram** (speech-to-text + voice recap) and **Redis Cloud** (vector store). ## Quick start See [`backend/README.md`](backend/README.md) and [`mobile/README.md`](mobile/README.md). ```bash # backend cd backend && npm install && cp .env.example .env && npm run dev # mobile cd mobile && npm install && npx expo start ``` ## Keys Every integration **degrades gracefully** when a key is missing, so the app always runs while keys are being provisioned. Fill in `backend/.env`: **Core (required for the full experience):** - `DEEPGRAM_API_KEY` — real-time multilingual speech-to-text + the spoken post-call recap - `REDIS_URL` — Redis Cloud: vector KNN scam-pattern search, caller fingerprints, call state **Optional (enhance summaries/embeddings; safe to leave blank):** - `ANTHROPIC_API_KEY`, `VOYAGE_API_KEY`, `TERAC_API_KEY` / `TERAC_BASE_URL` ## Multilingual Scams don't only happen in English. Loop transcribes calls natively across languages using **Deepgram's `nova-3` model with `language=multi`**, which handles **code-switching** — a caller mixing Hindi and English ("Hinglish"), or speaking entirely in Spanish, French, or German, all in one stream. Endpointing is tuned to 100 ms so each language shift finalizes promptly. Warnings and the post-call recap are rendered in the conversation's active language, so the protected user is always spoken to in a language they understand. ## Detected evidence (automated analysis) Indexed codebase: 68 recognized source files, 344 KB. - Anthropic (technology) — detected in the code - Express (technology) — detected in the code - React (technology) — detected in the code - Redis (technology) — detected in the code - TypeScript (language) — detected in the code - Node.js (technology) — claimed on Devpost, not found in the code - Python (language) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers - AI coding agent: Codex — evidence: config files committed to the repository - AI coding agent: Cursor — evidence: commit authorship or trailers ## Codebase structure (from repository index) ### Files (82 of 82) ``` .claude/launch.json .gitignore backend/.env.example backend/package.json backend/README.md backend/src/audio/transcode.ts backend/src/config.ts backend/src/data/kagglePatterns.json backend/src/data/scamPatterns.ts backend/src/index.ts backend/src/integrations/claude.ts backend/src/integrations/deepgram.ts backend/src/integrations/email.ts backend/src/integrations/redis.ts backend/src/integrations/terac.ts backend/src/integrations/twilio.ts backend/src/integrations/voiceMemo.ts backend/src/integrations/voyage.ts backend/src/logger.ts backend/src/permissions.ts backend/src/pipeline/orchestrator.ts backend/src/pipeline/redaction.ts backend/src/pipeline/riskEngine.ts backend/src/pipeline/rules.ts backend/src/pipeline/scamStages.ts backend/src/report/postCall.ts backend/src/scripts/seedKagglePatterns.ts backend/src/scripts/seedRedis.ts backend/src/scripts/simulateCall.ts backend/src/scripts/simulateHinglish.ts backend/src/scripts/testEmail.ts backend/src/scripts/testMultilingual.ts backend/src/scripts/testVoiceMemo.ts backend/src/server/audioCodec.ts backend/src/server/bus.ts backend/src/server/debugLog.ts backend/src/server/permissions.ts backend/src/server/rest.ts backend/src/server/trustedAdult.ts backend/src/server/twilioMedia.ts backend/src/server/twilioRoutes.ts backend/src/server/ws.ts backend/src/types.ts backend/tsconfig.json mobile/.claude/settings.json mobile/.gitignore mobile/AGENTS.md mobile/app.json mobile/App.tsx mobile/CLAUDE.md mobile/eas.json mobile/index.ts mobile/LICENSE mobile/package.json mobile/README.md mobile/src/audio/AudioPlayer.ts mobile/src/audio/AudioStreamer.ts mobile/src/components/RiskGauge.tsx mobile/src/components/RiskMeter.tsx mobile/src/components/StageMeter.tsx mobile/src/components/Transcript.tsx mobile/src/components/ui.tsx mobile/src/config.ts mobile/src/data/scripts.ts mobile/src/net/CallClient.ts mobile/src/net/DashboardClient.ts mobile/src/screens/family/FamilyFlow.tsx mobile/src/screens/HistoryScreen.tsx mobile/src/screens/protected/CallKilledOverlay.tsx mobile/src/screens/protected/IncomingCall.tsx mobile/src/screens/protected/ProtectedFlow.tsx mobile/src/screens/protected/ProtectionSettings.tsx mobile/src/screens/protected/ProtectivePause.tsx mobile/src/screens/ReportView.tsx mobile/src/screens/RoleSelectScreen.tsx mobile/src/theme.ts mobile/src/types.ts mobile/src/utils/storage.ts mobile/tsconfig.json README.md render.yaml requirements.txt ``` ### Dependencies - backend/package.json: @anthropic-ai/sdk@^0.32.1, @deepgram/sdk@^3.9.0, @types/cors@^2.8.17, @types/express@^4.17.21, @types/node@^22.10.5, @types/nodemailer@^8.0.1, @types/ws@^8.5.13, cors@^2.8.5, dotenv@^16.4.7, express@^4.21.2, ioredis@^5.11.1, nodemailer@^9.0.1, redis@^4.7.0, tsx@^4.19.2, twilio@^6.0.2, typescript@^5.7.3, ws@^8.18.0, zod@^3.24.1 - mobile/package.json: @edkimmel/expo-audio-stream@^0.6.4, @expo/vector-icons@^15.0.3, @types/react@~19.1.0, expo@~54.0.35, expo-audio@~1.1.1, expo-av@~16.0.8, expo-constants@~18.0.13, expo-file-system@~19.0.23, expo-haptics@~15.0.8, expo-linear-gradient@~15.0.8, expo-notifications@^0.32.17, expo-speech@~14.0.8, expo-status-bar@~3.0.9, react@19.1.0, react-native@0.81.5, react-native-live-audio-stream@^1.1.1, typescript@^5.9.3 ### Recent commits (newest first) - Update README.md - Merge branch 'agentaddition' — email + family phone alert - feat: fire family phone call at High risk threshold (score 61+) - call family - email sender - blank req - Remove stray root .py files so Render deploys as Node, not Python - Merge pull request #5 from CalHacksAIHackathon/complete-ui-haul - all the changes that were fixed/made works!! - Merge pull request #4 from CalHacksAIHackathon/feature-interjection-v2 - feature interjection mostly works - debug(ws): log user mic relay to Twilio (count + relay success) - fix(mobile): keep caller audio on speaker when mic is active - fix(twilio): app-as-endpoint inbound mode + mic format conversion - 0.0 port thing - fix(twilio): instrument inbound to diagnose dropped calls - fix all the merge issues - fixed git merge commits - interjections work, but broke audio hearing capaibilities - interjections work, but broke audio hearing capaibilities ## Key source files (fetched from GitHub, selected and truncated for size) ### mobile/CLAUDE.md ```markdown @AGENTS.md ``` ### mobile/AGENTS.md ```markdown # Expo HAS CHANGED Read the exact versioned docs at https://docs.expo.dev/versions/v56.0.0/ before writing any code. ``` ### mobile/package.json ``` { "name": "mobile", "version": "1.0.0", "main": "index.ts", "dependencies": { "@edkimmel/expo-audio-stream": "^0.6.4", "@expo/vector-icons": "^15.0.3", "expo": "~54.0.35", "expo-audio": "~1.1.1", "expo-av": "~16.0.8", "expo-constants": "~18.0.13", "expo-file-system": "~19.0.23", "expo-haptics": "~15.0.8", "expo-linear-gradient": "~15.0.8", "expo-notifications": "^0.32.17", "expo-speech": "~14.0.8", "expo-status-bar": "~3.0.9", "react": "19.1.0", "react-native": "0.81.5", "react-native-live-audio-stream": "^1.1.1" }, "scripts": { "start": "expo start", "android": "expo start --android", "ios": "expo start --ios", "web": "expo start --web" }, "private": true, "devDependencies": { "@types/react": "~19.1.0", "typescript": "^5.9.3" } } ``` ### backend/package.json ``` { "name": "loop-backend", "version": "0.1.0", "private": true, "description": "Loop backend — live scam intervention pipeline", "type": "module", "main": "dist/index.js", "scripts": { "dev": "tsx watch src/index.ts", "build": "tsc -p tsconfig.json", "start": "node dist/index.js", "seed": "tsx src/scripts/seedRedis.ts", "seed:kaggle": "tsx src/scripts/seedKagglePatterns.ts", "simulate": "tsx src/scripts/simulateCall.ts", "simulate:hinglish": "tsx src/scripts/simulateHinglish.ts", "test:multilingual": "tsx src/scripts/testMultilingual.ts", "typecheck": "tsc --noEmit", "test:email": "tsx src/scripts/testEmail.ts" }, "dependencies": { "@anthropic-ai/sdk": "^0.32.1", "@deepgram/sdk": "^3.9.0", "cors": "^2.8.5", "dotenv": "^16.4.7", "express": "^4.21.2", "ioredis": "^5.11.1", "nodemailer": "^9.0.1", "redis": "^4.7.0", "twilio": "^6.0.2", "ws": "^8.18.0", "zod": "^3.24.1" }, "devDependencies": { "@types/cors": "^2.8.17", "@types/express": "^4.17.21", "@types/node": "^22.10.5", "@types/nodemailer": "^8.0.1", "@types/ws": "^8.5.13", "tsx": "^4.19.2", "typescript": "^5.7.3" } } ``` ### mobile/index.ts ```typescript import { registerRootComponent } from 'expo'; import App from './App'; // registerRootComponent calls AppRegistry.registerComponent('main', () => App); // It also ensures that whether you load the app in Expo Go or in a native build, // the environment is set up appropriately registerRootComponent(App); ``` ### mobile/App.tsx ```typescript import React, { useState } from "react"; import { Pressable, SafeAreaView, StatusBar, StyleSheet, Text, View } from "react-native"; import { colors, radius, spacing } from "./src/theme"; import { RoleSelectScreen, type Role } from "./src/screens/RoleSelectScreen"; import { ProtectedFlow } from "./src/screens/protected/ProtectedFlow"; import { FamilyFlow } from "./src/screens/family/FamilyFlow"; export default function App() { const [role, setRole] = useState(null); return ( {role === null ? ( ) : (
{role === "protected" ? : setRole("protected")} />} )} ); } function Header({ role, onSwitch }: { role: Role; onSwitch: (r: Role | null) => void }) { return ( onSwitch(null)} hitSlop={12} style={styles.brandRow}> Loop onSwitch("protected")} /> onSwitch("family")} /> ); } function ToggleBtn({ label, active, onPress, }: { label: string; active: boolean; onPress: () => void; }) { return ( {label} ); } const styles = StyleSheet.create({ safe: { flex: 1, backgroundColor: colors.bg }, header: { flexDirection: "row", alignItems: "center", justifyContent: "space-between", paddingHorizontal: spacing(2), paddingVertical: spacing(1.25), borderBottomColor: colors.cardBorder, borderBottomWidth: 1, backgroundColor: colors.bgElevated, }, brandRow: { flexDirection: "row", alignItems: "center", gap: spacing(0.75) }, headerDot: { width: 8, height: 8, borderRadius: 4, backgroundColor: colors.brand, }, brand: { color: colors.text, fontSize: 20, fontWeight: "800", letterSpacing: -0.5, }, toggle: { flexDirection: "row", backgroundColor: colors.bg, borderRadius: radius.sm, padding: 2, borderColor: colors.cardBorder, borderWidth: 1, }, toggleBtn: { paddingHorizontal: spacing(1.75), paddingVertical: spacing(0.6), borderRadius: radius.sm - 2, }, toggleBtnActive: { backgroundColor: colors.brand, }, toggleText: { color: colors.textFaint, fontSize: 12, fontWeight: "600" }, toggleTextActive: { color: colors.bg }, }); ``` ### backend/src/index.ts ```typescript import http from "node:http"; import express from "express"; import cors from "cors"; import { config, integrationStatus } from "./config.js"; import { log } from "./logger.js"; import { connectRedis, seedPatterns } from "./integrations/redis.js"; import { SEED_SCAM_PATTERNS } from "./data/scamPatterns.js"; import { api } from "./server/rest.js"; import { attachWebSockets } from "./server/ws.js"; async function main() { const app = express(); app.use(cors({ origin: config.corsOrigin === "*" ? true : config.corsOrigin.split(",") })); app.use(express.json({ limit: "2mb" })); app.use(express.urlencoded({ extended: false })); app.use("/api", api); app.get("/", (_req, res) => res.json({ service: "loop-backend", status: "up" })); const server = http.createServer(app); attachWebSockets(server); // Connect Redis (or fall back to in-memory) and seed the scam corpus. await connectRedis(); await seedPatterns(SEED_SCAM_PATTERNS); server.listen(config.port, "0.0.0.0", () => { log.info(`Loop backend listening on 0.0.0.0:${config.port}`); log.info("Integrations", integrationStatus()); log.info("WebSockets: /ws/call (app) /ws/dashboard (family) /ws/twilio-media (twilio)"); }); const shutdown = () => { log.info("Shutting down…"); server.close(() => process.exit(0)); setTimeout(() => process.exit(0), 3000).unref(); }; process.on("SIGINT", shutdown); process.on("SIGTERM", shutdown); } main().catch((err) => { log.error("Fatal startup error", err); process.exit(1); }); ``` ### render.yaml ```yaml services: - type: web name: loop-backend runtime: node region: oregon rootDir: backend buildCommand: npm install --include=dev && npm run build startCommand: node dist/index.js envVars: - key: NODE_ENV value: production - key: PORT value: "8080" - key: CORS_ORIGIN value: "*" - key: DEEPGRAM_API_KEY sync: false - key: ANTHROPIC_API_KEY sync: false - key: VOYAGE_API_KEY sync: false - key: REDIS_URL sync: false - key: TWILIO_ACCOUNT_SID sync: false - key: TWILIO_AUTH_TOKEN sync: false - key: TWILIO_PHONE_NUMBER sync: false - key: TWILIO_FORWARD_TO sync: false - key: TWILIO_PUBLIC_URL sync: false - key: SMTP_HOST value: smtp.gmail.com - key: SMTP_PORT value: "587" - key: SMTP_USER sync: false - key: SMTP_PASS sync: false - key: SMTP_FROM sync: false ``` ### backend/src/logger.ts ```typescript /** Tiny structured logger — no dependency, readable in dev and Render logs. */ type Level = "debug" | "info" | "warn" | "error"; const COLORS: Record = { debug: "\x1b[90m", info: "\x1b[36m", warn: "\x1b[33m", error: "\x1b[31m", }; const RESET = "\x1b[0m"; function emit(level: Level, scope: string, msg: string, extra?: unknown) { const ts = new Date().toISOString(); const color = COLORS[level]; const head = `${color}[${level.toUpperCase()}]${RESET} ${ts} ${scope} —`; if (extra !== undefined) { // eslint-disable-next-line no-console console.log(head, msg, extra); } else { // eslint-disable-next-line no-console console.log(head, msg); } } export function makeLogger(scope: string) { return { debug: (msg: string, extra?: unknown) => emit("debug", scope, msg, extra), info: (msg: string, extra?: unknown) => emit("info", scope, msg, extra), warn: (msg: string, extra?: unknown) => emit("warn", scope, msg, extra), error: (msg: string, extra?: unknown) => emit("error", scope, msg, extra), }; } export const log = makeLogger("loop"); ``` ### backend/src/permissions.ts ```typescript export type PermissionLevel = "notify" | "notify_kill" | "notify_kill_barge"; export interface UserPairPermissions { protectedUserId: string; trustedAdultId: string; level: PermissionLevel; updatedAt: number; } const store = new Map(); // Seed a default permission for development store.set("Protected User", { protectedUserId: "Protected User", trustedAdultId: "trusted-adult-default", level: "notify_kill_barge", updatedAt: Date.now(), }); export function setPermissions( protectedUserId: string, trustedAdultId: string, level: PermissionLevel, ): void { store.set(protectedUserId, { protectedUserId, trustedAdultId, level, updatedAt: Date.now(), }); } export function getPermissions(protectedUserId: string): UserPairPermissions | null { return store.get(protectedUserId) ?? null; } export function canKill(protectedUserId: string): boolean { const perms = store.get(protectedUserId); if (!perms) return false; return perms.level === "notify_kill" || perms.level === "notify_kill_barge"; } export function canBarge(protectedUserId: string): boolean { const perms = store.get(protectedUserId); if (!perms) return false; return perms.level === "notify_kill_barge"; } ``` [57 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]