# Project export: ACCESSI This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: No One Left Off the Map . . . - Devpost: https://devpost.com/software/accessi-nbsd30 - GitHub: https://github.com/theltheinttheintsen/ACCESSI.git - Demo: https://radar-sauciness-agnostic.ngrok-free.dev/ - Video: https://www.youtube.com/embed/nx7y8eK9t00?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 1 GitHub contributor(s) — Hannay (1 commits) ## Devpost submission (written by the team) ### Inspiration Standard navigation apps assume everyone moves through the world the same way. They give a wheelchair user the same route as someone who can sprint up a flight of stairs, and they give a blind user distances in meters when what they actually need is a step count. We wanted to build something that asks "how do you move?" before it tells you where to go, and that listens, narrates, and adapts instead of just displaying a map ### What it does ACCESSI lets a user speak their destination, then plans a route tailored to how they actually move through the world and not a generic walking or driving route. Speak your destination instead of typing it. ACCESSI transcribes your voice and finds the right place even handling tricky cases like chains and franchises (e.g. making sure "Safeway" or "99 Ranch" resolves to the correct nearby branch, not just the most "popular" one). *Routes adapt to the traveler. * Select wheelchair, blind, elderly, disabled, or stroller, and ACCESSI picks the right routing profile wheelchair-accessible paths that avoid stairs and curbs, or standard walking routes where that's the safer/better option. AI-narrated directions, rewritten by Claude into short, warm, spoken-style guidance. Every turn paired with a clear distance ("in 10 meters, turn left"), ramps and stairs flagged for wheelchair users, pacing/rest suggestions for elderly users, and step-counts instead of meters for blind users. Personal step-length calibration. Instead of guessing that "1 step ≈ 0.75m" for every user, ACCESSI can analyze a short self-recorded video (using the person's height as a real-world scale reference) to estimate that specific user's actual stride length or wheelchair rolling speed — making step-count narration far more accurate. AI accessibility score. Before you even start walking, ACCESSI estimates how accessible a route is likely to be (based on turn count, distance, and route shape) and explains why in plain language. Obstacle reporting, with a warm, AI-generated acknowledgment and safety tip when someone flags a hazard. Tech Stack Backend / Framework Python + Flask — the web server Flask-CORS — cross-origin support so frontend/backend can talk python-dotenv — manages API keys via environment variables AI / Language Anthropic Claude API (Claude Haiku 4.5) — used in three places: Turning raw turn-by-turn directions into warm, spoken-style narration tailored to the user's disability type Estimating an accessibility "safety score" (0–100) for a route based on its shape (turns, distance, duration) Generating warm acknowledgments when a user reports an obstacle Speech Deepgram API Speech-to-text (model: nova-2) — lets users speak their destination instead of typing Text-to-speech (model: aura-2-thalia-en) — reads directions back out loud Maps / Routing OpenRouteService (ORS) Geocoding (text → coordinates, and coordinates → address) Turn-by-turn directions, with routing profiles that adapt to the user (wheelchair, foot-walking) Computer Vision OpenCV (cv2) — video frame processing Media pipe Pose Land marker — detects body landmarks from video NumPy — math for the calibration calculations Used to calibrate a user's real walking stride length or wheelchair speed from a short self-filmed video, using their height as a real-world scale reference ### Challenges we ran into Geocoding chains like "Safeway" kept returning the wrong branch. Fixed it by only ranking by distance among results that were already strong name matches. Keeping AI-narrated directions safe was another one — every instruction has to include a distance, no exceptions. Estimating stride length from video is naturally imprecise, so we kept the messaging honest about that instead of overselling accuracy. We also made sure the app never hard-crashes if an API call fails. ### Accomplishments we're proud of Routes that actually adapt to five different mobility needs. AI narration with real safety rules built in, not just nice wording. Step-by-step guidance calibrated to the actual user, not an average person. ### What we learned Writing prompts for safety-critical output is different from writing prompts for normal chat — small wording changes decide whether the directions are actually safe to follow. We also learned how messy "correct" results get with maps APIs once you factor in real user context, not just text matching. ### What's next Live narration while walking, not just one summary upfront. Crowdsourced obstacle reports feeding into the accessibility score. More real-world accessibility data, actual curb cuts, ramps, sidewalk conditions. Better stride calibration with longer or multi-angle video.  ## README (from the GitHub repository) # ACCESSI ACCESSI is an AI-powered accessible navigation app built for people with mobility, vision, or age-related needs: wheelchair users, blind users, elderly users, and stroller/disabled users. It generates turn-by-turn walking/rolling routes and turns them into short, warm, spoken-style directions (distances always included, never raw coordinates), estimates a route's accessibility, lets users report obstacles by voice, and can calibrate a person's real stride length or wheelchair speed from a short video. ## Features - **Accessible routing** — real turn-by-turn directions from OpenRouteService, using a routing profile matched to the traveler's needs (wheelchair vs. foot-walking). - **AI narration** — Claude rewrites raw directions into a warm, easy-to-follow spoken summary, always stating distances for every turn and flagging stairs/ramps where relevant. - **AI accessibility score** — a heuristic 0–100 estimate of how accessible a given route is, based on its shape (turn count, distance, duration), clearly presented as an estimate rather than measured data. - **Obstacle reporting** — report an obstacle by location/type and get a warm AI-generated acknowledgment and safety tip. - **Voice input/output** — speech-to-text and text-to-speech via Deepgram, so the app can be used hands-free. - **Video-based calibration (optional)** — estimate a user's real stride length (blind/elderly) or wheelchair speed (wheelchair/disabled/stroller) from a short video, using their height as a scale reference. ## Tech stack - **Backend:** Python, Flask, Flask-CORS, Gunicorn - **AI narration & scoring:** Anthropic Claude API (`claude-haiku-4-5`) - **Routing & geocoding:** OpenRouteService (ORS) API - **Speech:** Deepgram (speech-to-text + text-to-speech) - **Video calibration (optional):** OpenCV (`opencv-python-headless`), MediaPipe (pose landmark detection), NumPy - **Frontend:** HTML/JS (`index.html`) - **Config:** `python-dotenv` for environment variables - **Deployment:** `Procfile` (Heroku-style) ## Setup & running locally 1. **Clone the repo** ```bash git clone https://github.com/hannay-sen/ACCESSI.git cd ACCESSI ``` 2. **Install dependencies** ```bash pip install -r requirements.txt ``` 3. **Set up environment variables** Copy `.env.example` to `.env` and fill in your keys: ```bash cp .env.example .env ``` ``` ANTHROPIC_API_KEY=your_anthropic_key ORS_API_KEY=your_openrouteservice_key DEEPGRAM_API_KEY=your_deepgram_key ``` 4. **Run the app** ```bash python app.py ``` The app will start on `http://localhost:5000` (or the port set by the `PORT` environment variable). For production, use Gunicorn (as configured in the `Procfile`): ```bash gunicorn app:app ``` If these aren't installed, the app still runs fine — the `/calibrate-video` endpoint just returns a message telling you to install them. The first time `/calibrate-video` is called, it downloads a small (~5–6MB) pose-detection model automatically. ## Detected evidence (automated analysis) Indexed codebase: 2 recognized source files, 62 KB. - Anthropic (technology) — detected in the code - Flask (technology) — detected in the code - HTML (language) — detected in the code - Python (language) — detected in the code - CSS (language) — claimed on Devpost, not found in the code - JavaScript (language) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (6 of 6) ``` .env.example .gitignore app.py index.html Procfile requirements.txt ``` ### Dependencies - requirements.txt: anthropic, flask, flask-cors, gunicorn, mediapipe, numpy, opencv-python-headless, python-dotenv, requests ### Recent commits (newest first) - index.html - Update app.py - Add files via upload ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` flask anthropic requests python-dotenv gunicorn flask-cors opencv-python-headless mediapipe numpy ``` ### app.py ```python from flask import Flask, render_template, request, jsonify, Response from flask_cors import CORS from anthropic import Anthropic import os, requests, json, tempfile, math from dotenv import load_dotenv load_dotenv(override=True) print("Anthropic key starts with:", repr(os.getenv("ANTHROPIC_API_KEY", "")[:15])) print("ORS key starts with:", repr(os.getenv("ORS_API_KEY", "")[:8])) print("Deepgram key starts with:", repr(os.getenv("DEEPGRAM_API_KEY", "")[:8])) print("ORS key starts with:", repr(os.getenv("ORS_API_KEY", "")[:10])) app = Flask(__name__) CORS(app) claude_client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY")) CLAUDE_MODEL = "claude-haiku-4-5-20251001" ORS_KEY = os.getenv("ORS_API_KEY") DEEPGRAM_KEY = os.getenv("DEEPGRAM_API_KEY") # Video calibration deps are optional — the app still runs fine without them, # /calibrate-video just returns a clear error telling you to pip install them. try: import cv2 import numpy as np import mediapipe as mp from mediapipe.tasks import python as mp_tasks from mediapipe.tasks.python import vision as mp_vision VIDEO_CALIBRATION_AVAILABLE = True except ImportError: VIDEO_CALIBRATION_AVAILABLE = False POSE_MODEL_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'pose_landmarker_lite.task') POSE_MODEL_URL = 'https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_lite/float16/1/pose_landmarker_lite.task' def ensure_pose_model(): """Download the pose-detection model once, on first use. Subsequent calls reuse the saved file. ~5-6MB, one-time download.""" if not os.path.exists(POSE_MODEL_PATH): print("Downloading pose landmark model (one-time, ~5-6MB)...") r = requests.get(POSE_MODEL_URL, timeout=60) r.raise_for_status() with open(POSE_MODEL_PATH, 'wb') as f: f.write(r.content) print("Pose model downloaded to", POSE_MODEL_PATH) ORS_PROFILES = { "wheelchair": "wheelchair", "blind": "foot-walking", "elderly": "foot-walking", "disabled": "wheelchair", "stroller": "wheelchair", } def reverse_geocode(lat, lng): """Convert raw coordinates into a human-readable place name.""" try: res = requests.get( "https://api.openrouteservice.org/geocode/reverse", params={"api_key": ORS_KEY, "point.lon": lng, "point.lat": lat, "size": 1}, timeout=5 ) data = res.json() label = data["features"][0]["properties"].get("label") return label or "your current location" except Exception: return "your current location" def haversine_meters(lat1, lng1, lat2, lng2): """Great-circle distance between two points, in meters.""" R = 6371000 phi1, phi2 = math.radians(lat1), math.radians(lat2) d_phi = math.radians(lat2 - lat1) d_lambda = math.radians(lng2 - lng1) a = math.sin(d_phi / 2) ** 2 + math.cos(phi1) * math.cos(phi2) * math.sin(d_lambda / 2) ** 2 return R * 2 * math.atan2(math.sqrt(a), math.sqrt(1 - a)) def geocode_destination(query, focus_lat=None, focus_lng=None): """Convert a typed destination into coordinates. For chains/franchises ('Safeway', '99 Ranch', 'HeyTea') that have many locations, ORS's relevance ranking alone can return the wrong branch. To fix this properly: 1. We bias toward the user's area with focus.point — but as a soft hint only, NOT a hard radius cutoff. A hard radius would wrongly exclude the correct branch if the person explicitly names a different city (e.g. '99 Ranch Fremont' while testing from somewhere far from Fremont) — exactly the case this is meant to fix. 2. Pelias scores every result with a 'confidence' (0-1) measuring how well it actually matches the text query. We only compare DISTANCE among results that are genuinely good text matches — this stops an irrelevant-but-nearby place from beating the real, correctly-named, slightly farther store.""" params = {"api_key": ORS_KEY, "text": query, "size": 10 if focus_lat is not None else 1} if focus_lat is not None and focus_lng is not None: params["focus.point.lon"] = focus_lng params["focus.point.lat"] = focus_lat try: res = requests.get( "https://api.openrouteservice.org/geocode/search", params=params, timeout=5 ) data = res.json() features = data.get("features", []) if not features: print("ORS GEOCODE: no features returned. Status:", res.status_code, "Response:", data) return None if focus_lat is not None and focus_lng is not None and len(features) > 1: top_confidence = max(f["properties"].get("confidence", 0) for f in features) relevant = [ f for f in features if f["properties"].get("confidence", 0) >= max(top_confidence - 0.15, 0.4) ] if not relevant: relevant = [features[0]] best = min( relevant, key=lambda f: haversine_meters( focus_lat, focus_lng, f["geometry"]["coordinates"][1], f["geometry"]["coordinates"][0] ) ) else: best = features[0] lng, lat = best["geometry"]["coordinates"] label = best["properties"].get("label", query) return {"lat": lat, "lng": lng, "label": label} except Exception as e: print("ORS GEOCODE ERROR:", e) return None @app.route('/') def index(): return render_template('index.html') @app.route('/route', methods=['POST']) def get_route(): """Returns real GeoJSON route + plain-English step list.""" data = request.json start = data.get('start') destination_text = data.get('destination') user_type = data.get('userType', 'wheelchair') if not [truncated — 17246 more characters] ``` ### index.html ```html ACCESSI