Project Info

ACCESSI

Inspiration

Standard navigation apps assume everyone moves through the world the same way. They give a wheelchair user the same route as someone who can sprint up a flight of stairs, and they give a blind user distances in meters when what they actually need is a step count. We wanted to build something that asks "how do you move?" before it tells you where to go, and that listens, narrates, and adapts instead of just displaying a map

What it does

ACCESSI lets a user speak their destination, then plans a route tailored to how they actually move through the world and not a generic walking or driving route. Speak your destination instead of typing it. ACCESSI transcribes your voice and finds the right place even handling tricky cases like chains and franchises (e.g. making sure "Safeway" or "99 Ranch" resolves to the correct nearby branch, not just the most "popular" one). *Routes adapt to the traveler. * Select wheelchair, blind, elderly, disabled, or stroller, and ACCESSI picks the right routing profile wheelchair-accessible paths that avoid stairs and curbs, or standard walking routes where that's the safer/better option. AI-narrated directions, rewritten by Claude into short, warm, spoken-style guidance. Every turn paired with a clear distance ("in 10 meters, turn left"), ramps and stairs flagged for wheelchair users, pacing/rest suggestions for elderly users, and step-counts instead of meters for blind users. Personal step-length calibration. Instead of guessing that "1 step ≈ 0.75m" for every user, ACCESSI can analyze a short self-recorded video (using the person's height as a real-world scale reference) to estimate that specific user's actual stride length or wheelchair rolling speed — making step-count narration far more accurate. AI accessibility score. Before you even start walking, ACCESSI estimates how accessible a route is likely to be (based on turn count, distance, and route shape) and explains why in plain language. Obstacle reporting, with a warm, AI-generated acknowledgment and safety tip when someone flags a hazard. Tech Stack Backend / Framework Python + Flask — the web server Flask-CORS — cross-origin support so frontend/backend can talk python-dotenv — manages API keys via environment variables AI / Language Anthropic Claude API (Claude Haiku 4.5) — used in three places: Turning raw turn-by-turn directions into warm, spoken-style narration tailored to the user's disability type Estimating an accessibility "safety score" (0–100) for a route based on its shape (turns, distance, duration) Generating warm acknowledgments when a user reports an obstacle Speech Deepgram API Speech-to-text (model: nova-2) — lets users speak their destination instead of typing Text-to-speech (model: aura-2-thalia-en) — reads directions back out loud Maps / Routing OpenRouteService (ORS) Geocoding (text → coordinates, and coordinates → address) Turn-by-turn directions, with routing profiles that adapt to the user (wheelchair, foot-walking) Computer Vision OpenCV (cv2) — video frame processing Media pipe Pose Land marker — detects body landmarks from video NumPy — math for the calibration calculations Used to calibrate a user's real walking stride length or wheelchair speed from a short self-filmed video, using their height as a real-world scale reference

Challenges we ran into

Geocoding chains like "Safeway" kept returning the wrong branch. Fixed it by only ranking by distance among results that were already strong name matches. Keeping AI-narrated directions safe was another one — every instruction has to include a distance, no exceptions. Estimating stride length from video is naturally imprecise, so we kept the messaging honest about that instead of overselling accuracy. We also made sure the app never hard-crashes if an API call fails.

Accomplishments we're proud of

Routes that actually adapt to five different mobility needs. AI narration with real safety rules built in, not just nice wording. Step-by-step guidance calibrated to the actual user, not an average person.

What we learned

Writing prompts for safety-critical output is different from writing prompts for normal chat — small wording changes decide whether the directions are actually safe to follow. We also learned how messy "correct" results get with maps APIs once you factor in real user context, not just text matching.

What's next

Live narration while walking, not just one summary upfront. Crowdsourced obstacle reports feeding into the accessibility score. More real-world accessibility data, actual curb cuts, ramps, sidewalk conditions. Better stride calibration with longer or multi-angle video. 

ACCESSI

ACCESSI is an AI-powered accessible navigation app built for people with mobility, vision, or age-related needs: wheelchair users, blind users, elderly users, and stroller/disabled users. It generates turn-by-turn walking/rolling routes and turns them into short, warm, spoken-style directions (distances always included, never raw coordinates), estimates a route's accessibility, lets users report obstacles by voice, and can calibrate a person's real stride length or wheelchair speed from a short video.

Features

Accessible routing — real turn-by-turn directions from OpenRouteService, using a routing profile matched to the traveler's needs (wheelchair vs. foot-walking).
AI narration — Claude rewrites raw directions into a warm, easy-to-follow spoken summary, always stating distances for every turn and flagging stairs/ramps where relevant.
AI accessibility score — a heuristic 0–100 estimate of how accessible a given route is, based on its shape (turn count, distance, duration), clearly presented as an estimate rather than measured data.
Obstacle reporting — report an obstacle by location/type and get a warm AI-generated acknowledgment and safety tip.
Voice input/output — speech-to-text and text-to-speech via Deepgram, so the app can be used hands-free.
Video-based calibration (optional) — estimate a user's real stride length (blind/elderly) or wheelchair speed (wheelchair/disabled/stroller) from a short video, using their height as a scale reference.

Tech stack

Backend: Python, Flask, Flask-CORS, Gunicorn
AI narration & scoring: Anthropic Claude API (claude-haiku-4-5)
Routing & geocoding: OpenRouteService (ORS) API
Speech: Deepgram (speech-to-text + text-to-speech)
Video calibration (optional): OpenCV (opencv-python-headless), MediaPipe (pose landmark detection), NumPy
Frontend: HTML/JS (index.html)
Config: python-dotenv for environment variables
Deployment: Procfile (Heroku-style)

Setup & running locally

Clone the repo

git clone https://github.com/hannay-sen/ACCESSI.git
cd ACCESSI

Install dependencies
```
pip install -r requirements.txt
```

Set up environment variables

Copy .env.example to .env and fill in your keys:

cp .env.example .env

ANTHROPIC_API_KEY=your_anthropic_key
ORS_API_KEY=your_openrouteservice_key
DEEPGRAM_API_KEY=your_deepgram_key

Run the app
```
python app.py
```
The app will start on http://localhost:5000 (or the port set by the PORT environment variable).

For production, use Gunicorn (as configured in the Procfile):
```
gunicorn app:app
```

If these aren't installed, the app still runs fine — the /calibrate-video endpoint just returns a message telling you to install them.

The first time /calibrate-video is called, it downloads a small (~5–6MB) pose-detection model automatically.

Analysis

Compare with all teams

View

Metric

Hannay
1

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

AnthropicIn code
FlaskIn code
HTMLIn code
PythonIn code
CSSClaimed
JavaScriptClaimed

4 of 6 appear in the indexed code. 2 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

62 KB

Source files

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.

Feature verification

9 Verified1 Code-supported

AI accessibility safety score for a routeVerified
Before starting, ACCESSI estimates how accessible a route is likely to be (turn count, distance, route shape) and explains why
Claimed on Devposthigh confidence
- app.py:270 — /safety-score route sends route shape stats to Claude and returns a 0-100 score with reasons
- index.html:816 — fetchSafetyScore is called after route computation and hits /safety-score
AI-narrated turn-by-turn directions via Claude with mandatory distancesVerified
AI-narrated directions rewritten by Claude into short, warm, spoken-style guidance, every turn paired with a distance, ramps/stairs flagged for wheelchair users, pacing suggestions for elderly, step counts for blind users
Claimed on Devposthigh confidence
- app.py:210 — /ask route builds a system prompt enforcing mandatory per-turn distances, wheelchair ramp/stair flags, elderly pacing, and blind step-count conversion, then calls Claude Haiku 4.5
Chain/franchise geocoding disambiguation (e.g. Safeway, 99 Ranch)Verified
Handles chains and franchises so the correct nearby branch resolves, not just the most popular one
Claimed on Devposthigh confidence
- app.py:84 — geocode_destination filters ORS results by confidence threshold then picks nearest among strong text matches, with comments explicitly describing the Safeway/99 Ranch fix
Disability-specific routing profiles (wheelchair, blind, elderly, disabled, stroller)Verified
Select wheelchair, blind, elderly, disabled, or stroller and ACCESSI picks the right routing profile
Claimed on Devposthigh confidence
- app.py:50 — ORS_PROFILES maps all five user types to wheelchair or foot-walking ORS routing profiles
- index.html:250 — user-type select offers all five options and userType is sent to /route
Obstacle reporting with AI-generated acknowledgment and safety tipVerified
Obstacle reporting, with a warm AI-generated acknowledgment and safety tip when someone flags a hazard
Claimed on Devposthigh confidence
- app.py:339 — /report route reverse-geocodes the location and asks Claude for a warm 2-sentence acknowledgment plus safety tip
- index.html:1002 — frontend posts obstacle type and location to /report
Personal step-length / wheelchair speed calibration from self-recorded videoVerified
Analyzes a short self-recorded video using height as scale reference to estimate stride length or wheelchair rolling speed
Claimed on Devposthigh confidence
- app.py:366 — /calibrate-video uses OpenCV + MediaPipe PoseLandmarker on uploaded video, computes meters_per_px from user height, and derives stride_m or wheelchair_mph
- index.html:504 — frontend uploads video and height to /calibrate-video and stores the returned stride/speed in localStorage for later use in narration
Text-to-speech playback of directionsVerified
Text-to-speech (Deepgram aura-2-thalia-en) reads directions back out loud
Claimed on Devposthigh confidence
- app.py:572 — /speak route calls Deepgram TTS aura-2-thalia-en and streams back mp3 audio
- index.html:874 — frontend prefetches and plays audio from /speak
Turn-by-turn directions and geocoding via OpenRouteServiceVerified
Maps/Routing: OpenRouteService for geocoding and turn-by-turn directions adapting to the user (wheelchair, foot-walking)
Claimed on Devposthigh confidence
- app.py:149 — /route calls ORS directions API with the selected profile and returns step-by-step instructions and geometry
Voice-to-destination speech inputVerified
ACCESSI transcribes your voice to find your destination instead of typing it
Claimed on Devposthigh confidence
- app.py:530 — /transcribe route sends raw audio to Deepgram nova-2 STT and returns transcript
- index.html:654 — MediaRecorder captures audio and posts it to /transcribe
App never hard-crashes if an API call failsCode-supported
We also made sure the app never hard-crashes if an API call fails
Claimed on Devpostmedium confidence
- app.py:256 — Claude, Deepgram, and ORS calls throughout app.py are wrapped in try/except blocks returning fallback JSON instead of raising, e.g. /ask, /safety-score, /report, /transcribe, /speak

An AI agent derived these features from the project’s Devpost page and readme, then searched the code for each one. Verified features are backed by cited code; claimed-only features had no supporting code, which is not by itself proof a feature is missing.

Project Info

ACCESSI

Inspiration

What it does

Challenges we ran into

Accomplishments we're proud of

What we learned

What's next

ACCESSI

Features

Tech stack

Setup & running locally

Analysis

Technology

AI coding agents

Codebase size

Structure

Interface

Application logic

Languages

Dependencies

Feature verification