Project Info
Inspiration
A white cane is great, but it can’t “see” fast-moving hazards. We wanted a low-cost, phone-only co-pilot that narrates the scene and nudges safe movement—without sending video to the cloud.
What it does
PathSense runs on-device object detection and immediately speaks the object first (“person ahead”, “chair ahead”) and then a navigation cue (“slight left”, “stop”, “caution”). It streams lightweight event logs to a Fetch.ai Agentverse safety agent that: Detects danger states (stuck, repeated near-collisions, or device silence) Sends SMS alerts to a trusted contact Archives telemetry to ASI One for privacy-scoped insights that only the caregiver can access
How we built it
Android (Kotlin, CameraX, Foreground Service, Jetpack Compose) YOLOv8n → TFLite INT8 (320×320, NMS output [300×6]) via NNAPI/XNNPACK A planner that maps detections → tokens (STOP / CAUTION / SLIGHT LEFT / RIGHT) with smoothing and dead-zones TTS + haptics for clear, low-latency cues Agentverse client (OkHttp) to stream events; Agent triggers SMS and stores logs in ASI One with caregiver-only credentials
Challenges we ran into
Quantized model scaling & confidence thresholds across devices Label/index mismatches and coordinate normalization Reducing “chatter”: debouncing speech, stabilizing cues, handling orientation Balancing latency vs. battery in a persistent camera service
Accomplishments we're proud of
True on-device guidance with clear object-then-action narration End-to-end safety oversight via Agentverse + ASI One with strict access control Robust mobile pipeline that survives screen-off and poor connectivity
What we learned
Edge AI is as much systems work as model work: logging, smoothing, permissions, and UX (the right words at the right time) matter as much as mAP.
What's next
Stairs/curb depth cues Finer path planning Wearable audio integration Multilingual voices Caregiver dashboards on ASI One Structured pilot with blind users to iterate on safety and trust
This project did not link a GitHub repository.
Analysis
No indexed repository for this project, so there are no commit stats to show.
Technology
- KotlinUnchecked
- PythonUnchecked
- TensorFlowUnchecked
No repository was indexed for this project, so these Devpost claims have not been checked against code.
AI coding agents
No repository was indexed, so agent usage could not be checked.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
No repository was indexed, so there is no codebase to measure.
This project did not link a GitHub repository, so there is nothing to diagram.
This project did not link a GitHub repository, so its feature claims have not been checked against code.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.