# Project export: CaneYouSee

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: CruzHacks 2026
- Tagline: A smart-sensing assistive cane for visually impaired people in underdeveloped regions which determines elevation changes on a path and alerts the user, allowing them to avoid a potential accident.
- Devpost: https://devpost.com/software/caneyousee
- GitHub: https://github.com/j-silv/caneyousee
- Video: https://www.youtube.com/embed/XbRsmN1gluQ?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner (Health Hacks; [Sponsor - IF Magic] Best Use of IF Magic)
- Team: 2 GitHub contributor(s) — Justin Silver (13 commits), Vincent Tran (12 commits)

## Devpost submission (written by the team)

### Inspiration

We were inspired to work this project by looking at the lack of ADA certified infrastructure and walkways present throughout a large amount of underdeveloped communities in the country. The lack of this can hinder people with impaired vision from being able to access the resources they need throughout their city, town, or neighborhood. As such we wanted to design a system that can be taken with them wherever they need to go and would augment their navigation capabilities in these under-developed areas.

### What it does

The smart cane activates and uses a small scale LiDAR sensor to scan up to 6ft ahead of ground from the cane's position. The cane establishes a base height that will indicate that the ground being walked on is level, and if the LiDAR detects any significant change in the elevation or "flatness" of the ground, it will send a signal to a set of vibrating motors on the cane to gently alert the user that the direction they are pointing the cane towards has a curb, a hole, or some other change in the ground level.

### How we built it

We built our device using distance sensors and vibration motors from the IF Magic company. The distance sensors needed to be positioned at an angle in order to accurately calculate the distance from ground ledges and drop-offs. After finishing this, we used the information from the sensors to process the information on the main UI graphs which therefore vibrated the motors immediately alerting the blind user. The software stack we used were Python for the backend logic and device communication, JavaScript to handle real-time data flow, and CSS to design the user interface.

### Challenges we ran into

The main obstacles to this project were mechanical design, hardware compatibility, and integration. Mechanical design It was hard to attach the sensor to the cane in a way that accurately measured the change in ground level. This was because the sensor needed to be angled relative to the cane's axis. We iterated through several designs, but eventually landed on a design that would make MacGyver proud. Hardware compatibility We decided to use the IF Magic framework for connecting the physical world (distance sensors, vibration modules) and the software (frontend, algorithm to detect ground level change, etc.). The IF Magic team were incredible. We were pushing the limits of the software package and forcing the development team to quickly push updates and bug fixes in response to our use case. Integration We had many different machines to coordinate with each other. IF Magic provided us with 2 separate processor modules and 2 different sensors. Streaming the data was relatively straight-forward in Python (thanks to their library) but the main challenge was fetching the processed data and displaying into the frontend, in real-time. We were stuck for a while because there was a very noticeable 1 second delay between an object being detected and a vibration being enabled. To overcome this, we switched to using IF Magic's 'Equations', while still displaying real-time data via our own dashboard.

### Accomplishments we're proud of

Despite the challenges, we made a working prototype within 36 hours. It's not perfect, but the proof of concept is there and we see many opportunities for improvement. Also, against all odds, we affixed our sensor to our cane without damage. Finally, we are happy that we could make a project which has a positive impact on the community.

### What we learned

Integration is the hardest part of any project. We all were able to complete our individual tasks but in doing so we made many assumptions on the format and structure of the input/output data. It was only when we came together to try and connect the pieces that we had to establish clearer abstractions. We also learned that physical feedback to the user needs to have a delay of less than 100 ms, otherwise it's unusable.

### What's next

There are many directions we could take CaneYouSee. There are general improvements involving UI, latency, and false positive error rate. There are more involved endeavors including adding additional sensors and training ML models instead of hard-coded algorithms to know when to vibrate.

## README (from the GitHub repository)

# Can(e) you see?

![Project overview](./imgs/project.png)

A cane equipped with a distance sensor and a vibration module. The cane vibrates when it detects significant changes in ground-level elevation. 

For additional information, check out the [DevPost](https://devpost.com/software/caneyousee) for this project.

## Tech stack

### Hardware/firmware

Uses [IF Magic framework](https://docs.ifmagic.io/) which integrate ESP32 microcontrollers, sensors, and firmware. We specifically used 1 vibration module, 1 distance sensor, and a mesh network with 2 ESP32 microcontroller modules.

For the microcontroller attached to the cane, we used IF Magic's 'Equations' firmware. The device then connects to a mesh network and communicates with another ESP32 module talking to a MacOS via a serial port. 

### Software

We leverage IF Magic's Python API to stream the distance data directly to a Python program. Some calibration / debouncing is applied to this data, which is then packaged into a JSON file and saved to the local filesystem. 

Another Python script running a Flask app then reads this JSON file and serves any requests on a localhost /data endpoint. 

Finally, a simple HTML frontend fetches data from the /data endpoint and displays the real-time distance data in a web browser.

## Getting started

### 1. Environment and installation

*Note: project was only tested on MacOS running Sequoia 15.7.3 and Python 3.12.12*

First, clone this repo:

```
git clone https://github.com/j-silv/caneyousee.git
```

Next, setup a Python virtual environment and install the dependencies.

```
cd caneyousee
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```

To communicate with the IF Magic hardware, you will also need to setup the SLAB USB driver. Instructions are [provided here](https://docs.ifmagic.io/api/streaming#wired).

### 2. Hardware setup

Our system uses a mesh network of two [Bare^2 ESP32 modules](https://docs.ifmagic.io/the-device/bare). As of writing, there are currently no public docs on how to correctly set up this mesh network. Please reach out to IF Magic for more information. 

Assuming the mesh network is correctly configured and the driver installed, you can plug in one of devices into your computer and verify that the device is detected. You should see `/dev/cu.SLAB_USBtoUART` in the output

```
ls /dev/cu.*
```

The other Bare^2 module should be attached with a vibration sensor and a distance sensor at port 0 and port 3 respectively (the exact port numbers may be different, so refer to the source code in `sensor.py` and change if needed).

Another step not described here is the programming of the cane module using IF Magic's 'Equations'. This is described in their documentation, but essentially the remote cane module needs to be programmed with the following equation:

```
WHEN milimeters of distance IS LESS THAN 1000 turn relay to on once & WHEN milimeters of distance IS GREATER THAN 1000 turn relay to off once
```

### 3. Software 

The main script which gathers data from the IF Magic modules is `sensor.py`. The following command starts the data streaming process. For details on command-line arguments, pass in the `-h` option.

```
python sensor.py
```

After this runs, you should see distance data streaming through. If the mesh network was correctly configured, you should also be able to see real-time updates to the values when moving your hand around the distance sensor.

```
Position: 3.2 feet (baseline: 3)
Position: 3.1 feet (baseline: 3)
Position: 3.5 feet (baseline: 3)
Position: 3.4 feet (baseline: 3)
...
```

Next, you need to setup the Flask app which serves JSON data to an endpoint the frontend code can query. In another terminal, start up the Flask server:

```
python server.py
```

Finally, start up the frontend server and open up a web browser to `localhost:3000`.

```
python -m http.server 3000
```

You should be able to see streaming data in the window.

## Acknowledgements

Huge shout-out to the IF Magic team, especially to Lance and Paul. Without their help this project would of not been possible.

Thank you also to the CruzHacks 2026 team for organizing such a fun event.  

## Detected evidence (automated analysis)

Indexed codebase: 6 recognized source files, 19 KB.
- CSS (language) — detected in the code
- Flask (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code

## Codebase structure (from repository index)

### Files (8 of 8)

```
.gitignore
app.js
index.html
README.md
requirements.txt
sensor.py
server.py
styles.css
```

### Dependencies

- requirements.txt: flask, flask-cors, indistinguishable-from-magic, numpy, pyserial

### Recent commits (newest first)

- Remove obsolete test.py code
- Add link to devpost
- Update README and add project overview image
- Add final code for demo
- Add new mesh code. Also add horrible test code with experiments.
- Fix some weird bug where the frontend wouldn't update
- Add index.html changes
- new changes
- Update index.html
- Update styles.css
- Update styles.css
- Update styles notification
- Update index.htm\
- Update index bottom notification
- Change endpoint to /data
- Add python backend and test file
- Updated css show vibration calculation
- Merge branch 'main' of github.com:j-silv/caneyousee
- Add working code with serial port distance sensor wired on macOS
- fetch data from other module

## Key source files (fetched from GitHub, selected and truncated for size)

### requirements.txt

```
indistinguishable-from-magic
pyserial
numpy
flask
flask-cors
```

### server.py

```python
from flask import Flask, jsonify
from flask_cors import CORS
import json
import os

app = Flask(__name__)
CORS(app)  # Allow React frontend to access from different port

SHARED_DATA_FILE = '/tmp/sensor_data.json'

@app.route('/data')
def get_sensor_data():
    """Endpoint that returns the latest sensor data"""
    try:
        if os.path.exists(SHARED_DATA_FILE):
            with open(SHARED_DATA_FILE, 'r') as f:
                data = json.load(f)
            return jsonify(data)
        else:
            return jsonify({'error': 'No data available yet. Is main.py running?'}), 503
    except json.JSONDecodeError:
        return jsonify({'error': 'Data file is corrupted or being written'}), 500
    except Exception as e:
        return jsonify({'error': str(e)}), 500

@app.route('/health')
def health():
    """Health check endpoint"""
    return jsonify({'status': 'ok'})

if __name__ == '__main__':
    print("Starting Flask server on http://localhost:5000")
    print("Endpoints:")
    print("  - GET /data - Returns latest sensor data")
    print("  - GET /health - Health check")
    app.run(host='0.0.0.0', port=5000, debug=True)

```

### app.js

```javascript
const RED = "#E30000";
const GREEN = "#059400";

// keep only last N points in chart
const MAX_POINTS = 180;

// ---- UI refs ----
const statusText = document.getElementById("statusText");
const toggleBtn = document.getElementById("toggleBtn");
const notifBanner = document.getElementById("notif");
const baselineLabel = document.getElementById("baselineLabel");
const readout = document.getElementById("readout");
const vibeText = document.getElementById("vibeText");

// baseline label (static; you can update if you calibrate)
let baselineFt = 5.0;
baselineLabel.textContent = `${baselineFt.toFixed(1)} ft Baseline`;

// ---- Chart.js ----
const ctx = document.getElementById("chart").getContext("2d");
const chart = new Chart(ctx, {
  type: "line",
  data: {
    labels: [],
    datasets: [{
      data: [],
      borderWidth: 3,
      pointRadius: 0,
      tension: 0.35,
      borderColor: GREEN
    }]
  },
  options: {
    animation: false,
    plugins: { legend: { display: false } },
    scales: { x: { display: false }, y: { display: true } }
  }
});

// ---- UI states ----
function setModeLevel() {
  notifBanner.style.display = "none";
  statusText.textContent = "Level Ground";
  statusText.style.color = GREEN;
  chart.data.datasets[0].borderColor = GREEN;

  toggleBtn.textContent = "Activate";
  toggleBtn.classList.add("on");
  toggleBtn.classList.remove("off");
}

function setModeLedge() {
  notifBanner.style.display = "block";
  statusText.textContent = "Ledge Ahead";
  statusText.style.color = RED;
  chart.data.datasets[0].borderColor = RED;

  toggleBtn.textContent = "Deactivate";
  toggleBtn.classList.add("off");
  toggleBtn.classList.remove("on");
}

// ---- Chart point function (called by poll) ----
function addPointToChart(timestamp, distanceFt) {
  const label = (typeof timestamp === "number") ? timestamp : Date.now();

  chart.data.labels.push(label);
  chart.data.datasets[0].data.push(distanceFt);

  if (chart.data.labels.length > MAX_POINTS) {
    chart.data.labels.shift();
    chart.data.datasets[0].data.shift();
  }

  chart.update("none");
}

// ===================================================
// ✅ YOUR POLL FUNCTION (fetch JSON, update UI + chart)

const DATA_URL = "http://localhost:5000/data";

async function poll() {
  const res = await fetch(DATA_URL, { cache: "no-store" });
  const json = await res.json();

  // update chart + readout (backend uses "distance" not "distance_ft")
  const distance = json.distance || 0;
  const units = json.units || "ft";
  readout.textContent = `${distance.toFixed(2)} ${units}`;
  addPointToChart(json.timestamp, distance);

  // vibration true/false drives UI
  const vib = Boolean(json.vibration);
  vibeText.textContent = `Vibration: ${vib ? "TRUE" : "FALSE"}`;

  if (vib) setModeLedge();
  else setModeLevel();
}

// Only poll while activated
let active = true; // Start as true since we auto-start polling
let pollTimer = null;

function startPolling() {
  if (pollTimer) return;
  pollTimer = setInterval(() => {
    poll().catch((err) => {
      // If backend is down / CORS / network issue, don't crash UI
      console.error("Poll error:", err);
      readout.textContent = "Connection Error";
    });
  }, 50); // 20 requests/sec for fast real-time updates
}

function stopPolling() {
  if (!pollTimer) return;
  clearInterval(pollTimer);
  pollTimer = null;
}

// Activate/Deactivate
toggleBtn.addEventListener("click", () => {
  active = !active;

  if (active) {
    setModeLevel();
    startPolling();
  } else {
    stopPolling();
    setModeLevel();
    vibeText.textContent = "Vibration: --";
    readout.textContent = "-- ft";
  }
});

// init - start in activated state
setModeLedge(); // Properly show "active" state in UI
startPolling();
```

### index.html

```html
<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1" />
  <title>Ledge Detector</title>
  <link rel="stylesheet" href="./styles.css" />
</head>
<body>
  <div class="phone">
    <div class="pill"><span id="statusText">Level Ground</span></div>

    <div class="card">
      <canvas id="chart" height="250"></canvas>

      <div class="row">
        <div class="footnote"><span id="baselineLabel">5.0 ft Baseline</span></div>
        <div class="small" id="readout">-- ft</div>
        <div class="small" id="vibeText">Vibration: --</div>
      </div>
      <div class="small" id="vibeText">Vibration: --</div>
    </div>

    <!-- This becomes Deactivate when active -->
    <button id="toggleBtn" class="on">Activate</button>

    <!-- ✅ Notification is now UNDER the button -->
    <div id="notif" class="notif">
      <b>Notification</b><br />
      Edge of sidewalk reached, continue on your current approach but take note of crosswalks nearby.
    </div>
  </div>

  <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
  <script defer src="./app.js"></script>
</body>
</html>

```

### styles.css

```css
/* styles.css */
:root{
  --red:#E30000;
  --green:#059400;
  --blue:#35539D;

  --bg:#0f1115;
  --text:#0b0d12;
  --shadow: 0 18px 50px rgba(0,0,0,.35);
  --shadow-soft: 0 10px 22px rgba(0,0,0,.18);
}

* { box-sizing: border-box; }

html, body { height: 100%; }

body {
  margin: 0;
  font-family: system-ui, -apple-system, Segoe UI, Roboto, Arial, sans-serif;
  background: radial-gradient(1200px 800px at 50% 10%, rgba(53,83,157,.35), transparent 60%),
              linear-gradient(180deg, #0b0d12 0%, var(--bg) 100%);
  color: #fff;
  display: grid;
  place-items: start center;
  padding: 20px 12px;
}

.phone {
  width: min(360px, 92vw);
  height: min(720px, 92vh);
  background:
    radial-gradient(900px 500px at 30% 10%, rgba(255,255,255,.12), transparent 55%),
    linear-gradient(180deg, rgba(255,255,255,.08), rgba(0,0,0,.10)),
    var(--blue);
  border-radius: 30px;
  padding: 18px;
  box-shadow: var(--shadow);
  position: relative;
  overflow: hidden;
  border: 1px solid rgba(255,255,255,.18);
}

.pill {
  margin: 26px auto 12px;
  width: 180px;
  text-align: center;
  padding: 10px 14px;
  border-radius: 999px;
  background: rgba(255,255,255,.92);
  font-weight: 900;
  box-shadow: 0 8px 18px rgba(0,0,0,.18);
}

#statusText { color: var(--green); }

.card {
  background: rgba(255,255,255,.96);
  border-radius: 14px;
  padding: 12px;
  margin-top: 10px;
  border: 1px solid rgba(0,0,0,.10);
  box-shadow: var(--shadow-soft);
}

canvas {
  background: rgba(255,255,255,.88);
  border-radius: 12px;
  border: 1px solid rgba(0,0,0,.08);
}

.row {
  display: flex;
  justify-content: space-between;
  align-items: center;
  margin-top: 10px;
  gap: 10px;
}

.footnote, .small {
  font-size: 12px;
  color: var(--text);
  font-weight: 700;
  opacity: 0.85;
}

button {
  width: 100%;
  margin-top: 16px;
  border: 0;
  border-radius: 999px;
  padding: 16px 14px;
  font-weight: 950;
  font-size: 16px;
  color: #fff;
  cursor: pointer;
  box-shadow: 0 10px 22px rgba(0,0,0,.22);
}

#toggleBtn.on  { background: var(--green); }
#toggleBtn.off { background: var(--red); }


.notif {
  margin-top: 12px;
  background: rgba(255,255,255,.96);
  color: var(--text);
  border-radius: 14px;
  padding: 12px 14px;
  font-size: 12.5px;
  line-height: 1.25;
  display: none; /* JS toggles to block */
  box-shadow: var(--shadow-soft);
  border: 1px solid rgba(0,0,0,.08);

  transform: translateY(-6px);
  opacity: 0;
  animation: notifIn .22s ease-out forwards;
}

.notif b { color: var(--red); }

@keyframes notifIn {
  to { transform: translateY(0); opacity: 1; }
}

```

### sensor.py

```python
from indistinguishable_from_magic import magic as Magic
import time
import numpy as np
import argparse
import random
import json
import os
from enum import Enum

class State(Enum):
    NORMAL = "normal"
    THRESHOLD_EXCEEDED = "threshold_exceeded"

SHARED_DATA_FILE = '/tmp/sensor_data.json'

def convert(datum, units="mm"):
    """Convert units of distance data"""

    if units == "mm":
            return datum
    
    if units == "feet":
        return datum / 304.8
    
    raise ValueError("Invalid units", units)

def dummy_device_read(minval=600, maxval=1400, delay_ms=100):
    """Returns a random distance value (in mm)"""

    datum = random.uniform(minval, maxval)
    time.sleep(delay_ms / 1000)
    return datum

def device_read(device, port=3):
    """Returns a distance value (in mm)"""

    device_read = device.read()
    if "70B8F665C60C" in device.mesh:
        datum = float(device.mesh["70B8F665C60C"].modules[3].data.milimeters)
        return datum
    return 0

def calibrate(cal_time, device, port=6, dummy=True, units="mm"):
    """Measure and gather distance data from port for cal_time seconds"""

    start = time.time()
    data = []
    
    while (time.time() - start) < cal_time:

        if dummy: 
            datum = dummy_device_read()
        else:
            datum = device_read(device, port)

        datum = convert(datum, units)
        data.append(datum)

    return data

def main():
    parser = argparse.ArgumentParser(
        description="Calibrate and read sensor data from device",
        formatter_class=argparse.RawTextHelpFormatter
    )
    parser.add_argument(
        "-i",
        "--device",
        type=str,
        default="/dev/cu.SLAB_USBtoUART",
        help="Device path (default: %(default)s)"
    )
    parser.add_argument(
        "-c",
        "--cal-time",
        type=float,
        default=1.0,
        help="Calibration time in seconds (default: %(default)s)"
    )
    parser.add_argument(
        "-p",
        "--port",
        type=int,
        default=2,
        help="Port number to read from (default: %(default)s)"
    )
    parser.add_argument(
        "-t",
        "--threshold",
        type=float,
        default=3.28084,
        help=("Absolute distance threshold (in feet) for vibration detection.\n"
             "If the current distance is +/- within the average baseline distance,\n"
             "this counts as a vibration trigger.\n"
             "(default: %(default)s feet)")
    )
    parser.add_argument(
        "-u",
        "--units",
        type=str,
        choices=["mm", "feet"],
        default="feet",
        help="Units for distance measurements (default: %(default)s)"
    )
    parser.add_argument(
        "-b",
        "--debounce-count",
        type=int,
        default=1,
        help="Number of consecutive measurements exceeding threshold before triggering (default: %(default)s)"
    )
    parser.add_argument(
        "-d",
        "--dummy",
        action="store_true",
        help="Use dummy device read instead of actual device (default: False)"
    )
    args = parser.parse_args()


    with Magic.Hardware(args.device) as device:
        device.connect()

        curr = time.time()

        # to make sure device is up and running
        while time.time() - curr < 2:
            _ = device.read()
            

        print(f"Calibrating for {args.cal_time} seconds on port {args.port} device {args.device}...")
        data = calibrate(args.cal_time, device, args.port, dummy=args.dummy, units=args.units)

        baseline = np.mean(data)
        # Convert threshold to the selected units (threshold default is in feet)
        threshold_converted = convert(args.threshold * 304.8, args.units)
        print(f"Baseline: {baseline:.2f} {args.units}")
        print(f"Threshold: {threshold_converted:.2f} {args.units}")
        print(f"Debounce count: {args.debounce_count} measurements")
        print(f"Monitoring for trigger...\n")

        # State machine
        state = State.NORMAL
        exceeded_count = 0

        vibrate = 1

        # device.mesh["70B8F665C60C"].modules[0].out.setState(0)
        # time.sleep(1)

        while True:
            if args.dummy:
                distance = dummy_device_read()
            else:
                distance = device_read(device, args.port)




            distance = convert(distance, args.units)
            is_threshold_exceeded = distance < threshold_converted

            # device.mesh["70B8F665C60C"].modules[0].out.setState(exceeded_count >= args.debounce_count)

            # Prepare sensor data for export
            sensor_data = {
                'distance': float(distance),
                'units': args.units,
                'vibration': exceeded_count >= args.debounce_count,
                'baseline': float(baseline),
                'threshold': float(threshold_converted),
                'timestamp': time.time()
            }

            # Write to shared file atomically
            tmp_file = SHARED_DATA_FILE + '.tmp'
            with open(tmp_file, 'w') as f:
                json.dump(sensor_data, f)
            os.rename(tmp_file, SHARED_DATA_FILE)

            if state == State.NORMAL:
                if is_threshold_exceeded:
                    # Transition to THRESHOLD_EXCEEDED state
                    state = State.THRESHOLD_EXCEEDED
                    exceeded_count = 1
                    print(f"Position: {distance:.2f} {args.units} - Threshold exceeded (baseline: {baseline:.2f} {args.units})")
                    # device.mesh["70B8F665C60C"].modules[0].out.setState(1)
                else:
                    print(f"Position: {distance:.2f} {args.units} (baseline: {baseline:.2f})")

            elif state == State.THRESHOLD_EXCEEDED:
                if is_threshold_exceeded:
                    # Still exceeding threshold, increment count
                    exceeded_count += 1
                    if exceeded_count >= args.debounce_count:
[truncated — 1043 more characters]
```