Project Info

HiveSight: Real-Time Multi-Camera Spatial Intelligence

Inspiration

HiveSight started with a simple question: how do teams share awareness when visibility is low and information is fragmented? We were inspired by high-stakes situations like firefighters in smoke-filled buildings and police officers coordinating through bodycams—scenarios where communication can be chaotic and each person only sees a small slice of the environment. We wondered what would happen if those individual viewpoints could be combined into one shared, live map. Instead of describing what you see over the radio, what if your team could just see it?

What it does

HiveSight is a real-time multi-camera spatial awareness system that turns several iPhones into a collaborative tracking network. One iPhone acts as a spatial anchor, while the others act as mobile “bodycams.” Each phone: Determines its relative position to the anchor using Nearby Interaction (UWB) Measures its camera orientation using ARKit Streams its live camera feed via MJPEG to a central HTTPS server On a central computer, we run YOLO with BoT-SORT-ReID to detect and consistently track people across frames—even through partial occlusion or temporary exits from view. We then combine each phone’s detections to estimate the 2D positions of target people. The frontend is a map built with HTML, CSS, and JavaScript that shows team member locations and directions, detected people (including last-seen positions), and shortest paths from each team member to a target. The idea is similar to a minimap in a video game—but with real-world sensor and vision data!

How we built it

The project had four main components, developed in parallel: Mobile app (Swift) We built a custom iOS app using SwiftUI that uses Nearby Interaction for relative positioning, ARKit for orientation data, and AVFoundation to stream camera feeds via MJPEG. This creates our own lightweight streaming pipeline that updates every video frame. Detection and tracking (Python) On the backend, we used PyTorch with Ultralytics YOLO for person detection and integrated BoT-SORT with ReID for a consistent identity tracking method. OpenCV and NumPy helped with frame handling and preprocessing data. Spatial fusion logic We used bounding box detections to estimate 2D positions of detected people relative to each camera. We also implemented simple persistence logic to maintain “last known” positions when someone left the frame. Frontend visualization (Web stack) We built a 2D map using HTML, CSS, and JavaScript intended to display the final scene. Phones are rendered as dots with direction, and detected individuals are plotted in the space. We also added shortest-path visualization to demonstrate potential tactical use cases.

Challenges we ran into

Pivoting from laptops to phones Our original plan used laptops as cameras, but we quickly realized they couldn’t provide accurate indoor positioning or orientation data. Switching to iPhones meant rebuilding our streaming pipeline in Swift to ensure a more robust solution. Maintaining consistent IDs Early on, YOLO frequently reassigned IDs when someone left and re-entered the frame. We spent a significant amount of time tuning BoT-SORT-ReID parameters and experimenting with different configurations to improve stability. System integration Each component initially worked independently, and getting everything to operate together in real time was a significant challenge. Much of our hackathon time went into testing, refactoring, and reconnecting pieces, and we ultimately didn’t have the opportunity to fully combine each phone’s data into the live map.

Accomplishments we're proud of

Learning how to code custom apps with Swift in a weekend Successfully creating a pipeline to perform real-time computer vision across multiple devices Achieving persistent person tracking using computer vision

What we learned

This project taught us how challenging multi-view spatial reasoning can be in practice, and how finicky tracking systems across devices can be! Other things we learned include: How to connect mobile hardware sensing with ML pipelines How to debug cross-platform issues quickly under hackathon constraints How important clear delegation is in team development Overall, it was a great exercise in full-stack development and rapid prototyping and debugging.

What's next

If we continue developing HiveSight, we would focus on: Fully integrating camera data into a live global map Utilizing multiple anchor phones or a different tracking system for better positional data Incorporating a data gathering tool for unknown indoor environments Reducing latency for camera streams Experimenting with AR overlays for smart glasses HiveSight is still an early prototype, but we believe its real-world applications would be very impactful!

Priority Assignment Engine

Running the Frontend

Install dependencies:
```
npm install
```
Start the dev server:
```
./node_modules/.bin/vite
```
Open in browser: Visit the URL shown in the terminal (typically http://localhost:5173/).

Live Demo

When LIVE DEMO is pressed, the frontend polls fusion camera data and plots agents (cameras) and targets (fused tracks) on the map.

Option A: Real-time camera pipeline (camera position + target detection)

Start the live fusion server (optionally spawns camera automatically):
```
python -m fusion.live_fusion --with-camera
```
With --with-camera, live_fusion spawns camera.py (webcam + YOLO + MJPEG stream). Without it, start camera.py separately (see step 2).

Start camera.py (if not using --with-camera):

python computervision/camera.py --camera-id cam1 --source 0 --show \
    --emit camera+tracks --cam-x 0 --cam-y 0 --yaw-deg 0 --hfov-deg 70 \
    --udp-port 5055 --stream-port 5056

Start the frontend (npm run dev or ./node_modules/.bin/vite) and click LIVE DEMO. The Live Demo page shows camera feeds at the top and the fusion map (agents + targets) below.

Option B: Simulated data (no camera)

Start the cam_view server (simulated fusion data):
```
python -m fusion.cam_view.app
```
Runs on port 5051 by default.
Start the frontend and click LIVE DEMO.

If no server is running on 5051, the frontend falls back to mock data. The frontend proxies /api/fusion/map → http://127.0.0.1:5051/api/map. Cameras become agents; fused_tracks become targets. Coordinates are converted from the fusion room (0–12 m × 0–10 m) to the frontend canvas.

Running the Backend (optional)

python -m assignment_model.assignment              # Demo simulation
python -m assignment_model.assignment --serve      # Start HTTP + WebSocket server
uvicorn assignment_model.assignment:app --reload --port 8001   # Alternative server launch

Project Structure

├── index.html              # HTML entry point
├── main.jsx                # React entry point
├── frontend/
│   ├── App.jsx             # Main React component (UI, controls, tabs)
│   ├── config.js           # World constants, colors, thresholds
│   ├── utils.js            # Math utilities (euclidean, randomWalk)
│   ├── assignment.js       # Priority assignment algorithm (P1/P2/proximity)
│   ├── canvas.js           # Canvas renderer
│   ├── distances.js        # Distance matrix computation
│   └── pathfinding.js      # A* pathfinding
├── assignment_model/
│   ├── assignment.py       # Python entry point
│   ├── engine.py           # Assignment engine
│   ├── server.py           # FastAPI + WebSocket server
│   ├── demo.py             # Demo simulation
│   └── models.py           # Data models
├── package.json
└── vite.config.js

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

CSSIn code
FlaskIn code
HTMLIn code
JavaScriptIn code
PythonIn code
ReactIn code
SwiftIn code
PyTorchClaimed

7 of 8 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

CursorCommits

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

493 KB

Source files

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.