Project Info

Winner

Best Use of The Agentverse by Fetch AI; Best Use of QNX

Vanguard Telematics

GitHub Link: https://github.com/bencejdanko/VanguardTelematics Public ASI:One Shared Chat Link: https://asi1.ai/invite?channelInviteKey=EUZbyUu8nbYVK2xhekQN3EtGY60i_5mh0dGM9UuKV7I https://asi1.ai/invite?channelInviteKey=5Mb2Ov6_U4rAzxkd5rPNcIdd2zxD5tkx_E99IssFui4 Agentverse Profile: https://agentverse.ai/agents/details/agent1q05xhz7crc75lwt84v0u78mt2lswwr2a7n0wntp0zpuyedum80m3qwm4rnu/profile https://agentverse.ai/agents/details/agent1qgsr563q7gy2y97s4wp8rp3hjw3979xhcdjpnzu3f2vmf4c8pda0jkgsdsv/profile https://agentverse.ai/agents/details/agent1q0sfkv77ye39d8ctsj6rxgzapmulas4lx0ygd8ek2lxuw2ks3p3659kj04g/profile

Inspiration

82% of U.S. fire departments are all- or mostly-volunteer. Together they protect roughly 30% of the U.S. population — about 100 million people — concentrated in exactly the rural, lower-density areas where cell and radio coverage is weakest. Federal studies have found that communications outages hit within the first 24 hours in more than half of recent major U.S. wildfires. That gap is already being worked on, but not solved. FirstNet is the dominant public-safety connectivity platform and is investing in satellite extension and deployable cell sites — but those are ground-based assets an agency has to request, not something that rides on every truck by default. goTenna Pro/Pro X is real, proven off-grid mesh tech already used by humanitarian responders and law enforcement — it relays a message; it doesn't sense or diagnose anything on its own. We kept circling back to the same question: what happens in the minutes after an incident, before anyone has even picked up a radio? A vehicle that can sense and describe what just happened to it — without needing a human to triage it first — felt like a real, underserved piece of the problem.

What it does

Our project watches a vehicle's vitals in real time — accelerometer, vibration, temperature, and pressure from an industrial sensor rig, plus a proximity sensor and a camera that feeds live video the moment something gets too close. All of it streams into a live dashboard with a 3D model of the vehicle (we used a model Cybertruck) that visually reacts to the incoming data, so a human can see exactly what the truck is "feeling" at any moment. Underneath that, a custom Fetch.ai multi-agent system watches the same streams and decides, on its own, when something anomalous has actually happened — then generates a free-text, situation-specific description of the event instead of picking from a small, hardcoded list of crash types. The same pipeline is meant to generalize across first-responder vehicle types — an ambulance and a fire engine have very different normal operating signatures, and the system is built to account for that rather than assume one vehicle profile.

How we built it

On the hardware side: an industrial Nucleo sensor rig feeding accelerometer, vibration, temperature, and pressure data, plus a proximity sensor and camera, all running on a Raspberry Pi 5 under QNX — a real-time OS providing first responders reliability they need. Hence, event detection stays low-latency and deterministic instead of fighting a general-purpose scheduler. Redis sits behind everything as the live telemetry store both the dashboard and the agent pipeline read from. The crash-detection brain is a layered Fetch.ai (uAgents) pipeline. Since we didn't have a pre-built "sentry" crash-detection signal to lean on, the trigger agent had to derive one itself: it keeps a rolling, per-vehicle baseline (mean and standard deviation) for each sensor feature and computes a z-score, $$ z = \frac{x - \mu}{\sigma} $$ flagging an event only when multiple sensors cross threshold together, for several consecutive samples — which filters out single-sensor noise like a pothole or a slammed door while still catching real multi-sensor anomalies. That trigger hands a buffered pre/post-event window to downstream agents, which extract physics features, pull context for the specific vehicle, and feed an LLM-grounded synthesis step that writes an open-ended incident description rather than forcing the event into one of a few fixed categories.

Challenges we ran into

We originally wanted true off-grid relay — in the spirit of what goTenna does for mesh comms — so the system could report out even with no cell or wifi coverage. We tried wiring a radio link to the Pi 5 and hit a wall on two fronts at once: regulatory limits on unlicensed RF transmission, and practical RF interference that made it unreliable to get clean text through to and from the board. We made the call to descope it for the weekend and treat it as the natural next step rather than a hackathon deliverable. The other real challenge was the crash detector itself. Without an existing crash-detection signal to wrap, we had to build trigger logic from raw sensor physics — and do it in a way that wouldn't quietly bake in assumptions about one specific vehicle or one specific kind of accident. Getting a rolling baseline that adapts per vehicle, instead of a single hardcoded threshold, took more iteration than we expected, especially separating real signal from sensor noise without throwing away genuine events.

Accomplishments we're proud of

Building a dashboard visualizer that showed us our data is realistic and reliable to actually generate meaningful analytics from. Getting real sensor hardware talking to a real-time OS on actual embedded hardware, end to end, in a weekend. Building an anomaly detector from first principles — rolling baselines and multi-sensor z-scores — instead of a hardcoded if/else crash classifier, paired with a reasoning layer that produces open-ended, vehicle-specific incident descriptions instead of forcing every event into one of three canned labels. And doing the homework before writing code: we went in with real numbers on who this actually affects and a clear-eyed, so we knew what gap we were actually trying to close.

What we learned

Radio is a much harder problem than "just send a signal" — between licensing and interference, it's a real compliance and RF-engineering problem, not a wiring exercise you can finish in a weekend. We also learned that you can't shortcut crash detection with a single magic threshold number if you actually care about it generalizing across vehicles — you have to understand what a real anomaly looks like across multiple sensors at once, which pushed us toward a statistically grounded approach instead of guessed constants. And splitting the pipeline into agents — deterministic, testable signal processing in one layer, open-ended LLM reasoning in another — made the whole system much easier to debug than one big classifier would have been.

What's next

Solving off-grid relay properly — likely by pairing with certified mesh hardware like goTenna rather than rolling raw radio from scratch, so we're not fighting licensing and interference ourselves. Real-world impact and drop testing across more vehicle types (ambulance, fire engine) to calibrate the rolling baselines per vehicle class instead of relying on bench simulation. And closing the loop with multi-agent system to gain more meaningful analytics to aide second responders, and inform authorities of the disaster.

Vanguard Telematics: UC Berkeley AI Hackathon

1st place prize QNX track + honorable mentions @ Fetch AI

They protect us. Who protects them? This is more than a hackathon idea... Vanguard Telematics is our effort to ensure that those who run toward danger are never left behind in the data silence.

Quick Links

ASI:One Shared Chat Links:
- Link 1
- Link 2
Agentverse Profiles:
- Agent 1
- Agent 2
- Agent 3

The Blind Spot in Disaster Response (Inspiration)

82% of U.S. fire departments are all- or mostly-volunteer, protecting roughly 100 million people concentrated exactly in the rural, lower-density areas where cell and radio coverage is weakest. Federal studies have found that communications outages hit within the first 24 hours in more than half of recent major U.S. wildfires.

Current solutions fall short:

Infrastructure Dependencies: FirstNet is the dominant public-safety connectivity platform and invests in deployable cell sites—but those are requested ground assets, not native gear riding on every truck by default.
Blind Communications Pipes: Mesh tech like goTenna Pro/Pro X is real, proven off-grid tech used by humanitarian responders. However, it relays messages perfectly but acts solely as a pipe. It doesn't sense or diagnose anomalies on its own.
The Operator Dependency: If an incident occurs and the operator is incapacitated, the best radio network in the world cannot transmit a cry for help.

We kept circling back to the same question: what happens in the minutes after an incident, before anyone has even picked up a radio? A vehicle that can sense and describe what just happened to it—without needing a human to triage it first—is a massive, underserved need.

What It Does: Live 3D Telemetry & Edge Sensing

All of it streams into a live dashboard with a 3D model (we used a Cybertruck) that visually reacts to the incoming data. A human supervisor can see exactly what the truck is "feeling" at any exact millisecond, without relying on verbal reports.

Underneath that, a custom Fetch.ai multi-agent system watches the same streams and decides, on its own, when something anomalous has actually happened. It then generates a free-text, situation-specific description of the event instead of picking from a small, hardcoded list of crash types. The pipeline generalizes across first-responder vehicle types, accounting for the very different normal operating signatures of an ambulance versus a fire engine.

Deterministic Edge Architecture: How We Built It

Hardware & Real-Time Foundation

Raspberry Pi 5 + QNX: The rig runs under QNX—a real-time OS providing the strict reliability first responders need. Event detection stays low-latency and deterministic instead of fighting a general-purpose scheduler.
Industrial Nucleo Rig: Feeds raw accelerometer, vibration, temperature, and pressure data seamlessly. It processes vehicular physics directly, remaining completely agnostic to specific vehicle CAN buses.
Redis Telemetry Store: Redis sits behind everything as the live, ultra-low latency telemetry store. Both the dashboard visualizer and the downstream agent pipeline read directly from this high-speed memory layer.

The Crash-Detection Brain

Built as a layered Fetch.ai (uAgents) pipeline. Because we didn't have a pre-built "sentry" crash-detection signal to lean on, the trigger agent had to derive one itself. It maintains a rolling, per-vehicle baseline (mean and standard deviation) for each sensor feature and computes a z-score. It flags an event only when multiple sensors cross the threshold together for several consecutive samples. This filters out single-sensor noise (like a pothole or a slammed door) while catching real multi-sensor anomalies.

That trigger hands a buffered pre/post-event window to downstream agents, which extract physics features, pull context for the specific vehicle, and feed an LLM-grounded synthesis step that writes an open-ended incident description rather than forcing the event into a few fixed categories.

Challenges & Technical Realities

The Radio Reality: We originally wanted true off-grid relay—in the spirit of what goTenna does for mesh comms—so the system could report out even with no cell or Wi-Fi coverage. We tried wiring a radio link to the Pi 5 and hit a wall: strict FCC regulatory limits on unlicensed RF and massive practical RF interference made it unreliable to get clean text through to and from the board. We made the call to descope it for the weekend and treat it as a natural next step.
Generalized Detection: Without an existing crash-detection signal to wrap, we had to build trigger logic from raw sensor physics. We wanted to avoid arbitrary thresholds and generalizing events across vehicles. Getting a rolling baseline that adapts per vehicle—separating real multi-sensor signals from noise without missing genuine events—took immense iterations.

Accomplishments We're Proud Of

Building a dashboard visualizer that showed our data was realistic and reliable enough to generate meaningful analytics from.
Getting real sensor hardware talking to a real-time OS on actual embedded hardware, end-to-end, in a single weekend.
Building an anomaly detector from first principles (rolling baselines and multi-sensor z-scores) instead of a hardcoded if/else crash classifier.
Pairing deterministic signal processing with a reasoning layer that produces open-ended, vehicle-specific incident descriptions instead of forcing every event into canned labels.
Doing the homework before writing code: we went in with real numbers on who this actually affects and a clear-eyed view of the gap we were actually trying to close.

What We Learned

Radio is a much harder problem than "just send a signal"—between licensing and interference, it's a real compliance and RF-engineering problem, not a wiring exercise you can finish in a weekend. We also learned that you can't shortcut crash detection with a single magic threshold number if you actually care about it generalizing across vehicles. You have to understand what a real anomaly looks like across multiple sensors at once, which pushed us toward a statistically grounded approach instead of guessed constants. And splitting the pipeline into agents—deterministic, testable signal processing in one layer, open-ended LLM reasoning in another—made the whole system much easier to debug than one big classifier would have been.

The Road Ahead

Certified Mesh Integration: Solving off-grid relay properly by pairing our data pipeline with certified mesh hardware like goTenna Pro, rather than fighting licensing constraints and raw RF interference ourselves.
Real-World Drop Testing: Executing physical impact testing across more vehicle types (ambulance, fire engine) to calibrate the rolling baselines per vehicle class instead of relying purely on bench simulation.
Analytics Loop: Closing the loop with the multi-agent system to gain more meaningful macro-analytics to aid second responders, helping inform authorities of shifting disaster zones.

Technical Details

DataLogFusion Clone

Repurposed from https://www.st.com/en/embedded-software/x-cube-mems1.html. Modified clone of the DataLogFusion firmware from STMicroElectronics: \x-cube-mems1\Projects\NUCLEO-F401RE\Applications\IKS5A1\DataLogFusion

Logs to Serial port - 115200 8N1

The sampling frequency of the measurements is 100 Hz (a sampling period of 10 ms).

# app_mems.c

#define ALGO_FREQ  100U /* Algorithm frequency 100Hz */
#define ALGO_PERIOD  (1000U / ALGO_FREQ) /* Algorithm period [ms] = 10ms */

Output format

timestamp,acc_x_mg,acc_y_mg,acc_z_mg,gyr_x_mdps,gyr_y_mdps,gyr_z_mdps,mag_x_mgauss,mag_y_mgauss,mag_z_mgauss,press_hpa,roll_deg,pitch_deg,yaw_deg
00:02:54.60,-50,-21,996,0,210,-210,526,-162,-345,1005.83,17.68,2.88,1.12
00:02:54.61,-51,-22,997,-210,70,-210,531,-165,-345,1005.83,17.68,2.88,1.12

CSV Header Summary

Column Name	Data Type	Units	Description
`timestamp`	String	`HH:MM:SS.hh`	Real-Time Clock (RTC) timestamp (Hours:Minutes:Seconds.hundredths)
`acc_x_mg` / `y` / `z`	Integer	$mg$ (milli-g)	Accelerometer axes value ($1000\text{ mg} \approx 9.81\text{ m/s}^2$)
`gyr_x_mdps` / `y` / `z`	Integer	$mdps$	Gyroscope axes angular rate value (milli-degrees per second)
`mag_x_mgauss` / `y` / `z`	Integer	$mgauss$	Magnetometer axes magnetic field intensity value (milli-gauss)
`press_hpa`	Float	$hPa$	Ambient atmospheric pressure (hectopascals)
`roll_deg`	Float	Degrees	Estimated device Roll angle from MotionFX Sensor Fusion
`pitch_deg`	Float	Degrees	Estimated device Pitch angle from MotionFX Sensor Fusion
`yaw_deg`	Float	Degrees	Estimated device Yaw (heading) angle from MotionFX Sensor Fusion

QNX configuration

Plug in the kit.

[qnxuser@qnxpi18 ~]$ usb -v
USB 0 (XHCI) v10.00, v1.01 DDK, v2.00 HCD, DLL: Active
    Control, Interrupt, Bulk(SG), Isoch(Stream), High Speed, Super Speed, DMA:32-bit

Device Address             : 1
Upstream Host Controller   : 0
Upstream Device Address    : 0
Upstream Port              : 1
Upstream Port Speed        : Full
Vendor                     : 0x0483 (STMicroelectronics)
Product                    : 0x374b (STM32 STLink)
Device Release             : r1.00
Class                      : 0xef (Miscellaneous)
Subclass                   : 0x02
Protocol                   : 0x01
Max PacketSize0            : 64
Configurations             : 1
  Configuration            : 1
    Attributes             : 0x80 (Bus-powered)
    Max Power              : 300 mA

USB 1 (XHCI) v10.00, v1.01 DDK, v2.00 HCD, DLL: Active
    Control, Interrupt, Bulk(SG), Isoch(Stream), High Speed, Super Speed, DMA:32-bit

You must configure the the driver tool to help the pi/qnx to understand the USB

https://devblog.qnx.com/get-gps-data-on-qnx-with-a-usb-gps/

# you can request the binary from john in the discord
# or it is in the tools somewhere :D
scp devc-serusb qnxuser@172.20.10.7:/tmp/

# give permissions to the binary and configure anything active
chmod +x /tmp/devc-serusb
sudo /tmp/devc-serusb

# check what serial line the USB device is using
ls -l /dev/ser*

# ex gives me /dev/serusb1

# set the correct baud rate to that line
stty baud=115200 < /dev/serusb1

# verify 
stty < /dev/serusb1

Raspberry Pi Hardware Sensor Code

We have cloned QNX's example repository for sensors:

https://github.com/qnx/Raspberry-Pi-Hardware-Component-Samples (pi sensor sample code)

https://gitlab.com/qnx/projects/camera-projects/applications/camera-dump-frame-no-screen (camera sample code)

And modified for "poor mans sentry mode", where we use an infrared sensor to start a camera only when infrared activity is detected

Wiring IR Obstacle Sensor

Wiring: Connected VCC to 3.3V to protect the Pi's GPIO pins, GND to ground, and SIG (Signal / OUT) to GPIO 21.

# ssh helper shortcut 
ssh qnxuser@172.20.10.7

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

AnthropicIn code
CIn code
CSSIn code
FastAPIIn code
HTMLIn code
OpenAIIn code
PythonIn code
ReactIn code
RedisIn code
TypeScriptIn code

10 of 10 appear in the indexed code.

AI coding agents

Claude CodeConfig · Commits
CursorConfig

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

1.3 MB

Source files

218

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.

Feature verification

7 Verified3 Code-supported1 Claimed only

Fetch.ai multi-agent crash trigger: rolling per-vehicle baseline and multi-sensor z-scoreVerified
A trigger agent keeps a rolling per-vehicle baseline (mean/std) per sensor feature and computes a z-score, flagging an event only when multiple sensors cross threshold together for several consecutive samples
Claimed on readmehigh confidence
- backend/agent/agent.py:89 — RollingBaseline.z_score computes (value-mean)/std from real rolling statistics
- backend/agent/agent.py:173 — requires len(triggered features) >= MIN_FEATURES_TRIGGERED for CONSECUTIVE_SAMPLES_REQUIRED ticks before firing
- backend/agent/agent.py:236 — _detectors dict keyed by vehicle_id gives each vehicle its own baseline instance
IR obstacle sensor triggers camera capture on Raspberry Pi (poor man's sentry mode)Verified
An infrared sensor wired to GPIO 21 starts a camera only when infrared/proximity activity is detected
Claimed on readmehigh confidence
- sentry-mode/pi_sentry_client.py:239 — GPIO.setup(SIG_PIN,...) configures the IR sensor pin
- sentry-mode/pi_sentry_client.py:246 — if GPIO.input(SIG_PIN)==0 triggers camera capture on obstacle detection
Live 3D dashboard with vehicle model reacting to sensor dataVerified
A live dashboard with a 3D model of the vehicle (Cybertruck) that visually reacts to incoming data
Claimed on readmehigh confidence
- frontend/src/components/CybertruckViewer.tsx:21 — loads real cybertruck.glb via useGLTF
- frontend/src/components/CybertruckViewer.tsx:82 — useFrame slerps model rotation each frame toward live pitch/roll/yaw props sourced from the sensor stream
Live analytics charts on dashboardVerified
Dashboard shows meaningful real-time analytics/charts of vehicle telemetry
Claimed on Devpostmedium confidence
- frontend/src/components/TelemetryDashboard.tsx:93 — recharts LineChart components render live gyroscope/accelerometer data from the stream
Real-time vehicle vitals sensing (accelerometer, vibration, temperature, pressure)Verified
Watches a vehicle's vitals in real time: accelerometer, vibration, temperature, and pressure from an industrial sensor rig
Claimed on readmehigh confidence
- backend/agent/agent.py:123 — _extract_features computes accel_mag, accel_jerk, vibration, temp_rate, pressure_rate from telemetry
- frontend/src/hooks/useSensorStream.ts:49 — EventSource SSE stream parses live accel/gyro/pressure JSON into SensorData; temperature field present but parsing left commented out
Redis as shared live telemetry store for dashboard and agent pipelineVerified
Redis sits behind everything as the live telemetry store both the dashboard and the agent pipeline read from
Claimed on readmehigh confidence
- backend/api.py:72 — xadd/xread against Redis stream backs the SSE dashboard feed
- backend/agent/agent.py:245 — trigger agent polls the same Redis stream via xread using shared REDIS_STREAM_KEY
STM32 Nucleo industrial sensor rig running modified DataLogFusion firmware at 100Hz CSV outputVerified
An industrial Nucleo sensor rig feeding accelerometer, vibration, temperature, and pressure data, a modified DataLogFusion clone sampling at 100Hz over serial CSV
Claimed on readmehigh confidence
- Src/app_mems.c:39 — ALGO_FREQ 100U defines 100Hz sampling as documented
- Src/app_mems.c:232 — prints CSV header and formatted rows matching README's documented output format
Camera feed triggered by proximity sensor shown in dashboardCode-supported
A proximity sensor and a camera that feeds live video the moment something gets too close, visible in the dashboard
Claimed on readmemedium confidence
- sentry-mode/pi_sentry_client.py:246 — IR sensor on GPIO reads SIG_PIN and triggers camera capture on obstacle detection (poor man's sentry mode)
Open-ended LLM-generated incident description instead of a small hardcoded list of crash typesCode-supported
Generates a free-text, situation-specific description of the event instead of picking from a small, hardcoded list of crash types
Claimed on readmemedium confidence
- backend/agent/agent.py:501 — OpenAI-compatible client call to api.asi1.ai produces free-text incident synthesis grounded in telemetry
- backend/agent/agent.py:432 — classify_event() is a hardcoded if/elif tree over 6 fixed labels, and the LLM prompt forces the response to begin with that fixed classification string, so the free text wraps a small hardcoded taxonomy rather than replacing it
Raspberry Pi 5 running QNX real-time OS for low-latency deterministic event detectionCode-supported
Hardware runs on a Raspberry Pi 5 under QNX, a real-time OS providing deterministic, low-latency event detection
Claimed on readmemedium confidence
- sentry-mode/README.md:1 — documents QNX toolchain build commands (qcc -Vgcc_ntoaarch64le) and QNX sensor driver invocation for the Pi target
Pipeline generalizes across vehicle classes (ambulance vs fire engine) with per-class baselinesClaimed only
The pipeline generalizes across first-responder vehicle types, accounting for different normal operating signatures of an ambulance versus a fire engine
Claimed on readmehigh confidence

An AI agent derived these features from the project’s Devpost page and readme, then searched the code for each one. Verified features are backed by cited code; claimed-only features had no supporting code, which is not by itself proof a feature is missing.