Project Info

Winner

GRAND PRIZE - Social Impact Track; Cal Hacks: Hacker's Choice

pFOG: Parkinson's Medical Aid Device

Inspiration

Someone we know has Parkinson’s disease, and like many patients, he suffers from Freezing of Gait (FOG) — a sudden, involuntary inability to walk. Recent research (N.E. Ibrahim, Nature, 2024) has shown that targeted muscle stimulation can effectively eliminate FOG. Inspired by this, his family noticed that gently poking the back of his leg with a stick during an episode helped him move again. This simple action inspired us to develop an automated, predictive solution.

What it does

Our project, pFOG, is a wearable device that predicts Freezing of Gait using sensor data and machine learning. Once FOG is detected, it gives targeted muscle stimulation to the frozen leg, mimicking the effect of a physical "poke" and helping the patient resume walking. We also built a web interface for easy data collection and labeling, which allows users to fine-tune a model using their own walking data—essential for adapting to gait variability across patients.

How we built it

We collected accelerometer and gyroscope data using an Adafruit MPU-6050 IMU mounted on a wearable module. A 1D Convolutional Neural Network (1D-CNN) extracts rich spatial features, which are fed into a Long Short-Term Memory (LSTM) model with learned temporal patterns. Together, these are able to recognize the onset of FOG from real-time motion data. Upon detection, a servo-actuated mechanism delivers localized muscle stimulation to the leg using a robotic arm, replicating the therapeutic “poke.”

Challenges we ran into

Our challenges began before the hackathon even started—we didn’t have a 3D printer! Huge thanks to Nour and Team AR Glasses for letting us run a quick print on theirs! From there: We had to improvise wiring and mechanical assemblies with whatever was available (true hacker spirit!). UART communication was not as “plug and play” as advertised—serial timing issues cost us a lot of debugging hours. Training a machine learning model that didn’t overfit and could generalize across different walking styles was tricky, especially with limited, noisy data.

Accomplishments we're proud of

We are very proud to demonstrate a working prototype that successfully detects FOG episodes with >90% accuracy and responds in real time!

FOG Detection System

A comprehensive real-time system for collecting and annotating IMU data for Freezing of Gait (FOG) detection in Parkinson's patients.

System Architecture

ESP32 → Serial → Backend (ESP32 Connector) → Flask API → Next.js Frontend
                       ↓                        ↓
                 Terminal Keys              Real-time WebSocket
                       ↓                        ↓
                 HTTP API ← ← ← ← ← ← ← ← Database (SQLite)

Components

🔧 Backend (`/backend/`)

Flask API (app.py) - REST endpoints and WebSocket server
ESP32 Connector (esp32_connector.py) - Serial bridge for real ESP32 data
SQLite Database - Stores all IMU data with 3-way classification labels
PyTorch 1d-CNN + LSTM Hybrid model - CNN extracts rich spatial features, then feeds those features to the LSTM to learn temporal patterns.

🖥️ Frontend (`/frontend/`)

Next.js App - Modern React-based web interface
Real-time Charts - Live visualization of IMU data (accelerometer & gyroscope)
State Annotation - 3-way classification: Walking / Standing / Freezing

Quick Start

1. Backend Setup

# Install backend dependencies
cd backend
pip install -r backend_requirements.txt

# Start Flask server
python app.py

2. Frontend Setup

# Install frontend dependencies
cd frontend
npm install

# Start Next.js development server
npm run dev

3. Access the System

Web Interface: http://localhost:3000
Backend API: http://localhost:6000

Usage Modes

🧪 Simulated Data (Testing)

Start both backend and frontend
Open web interface
Click "Start Recording"
Use keyboard shortcuts to annotate states:
- W - Walking
- S - Standing
- F - Freezing

🔌 Real ESP32 Data

Connect ESP32 via USB
Update serial port in backend/esp32_connector.py
Start backend and frontend
Start ESP32 connector:
```
cd backend
python esp32_connector.py
```
Real data will appear automatically in web interface

ESP32 Data Format

Your ESP32 should send CSV data over serial:

ax,ay,az,gx,gy,gz
1.23,-0.45,9.67,12.34,-5.67,8.90

Where:

ax, ay, az = Accelerometer (m/s²)
gx, gy, gz = Gyroscope (°/s)

Features

📊 Real-time Visualization

Live accelerometer and gyroscope charts with X,Y,Z legends
Color-coded state indicators
Sample counters for each state

🏷️ Data Annotation

3-way classification: Walking vs Standing vs Freezing
Keyboard shortcuts: W/S/F keys
Real-time feedback with immediate visual updates

💾 Data Management

Session recording with unique IDs
SQLite storage with timestamps and labels
CSV export for machine learning model training
Session history with statistics

🔄 Multiple Input Sources

Simulated data for testing and development
Real ESP32 data via serial connection
Manual annotation via web interface or ESP32 connector

API Endpoints

GET / - Health check
POST /start_session - Start recording session
POST /stop_session - Stop recording session
POST /annotate_state - Annotate current state ({'state': 'walking'|'standing'|'freezing'})
GET /get_sessions - Get all recording sessions
GET /get_session_data/<session_id> - Get data for specific session

WebSocket Events

imu_data - Real-time IMU data stream
state_annotation - State annotation updates
esp32_status - ESP32 connector status

Development

File Structure

parkinsons/
├── backend/
│   ├── app.py                    # Flask API server
│   ├── esp32_connector.py        # ESP32 serial bridge
│   ├── backend_requirements.txt  # Python dependencies
│   └── esp32_requirements.txt    # ESP32 connector dependencies
├── frontend/
│   ├── app/
│   │   └── page.tsx             # Main Next.js page
│   ├── lib/
│   │   └── api.ts               # API service layer
│   └── package.json             # Node.js dependencies
└── README.md                    # This file

Environment Variables

Create .env.local in /frontend/:

NEXT_PUBLIC_BACKEND_URL=http://localhost:5000

Database Schema

CREATE TABLE imu_data (
    id INTEGER PRIMARY KEY,
    timestamp TEXT,
    acc_x REAL, acc_y REAL, acc_z REAL,
    gyro_x REAL, gyro_y REAL, gyro_z REAL,
    label TEXT,  -- 'walking', 'standing', 'freezing'
    session_id TEXT
);

Troubleshooting

Backend Issues

Port 6000 in use: Change port in app.py
Database errors: Delete fog_data.db to reset
CORS errors: Ensure Flask-CORS is installed

ESP32 Issues

Serial connection failed: Check port in esp32_connector.py
Data format errors: Ensure ESP32 sends correct CSV format
Permission denied: Run with admin/sudo privileges

Frontend Issues

API connection failed: Check backend is running on port 5000
WebSocket disconnects: Check firewall settings
Build errors: Run npm install to update dependencies

Contributing

Fork the repository
Create a feature branch
Make your changes
Test with both simulated and real data
Submit a pull request

License

This project is licensed under the GPL 3.0 License - see the LICENSE file for details.

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

CSSIn code
FlaskIn code
Next.jsIn code
PythonIn code
PyTorchIn code
ReactIn code
Tailwind CSSIn code
TypeScriptIn code

8 of 8 appear in the indexed code.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

171 KB

Source files

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.