# Project export: Slouching Slugs

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: CruzHacks 2025
- Tagline: "Sit Right, Work Bright – Your AI Posture Coach"
- Devpost: https://devpost.com/software/slouching-slugs
- GitHub: https://github.com/NguyenEvan/slouching-slugs
- Demo: https://docs.google.com/presentation/d/1R5XnjSk3gPbGRf6DyGGSn5Fg7JppibTeHAiDHtz8LWg/edit?usp=sharing
- Team: 1 GitHub contributor(s) — Evan (5 commits)

## Devpost submission (written by the team)

### Inspiration

As STEM students, we spend countless hours hunched over screens, often ignoring our posture until the pain kicks in. Yet, we easily notice when others slouch! We realized that if we had a real-time reminder to sit straight, we could prevent long-term damage. That’s why we built Slouching Slugs—an AI-powered posture guard that nudges you before bad habits occur.

### What it does

We utilize the built-in webcam or an external one and send a live feed to our program that detects if the user is slouching or not, and if so, a notification is sent to the user's desktop as well as a pop-up with advice on how they can improve

### How we built it

We engineered Slouching Slugs entirely in Python, using OpenCV for real-time webcam processing and MediaPipe to track critical posture landmarks (shoulders, nose, eyes). Our system implements a 200-frame image buffer that analyzes posture state across multiple frames - only triggering Gemini API feedback when 75% of buffered frames (150+ images) confirm slouching, preventing false alarms. After generating personalized corrective advice (e.g., "Ensure your monitor is at eye level to avoid hunching over"), the deque automatically clears to avoid notification spam. The frontend delivers these insights through a Tkinter GUI and subprocess-powered desktop alerts, creating a seamless feedback loop.

### Challenges we ran into

One challenge we encountered was adjusting the sensitivity of the slouch detector; at times, it was too sensitive, and a slight movement would cause it to send a notification.

### Accomplishments we're proud of

We are most proud of being able to fully commit ourselves to this project and achieve what we have done today through 48 hours of coding, debugging, learning, and collaboration. This was our first hackathon, so we did not know what to expect. We learned a lot about building an application from scratch.

### What we learned

We learned how to utilize more technologies, such as the variety of Python libraries we used, and gained more experience with AI and LLMS through Gemini. Beyond tech, we also better understood how our postures can affect us in the long run and key points to look out for. Alongside how we will approach the next hackathon with better time management and organization and not be afraid to branch out and go all-in on an ambitious project.

### What's next

We plan on scaling this more through creating our own convolutional neural network and training it to our standards by having actual people be subjects and documenting their posture to provide our program with much better accuracy. Also, we would want to improve our user interface and include more features and customization for our users.

## README (from the GitHub repository)

# CruzHacks2025

## By: Wilson Xie, Evan Nguyen, Austin Lien

## 💻 How to Run

To launch Slouching Slugs, make sure you have Python 3.10+ and the required packages installed.

1. Clone the repository and navigate to the project directory:

```bash
git clone https://github.com/your-username/slouching-slugs.git
cd slouching-slugs
pip install -r requirements.txt
python ./app/gui_app.py
```


## Inspiration

As STEM students, we spend countless hours hunched over screens, often ignoring our posture until the pain kicks in. Yet, we easily notice when others slouch! We realized that if we had a real-time reminder to sit straight, we could prevent long-term damage. That’s why we built Slouching Slugs—an AI-powered posture guard that nudges you before bad habits occur.

## What it does

We utilize the built-in webcam or an external one and send a live feed to our program that detects if the user is slouching or not, and if so, a notification is sent to the user's desktop as well as a pop-up with advice on how they can improve 

## How we built it

We only used Python to code this, as it offers the most diverse range of libraries we can utilize. Some important libraries we used were mediapipe to help map out the important points on the body, such as the shoulders, nose, and eyes. Then, we created our own program to determine if the user is slouching or not utilizing those points. We then take a snapshot of the slouch and send that to gemini AI to provide advice to the user on what and how to improve. 

## Challenges we ran into

A big challenge we encountered was adjusting the sensitivity of the slouch detector; at times, it was too sensitive, and just a slight movement would cause it to send a notification. 

## Accomplishments that we're proud of

We are most proud of being able to fully commit ourselves to this project and achieve what we have done today within 48 hours. As this was our first hackathon, we came in with low expectations; however, those were easily exceeded. 

## What we learned

We learned how to utilize more technologies, such as the variety of Python libraries we used, and gained more experience with AI and LLMS through Gemini. Beyond tech, we also better understood how our postures can affect us in the long run and key points to look out for. Alongside how we will approach the next hackathon with better time management and organization and not be afraid to branch out and go all-in on an ambitious project. 

## What's next for Slouching Slugs

We plan on scaling this more through creating our own convolutional neural network and training it to our standards by having actual people be subjects and documenting their posture to provide our program with much better accuracy. Also, we would want to improve our user interface and include more features and customization for our users. 

## Detected evidence (automated analysis)

Indexed codebase: 6 recognized source files, 18 KB.
- Python (language) — detected in the code
- Google Gemini (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (8 of 8)

```
.gitignore
app/gemini/feedback.py
app/gui_app.py
app/pose/detector.py
app/posture_loop.py
app/slouch_logic.py
README.md
requirements.txt
```

### Dependencies

- requirements.txt: google, mediapipe, opencv-python, PIL, python-dotenv, win10toast-click

### Recent commits (newest first)

- update readme
- delete init.py
- vfinal
- v1
- Template
- first commit

## Key source files (fetched from GitHub, selected and truncated for size)

### requirements.txt

```
opencv-python
mediapipe
win10toast-click
python-dotenv
google
PIL

```

### app/slouch_logic.py

```python
# app/slouch_logic.py
import mediapipe as mp

# Pose landmark indexes
mp_pose = mp.solutions.pose

# Pose landmark indexes
LEFT_SHOULDER = mp_pose.PoseLandmark.LEFT_SHOULDER.value
RIGHT_SHOULDER = mp_pose.PoseLandmark.RIGHT_SHOULDER.value
NOSE = mp_pose.PoseLandmark.NOSE.value
LEFT_EAR = mp_pose.PoseLandmark.LEFT_EAR.value
RIGHT_EAR = mp_pose.PoseLandmark.RIGHT_EAR.value


def detect_slouch(landmarks):
    left_shoulder = landmarks[LEFT_SHOULDER]
    right_shoulder = landmarks[RIGHT_SHOULDER]
    nose = landmarks[NOSE]
    avg_shoulder_x = (left_shoulder.x + right_shoulder.x) / 2
    # print("left shoulder x", left_shoulder.x)
    # print("right shoulder x", right_shoulder.x)
    z_face_values = []
    for i in range(0, 11):
        z_face_values.append(landmarks[i].z)
    average_z = sum(z_face_values) / len(z_face_values)

    head_forward_displacement = nose.z - (left_shoulder.z + right_shoulder.z) / 2
    # print("head forward displacement", head_forward_displacement)


    z_to_should_displace = average_z - (left_shoulder.z + right_shoulder.z) / 2 
    #print ("average z to shoulder didsplacement", z_to_should_displace)
    slouching_z = z_to_should_displace < -0.80 #adjust based on testing

    slouching_1 = head_forward_displacement < -1.00  # adjust based on testing
    slouching_2 = abs(nose.x - avg_shoulder_x) > 0.04
    # print("slouching 2", abs(nose.x - avg_shoulder_x))
    #print("slouching 1", head_forward_displacement)
    slouching_features = None
    return z_to_should_displace
    if slouching:
        slouching_features = detect_slouch_gemini(landmarks=landmarks)
    
    if slouching_features and slouching_features["is_slouching"]:
        return True
    return False

```

### app/posture_loop.py

```python
from pose.detector import process_frame, draw_landmarks
from slouch_logic import detect_slouch
from gemini.feedback import send_posture_alert, analyze_posture_with_gemini
from collections import deque


MAX_SCORE = 200
slouch_score = 0
slouch_triggered = False

def process_frame_with_posture(frame, pose_model, frame_buffer, landmarks_callback=None):
    global slouch_score, slouch_triggered
    processed_frame, results = process_frame(frame, pose_model)

    if results.pose_landmarks:
        draw_landmarks(processed_frame, results.pose_landmarks)
        landmarks = results.pose_landmarks.landmark
        z_score = detect_slouch(landmarks)
        slouching = z_score < -0.85
        # print("score", z_score)
        if slouching:
            slouch_score = min(slouch_score + 1, MAX_SCORE)
        else:
            slouch_score = max(slouch_score - 1, 0)

        report = ""

        if slouch_score > 0.75 * MAX_SCORE:
            print("⚠️ Slouching detected!")
            #placeholder
            #get lowest z_average from frame buffer
            min_z = float('inf')
            min_img = None
            for z_score, img in reversed(frame_buffer):
                # print("z_score type:", type(z_score))
                if z_score < min_z:
                    min_z = z_score 
                    min_img = img
            
            report = analyze_posture_with_gemini(min_img)
            print(report)
            #done doing gemini report

            frame_buffer = deque()

            send_posture_alert()
            slouch_score = 0

        if slouching and not slouch_triggered:
            slouch_triggered = True
        elif not slouching:
            slouch_triggered = False

    return processed_frame, z_score, report

```

### app/gui_app.py

```python

import cv2
import tkinter as tk
from tkinter import ttk
from PIL import Image, ImageTk
from pose.detector import init_pose
from posture_loop import process_frame_with_posture
from collections import deque


class SlouchDetectorApp:
    def __init__(self, window):
        self.window = window
        self.window.title("Posture Detector")
        self.window.geometry("1200x680")
        self.running = False
        self.pose_model = init_pose()
        self.z_buffer = deque(maxlen=100)

        self.style = ttk.Style()
        self.style.theme_use("clam")
        self._configure_pastel_theme()

        self.setup_ui()

    #cream pastel background
    def _configure_pastel_theme(self):
        self.window.configure(bg="#fdf6f0")
        self.style.configure("TFrame", background="#fdf6f0")
        self.style.configure("TLabel", background="#fdf6f0", foreground="#4a4a4a", font=("Segoe UI", 10))
        self.style.configure("TButton",
                             background="#ffeaa7",
                             foreground="#4a4a4a",
                             font=("Segoe UI", 10),
                             padding=6)
        self.style.map("TButton", background=[("active", "#fab1a0")])

    def setup_ui(self):
        self.main_frame = ttk.Frame(self.window, padding=10)
        self.main_frame.pack(fill=tk.BOTH, expand=True)

        #video feed
        self.video_label = ttk.Label(self.main_frame, anchor="center", relief=tk.SOLID, borderwidth=2)
        self.video_label.grid(row=0, column=0, padx=(10, 5), pady=10, sticky="nsew")

        #sidebar
        self.sidebar = tk.Text(
            self.main_frame,
            width=45,
            wrap=tk.WORD,
            font=("Segoe UI", 11),
            bg="#c8d6e5",     #pastel blue
            fg="#4a4a4a",     #soft dark gray color text
            insertbackground="#4a4a4a",
            relief=tk.FLAT,
            padx=10,
            pady=10
        )
        self.sidebar.grid(row=0, column=1, padx=(5, 10), pady=10, sticky="nsew")
        self.sidebar.insert(tk.END, "🧠 Waiting for feedback...\n")
        self.sidebar.config(state=tk.DISABLED)

        self.main_frame.columnconfigure(0, weight=3)
        self.main_frame.columnconfigure(1, weight=2)
        self.main_frame.rowconfigure(0, weight=1)

        #buttons
        self.button_frame = ttk.Frame(self.window, padding=(10, 5))
        self.button_frame.pack(fill=tk.X)

        self.start_button = ttk.Button(self.button_frame, text="▶ Start", command=self.start_camera)
        self.start_button.pack(side=tk.LEFT, padx=10)

        self.stop_button = ttk.Button(self.button_frame, text="⏹ Stop", command=self.stop_camera)
        self.stop_button.pack(side=tk.LEFT, padx=10)

    def start_camera(self):
        if not self.running:
            self.cap = cv2.VideoCapture(0)
            self.running = True
            self.update_frame()

    def stop_camera(self):
        self.running = False
        if self.cap:
            self.cap.release()
        self.video_label.config(image='')
        self.sidebar.config(state=tk.NORMAL)
        self.sidebar.delete(1.0, tk.END)
        self.sidebar.insert(tk.END, "⛔ Camera stopped.\n")
        self.sidebar.config(state=tk.DISABLED)

    def update_frame(self):
        try:
            if self.running and self.cap and self.cap.isOpened():
                ret, frame = self.cap.read()
                if ret:
                    processed_frame, z_score, gemini_feedback = process_frame_with_posture(
                        frame, self.pose_model, self.z_buffer
                    )

                    frame_rgb = cv2.cvtColor(processed_frame, cv2.COLOR_BGR2RGB)
                    image = Image.fromarray(frame_rgb)
                    image_tk = ImageTk.PhotoImage(image)

                    self.video_label.configure(image=image_tk)
                    self.video_label.image = image_tk

                    self.z_buffer.append((z_score, image))

                    if gemini_feedback:
                        self.sidebar.config(state=tk.NORMAL)
                        self.sidebar.delete(1.0, tk.END)
                        self.sidebar.insert(tk.END, "🧠 Gemini Feedback:\n\n")
                        self.sidebar.insert(tk.END, gemini_feedback.strip())
                        self.sidebar.config(state=tk.DISABLED)

            if self.running:
                self.window.after(15, self.update_frame)
        except Exception as e:
            print(f"[update_frame ERROR] {e}")


if __name__ == "__main__":
    root = tk.Tk()
    app = SlouchDetectorApp(root)
    root.mainloop()

```

### app/pose/detector.py

```python
import cv2
import mediapipe as mp

# Initialize MediaPipe pose components globally
mp_pose = mp.solutions.pose
mp_drawing = mp.solutions.drawing_utils

def init_pose(static_image_mode=False, model_complexity=1, enable_segmentation=False, min_detection_confidence=0.5):
    """
    Initialize the MediaPipe Pose model.
    """
    return mp_pose.Pose(
        static_image_mode=static_image_mode,
        model_complexity=model_complexity,
        enable_segmentation=enable_segmentation,
        min_detection_confidence=min_detection_confidence
    )

def process_frame(frame, pose_model):
    """
    Process a frame to extract pose landmarks using MediaPipe.
    Returns:
        - Annotated image
        - Pose landmarks result object
    """
    # Convert to RGB for MediaPipe
    rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    image_height, image_width, _ = rgb.shape
    rgb.flags.writeable = False
    results = pose_model.process(rgb)
    rgb.flags.writeable = True

    # Convert back to BGR
    output_frame = cv2.cvtColor(rgb, cv2.COLOR_RGB2BGR)

    return output_frame, results

def draw_landmarks(image, landmarks):
    """
    Draw pose landmarks and connections on the image.
    """
    if landmarks:
        mp_drawing.draw_landmarks(
            image,
            landmarks,
            mp_pose.POSE_CONNECTIONS,
            mp_drawing.DrawingSpec(color=(0, 255, 0), thickness=2, circle_radius=2),
            mp_drawing.DrawingSpec(color=(0, 0, 255), thickness=2)
        )

```

### app/gemini/feedback.py

```python
import platform
import subprocess
from win10toast_click import ToastNotifier 
import cv2
import mediapipe as mp
import time
from math import degrees, acos
# import google.generativeai as genai
from dotenv import load_dotenv
import os
from google import genai
from google.genai import types

import PIL.Image


def analyze_posture_with_gemini(pil_image):
    load_dotenv()  # Load variables from .env file
    api_key = os.getenv("GEMINI_API_KEY")
    client = genai.Client(api_key=api_key)

    prompt = f"""Analyze the person's body posture and give some feedback 
    on what's specifically wrong with the posture and 3 bullet points of actional steps to improve the posture. 
    Be specific in your response, limit it to 4-5 sentences. 

    #     Features You May Want to Observe:
    #     - Shoulder Angle 
    #     - Head Forward Displacement 
    #     - Trunk Inclination (degrees from vertical)
    #     - Left Ear-Shoulder Horizontal Distance (relative x)
    #     - Right Ear-Shoulder Horizontal Distance (relative x): 
    #     - Anything else you observe!

    #     Use reasoning to come up with actionable steps the person can take to fix their posture.
    #         - e.g. Move your shoulders back, tilt your head up, stop tilting your head to the side etc.
    #     Write your diagnosis as if you are addressing the person in the picture. 
    #     Don't include markdown formatting, such as asterisks.
    #     Use bullet points.
    #     """
    # image = PIL.Image.open('Screenshot 2024-02-20 220151.png')
    # print("image type", image)

    # print ("pil image type", type(pil_image))
    # print("pil image", pil_image)


    response = client.models.generate_content(
        model="gemini-2.0-flash",
        contents=[prompt, pil_image])

    print(response.text)

    return response.text

def send_posture_alert():
    message = "You're slouching! Sit up straight for better posture 🧍"
    title = "Posture Alert"
    system = platform.system()
    
    if system == "Darwin":  # macOS
        subprocess.run([
            "osascript", "-e",
            f'display notification "{message}" with title "Posture Alert"'
        ])
    
    elif system == "Linux":
        subprocess.run([
            "notify-send", "Posture Alert", message
        ])
    
    elif system == "Windows":
        toaster = ToastNotifier()
        toaster.show_toast(
            title,
            message,
            duration=5,         # seconds
            threaded=True       # allows notification while your main loop runs
        )

       
```