# Project export: Foodprint
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: UC Berkeley AI Hackathon 2026
- Tagline: An AI-powered dining hall analytics platform that tracks plate waste
- Devpost: https://devpost.com/software/foodprint-uyrcf5
- GitHub: https://github.com/anishka-v/food-print
- Video: https://www.youtube.com/embed/YxeaFw62zr0?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) — Anishka Vissamsetty (4 commits)
## Devpost submission (written by the team)
### Inspiration
Food waste is a huge problem. College campuses in the United States generate an estimated 22 million pounds of food waste annually. Every day in dining halls, students take food they don't end up eating, and most schools have very little visibility into what those leftovers actually are. They might know how much food they purchased or how much waste was thrown away overall, but they usually don't know which specific foods students consistently leave on their plates.
### What it does
Foodprint is an AI-powered dining hall waste-tracking system that analyzes leftover food on student plates. A Raspberry Pi camera monitors plates, Overshoot detects when a full plate is present, and Gemini analyzes captured images to identify leftover foods. The system combines this data with scraped dining hall menus using Browserbase and stores the results in a database. Staff can then view waste trends, meal-specific waste patterns, recent plate events, and actionable insights through a dashboard, helping them understand what foods are being wasted most frequently.
### How we built it
We built Foodprint using a Raspberry Pi camera stream, Python, FastAPI, SQLite, and OpenCV. Overshoot handles full-plate detection, while Gemini generates detailed leftover descriptions from captured images. Browserbase and Stagehand automatically scrape dining hall menus and store them as structured data. The backend aggregates waste events and serves analytics through a FastAPI dashboard, allowing dining hall staff to explore trends and menu-specific waste patterns.
### Challenges we ran into
One of our biggest challenges was how classifying against the entire Berkeley Dining menu was far too broad a problem, which pushed us to scope classification down to one dining hall and meal period at a time. Scraping the menu data itself was its own challenge, since the site is JS-rendered and filter-based, requiring Browserbase and Stagehand instead of a simple static scrape.
### Accomplishments we're proud of
We're proud that we built a working end-to-end pipeline in a short timeframe, taking a live Pi camera feed all the way through AI-based food classification to a working dashboard. We solved the dynamic menu problem by using Browserbase to automatically scrape Berkeley Dining's daily menus, and we designed two physical deployment options, a conveyor belt mount and a trash can mount, so the system could realistically fit different dining hall layouts.
### What we learned
Working on Foodprint taught us a lot about the gap between planning hardware on paper and actually wiring it up, since things like camera mounting and color format only became obvious once we had real components in hand. We also learned that scoping a problem well matters as much as the model you choose, which is why we narrowed classification down to one dining hall and meal period at a time.
### What's next
Our next goal is to move beyond generic leftover descriptions and improve menu-item-level identification accuracy. We also would love to bring this to life and expand it to dining halls across the world so that we can help universities reduce food waste at scale by turning everyday dining hall activity into actionable sustainability insights.
## README (from the GitHub repository)
# foodprint
`foodprint` is a dining hall waste-tracking system for plate-level feedback.
It combines:
- a Raspberry Pi camera stream
- Overshoot for full-plate detection
- Gemini for leftover description
- Browserbase for daily menu scraping
- a FastAPI dashboard for dining hall staff
The goal is to show what students are actually leaving on their plates, meal by meal, so dining teams can make better decisions about portions, menus, and purchasing.
## What It Does
`foodprint` has four main parts:
1. `camera_server.py`
Runs on the Raspberry Pi and exposes the camera as an MJPEG stream at `/video`.
2. `watch_pi_camera.py`
Runs on a Mac or laptop, watches the Pi stream, uses Overshoot to decide when a full plate is in view, captures an image, sends the image to Gemini for leftover analysis, and stores the result in `foodprint.db`.
3. `browserbase_crossroads_menu_agent.mjs`
Uses Browserbase + Stagehand to scrape Berkeley Dining menus into JSON files in `menus/`.
4. `dining_waste_tracker_gemini.py` + `staff_dashboard.html`
Serves the staff dashboard and APIs for menu logs, waste trends, recent plate events, and waste by meal.
## Architecture
```text
Raspberry Pi Camera
-> Flask MJPEG stream
-> watch_pi_camera.py
-> Overshoot: plate present / no plate
-> Gemini: leftover description from image
-> SQLite: foodprint.db
-> FastAPI dashboard
-> staff_dashboard.html
-> dining hall analytics
Browserbase
-> scrape Berkeley Dining menus
-> menus/*.json
-> dashboard menu log import
```
## Repository Layout
- [camera_server.py](/Users/anishka/Desktop/projects/eco-dining/camera_server.py)
- [watch_pi_camera.py](/Users/anishka/Desktop/projects/eco-dining/watch_pi_camera.py)
- [dining_waste_tracker_gemini.py](/Users/anishka/Desktop/projects/eco-dining/dining_waste_tracker_gemini.py)
- [staff_dashboard.html](/Users/anishka/Desktop/projects/eco-dining/staff_dashboard.html)
- [browserbase_crossroads_menu_agent.mjs](/Users/anishka/Desktop/projects/eco-dining/browserbase_crossroads_menu_agent.mjs)
- [requirements.txt](/Users/anishka/Desktop/projects/eco-dining/requirements.txt)
- [package.json](/Users/anishka/Desktop/projects/eco-dining/package.json)
- `menus/`
- `captures/`
- `foodprint.db`
## Requirements
### Python
Install:
```bash
python3 -m pip install -r requirements.txt
```
Key Python dependencies:
- `fastapi`
- `uvicorn`
- `opencv-python`
- `google-generativeai`
- `requests`
- `livekit`
### Node
Install:
```bash
npm install
```
Key Node dependencies:
- `@browserbasehq/stagehand`
- `zod`
## Environment Variables
Set the API keys you need before running the watcher or scraper:
```bash
export OVERSHOOT_API_KEY="your_overshoot_key"
export GEMINI_API_KEY="your_gemini_key"
export BROWSERBASE_API_KEY="your_browserbase_key"
```
Optional:
```bash
export GEMINI_MODEL="gemini-1.5-flash"
export DINING_HALL="Crossroads"
export FOODPRINT_DB_PATH="foodprint.db"
export BROWSERBASE_MODEL="google/gemini-2.5-flash"
```
## 1. Run the Pi Camera Server
Run this on the Raspberry Pi:
```bash
pip3 install flask picamera2 opencv-python
python3 camera_server.py
```
The stream should be available at:
```text
http://raspberrypi.local:8080/video
```
If colors look wrong, try setting:
```bash
PI_COLOR_MODE=raw python3 camera_server.py
```
Valid values:
- `raw`
- `rgb_to_bgr`
- `bgr_to_rgb`
## 2. Scrape the Dining Menu
Scrape Crossroads:
```bash
npm run scrape:crossroads-menu
```
Scrape all supported halls:
```bash
npm run scrape:all-menus
```
Output is written to `menus/*.json`.
## 3. Run the Plate Watcher
Run this on your laptop or Mac:
```bash
python3 watch_pi_camera.py \
--url http://raspberrypi.local:8080/video \
--dining-hall Crossroads \
--db-path foodprint.db \
--show-window
```
What it does:
- waits until a full plate is clearly in frame
- uses Overshoot only for plate detection
- captures an image into `captures/`
- sends the image to Gemini for leftover analysis
- normalizes leftovers into tracked items such as `Banana`, `Bread`, or `Yogurt`
- stores the event in `foodprint.db`
## 4. Run the Dashboard
Start the backend:
```bash
python3 dining_waste_tracker_gemini.py
```
Open the dashboard:
```text
http://localhost:8000/staff
```
The dashboard includes:
- daily menu log
- waste by meal
- daily trend graph
- dining hall menu waste
- recent plate events with thumbnails
- insights
## Current Detection Behavior
The watcher is intentionally conservative:
- Overshoot only answers: is a real full plate present?
- Gemini describes leftovers from the captured image
- menu matching is not forced
- if the exact dish is unclear, generic leftovers are preserved
Examples:
- `banana slices` -> `Banana`
- `bread with spread` -> `Bread`
- `white creamy substance` -> often `Yogurt` or another generic dairy-style label if that is all that is visually supported
This is deliberate. It is better to keep a generic truthful label than invent the wrong menu item.
## Data Storage
Captured events are stored in SQLite at `foodprint.db`.
Main tables:
- `plate_events`
- `leftover_items`
Captured images and metadata sidecars are stored in:
- `captures/*.jpg`
- `captures/*.json`
## Dashboard Data Rules
- `Daily Menu Log` stays based on scraped or manual menu items.
- `Dining Hall Menu Waste` switches to real DB-backed waste data as soon as at least one real plate event exists for the selected hall and window.
- The dashboard will stop showing test placeholder waste rows once real captured data exists.
## Common Commands
Run watcher:
```bash
python3 watch_pi_camera.py --url http://raspberrypi.local:8080/video --dining-hall Crossroads --db-path foodprint.db --show-window
```
Run dashboard:
```bash
python3 dining_waste_tracker_gemini.py
```
Scrape menus:
```bash
npm run scrape:all-menus
```
Clear the database:
```bash
sqlite3 foodprint.db "DELETE FROM leftover_items; DELETE FROM plate_events;"
```
## Notes
- The Pi camera stream must be stable before the watcher starts.
- If `watch_pi_camera.py` times out waiting for frames, confirm `http://raspberrypi.local:8080/video` works in a browser first.
- If Overshoot fails to connect, the issue is usually network or WebRTC connectivity, not the Pi stream.
- If `GEMINI_API_KEY` is missing, the watcher falls back to the generic Overshoot leftover prompt.
## Related Docs
- [PI_CAMERA_SETUP.md](/Users/anishka/Desktop/projects/eco-dining/PI_CAMERA_SETUP.md)
- [PERSON_SNAPSHOT_OVERSHOOT.md](/Users/anishka/Desktop/projects/eco-dining/PERSON_SNAPSHOT_OVERSHOOT.md)
- [BROWSERBASE_MENU_AGENT.md](/Users/anishka/Desktop/projects/eco-dining/BROWSERBASE_MENU_AGENT.md)
## Detected evidence (automated analysis)
Indexed codebase: 9 recognized source files, 156 KB.
- FastAPI (technology) — detected in the code
- HTML (language) — detected in the code
- Python (language) — detected in the code
- Google Gemini (technology) — claimed on Devpost, not found in the code
## Codebase structure (from repository index)
### Files (13 of 13)
```
.gitignore
browserbase_crossroads_menu_agent.mjs
BROWSERBASE_MENU_AGENT.md
camera_server.py
dining_waste_tracker_gemini.py
package.json
PERSON_SNAPSHOT_OVERSHOOT.md
person_snapshot_overshoot.py
PI_CAMERA_SETUP.md
README.md
requirements.txt
staff_dashboard.html
watch_pi_camera.py
```
### Dependencies
- package.json: @browserbasehq/stagehand@^3.0.0, zod@^3.23.8
- requirements.txt: fastapi@>=0.115.0, google-generativeai@>=0.8.0, livekit@>=0.12.0, numpy@>=2.1.0, opencv-python@>=4.10.0.84, Pillow@>=11.0.0, python-dotenv@>=1.0.1, python-multipart@>=0.0.9, requests@>=2.32.3, uvicorn@>=0.30.0
### Recent commits (newest first)
- readme
- final
- test
- Initial push with Browserbase integration
- Add files via upload
## Key source files (fetched from GitHub, selected and truncated for size)
### BROWSERBASE_MENU_AGENT.md
```markdown
# Browserbase Menu Agent
This script uses Browserbase + Stagehand to open the Berkeley Dining menu site, select a dining hall and date, and extract breakfast, lunch, and dinner items into JSON.
## Install
```bash
npm install
```
## Environment
```bash
export BROWSERBASE_API_KEY="your_browserbase_api_key"
```
Optional:
```bash
export BROWSERBASE_MODEL="google/gemini-2.5-flash"
```
## Run
Crossroads for today:
```bash
npm run scrape:crossroads-menu
```
All supported Berkeley dining halls:
```bash
npm run scrape:all-menus
```
Print JSON too:
```bash
node browserbase_crossroads_menu_agent.mjs --stdout
```
Pick a different date label shown on the site:
```bash
node browserbase_crossroads_menu_agent.mjs --date "Tomorrow"
```
Pick a different hall:
```bash
node browserbase_crossroads_menu_agent.mjs --location "Cafe 3"
```
## Output
The script writes JSON files into `menus/`, for example:
```text
menus/crossroads-today.json
```
The JSON shape is:
```json
{
"scrapedAt": "2026-06-20T00:00:00.000Z",
"sourceUrl": "https://dining.berkeley.edu/menus/",
"location": "Crossroads",
"dateLabel": "Today",
"breakfast": ["Item 1", "Item 2"],
"lunch": ["Item 1", "Item 2"],
"dinner": ["Item 1", "Item 2"]
}
```
```
### PI_CAMERA_SETUP.md
```markdown
# Pi Camera Motion Capture
This setup uses a small Flask server on the Raspberry Pi and a Mac watcher that:
- reads the MJPEG stream locally
- uses OpenCV motion detection as a cheap filter
- asks Overshoot whether the motion actually contains a person before saving a photo
## Pi setup
Install dependencies on the Pi:
```bash
pip3 install flask picamera2 opencv-python
```
Run the server on the Pi:
```bash
python3 camera_server.py
```
Test from the Mac in a browser:
```text
http://10.0.0.217:8080/video
```
## Mac setup
Install dependencies on the Mac:
```bash
pip3 install opencv-python requests livekit python-dotenv
```
Run the watcher:
```bash
export OVERSHOOT_API_KEY="ovs-..."
python3 watch_pi_camera.py --url http://10.0.0.217:8080/video --show-window
```
Photos are saved into `captures/`.
Each saved image also gets a JSON sidecar with the Overshoot confidence and response text.
## Tuning
- Increase `--min-changed-pixels` if small lighting shifts are causing too many Overshoot checks.
- Increase `--cooldown-seconds` if one person creates too many photos.
- Lower `--min-changed-pixels` if real motion is being missed before Overshoot gets a chance to check.
- Raise `--min-confidence` if you want fewer borderline person detections.
- Omit `--show-window` if you want it to run headless.
```
### requirements.txt
```
fastapi>=0.115.0
uvicorn>=0.30.0
python-multipart>=0.0.9
opencv-python>=4.10.0.84
numpy>=2.1.0
Pillow>=11.0.0
google-generativeai>=0.8.0
python-dotenv>=1.0.1
requests>=2.32.3
livekit>=0.12.0
```
### package.json
```
{
"name": "eco-dining-browserbase-agent",
"private": true,
"type": "module",
"scripts": {
"scrape:crossroads-menu": "node browserbase_crossroads_menu_agent.mjs",
"scrape:all-menus": "node browserbase_crossroads_menu_agent.mjs --all-locations"
},
"dependencies": {
"@browserbasehq/stagehand": "^3.0.0",
"zod": "^3.23.8"
}
}
```
### camera_server.py
```python
#!/usr/bin/env python3
"""
Simple Raspberry Pi camera server that exposes an MJPEG stream at /video.
Run this file on the Pi, not on the Mac.
"""
from __future__ import annotations
import os
import time
import cv2
from flask import Flask, Response
from picamera2 import Picamera2
app = Flask(__name__)
COLOR_MODE = os.getenv("PI_COLOR_MODE", "bgr_to_rgb").lower()
picam2 = Picamera2()
config = picam2.create_video_configuration(
main={"size": (640, 480), "format": "BGR888"}
)
picam2.configure(config)
picam2.start()
time.sleep(1)
def prepare_frame(frame):
if COLOR_MODE == "raw":
return frame
if COLOR_MODE == "rgb_to_bgr":
return cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
if COLOR_MODE == "bgr_to_rgb":
return cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
raise ValueError(
"Unsupported PI_COLOR_MODE. Use raw, rgb_to_bgr, or bgr_to_rgb."
)
def generate_frames():
while True:
frame = picam2.capture_array()
frame = prepare_frame(frame)
ok, jpeg = cv2.imencode(".jpg", frame)
if not ok:
continue
yield (
b"--frame\r\n"
b"Content-Type: image/jpeg\r\n\r\n"
+ jpeg.tobytes()
+ b"\r\n"
)
@app.route("/video")
def video():
return Response(
generate_frames(),
mimetype="multipart/x-mixed-replace; boundary=frame",
)
@app.route("/")
def index():
return f"Pi camera server running. Open /video. color_mode={COLOR_MODE}"
if __name__ == "__main__":
app.run(host="0.0.0.0", port=8080)
```
### person_snapshot_overshoot.py
```python
#!/usr/bin/env python3
"""
Watch a network camera stream, publish it to Overshoot, and save a photo
whenever a plate of food newly enters the frame.
"""
from __future__ import annotations
import argparse
import asyncio
import json
import os
import signal
import threading
import time
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import Any, Dict, Optional
import cv2
import requests
from dotenv import load_dotenv
OVERSHOOT_BASE_URL = "https://api.overshoot.ai/v1"
DEFAULT_PROMPT = (
"You are monitoring a fixed dining hall style camera. Decide whether a real "
"plate of food is visible in the latest frame right now only if the full plate is visible in the frame. Ignore empty plates, bowls, "
"cups, printed pictures of food, and food advertisements. Use the short "
"video clip only for motion context. Return JSON only with this exact "
'schema: {"plate_of_food_visible": true, "confidence": 0.0, "reason": "short phrase"}'
)
class OvershootError(RuntimeError):
"""Raised when the Overshoot API returns an unexpected response."""
def parse_json_fragment(text: str) -> Dict[str, Any]:
text = text.strip()
if text.startswith("```"):
parts = text.split("```")
if len(parts) >= 3:
text = parts[1]
if text.startswith("json"):
text = text[4:]
start = text.find("{")
end = text.rfind("}")
if start == -1 or end == -1 or end < start:
raise ValueError(f"Could not find JSON object in model response: {text!r}")
return json.loads(text[start : end + 1])
@dataclass
class DetectionResult:
plate_of_food_visible: bool
confidence: float
reason: str
raw_response: str
class OvershootClient:
def __init__(self, api_key: str, model: Optional[str] = None) -> None:
self.api_key = api_key
self.requested_model = model
self.session = requests.Session()
self.session.headers.update({"Authorization": f"Bearer {api_key}"})
def _raise_for_status(self, response: requests.Response) -> None:
if response.ok:
return
detail = response.text
try:
detail = response.json().get("detail", detail)
except Exception:
pass
raise OvershootError(f"Overshoot API error {response.status_code}: {detail}")
def list_ready_models(self) -> list[str]:
response = self.session.get(f"{OVERSHOOT_BASE_URL}/models", timeout=20)
self._raise_for_status(response)
payload = response.json()
return [
item["id"]
for item in payload.get("data", [])
if item.get("status") == "ready"
]
def choose_model(self) -> str:
models = self.list_ready_models()
if not models:
raise OvershootError("Overshoot returned no ready models.")
if self.requested_model:
if self.requested_model not in models:
raise OvershootError(
f"Requested model {self.requested_model!r} is not ready. "
f"Ready models: {', '.join(models)}"
)
return self.requested_model
preferred = [
"Qwen/Qwen3.5-9B",
"google/gemma-4-E4B-it",
]
for candidate in preferred:
if candidate in models:
return candidate
return models[0]
def create_stream(self) -> Dict[str, Any]:
response = self.session.post(f"{OVERSHOOT_BASE_URL}/streams", timeout=20)
self._raise_for_status(response)
return response.json()
def keepalive(self, stream_id: str) -> Dict[str, Any]:
response = self.session.post(
f"{OVERSHOOT_BASE_URL}/streams/{stream_id}/keepalive",
timeout=20,
)
self._raise_for_status(response)
return response.json()
def get_stream(self, stream_id: str) -> Dict[str, Any]:
response = self.session.get(
f"{OVERSHOOT_BASE_URL}/streams/{stream_id}",
timeout=20,
)
self._raise_for_status(response)
return response.json()
def delete_stream(self, stream_id: str) -> None:
response = self.session.delete(
f"{OVERSHOOT_BASE_URL}/streams/{stream_id}",
timeout=20,
)
self._raise_for_status(response)
def detect_plate_of_food(
self,
*,
stream_id: str,
model: str,
prompt: str,
window_ms: int,
max_completion_tokens: int,
) -> DetectionResult:
payload = {
"model": model,
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": prompt},
{
"type": "video_url",
"video_url": {
"url": (
f"ovs://streams/{stream_id}"
f"?start_offset_ms=-{window_ms}&max_fps=1"
)
},
},
{
"type": "image_url",
"image_url": {
"url": f"ovs://streams/{stream_id}?frame_index=-1"
},
},
],
}
],
"max_completion_tokens": max_completion_tokens,
}
response = self.session.post(
f"{OVERSHOOT_BASE_URL}/chat/completions",
json=payload,
timeout=45,
)
self._raise_for_status(response)
content = response.json()["choices"][0]["message"]["content"]
data = parse_json_fragment(content)
return DetectionResult(
plate_of_food_visible=bool(data.g
[truncated — 15246 more characters]
```
### staff_dashboard.html
```html
foodprint staff dashboard