# Project export: Savor This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: Savor uses computer vision to transform food waste into real-time purchasing and prep decisions, cutting costs and reducing waste across university dining operations. - Devpost: https://devpost.com/software/savor-2qn7ax - GitHub: https://github.com/Deanbarrow10/savor - Video: https://www.youtube.com/embed/Hy9XcJwNLZ0?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — Edward Lu (16 commits), Dino (3 commits) ## Devpost submission (written by the team) ### Inspiration U.S. Colleges and Universities produce over 22 million pounds of food waste annually, with the average student producing 142 pounds of food waste per year. This is substantial waste, and with studies showing that 30-40% of students experience food insecurity, it is simply unacceptable to let all of this high-quality food go to complete waste. Furthermore, that 22 million pounds also contributes to methane emissions on the order of 55 million additional pounds of greenhouse gases. We believe that this issue can be addressed. ### What it does Savor is an AI-powered plate waste analytics and supply optimization platform that transforms any camera into a food waste management system. Our product identifies the quantity and makeup of food waste by college students. We leverage a camera mounted above the food deposit zone, which segments the plate into its constituent food groups. We approximate the quantity of each different food left on the plate before it is deposited into the trash can. This gives us a viable reading on the precise amount of waste being produced and specifically which foods are producing the most waste, so that a university can adjust its budgeting accordingly. Waste information is then processed in our cloud analytics engine, providing critical long-term tracking information, recommended budgeting updates in accordance with individual waste statistics for all food types, and providing rewards for students who minimize their food waste. How it works in flow: Waste Recognition: When students scrape their plates at the waste station, our camera captures the plate. The image is sent to our detection layer, a QWEN computer vision model hosted on Modal running on GPU infrastructure, which identifies and quantifies wasted items in real-time with 95%+ accuracy. We input menu context via OpenAI API to understand what specific dishes are being wasted, then store everything in our database. Waste Analytics Dashboard: Every data point feeds into our cloud analytics engine, generating charts and catalog rankings of the most wasted items. Dining managers get instant visibility into waste trends by time, day, and dish-visualized in an intuitive dashboard. Smart Procurement Engine: Our linear regression model learns waste patterns and predicts optimal order quantities for each menu item. The system ranks dishes by waste probability and suggests precise purchasing adjustments. Managers can approve orders with one click or use our voice-enabled Savor agent to reorder hands-free. Provides Incentives for Students to Reduce Waste: We offer rewards to students in the form of cashback if they do not produce any waste! Essentially, when students purchase a meal in the dining hall, they are essentially putting up a deposit, which will get repaid in accordance with how much waste they create in that dining hall session. Features Detects food waste from live camera frames using OpenAI vision analysis. Prevents duplicate detections using SSIM + cooldown checks. Stores events in SQLite and generates aggregate stats. Streams live camera and scanner overlays (MJPEG endpoints). Tracks student credit scans with barcode/OCR and leaderboard ranking. Incentivizes students to waste less by rewarding them for consistently cleaning their plates. Syncs Stanford dining hall menus (all halls, all meals, 3-day window with fallback) into JSON. Produces historical waste inventory datasets for dashboard charts. Generates smart-shopping recommendations using linear regression predictions. Includes an ordering chat assistant endpoint. Supports Whisper transcription and Google TTS for voice experiences. Tech Stack Backend: FastAPI, Uvicorn, SQLite, OpenCV, OpenAI SDK, httpx, BeautifulSoup. Frontend: React 19 + Vite, Recharts, Three.js/react-three-fiber. Data: JSON inventory files + SQLite event/credits tables. What we Learned Tech & Development framework: Real-time AI systems need explicit fallback paths (timeouts, forced cadence, restart logic) to handle edge cases gracefully. UX clarity matters as much as model quality for operational adoption; beautiful ML means nothing if operators can't onboard and integrate it. Sustainability understanding: After talking to TreeHacks mentors, we realized how our solution can serve both B2B (dining institutions) and B2C (dining hall customers). This exactly reminded us how we should better reduce food waste from both the demand and supply side, as institutions need procurement optimization and students need behavioral incentives. Team collaboration: We understand the importance of working as a team, not just physically sitting together side by side, but granting enough trust and responsibility to work towards the same goal. What it Enables Savor prevents waste through carefully designed incentive structures. By giving dining halls the power to forecast demand accurately, control portions based on real data, and implement just-in-time shopping planning. It enables: Cost reduction through optimized purchasing and reduced over-preparation Sustainability impact with measurable waste reduction tied to institutional goals Behavioral change through student incentives that make sustainability rewarding for students Data-driven operations replacing guesswork with actionable analytics Unlike expensive hardware solutions that cost $100K+, Savor is a lightweight, camera-only SaaS platform that's flexible, easy to deploy, and affordable for any institution. ### Challenges we ran into Camera backend instability and frame-grab failures on Windows- getting reliable real-time plate captures proved harder than expected Preventing false duplicates while still producing near-live updates without overwhelming the system Keeping AI calls fast enough for operational use while controlling noisy triggers that could spike costs Ensuring smooth mode transitions between credits scanning and food-waste analysis without breaking the detection pipeline Designing dense dashboards that fit high-value information in one screen without clutter balancing data richness with visual clarity ### What's next Vision modules for people to deploy at their own homes Pilot deployment at Stanford dining halls to validate real-world performance Expand ML model to recognize more food types and handle edge cases Develop a mobile app for users to track their personal waste reduction and earnings Build deeper integrations with food service procurement systems Outreach and partner with additional universities and institutional dining providers Add predictive features for menu planning based on historical trends and external factors (weather, events, etc.) ## README (from the GitHub repository) # Savor Savor is a real-time food waste intelligence platform for dining operations. It combines camera-based waste detection, inventory/menu sync, historical analytics, student credits tracking, and an ordering assistant into one full-stack application. ## What It Does - Detects food waste from live camera frames using OpenAI vision analysis. - Prevents duplicate detections using SSIM + cooldown checks. - Stores events in SQLite and generates aggregate stats. - Streams live camera and scanner overlays (MJPEG endpoints). - Tracks student credit scans with barcode/OCR and leaderboard ranking. - Syncs Stanford dining hall menus (all halls, all meals, 3-day window with fallback) into JSON. - Produces historical waste inventory datasets for dashboard charts. - Generates smart-shopping recommendations using linear regression predictions. - Includes an ordering chat assistant endpoint. - Supports Whisper transcription and Google TTS for voice experiences. ## Tech Stack - Backend: FastAPI, Uvicorn, SQLite, OpenCV, OpenAI SDK, httpx, BeautifulSoup. - Frontend: React 19 + Vite, Recharts, Three.js/react-three-fiber. - Data: JSON inventory files + SQLite event/credits tables. ## Repository Layout ```text savor/ backend/ main.py # FastAPI app and background loops analyzer.py # GPT-based plate and card analysis stanford_inventory.py # Stanford menu sync + caching database.py # SQLite queries + historical inventory builder config.py # Central environment config requirements.txt # Python dependencies frontend/ src/App.jsx # Main UI routes/pages src/api.js # Backend API wrappers package.json # Frontend scripts/deps README.md # Single project documentation source ``` ## Frontend Pages - `/` Live Tracking - `/dashboard` Dashboard - `/reports` Reports - `/credits` Credits - `/smart-shopping` Smart Commerce - `/menu` Menu Operations - `/inventory` Inventory ## Backend API ### Camera and system - `GET /api/video_feed` - `POST /api/camera/start` - `POST /api/camera/stop` - `POST /api/system/reactivate` - `GET /api/status` ### Waste and inventory analytics - `GET /api/events` - `GET /api/stats` - `GET /api/inventory-history` - `GET /api/inventory` - `POST /api/inventory/refresh` - `GET /api/smart-shopping` ### Credits scanner - `GET /api/credits/status` - `POST /api/credits/scanner/start` - `POST /api/credits/scanner/stop` - `GET /api/credits/video_feed` - `GET /api/credits/leaderboard` ### Ordering and voice - `POST /api/order-chat` - `POST /api/order-chat/greeting` - `POST /api/transcribe` - `POST /api/tts` ## Environment Variables Create a root `.env` file (same level as this README). ### Required - `OPENAI_API_KEY` - required for vision analysis and transcription endpoint. ### Optional (backend behavior) - `CAMERA_INDEX` (default `0`) - `CAMERA_BACKEND` (default `dshow` on Windows, `default` otherwise; use `dshow` to avoid MSMF grab warnings) - `CAPTURE_INTERVAL` (default `3.0`) - `ANALYSIS_FORCE_INTERVAL_SECONDS` (default `8.0`; forces periodic analysis even without large scene change) - `ANALYSIS_REQUEST_TIMEOUT_SECONDS` (default `22.0`; max wait per OpenAI vision call) - `SSIM_THRESHOLD` (default `0.85`) - `COOLDOWN_SECONDS` (default `10`) - `GOOGLE_TTS_API_KEY` (required only if using `/api/tts`) ### Optional (Stanford menu sync) - `STANFORD_DINING_MENU_URL` (default `https://rdeapps.stanford.edu/dininghallmenu/`) - `STANFORD_DINING_HALL` (default `Wilbur`) - `STANFORD_DINING_MEAL` (default `Lunch`) - `STANFORD_DINING_DAY_OFFSET` (default `0`) ### Legacy (currently unused by menu sync) - `BRIGHTDATA_API_KEY` - `BRIGHTDATA_BROWSER_WSS` ## Local Development ### 1) Backend ```powershell cd backend python -m venv .venv .venv\Scripts\Activate.ps1 pip install -r requirements.txt uvicorn main:app --reload --host 0.0.0.0 --port 8000 ``` ### 2) Frontend ```powershell cd frontend npm install npm run dev ``` Vite runs on `http://localhost:5173` and calls backend `http://localhost:8000` through `/api/*`. ## Data and Outputs - `backend/vision_bin.db` - waste events + student credits. - `backend/food_inventory.json` - aggregate inventory summary. - `backend/food_inventory_historical.json` - chart-friendly historical dataset. - `backend/stanford_inventory.json` - latest Stanford menu sync payload. ## Notes - Inventory sync now uses direct HTTP form postbacks (no browser automation required). - `/api/inventory/refresh` forces a live menu sync and bypasses cache. - If Stanford does not expose full past dates, sync includes fallback window metadata in JSON. - Backend startup auto-seeds demo rows if `backend/vision_bin.db` has zero `waste_events`, so analytics pages never start from a fully empty DB. ## Detected evidence (automated analysis) Indexed codebase: 27 recognized source files, 396 KB. - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - SQL (language) — claimed on Devpost, not found in the code - TypeScript (language) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: config files committed to the repository ## Codebase structure (from repository index) ### Files (39 of 39) ``` .claude/settings.local.json .env.example .gitignore backend/analyzer.py backend/camera.py backend/config.py backend/database.py backend/detector.py backend/food_inventory_historical.json backend/food_inventory.json backend/main.py backend/models.py backend/order_agent.py backend/requirements.txt backend/stanford_inventory.json backend/stanford_inventory.py backend/waste_model.py devpost_talking_points.txt devpost.txt frontend/.gitignore frontend/assets/product.gltf frontend/eslint.config.js frontend/index.html frontend/package.json frontend/src/api.js frontend/src/App.jsx frontend/src/components/CameraFeed.jsx frontend/src/components/EventCard.jsx frontend/src/components/EventFeed.jsx frontend/src/components/PersonalVisionShowcase.jsx frontend/src/components/StatsDashboard.jsx frontend/src/components/StatusBadge.jsx frontend/src/hooks/usePolling.js frontend/src/index.css frontend/src/main.jsx frontend/vite.config.js README.md references/dashboard.html references/menu_ops.html ``` ### Dependencies - backend/requirements.txt: beautifulsoup4, fastapi, httpx, openai, opencv-python, Pillow, python-dotenv, python-multipart, scikit-image, uvicorn[standard] - frontend/package.json: @eslint/js@^9.39.1, @react-three/fiber@^9.5.0, @tailwindcss/vite@^4.1.18, @types/react@^19.2.7, @types/react-dom@^19.2.3, @vitejs/plugin-react@^5.1.1, eslint@^9.39.1, eslint-plugin-react-hooks@^7.0.1, eslint-plugin-react-refresh@^0.4.24, globals@^16.5.0, react@^19.2.0, react-dom@^19.2.0, recharts@^3.7.0, tailwindcss@^4.1.18, three@^0.182.0, vite@^7.3.1 ### Recent commits (newest first) - some fixes - fin ui - new ui fix bugs - fixes - fix cam change - fixes - fix merge conflicts - storage - made agent - changes - How did a voice agent for smart shopping - update credits - credits - dean changes - fix some ui 3d model - ui-updates - fix some issues - before move - ui-update - fixed up the human detection issue, now will only detect food ## Key source files (fetched from GitHub, selected and truncated for size) ### backend/requirements.txt ``` fastapi uvicorn[standard] opencv-python openai scikit-image Pillow python-dotenv httpx python-multipart beautifulsoup4 ``` ### frontend/package.json ``` { "name": "frontend", "private": true, "version": "0.0.0", "type": "module", "scripts": { "dev": "vite", "build": "vite build", "lint": "eslint .", "preview": "vite preview" }, "dependencies": { "@react-three/fiber": "^9.5.0", "react": "^19.2.0", "react-dom": "^19.2.0", "recharts": "^3.7.0", "three": "^0.182.0" }, "devDependencies": { "@eslint/js": "^9.39.1", "@tailwindcss/vite": "^4.1.18", "@types/react": "^19.2.7", "@types/react-dom": "^19.2.3", "@vitejs/plugin-react": "^5.1.1", "eslint": "^9.39.1", "eslint-plugin-react-hooks": "^7.0.1", "eslint-plugin-react-refresh": "^0.4.24", "globals": "^16.5.0", "tailwindcss": "^4.1.18", "vite": "^7.3.1" } } ``` ### frontend/src/main.jsx ```javascript import { StrictMode } from 'react' import { createRoot } from 'react-dom/client' import './index.css' import App from './App.jsx' createRoot(document.getElementById('root')).render( , ) ``` ### backend/main.py ```python """ main.py — FastAPI application entry point for Savor. This is the central orchestrator that: 1. Initializes core services on startup (passive until API start calls) 2. Runs background loops for analysis/scanning only when requested 3. Exposes REST API endpoints for the frontend: - GET /api/video_feed → live MJPEG camera stream - GET /api/events → recent waste detection events - GET /api/stats → aggregate statistics - GET /api/status → system health/status - GET /api/images/{f} → serve saved plate images Run with: uvicorn main:app --reload --host 0.0.0.0 --port 8000 """ import asyncio import base64 import os import re import time from concurrent.futures import ThreadPoolExecutor from contextlib import asynccontextmanager from datetime import datetime from threading import Lock import cv2 import httpx from fastapi import FastAPI, Query, UploadFile, File from fastapi.middleware.cors import CORSMiddleware from fastapi.responses import Response, StreamingResponse from fastapi.staticfiles import StaticFiles from pydantic import BaseModel import openai from config import ( ANALYSIS_FORCE_INTERVAL_SECONDS, ANALYSIS_REQUEST_TIMEOUT_SECONDS, CAPTURE_INTERVAL, GOOGLE_TTS_API_KEY, IMAGE_SAVE_DIR, OPENAI_API_KEY, ) from database import ( award_student_credit, ensure_non_empty_database, get_events, get_food_inventory_historical, regenerate_demo_data, get_student_credits_leaderboard, get_stats, init_db, insert_event, reset_waste_events, update_food_inventory, ) from models import EventsResponse, StatsResponse, StatusResponse from camera import CameraManager from detector import ChangeDetector from analyzer import analyze_card_id, analyze_plate from stanford_inventory import get_stanford_inventory, sync_stanford_inventory from waste_model import build_waste_predictions from order_agent import get_greeting, process_message # --------------------------------------------------------------------------- # Global instances — shared across the app # --------------------------------------------------------------------------- # These are module-level singletons. The camera manager handles the physical # camera, the detector prevents duplicate plate analysis. camera_manager = CameraManager() change_detector = ChangeDetector() # Track the last successful analysis time as an ISO string for the status endpoint last_analysis_iso: str | None = None CREDITS_SCAN_COOLDOWN_SECONDS = 4.0 SCANNER_AI_INTERVAL_SECONDS = 0.9 SCANNER_CONFIRMATION_REQUIRED = 1 SCANNER_MIN_CONFIDENCE = 0.55 SCANNER_RENDER_FPS = 15 ANALYSIS_FRAME_MAX_WIDTH = 1024 CAMERA_START_TIMEOUT_SECONDS = 8.0 CAMERA_REBOOT_TIMEOUT_SECONDS = 12.0 CAMERA_STOP_TIMEOUT_SECONDS = 5.0 credits_scanner_lock = Lock() credits_scanner_active = False analysis_paused_for_credits = False analysis_task_handle: asyncio.Task | None = None credits_scan_task_handle: asyncio.Task | None = None credits_last_scan_iso: str | None = None credits_last_scan_id: str | None = None credits_last_award_by_id: dict[str, float] = {} credits_last_award_ts = 0.0 BARCODE_COLOR_RED = (48, 56, 235) BARCODE_COLOR_YELLOW = (0, 210, 255) BARCODE_COLOR_GREEN = (24, 201, 100) BARCODE_BORDER_THICKNESS = 8 scanner_status = "red" scanner_status_text = "No code/ID detected" scanner_last_source = "none" scanner_last_confidence = 0.0 scanner_last_detected_id: str | None = None scanner_streak_id = "" scanner_streak_count = 0 analysis_executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix="waste-analysis") def _set_credits_scanner_active(is_active: bool) -> None: global analysis_paused_for_credits, credits_scanner_active global scanner_status, scanner_status_text, scanner_last_source global scanner_last_confidence, scanner_last_detected_id global scanner_streak_id, scanner_streak_count global credits_last_award_ts with credits_scanner_lock: credits_scanner_active = bool(is_active) analysis_paused_for_credits = bool(is_active) scanner_status = "red" scanner_status_text = "No code/ID detected" scanner_last_source = "none" scanner_last_confidence = 0.0 scanner_last_detected_id = None scanner_streak_id = "" scanner_streak_count = 0 credits_last_award_ts = 0.0 def _is_credits_scanner_active() -> bool: with credits_scanner_lock: return credits_scanner_active def _is_analysis_paused_for_credits() -> bool: with credits_scanner_lock: return analysis_paused_for_credits def _is_analysis_task_running() -> bool: return analysis_task_handle is not None and not analysis_task_handle.done() def _is_credits_scan_task_running() -> bool: return credits_scan_task_handle is not None and not credits_scan_task_handle.done() async def _start_analysis_task() -> bool: global analysis_task_handle if _is_analysis_task_running(): return False change_detector.reset() analysis_task_handle = asyncio.create_task(analysis_loop()) print("[main] Analysis loop task started") return True async def _stop_analysis_task() -> bool: global analysis_task_handle if analysis_task_handle is None: return False task = analysis_task_handle analysis_task_handle = None if task.done(): return False task.cancel() try: await task except asyncio.CancelledError: pass except Exception as error: print(f"[main] Analysis task shutdown error: {error}") print("[main] Analysis loop task stopped") return True def _status_to_color(status: str) -> tuple[int, int, int]: normalized = str(status or "red").strip().lower() if normalized == "green": return BARCODE_COLOR_GREEN if normalized == "yellow": return BARCODE_COLOR_YELLOW return BARCODE_COLOR_RED def _set_scanner_view_state( *, status: str, text: str, source [truncated — 37129 more characters] ``` ### frontend/vite.config.js ```javascript import { defineConfig, loadEnv } from "vite"; import react from "@vitejs/plugin-react"; import tailwindcss from "@tailwindcss/vite"; export default defineConfig(({ mode }) => { const env = loadEnv(mode, ".", ""); const apiProxyTarget = env.VITE_API_PROXY_TARGET || "http://127.0.0.1:8000"; return { plugins: [react(), tailwindcss()], server: { proxy: { "/api": { target: apiProxyTarget, changeOrigin: true, }, }, }, }; }); ``` ### frontend/index.html ```html Savor — Food Waste Detection
``` ### frontend/eslint.config.js ```javascript import js from '@eslint/js' import globals from 'globals' import reactHooks from 'eslint-plugin-react-hooks' import reactRefresh from 'eslint-plugin-react-refresh' import { defineConfig, globalIgnores } from 'eslint/config' export default defineConfig([ globalIgnores(['dist']), { files: ['**/*.{js,jsx}'], extends: [ js.configs.recommended, reactHooks.configs.flat.recommended, reactRefresh.configs.vite, ], languageOptions: { ecmaVersion: 2020, globals: globals.browser, parserOptions: { ecmaVersion: 'latest', ecmaFeatures: { jsx: true }, sourceType: 'module', }, }, rules: { 'no-unused-vars': ['error', { varsIgnorePattern: '^[A-Z_]' }], }, }, ]) ``` ### backend/models.py ```python """ models.py — Pydantic models for API request/response shapes. These models define the exact JSON structure returned by each API endpoint, giving us automatic validation and OpenAPI documentation. """ from pydantic import BaseModel class WasteEvent(BaseModel): """A single food waste detection event.""" id: int timestamp: str # ISO 8601 datetime string image_path: str # Filename of the saved plate image food_items: list[str] # e.g. ["rice", "chicken", "broccoli"] waste_percentage: int # 0-100 summary: str # One-sentence description from GPT-4o class EventsResponse(BaseModel): """Response for the GET /api/events endpoint.""" events: list[WasteEvent] class TopWastedItem(BaseModel): """A single item in the top-wasted-foods list.""" item: str # Food name (lowercase) count: int # How many times this item appeared across all events class StatsResponse(BaseModel): """Response for the GET /api/stats endpoint.""" total_events: int avg_waste_percentage: float events_today: int top_wasted_items: list[TopWastedItem] class StatusResponse(BaseModel): """Response for the GET /api/status endpoint.""" camera_active: bool # Whether the camera is currently capturing camera_index: int | None = None # Active OpenCV camera index (if known) camera_backend: str | None = None # Backend label (e.g. DSHOW/DEFAULT) last_analysis_time: str | None # ISO timestamp of last successful analysis events_count: int # Total events in the database analysis_active: bool # Whether the waste analysis task is running scanner_active: bool # Whether credits scanner mode is active ``` ### backend/detector.py ```python """ detector.py — Change detection to prevent duplicate plate analysis. This is the core logic that prevents the same plate from being analyzed multiple times. It works by comparing each new camera frame against the last frame that was successfully analyzed, using SSIM (Structural Similarity Index). If the frames are too similar, we skip analysis. Additionally, a cooldown timer ensures we don't fire rapid API calls during scene transitions (e.g., a hand moving across the frame). The flow: 1. Camera grabs a frame every ~3 seconds 2. detector.should_analyze(frame) checks: a) Is the frame different enough from the last analyzed frame? (SSIM < threshold) b) Has enough time passed since the last analysis? (cooldown) 3. If both conditions are met → analyze the frame 4. After analysis, call detector.mark_analyzed(frame) to update the reference """ import time import cv2 import numpy as np from skimage.metrics import structural_similarity as ssim from config import SSIM_THRESHOLD, COOLDOWN_SECONDS class ChangeDetector: """ Detects meaningful scene changes between camera frames. Uses SSIM (Structural Similarity Index) to compare frames. SSIM produces a value between 0.0 (completely different) and 1.0 (identical). We consider frames with SSIM > threshold to be "the same scene" (same plate still in view). """ def __init__(self): # The last frame that was successfully sent for analysis, # stored as a small grayscale image for fast SSIM computation self._last_frame_gray: np.ndarray | None = None # Timestamp of the last successful analysis self._last_analysis_time: float = 0.0 @property def last_analysis_time(self) -> float: """Unix timestamp of the last successful analysis.""" return self._last_analysis_time def reset(self) -> None: """ Clear detector history so the next frame is analyzed immediately. Useful when switching camera modes/devices, where reusing the previous reference frame can suppress new analysis after restart. """ self._last_frame_gray = None self._last_analysis_time = 0.0 def should_analyze(self, frame: np.ndarray) -> bool: """ Decide whether a new frame should be sent to GPT-4o for analysis. Args: frame: The current camera frame (BGR numpy array from OpenCV) Returns: True if this frame represents a new scene worth analyzing. """ # Preprocess: convert to grayscale and resize to a small fixed # size for fast SSIM computation. 320x240 is plenty for detecting # whether the scene has changed. gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) gray = cv2.resize(gray, (320, 240)) # First frame ever — always analyze it if self._last_frame_gray is None: return True # Compute SSIM between the current frame and the last analyzed frame. # Higher score = more similar. If score is above the threshold, # the scene hasn't changed enough to warrant a new analysis. similarity = ssim(self._last_frame_gray, gray) if similarity > SSIM_THRESHOLD: # Scene looks the same — skip return False # Scene has changed! But check the cooldown to avoid rapid-fire # calls during transitions (e.g., someone's hand is in the frame). elapsed = time.time() - self._last_analysis_time if elapsed < COOLDOWN_SECONDS: print(f"[detector] Scene changed (SSIM={similarity:.3f}) but cooldown active ({elapsed:.0f}s / {COOLDOWN_SECONDS:.0f}s)") return False print(f"[detector] Scene changed (SSIM={similarity:.3f}) — triggering analysis") # Different scene + cooldown elapsed → go ahead and analyze return True def mark_analyzed(self, frame: np.ndarray) -> None: """ Update the reference frame and timestamp after a successful analysis. Call this AFTER the GPT-4o analysis succeeds, not before, so that a failed API call doesn't prevent retrying with the same frame. Args: frame: The frame that was just analyzed (BGR numpy array) """ self._last_frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) self._last_frame_gray = cv2.resize(self._last_frame_gray, (320, 240)) self._last_analysis_time = time.time() ``` ### backend/config.py ```python """ config.py — Central configuration for the Savor backend. Loads environment variables from a .env file and exposes all tuneable settings as module-level constants. Every other backend module imports from here rather than reading env vars directly. """ import os from dotenv import load_dotenv # Load .env file from the project root (one level above /backend) load_dotenv(os.path.join(os.path.dirname(__file__), "..", ".env")) # --------------------------------------------------------------------------- # OpenAI # --------------------------------------------------------------------------- # Required — the GPT-4o Vision API key used by analyzer.py OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "") # --------------------------------------------------------------------------- # Database # --------------------------------------------------------------------------- # SQLite database file lives inside the backend/ directory DATABASE_PATH = os.path.join(os.path.dirname(__file__), "vision_bin.db") # --------------------------------------------------------------------------- # Camera & capture # --------------------------------------------------------------------------- # Which camera device index to use (0 = default/built-in webcam) CAMERA_INDEX = int(os.getenv("CAMERA_INDEX", "1")) # OpenCV camera backend preference. # Windows defaults to DirectShow ("dshow") to avoid MSMF grab instability. # Supported values: dshow, msmf, default, auto CAMERA_BACKEND = os.getenv( "CAMERA_BACKEND", "dshow" if os.name == "nt" else "default", ).strip().lower() # How often (in seconds) the background loop grabs a frame for analysis CAPTURE_INTERVAL = float(os.getenv("CAPTURE_INTERVAL", "3.0")) # Food item blocklist (comma-separated substrings). Any detected food item # containing one of these tokens is ignored for storage + analytics. FOOD_ITEM_BLOCKLIST = os.getenv("FOOD_ITEM_BLOCKLIST", "burrito") # Force a plate analysis if no analysis has been run for this many seconds, # even when scene-change detection says frames are similar. ANALYSIS_FORCE_INTERVAL_SECONDS = float( os.getenv("ANALYSIS_FORCE_INTERVAL_SECONDS", "8.0") ) # Max seconds to wait for a single OpenAI plate-analysis request. ANALYSIS_REQUEST_TIMEOUT_SECONDS = float( os.getenv("ANALYSIS_REQUEST_TIMEOUT_SECONDS", "22.0") ) # --------------------------------------------------------------------------- # Change detection (duplicate plate prevention) # --------------------------------------------------------------------------- # SSIM (structural similarity) threshold — frames more similar than this # value are considered "the same scene" and won't trigger a new analysis. # Range is 0.0 (completely different) to 1.0 (identical). SSIM_THRESHOLD = float(os.getenv("SSIM_THRESHOLD", "0.85")) # Minimum seconds between two consecutive analyses, even if the scene # changes. Prevents rapid-fire API calls during transitions. COOLDOWN_SECONDS = float(os.getenv("COOLDOWN_SECONDS", "10")) # --------------------------------------------------------------------------- # Image storage # --------------------------------------------------------------------------- # Directory where captured plate images are saved IMAGE_SAVE_DIR = os.path.join(os.path.dirname(__file__), "captured_images") # JSON file that tracks cumulative food waste inventory FOOD_INVENTORY_PATH = os.path.join(os.path.dirname(__file__), "food_inventory.json") # JSON file for historical/filler inventory data used by dashboard trends FOOD_INVENTORY_HISTORICAL_PATH = os.path.join( os.path.dirname(__file__), "food_inventory_historical.json", ) # --------------------------------------------------------------------------- # Stanford dining inventory sync # --------------------------------------------------------------------------- # Legacy Bright Data settings (currently not required by Stanford inventory sync). BRIGHTDATA_API_KEY = os.getenv("BRIGHTDATA_API_KEY", "") BRIGHTDATA_BROWSER_WSS = os.getenv("BRIGHTDATA_BROWSER_WSS", "") # Source menu URL and scrape selection defaults. STANFORD_DINING_MENU_URL = os.getenv( "STANFORD_DINING_MENU_URL", "https://rdeapps.stanford.edu/dininghallmenu/", ) STANFORD_DINING_HALL = os.getenv("STANFORD_DINING_HALL", "Wilbur") STANFORD_DINING_MEAL = os.getenv("STANFORD_DINING_MEAL", "Lunch") # Offset from today's date for selected menu day (0=today, 1=tomorrow, etc). STANFORD_DINING_DAY_OFFSET = int(os.getenv("STANFORD_DINING_DAY_OFFSET", "0")) # Cached inventory menu JSON used by /api/inventory and frontend inventory UI. STANFORD_INVENTORY_PATH = os.path.join( os.path.dirname(__file__), "stanford_inventory.json", ) # --------------------------------------------------------------------------- # Google Cloud Text-to-Speech # --------------------------------------------------------------------------- GOOGLE_TTS_API_KEY = os.getenv("GOOGLE_TTS_API_KEY", "") ``` [16 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]