# Project export: FoodAllerGuard

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 12.0
- Tagline: One scan a day keeps the allergies away!
- Devpost: https://devpost.com/software/foodallerguard-quhexj
- GitHub: https://github.com/thamizarasus/CalHacks-web
- Team: 3 GitHub contributor(s) — Thamizarasu (22 commits), David Lei (6 commits), Ramanuja Mohanty (5 commits)

## Devpost submission (written by the team)

### Inspiration

One of our group members had an allergic reaction to food during an outing. The menu did not contain any listed allergens.

### What it does

Scans text files (.txt, .pdf, etc), live scans, and photos of menus. Uses text recognition models to identify ingredients in food items. Cross-references with user-provided list of allergens and then recommends to user if food is safe to consume or not.

### How we built it

Frontend: React, Tailwind CSS, Vite | Backend: Flask, OpenCV for training data, pytesseract for text recognition module.

### Challenges we ran into

Lack of time, lack of experience, connection issues with camera, problems with integrating pytesseract.

### Accomplishments we're proud of

Furthered knowledge of AI models, finished despite lack of time, implementation of solution

### What we learned

Text detection & recognition, image processing, computer vision, natural language processing.

### What's next

Develop a mobile app, partner with restaurants to show allergens on their menus.

## README (from the GitHub repository)

# FoodAllerGuard

**One scan a day keeps the allergies away!**

---
## Description

FoodAllerGuard is a web application that analyzes the ingredients in a food item (either via live scan or still photos or text files) and alerts the user if the food item being evaluated contains any allergens or other possibly harmful ingredients.

---
## Tech Stack
- ### Frontend
	- **React** (^18.2.0)
	- **React Router DOM** (^7.9.4)
	- **Tailwind CSS** (^3.4.0)
	- **Vite** (^5.0.8)
- ### Backend
	- **Flask** (Python)
	- **Flask-CORS** (^4.0.0)
	- **OpenCV**
	- **pytesseract**

---
## Instructions

### Installing Dependencies

#### Frontend (JavaScript)
Navigate to the frontend directory and install dependencies:
```bash
cd foodallerguard/frontend
npm install
```

This will install all JavaScript resources including:
- **React** (^18.2.0) - UI library
- **React DOM** (^18.2.0) - React rendering
- **React Router DOM** (^7.9.4) - Client-side routing
- **Vite** (^5.0.8) - Build tool and dev server
- **Tailwind CSS** (^3.4.0) - Utility-first CSS framework
- **PostCSS** & **Autoprefixer** - CSS processing
- **ESLint** - Code linting
- **@vitejs/plugin-react** - React plugin for Vite

#### Backend (Python)
Navigate to the backend directory and install dependencies:
```bash
cd foodallerguard/backend
pip install -r requirements.txt
```

Or using pipenv:
```bash
pipenv install
```

This will install:
- **Flask** (^2.3.3) - Web framework
- **Flask-CORS** (^4.0.0) - Cross-origin resource sharing
- **python-dotenv** (^1.0.0) - Environment variables
- **pytesseract** - OCR for text extraction
- **opencv-python** - Computer vision library
- **Pillow** - Image processing

### Running the Application

1. Start the backend server:
```bash
cd foodallerguard/backend
python app.py
```

2. In a separate terminal, start the frontend development server:
```bash
cd foodallerguard/frontend
npm run dev
```

The application will be available at http://localhost:3000

---
## Developers

|Name|Role|Email|GitHub|
|-------------------|------------------------|-----------------------|------------|
|Thamizarasu Sankara|Team Lead & Frontend Dev| thamizarasus@gmail.com|thamizarasus|
|Ramanuja Mohanty|Backend Dev| ramanuja.mohanty@gmail.com|RamanujaMohanty|
|David Lei|Backend Dev| DavidtheDefender2003@gmail.com|CaliCanine|

---
## About Us
The concept for FoodAllerGuard began around 2024. We were having lunch at a restaurant when, suddenly, one of our members had an adverse reaction to the item they were eating.
After the affected individual received all relevant care, we were able to determine why they had that reaction: the allergens
were not clearly listed in the menu. Hence, FoodAllerGuard came into being, a web app (soon to be mobile) that allows users to
upload photos of their menus or menu PDFs and learn exactly which allergens are present in the food that can affect them.

---
## Credits
- OpenCV - Training Data
- pytesseract - Image Recognition Module

---
# <center>THANK YOU!</center>


## Detected evidence (automated analysis)

Indexed codebase: 27 recognized source files, 92 KB.
- CSS (language) — detected in the code
- Flask (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code

## Codebase structure (from repository index)

### Files (38 of 38)

```
.gitignore
foodallerguard/backend/allergens.json
foodallerguard/backend/app.py
foodallerguard/backend/backend.log
foodallerguard/backend/frozen_east_text_detection.pb
foodallerguard/backend/models/__init__.py
foodallerguard/backend/models/scans.py
foodallerguard/backend/README.md
foodallerguard/backend/requirements.txt
foodallerguard/frontend/index.html
foodallerguard/frontend/package.json
foodallerguard/frontend/postcss.config.cjs
foodallerguard/frontend/src/App.jsx
foodallerguard/frontend/src/components/AllergySelector.jsx
foodallerguard/frontend/src/components/CameraScanningPage.jsx
foodallerguard/frontend/src/components/CustomAllergenInput.jsx
foodallerguard/frontend/src/components/FeaturesPage.jsx
foodallerguard/frontend/src/components/Header.jsx
foodallerguard/frontend/src/components/HomePage.jsx
foodallerguard/frontend/src/components/Results.jsx
foodallerguard/frontend/src/components/ResultsPage.jsx
foodallerguard/frontend/src/components/ScanningPage.jsx
foodallerguard/frontend/src/components/SelectedAllergies.jsx
foodallerguard/frontend/src/components/TopNavTabs.jsx
foodallerguard/frontend/src/components/UploadCard.jsx
foodallerguard/frontend/src/context/ScanContext.jsx
foodallerguard/frontend/src/index.css
foodallerguard/frontend/src/lib/api.js
foodallerguard/frontend/src/lib/firebase.js.example
foodallerguard/frontend/src/lib/mockScanner.js
foodallerguard/frontend/src/main.jsx
foodallerguard/frontend/src/pages/About.jsx
foodallerguard/frontend/src/pages/Contact.jsx
foodallerguard/frontend/tailwind.config.cjs
foodallerguard/frontend/vite.config.js
Pipfile
Pipfile.lock
README.md
```

### Dependencies

- foodallerguard/backend/requirements.txt: Flask@==2.3.3, Flask-CORS@==4.0.0, numpy@==1.24.3, opencv-python@==4.8.1.78, Pillow@==10.0.0, pytesseract@==0.3.10, python-dotenv@==1.0.0
- foodallerguard/frontend/package.json: @types/react@^18.2.43, @types/react-dom@^18.2.17, @vitejs/plugin-react@^4.2.1, autoprefixer@^10.4.16, eslint@^8.55.0, eslint-plugin-react@^7.33.2, eslint-plugin-react-hooks@^4.6.0, eslint-plugin-react-refresh@^0.4.5, postcss@^8.4.32, preview@^0.1.3, react@^18.2.0, react-dom@^18.2.0, react-router-dom@^7.9.4, tailwindcss@^3.4.0, vite@^5.0.8

### Recent commits (newest first)

- Fix: Add missing dependencies and improve error handling
- Local host error fixed
- fix backend log
- fixed local host issue
- hide restaurant menu for cam scan
- Fix camera scan results to show only allergens without menu mock data
- cam scan
- Add /api/scan-image endpoint for camera scanning with OCR
- Added final README.md version.
- Added .gitignore to repository.
- backend log fix
- Fix some errors
- Delete duplicate scans
- Scan History
- update on the home and result pages
- emergency stop scan
- remove unwanted custome allergy, and result page upgrade
- Contact page
- about section update
- improving the front trencH

## Key source files (fetched from GitHub, selected and truncated for size)

### foodallerguard/backend/requirements.txt

```
Flask==2.3.3
Flask-CORS==4.0.0
python-dotenv==1.0.0
Pillow==10.0.0
opencv-python==4.8.1.78
numpy==1.24.3
pytesseract==0.3.10

```

### foodallerguard/frontend/package.json

```
{
  "name": "foodallerguard-frontend",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "vite build",
    "lint": "eslint . --ext js,jsx --report-unused-disable-directives --max-warnings 0",
    "preview": "vite preview"
  },
  "dependencies": {
    "preview": "^0.1.3",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-router-dom": "^7.9.4"
  },
  "devDependencies": {
    "@types/react": "^18.2.43",
    "@types/react-dom": "^18.2.17",
    "@vitejs/plugin-react": "^4.2.1",
    "autoprefixer": "^10.4.16",
    "eslint": "^8.55.0",
    "eslint-plugin-react": "^7.33.2",
    "eslint-plugin-react-hooks": "^4.6.0",
    "eslint-plugin-react-refresh": "^0.4.5",
    "postcss": "^8.4.32",
    "tailwindcss": "^3.4.0",
    "vite": "^5.0.8"
  }
}

```

### foodallerguard/backend/app.py

```python
from flask import Flask, request, jsonify
from flask_cors import CORS
import json
import os
import cv2 as cv
import numpy as np
import pytesseract
import random
import base64
import io
from PIL import Image
from models.scans import save_scan, get_scans, clear_scans

app = Flask(__name__)
CORS(app)

# Configure logging to file and console
import logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('backend.log'),
        logging.StreamHandler()
    ]
)
logger = logging.getLogger(__name__)

@app.before_request
def log_request_info():
    logger.info(f'{request.method} {request.path} - {request.remote_addr}')
    if request.is_json:
        logger.info(f'Body: {request.get_json()}')

# Get the directory of the current file
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
MODEL_PATH = os.path.join(BASE_DIR, "frozen_east_text_detection.pb")

# Initialize the text detector only if the model file exists
textDetector = None
if os.path.exists(MODEL_PATH):
    textDetector = cv.dnn_TextDetectionModel_EAST(MODEL_PATH)
    textDetector.setNMSThreshold(0.4)
    textDetector.setConfidenceThreshold(0.8)
else:
    print(f"Warning: Model file not found at {MODEL_PATH}")

# ✅ Fake OCR Text Data (customize if needed)
MOCK_OCR_TEXT = """
    Crispy Fried Tilapia
    Southern Fried Catfish
    Tempura Haddock
    Asian-Style Fried Sole
    Spicy Fried Mahi Mahi
    Garden Salad
    Fries
    Buttercrust Wahoo
    Grilled Chicken
    Peanut Satay
    Crab Cakes
    Buffalo Wings
    Cheeseburger
    Mozzarella Sticks
    Caesar Salad
    Tofu Bowl
    Shrimp Tacos
    Clam Chowder
    Garlic Bread
    Spaghetti
"""

# ✅ Allergen Keyword Mapping
ALLERGEN_KEYWORDS = {
    "fish": ["fish", "tilapia", "catfish", "mahi", "sole", "snapper", "haddock", "wahoo"],
    "dairy": ["cheese", "milk", "cream", "butter", "buffalo"],
    "peanuts": ["peanut", "satay"],
    "eggs": ["egg", "caesar"],
    "soy": ["soy", "tofu"],
    "wheat": ["bread", "wrap", "flour", "wheat", "garlic bread", "spaghetti"],
    "shellfish": ["crab", "shrimp", "lobster", "clam"],
    "sesame": ["sesame"],
    "nuts": ["peanut", "almond", "cashew", "walnut"],
}

# Load allergens data
def load_allergens():
    try:
        with open('allergens.json', 'r') as f:
            return json.load(f)
    except FileNotFoundError:
        return {}

@app.route("/", methods=["GET"])
def home_page():
    """API Information"""
    return jsonify({
        "message": "FoodAllerGuard API",
        "endpoints": [
            "/api/health - Health check",
            "/api/allergens - Get available allergens",
            "/api/check - Check ingredients for allergens"
        ]
    })

@app.route("/camera", methods=["GET"])
def camera_page():
    """Camera access for scanning"""
    if textDetector is None:
        return jsonify({"error": "Text detection model not available"}), 503
    
    try:
        # Check camera access
        camera = cv.VideoCapture(0)
        if not camera.isOpened():
            return jsonify({"error": "Camera cannot be opened"}), 500
        
        ret, frame = camera.read()
        camera.release()
        
        if not ret:
            return jsonify({"error": "Unable to receive frame"}), 500
        
        return jsonify({"status": "Camera is ready"})
    except Exception as e:
        return jsonify({"error": str(e)}), 500

@app.route('/api/allergens', methods=['GET'])
def get_allergens():
    """Get list of all available allergens"""
    allergens = load_allergens()
    return jsonify(list(allergens.keys()))

@app.route('/api/check', methods=['POST'])
def check_ingredients():
    """Check ingredients against user's allergens"""
    data = request.get_json()
    user_allergens = data.get('allergens', [])
    ingredients = data.get('ingredients', [])
    
    allergens_db = load_allergens()
    found_allergens = []
    
    for ingredient in ingredients:
        ingredient_lower = ingredient.lower()
        for allergen in user_allergens:
            if allergen in allergens_db:
                allergen_ingredients = allergens_db[allergen]
                for allergen_ingredient in allergen_ingredients:
                    if allergen_ingredient.lower() in ingredient_lower:
                        found_allergens.append({
                            'allergen': allergen,
                            'ingredient': ingredient,
                            'matched_term': allergen_ingredient
                        })
    
    return jsonify({
        'safe': len(found_allergens) == 0,
        'found_allergens': found_allergens
    })

@app.route('/api/health', methods=['GET'])
def health_check():
    """Health check endpoint"""
    return jsonify({'status': 'healthy'})

@app.route('/api/scan-image', methods=['POST'])
def scan_image():
    """Scan uploaded image and detect allergens using OCR"""
    data = request.get_json()
    selected_allergens = [a.lower() for a in data.get("allergens", [])]
    image_data = data.get("image", "")
    
    if not image_data:
        return jsonify({"error": "No image provided"}), 400
    
    try:
        # Decode base64 image
        # Remove data URL prefix if present
        if ',' in image_data:
            image_data = image_data.split(',')[1]
        
        # Decode base64
        image_bytes = base64.b64decode(image_data)
        image = Image.open(io.BytesIO(image_bytes))
        
        # Convert to RGB if necessary
        if image.mode != 'RGB':
            image = image.convert('RGB')
        
        # Convert PIL Image to OpenCV format
        img_array = np.array(image)
        opencv_img = cv.cvtColor(img_array, cv.COLOR_RGB2BGR)
        
        # Extract text using Tesseract OCR
        extracted_text = pytesseract.image_to_string(opencv_img)
        
        # Clean and split text into lines
        lines = [line.strip() for line in extracted_text.split('\n') 
[truncated — 5278 more characters]
```

### foodallerguard/frontend/src/main.jsx

```javascript
import React from 'react'
import ReactDOM from 'react-dom/client'
import App from './App.jsx'
import './index.css'

ReactDOM.createRoot(document.getElementById('root')).render(
  <React.StrictMode>
    <App />
  </React.StrictMode>,
)

```

### foodallerguard/frontend/src/App.jsx

```javascript
import { BrowserRouter as Router, Routes, Route } from "react-router-dom";
import { ScanProvider } from "./context/ScanContext";
import Header from "./components/Header";
import HomePage from "./components/HomePage";
import FeaturesPage from "./components/FeaturesPage";
import ScanningPage from "./components/ScanningPage";
import CameraScanningPage from "./components/CameraScanningPage";
import ResultsPage from "./components/ResultsPage";
import About from "./pages/About";
import Contact from "./pages/Contact";

export default function App() {
  return (
    <ScanProvider>
      <Router>
        <Header />
        <Routes>
          <Route path="/" element={<HomePage />} />
          <Route path="/features" element={<FeaturesPage />} />
          <Route path="/scanning" element={<ScanningPage />} />
          <Route path="/camera-scan" element={<CameraScanningPage />} />
          <Route path="/results" element={<ResultsPage />} />
          <Route path="/about" element={<About />} />
          <Route path="/contact" element={<Contact />} />
        </Routes>
      </Router>
    </ScanProvider>
  );
}

```

### foodallerguard/frontend/vite.config.js

```javascript
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [react()],
  server: {
    port: 3000,
    proxy: {
      '/api': {
        target: 'http://127.0.0.1:6000',
        changeOrigin: true,
        secure: false,
      },
    },
  },
})

```

### foodallerguard/frontend/index.html

```html
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" type="image/svg+xml" href="/vite.svg" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>FoodAllerGuard</title>
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="/src/main.jsx"></script>
  </body>
</html>

```

### foodallerguard/backend/models/__init__.py

```python
# Models package


```

### foodallerguard/backend/models/scans.py

```python
# In-memory scan storage
scans_db = []

def save_scan(scan_data):
    """Save a scan to the database"""
    scans_db.append(scan_data)
    return scan_data

def get_scans():
    """Get all scans"""
    return scans_db

def clear_scans():
    """Clear all scans"""
    scans_db.clear()


```

### foodallerguard/frontend/src/index.css

```css
@tailwind base;
@tailwind components;
@tailwind utilities;

body {
  margin: 0;
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
    'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
    sans-serif;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

code {
  font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New',
    monospace;
}

```

[17 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]