# Project export: NutriVision
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: TreeHacks 2025
- Tagline: Zero Obese
- Devpost: https://devpost.com/software/nutrivision-msxjyl
- GitHub: https://github.com/enkhbold470/nutrivision
- Video: https://www.youtube.com/embed/57Bno38DyZU?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 4 GitHub contributor(s) — enkhbold470 (46 commits), shuixianli (5 commits), Archita Nemalikanti (2 commits), akaashrp (1 commits)
## Devpost submission (written by the team)
### Overview
I (Archita) have tried countless ways to count my calories, but every time, I ended up giving up—it was just too much work. Manually adding up numbers everyday became overwhelming. Yes, iPhone apps exist that let you take pictures of your food, but they aren’t live, or capable of tracking your daily caloric intake–-I would have to end up writing down and adding my total calories myself by hand, and existing iPhone applications do not already do this for me. They also don’t tell you whether you’re actually working toward your food-intake goals and making progress. The bottom line problem: I want to lose weight, but I need a live assistant to watch what I eat and count my calories because I am too busy to manually add up the numbers. While iPhone applications make you take a picture of your food, and tell you the calories for a particular meal, users want a truly effortless experience. I (Archita) have always dreamed of someone to watch what I eat, and give me a notification whenever I binge eat chocolates, or when I cross my daily caloric intake. Our application provides an effortless way to track food intake in real time. Once you use it, you’ll wonder how you ever lived without it! And we pulled it off! 🚀 The report below follows the assumption that we are deploying our application in the Ray-Ban Meta Glasses—a sleek pair of glasses. In our demo, our application runs on Meta Quest 2. We assume that these glasses will be as common as a smartphone in the future. Our AI-powered VR application will automatically track what you eat, logging calories, protein, fat, potassium, and carbs in real time. If a user is binge-eating unhealthy foods, the system sends a live notification, telling them to stop and helping them work towards their goals. User Experience Workflow The user first enters their weight loss goal: for example, lose 5 pounds by the next month. The app will calculate how many calories they need to consume everyday to reach this weight loss goal. The VR headset detects when the person is eating and will log in the calories, potassium, protein, carbs, and fat of that meal. The user will get a live notification—as they are eating—if they reach above their daily caloric limit. The UI displays overall results/trends about the user's daily food intake. What we are proud of, and how we built it It’s incredible that we created a novel application with the potential to revolutionize the future of health tracking! Wow! We will most definitely use this app to track our calories when VR/AR glasses become more common. We developed an object detection model based on YOLOv11 to perform initial food detection and integrated our application with the OpenAI API for calorie counting. Since the original YOLO model was trained on the COCO dataset, which lacks class labels corresponding to different kinds of food, we scraped food images from online datasets and used the Grounding DINO model to annotate them with bounding boxes for YOLO finetuning. The model was fine tuned on a Jetson Orin Nano. The model is deployed on a web app on a Meta Quest 2 using the onnxruntime JS library and accesses the live video feed to make predictions. Specifically, the model detects when food is in the frame (future support will be added for detecting when a person is eating) and the web app automatically captures a picture to be used as input for a query to the OpenAI API (the YOLO model makes this process less expensive by reducing the number of queries). The OpenAI API call allows us to determine what food the user is eating, its weight, and the approximate nutrient content. We make a further API call to the USDA caloric food database to ensure that our nutrient predictions are more accurate. This nutrient information is sent to an SQL database and stored for future reference. If the user exceeds their daily calorie intake goal, they will receive a notification from the app. Business Model $10 Billion Dollar Market Cap estimation. We expect our app to be used on the Meta Ray-Ban Glasses, and aggressive AI/VR adoption in the next 5-10 years. As if they aren’t already becoming common, the Meta Ray-Ban Glasses are rapidly gaining traction and will soon be ubiquitous. We solve the core calorie-tracking pain point better than any other app, and this could be the next MyFitnessPal but smarter, faster, and fully automated. Our app assumes aggressive AI/VR adoption, seamless integration into daily life, and solving the core calorie-tracking pain point better than any other app. If executed well, this could be the next MyFitnessPal but smarter, faster, and fully automated. Inky - De Anza, Specializes in VR Frontend and Backend Development Susan - Stanford, specializes in Hardware Camera Integration and ML/AI Archita - Cornell, specializes in Backend Development and ML/AI Akaash - Georgia Tech, specializes in Vision and Backend Development Regularly watching when the user is consuming addictive substances, such as nicotine, alcohol, marijuana, etc, and giving a notification to say “that’s enough substance for today”. We plan to expand it to an overall live health monitoring assistant. Integrate with technology such as Meta AR glasses, which is a much more natural item to wear on a daily basis. The Meta Quest has privacy issues with live streaming. We are not allowed to live stream on the Meta Quest, and we would need to submit a petition to get our use-case approved. It is possible to get our livestream purpose approved, but the approval process takes a very long time and is too long for the duration of this hackathon. So, to demonstrate a proof of concept, we quickly pivoted to using a web-cam that can stream onto MetaQuest. The Webcam connects to our local computer for the server, and streams to the MetaQuest via RTSP protocol. The webcam is mounted on top of the Meta Quest, and we behave as if the Meta Quest itself is live-streaming data. A ton! Live Streaming a peripheral hardware webcam using RTSP protocol, programming VR and AR on the Meta Quest 2, and training YOLOv11 to detect when a person is eating food! Our pipeline is built ground up and novel—we figured it out along the way with little documentation to lean on! OpenAI API: to retrieve nutrition information about food NVIDIA Jetson Orin Nano: to train the YOLOv11 model Perplexity Sonar: to display eating habits insights. Other Technologies: HTML, CSS, Javascript, Flask, Postgres, YOLOv11, hardware camera webcam In the future, we plan to integrate this application with Terra API as it is very easy to retrieve other health data from the user, such as daily calories burnt. We were looking forward to integrating with TerraAPI.
## README (from the GitHub repository)
# NurtiVision
# The Problem
I (Archita) have tried countless ways to count my calories, but every time, I ended up giving up—it was just too much work. Manually adding up numbers everyday became overwhelming. Yes, iPhone apps exist that let you take pictures of your food, but they aren’t live, or capable of tracking your daily caloric intake–-I would have to end up writing down and adding my total calories myself by hand, and existing iPhone applications do not already do this for me. They also don’t tell you whether you’re actually working toward your food-intake goals and making progress.
The bottom line problem: I want to lose weight, but I need a live assistant to watch what I eat and count my calories because I am too busy to manually add up the numbers. While iPhone applications make you take a picture of your food, and tell you the calories for a particular meal, users want a truly effortless experience. I (Archita) have always dreamed of someone to watch what I eat, and give me a notification whenever I binge eat chocolates, or when I cross my daily caloric intake.
# The Solution
The report below follows the assumption that we are deploying our application in the Ray-Ban Meta Glasses—a sleek pair of glasses. In our demo, our application runs on Meta Quest 2.
We assume that these glasses will be as common as a smartphone in the future.
Our AI-powered VR application will automatically track what you eat, logging calories, protein, fat, potassium, and carbs in real time. If a user is binge-eating unhealthy foods, the system sends a live notification, telling them to stop and helping them work towards their goals.
## User Experience Workflow
1. The user first enters their weight loss goal: for example, lose 5 pounds by the next month.
2. The app will calculate how many calories they need to consume everyday to reach this weight loss goal.
3. The VR headset detects when the person is eating and will log in the calories, potassium, protein, carbs, and fat of that meal.
4. The user will get a live notification—as they are eating—if they reach above their daily caloric limit.
5. The UI displays overall results/trends about the user's daily food intake.
## What we are proud of, and how we built it
It’s incredible that we created a novel application with the potential to revolutionize the future of health tracking! Wow! We will most definitely use this app to track our calories when VR/AR glasses become more common.
We developed an object detection model based on YOLOv11 to perform initial food detection and integrated our application with the OpenAI API for calorie counting. Since the original YOLO model was trained on the COCO dataset, which lacks class labels corresponding to different kinds of food, we scraped food images from online datasets and used the Grounding DINO model to annotate them with bounding boxes for YOLO finetuning. The model was fine tuned on a Jetson Orin Nano. The model is deployed on a web app on a Meta Quest 2 using the onnxruntime JS library and accesses the live video feed to make predictions. Specifically, the model detects when food is in the frame (future support will be added for detecting when a person is eating) and the web app automatically captures a picture to be used as input for a query to the OpenAI API (the YOLO model makes this process less expensive by reducing the number of queries). The OpenAI API call allows us to determine what food the user is eating, its weight, and the approximate nutrient content. We make a further API call to the USDA caloric food database to ensure that our nutrient predictions are more accurate. This nutrient information is sent to an SQL database and stored for future reference. If the user exceeds their daily calorie intake goal, they will receive a notification from the app.
## Business Model
$10 Billion Dollar Market Cap estimation.
We expect our app to be used on the Meta Ray-Ban Glasses, and aggressive AI/VR adoption in the next 5-10 years.
As if they aren’t already becoming common, the Meta Ray-Ban Glasses are rapidly gaining traction and will soon be ubiquitous.
We solve the core calorie-tracking pain point better than any other app, and this could be the next MyFitnessPal but smarter, faster, and fully automated.
Our app assumes aggressive AI/VR adoption, seamless integration into daily life, and solving the core calorie-tracking pain point better than any other app. If executed well, this could be the next MyFitnessPal but smarter, faster, and fully automated.
# The Team (4 people, 4 schools represented)
- **Inky** - De Anza, Specializes in VR Frontend and Backend Development
- **Susan** - Stanford, specializes in Hardware Camera Integration and ML/AI
- **Archita** - Cornell, specializes in Backend Development and ML/AI
- **Akaash** - Georgia Tech, specializes in Vision and Backend Development
# Next Steps
- Regularly watching when the user is consuming addictive substances, such as nicotine, alcohol, marijuana, etc, and giving a notification to say “that’s enough substance for today”. We plan to expand it to an overall live health monitoring assistant.
- Integrate with technology such as Meta AR glasses, which is a much more natural item to wear on a daily basis.
# Challenges we ran into
The Meta Quest has privacy issues with live streaming. We are not allowed to live stream on the Meta Quest, and we would need to submit a petition to get our use-case approved.
It is possible to get our livestream purpose approved, but the approval process takes a very long time and is too long for the duration of this hackathon.
So, to demonstrate a proof of concept, we quickly pivoted to using a web-cam that can stream onto MetaQuest. The Webcam connects to our local computer for the server, and streams to the MetaQuest via RTSP protocol. The webcam is mounted on top of the Meta Quest, and we behave as if the Meta Quest itself is live-streaming data.
# What we learned
A ton! Live Streaming a peripheral hardware webcam using RTSP protocol, programming VR and AR on the Meta Quest 2, and training YOLOv11 to detect when a person is eating food! Our pipeline is built ground up and novel—we figured it out along the way with little documentation to lean on!
# Built With
- **OpenAI API**: to retrieve nutrition information about food
- **NVIDIA Jetson Orin Nano**: to train the YOLOv11 model
- **Perplexity Sonar**: to display eating habits insights.
- **Other Technologies**: Unity, React, Node.js, Flask, SQLAlchemy, YOLOv11, USDA Food API, hardware camera webcam
# Side Note
In the future, we plan to integrate this application with Terra API as it is very easy to retrieve other health data from the user, such as daily calories burnt. We were looking forward to integrating with TerraAPI.
# Try it Out!
- Insert a public github link.
- Insert Youtube video link.
## Detected evidence (automated analysis)
Indexed codebase: 93 recognized source files, 1081 KB.
- CSS (language) — detected in the code
- Flask (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Next.js (technology) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- PyTorch (technology) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- PostgreSQL (technology) — claimed on Devpost, not found in the code
## Codebase structure (from repository index)
### Files (119 of 119)
```
.DS_Store
backend/app.py
backend/db.py
backend/requirements.txt
client-side/.eslintrc.json
client-side/.gitignore
client-side/.vscode/settings.json
client-side/app/api/analyze-food/route.ts
client-side/app/api/capture/route.ts
client-side/app/api/proxy-camera/route.ts
client-side/app/camera/page.tsx
client-side/app/globals.css
client-side/app/goal/page.tsx
client-side/app/layout.tsx
client-side/app/not-found.tsx
client-side/app/page.tsx
client-side/app/settings/page.tsx
client-side/app/test/page.tsx
client-side/app/types/nutrition.ts
client-side/components.json
client-side/components/AutoRecord.tsx
client-side/components/DarkMode.tsx
client-side/components/face-model-changer/FaceModelSelect.tsx
client-side/components/face-model-changer/FaceModelSelectContent.tsx
client-side/components/FlipCamera.tsx
client-side/components/header.tsx
client-side/components/hooks/useInterval.tsx
client-side/components/infoCard.tsx
client-side/components/InformationDetailDialog.tsx
client-side/components/InformationDialog.tsx
client-side/components/landing.tsx
client-side/components/loading.tsx
client-side/components/mobile-settings.tsx
client-side/components/model-changer/ModelSelect.tsx
client-side/components/model-changer/ModelSelectContent.tsx
client-side/components/model-settings/CameraSelect.tsx
client-side/components/model-settings/FaceLandmarkSetting.tsx
client-side/components/model-settings/FaceModelSetting.tsx
client-side/components/model-settings/GestureModelSetting.tsx
client-side/components/model-settings/InterfaceDelegate.tsx
client-side/components/model-settings/ModelSetting.tsx
client-side/components/model-settings/ModelSettingDialog.tsx
client-side/components/model-settings/ObjectModelSetting.tsx
client-side/components/navbar/bottom-navbar.tsx
client-side/components/plant-card.tsx
client-side/components/plant-notification.tsx
client-side/components/RecordVideo.tsx
client-side/components/ScreenShot.tsx
client-side/components/sensorInfo.tsx
client-side/components/sign-in.tsx
client-side/components/sign-up.tsx
client-side/components/theme-provider.tsx
client-side/components/theme-toggle.tsx
client-side/components/ui/button.tsx
client-side/components/ui/dialog.tsx
client-side/components/ui/dropdown-menu.tsx
client-side/components/ui/hover-card.tsx
client-side/components/ui/input.tsx
client-side/components/ui/label.tsx
client-side/components/ui/popover.tsx
client-side/components/ui/select.tsx
client-side/components/ui/separator.tsx
client-side/components/ui/slider.tsx
client-side/components/ui/sonner.tsx
client-side/components/ui/switch.tsx
client-side/components/Volume.tsx
client-side/components/YourPlants.tsx
client-side/lib/Drawing3d.ts
client-side/lib/loadModel.ts
client-side/lib/types.ts
client-side/lib/utils.ts
client-side/mediapipe/face-detection.ts
client-side/mediapipe/face-landmark.ts
client-side/mediapipe/gesture-recognition.ts
client-side/mediapipe/mediapipe-vision.ts
client-side/mediapipe/object-detection.ts
client-side/next.config.mjs
client-side/package.json
client-side/postcss.config.mjs
client-side/providers/CameraDevicesProvider.tsx
client-side/public/site.webmanifest
client-side/public/sw.js
client-side/README.md
client-side/tailwind.config.ts
client-side/tsconfig.json
client-side/utils/audio.ts
client-side/utils/definitions.ts
plain-js/index.html
plain-js/model.json
README.md
stream/.env.example
stream/.gitignore
stream/app.py
stream/captured_images/image.md
stream/database.py
stream/docker-compose.yml
stream/Dockerfile
stream/nutrition_data.db
stream/README.md
stream/requirements.txt
stream/static/manifest.json
stream/static/serviceWorker.js
stream/static/sw.js
stream/templates/base.html
stream/templates/index.html
stream/templates/login.html
stream/templates/logs.html
stream/templates/profile.html
stream/templates/register.html
stream/utils.py
stream/yolo11n_food_2.onnx
stream/yolo11n_food_2.pt
stream/yolov8n.pt
trained-model/model.json
trained-model/yolov8_quant.tflite
trained-model/yolov8.tflite
vlm/.env.example
vlm/.gitignore
vlm/photo.py
```
### Dependencies
- backend/requirements.txt: blinker@==1.9.0, click@==8.1.8, Flask@==3.1.0, Flask-Cors@==5.0.0, itsdangerous@==2.2.0, Jinja2@==3.1.5, MarkupSafe@==3.0.2, Werkzeug@==3.1.3
- client-side/package.json: @mediapipe/tasks-vision@^0.10.21, @radix-ui/react-dialog@^1.1.6, @radix-ui/react-dropdown-menu@^2.1.6, @radix-ui/react-hover-card@^1.1.6, @radix-ui/react-icons@^1.3.0, @radix-ui/react-popover@^1.1.6, @radix-ui/react-select@^2.1.6, @radix-ui/react-separator@^1.1.2, @radix-ui/react-slider@^1.2.3, @radix-ui/react-switch@^1.1.1, @tensorflow-models/coco-ssd@^2.2.3, @tensorflow/tfjs@^4.22.0, @types/node@^20, @types/react@^18, @types/react-dom@^18, @types/three@^0.173.0, axios@^1.7.9, class-variance-authority@^0.7.1, clsx@^2.1.1, eslint@^8, eslint-config-next@14.2.14, lucide-react@^0.447.0, next@14.2.14, next-pwa@^5.6.0, next-themes@^0.4.4, openai@^4.85.1, postcss@^8, react@^18, react-dom@^18, react-loader-spinner@^6.1.6, react-webcam@^7.2.0, sharp@^0.33.5, sonner@^1.7.4, tailwind-merge@^2.6.0, tailwindcss@^3.4.1, tailwindcss-animate@^1.0.7, three@^0.161.0, typescript@^5
- stream/requirements.txt: Flask@==3.0.2, Flask-Bcrypt@==1.0.1, Flask-Login@==0.6.3, Flask-SQLAlchemy@==3.1.1, Flask-WTF@==1.2.1, openai@==1.12.0, opencv-python@==4.9.0.80, psycopg2-binary@==2.9.9, python-dotenv@==1.0.1, torch@==2.2.1, torchvision@==0.17.1, ultralytics@==8.1.28, Werkzeug@==3.0.1
### Recent commits (newest first)
- Update README.md
- Update README.md
- Merge pull request #5 from enkhbold470/yolo
- feat: integrate YOLO model for food detection in video stream
- feat: enhance loading experience with modal and progress steps for profile navigation
- Merge pull request #4 from enkhbold470:sqlalchemy
- feat: update profile page and enhance feedback retrieval with improved data formatting and loading states
- feat: add PostgreSQL support with Docker configuration and environment setup
- feat: enhance profile page with skeleton loading animations and activity tracker improvements
- Merge pull request #3 from enkhbold470/sqlalchemy
- feat: implement personalized feedback feature and improve database connection management
- feat: update nutrition data database
- asdf
- Merge pull request #2 from enkhbold470/major
- feat: update .gitignore to exclude additional temporary files
- feat: enhance user-specific data retrieval and improve manifest icons
- feat: update .gitignore to exclude database files and Python cache directories
- feat: add service worker and manifest for offline support and caching
- Merge pull request #1 from enkhbold470/major
- feat: add login and registration templates with form validation
## Key source files (fetched from GitHub, selected and truncated for size)
### stream/captured_images/image.md
```markdown
#md
```
### backend/requirements.txt
```
blinker==1.9.0
click==8.1.8
Flask==3.1.0
Flask-Cors==5.0.0
itsdangerous==2.2.0
Jinja2==3.1.5
MarkupSafe==3.0.2
Werkzeug==3.1.3
```
### stream/requirements.txt
```
psycopg2-binary==2.9.9
openai==1.12.0
python-dotenv==1.0.1
Werkzeug==3.0.1
Flask==3.0.2
Flask-Login==0.6.3
Flask-WTF==1.2.1
Flask-SQLAlchemy==3.1.1
Flask-Bcrypt==1.0.1
opencv-python==4.9.0.80
ultralytics==8.1.28
torch==2.2.1
torchvision==0.17.1
```
### stream/Dockerfile
```
FROM python:3.11-slim
# Install system dependencies
RUN apt-get update && apt-get install -y \
libgl1-mesa-glx \
libglib2.0-0 \
&& rm -rf /var/lib/apt/lists/*
# Set working directory
WORKDIR /app
# Copy requirements first to leverage Docker cache
COPY requirements.txt .
# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Copy the rest of the application
COPY . .
# Create directory for captured images
RUN mkdir -p captured_images
# Expose port
EXPOSE 5001
# Command to run the application
CMD ["python", "app.py"]
```
### stream/docker-compose.yml
```yaml
version: "3.8"
services:
web:
build: .
ports:
- "5001:5001"
environment:
- IS_DOCKER=true
- DB_HOST=postgres
- DB_PORT=5432
- DB_NAME=postgres
- DB_USER=postgres
- DB_PASSWORD=asdf
- OPENAI_API_KEY=${OPENAI_API_KEY}
- PERPLEXITY_API_KEY=${PERPLEXITY_API_KEY}
- SECRET_KEY=${SECRET_KEY}
volumes:
- .:/app
- ./captured_images:/app/captured_images
depends_on:
- postgres
postgres:
image: postgres:16
environment:
- POSTGRES_PASSWORD=asdf
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- "5432:5432"
volumes:
postgres_data:
```
### client-side/package.json
```
{
"name": "nurtiVision",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint"
},
"dependencies": {
"@mediapipe/tasks-vision": "^0.10.21",
"@radix-ui/react-dialog": "^1.1.6",
"@radix-ui/react-dropdown-menu": "^2.1.6",
"@radix-ui/react-hover-card": "^1.1.6",
"@radix-ui/react-icons": "^1.3.0",
"@radix-ui/react-popover": "^1.1.6",
"@radix-ui/react-select": "^2.1.6",
"@radix-ui/react-separator": "^1.1.2",
"@radix-ui/react-slider": "^1.2.3",
"@radix-ui/react-switch": "^1.1.1",
"@tensorflow-models/coco-ssd": "^2.2.3",
"@tensorflow/tfjs": "^4.22.0",
"axios": "^1.7.9",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"lucide-react": "^0.447.0",
"next": "14.2.14",
"next-pwa": "^5.6.0",
"next-themes": "^0.4.4",
"openai": "^4.85.1",
"react": "^18",
"react-dom": "^18",
"react-loader-spinner": "^6.1.6",
"react-webcam": "^7.2.0",
"sharp": "^0.33.5",
"sonner": "^1.7.4",
"tailwind-merge": "^2.6.0",
"tailwindcss-animate": "^1.0.7",
"three": "^0.161.0"
},
"devDependencies": {
"@types/node": "^20",
"@types/react": "^18",
"@types/react-dom": "^18",
"@types/three": "^0.173.0",
"eslint": "^8",
"eslint-config-next": "14.2.14",
"postcss": "^8",
"tailwindcss": "^3.4.1",
"typescript": "^5"
}
}
```
### backend/app.py
```python
from backend import db
from backend.db import User
from flask import Flask, request, jsonify
from flask_cors import CORS
import os
from datetime import datetime
app = Flask(__name__)
CORS(app)
# change
def calculate_caloric_deficit(goal_weight_loss, target_date):
"""Calculate required daily caloric deficit based on goal weight loss and timeline."""
today = datetime.today()
target_date = datetime.strptime(target_date, "%Y-%m-%d") # Convert string to date
days_to_target = (target_date - today).days
if days_to_target <= 0:
return None, "Target date must be in the future."
# Convert lbs to kg
weight_loss_kg = goal_weight_loss * 0.453592
calorie_deficit_needed = weight_loss_kg * 7700 # 1 kg ≈ 7700 kcal
daily_deficit = calorie_deficit_needed / days_to_target
return round(daily_deficit, 2), None
@app.route("/" , methods=['GET'])
def home():
return "Hello World"
@app.route('/api/analyze-food', methods=['POST'])
def analyze_food():
try:
image = request.files.get('image')
if not image:
return jsonify({'error': 'No image provided'}), 400
# TODO: Implement AI food recognition and nutrition analysis
# Placeholder response
nutrition_data = {
'food_name': 'Sample Food',
'calories': 250,
'protein': 10,
'carbs': 30,
'fat': 8
}
return jsonify(nutrition_data)
except Exception as e:
return jsonify({'error': str(e)}), 500
@app.route('/api/track-meal', methods=['POST'])
def track_meal():
try:
data = request.get_json()
# TODO: Implement meal tracking logic with database
return jsonify({'message': 'Meal tracked successfully'})
except Exception as e:
return jsonify({'error': str(e)}), 500
@app.route('/api/get-recommendations', methods=['GET'])
def get_recommendations():
try:
user_id = request.args.get('user_id')
# TODO: Implement personalized recommendations
recommendations = {
'daily_calories': 2000,
'suggestions': [
'Increase protein intake',
'Add more vegetables to your diet',
'Stay hydrated'
]
}
return jsonify(recommendations)
except Exception as e:
return jsonify({'error': str(e)}), 500
@app.route('/api/calculate-if-healthy', methods=['GET'])
def isGoalHealthy():
try:
goal_weight_loss = float(request.args.get('goal')) # Weight loss in lbs
target_date = request.args.get('targetDate') # Target date (YYYY-MM-DD)
# Calculate daily caloric deficit needed
daily_deficit, error = calculate_caloric_deficit(goal_weight_loss, target_date)
if error:
return jsonify({'error': error}), 400
# Convert daily deficit to weekly weight loss
weekly_loss = (daily_deficit * 7) / 3500 # Convert kcal to lbs
is_healthy = 0.5 <= weekly_loss <= 2 # Healthy range: 0.5–2 lbs per week
if is_healthy is True:
return jsonify({
'goal_weight_loss': goal_weight_loss,
'weekly_weight_loss': round(weekly_loss, 2),
'is_healthy_goal': 'This is a healthy goal.'
})
else:
return jsonify({
'goal_weight_loss': goal_weight_loss,
'weekly_weight_loss': round(weekly_loss, 2),
'is_healthy_goal': 'This is not a healthy goal.'
})
except Exception as e:
return jsonify({'error': str(e)}), 500
#fitness calculator
@app.route('/api/track-progress', methods=['GET'])
def track_progress():
if User.calories <= User.max_daily_calories:
status = None
else:
status = "Not reaching goal."
return jsonify({
'status': status
})
#display nutrients
@app.route('/api/display-nutrients', methods=['GET'])
def return_nutrients():
return jsonify({
'calories': User.calories,
'potassium': User.potassium,
'protein': User.protein,
'carbs': User.carbs,
'totalfat': User.totalfat
})
#display calories
@app.route('/api/calories', methods=['GET'])
def return_nutrients():
return jsonify({
'calories': User.calories,
})
#you've exceeded the intake of fat/cholestoral for today.
#take the food data from the open ai. increment values for calories, fat, and carbs. if they reach a max value, then return a message saying that they've exceeded the intake of that nutrient, to the frontend.
# Define daily intake limits (can be adjusted per dietary guidelines)
#a method that defines user's daily limits based on their goal weight, and their goal date:
def daily_limits(user):
"""
Calculate daily calorie and protein intake limits based on user's goal weight and target date.
Returns:
- daily_calories (int): The daily calorie limit needed to achieve the goal.
- daily_protein (int): The recommended daily protein intake in grams.
"""
try:
# Convert targetDate from datetime object
today = datetime.today()
days_remaining = (user.targetDate - today).days
if days_remaining <= 0:
return None, None # Target date must be in the future
# Convert goal weight loss to kg
weight_loss_kg = user.goal * 0.453592 # 1 lb = 0.453592 kg
# Calculate required daily caloric deficit
total_calorie_deficit = weight_loss_kg * 7700 # 1 kg ≈ 7700 kcal
daily_caloric_deficit = total_calorie_deficit / days_remaining
# Calculate BMR (Mifflin-St Jeor Equation)
height_cm = user.height * 2.54 # Convert inches to cm
weight_kg = user.weight * 0.453592 # Convert lbs to kg
if user.gender.lower() == "male":
bmr = 10 * weight_kg + 6.25 * height
[truncated — 3290 more characters]
```
### client-side/app/page.tsx
```typescript
import Landing from "@/components/landing";
import Dark from "@/components/DarkMode";
export default function Home() {
return(
<>
{/* */}
);
}
```
### client-side/app/layout.tsx
```typescript
import { ThemeProvider } from "@/components/theme-provider";
import { Toaster } from "@/components/ui/sonner";
import CameraDevicesProvider from "@/providers/CameraDevicesProvider";
import type { Metadata, Viewport } from "next";
import { Roboto } from "next/font/google";
// import localFont from "next/font/local";
import "./globals.css";
import BottomNavbar from "@/components/navbar/bottom-navbar";
// import PlantNotification from "@/components/plant-notification";
// import Dark from "@/components/DarkMode";
const roboto = Roboto({
subsets: ["latin"],
weight: ["400", "900"],
});
export const viewport: Viewport = {
themeColor: "#8FCE00",
};
export const metadata: Metadata = {
title: "nurtiVision",
description:
"nurtiVision is nutrition tracking app that helps you to track your daily nutrition intake and helps you to maintain a healthy lifestyle.",
};
export default function RootLayout({
children,
}: Readonly<{
children: React.ReactNode;
}>) {
return (
{/* */}
{/*
*/}
{children}
);
}
```
### stream/app.py
```python
import cv2
from flask import Flask, Response, request, send_from_directory, render_template, redirect, url_for, flash, jsonify
import os
import uuid
import base64
from flask_login import current_user, LoginManager, login_required, login_user, logout_user
from openai import OpenAI
import json
from dotenv import load_dotenv
import threading
from threading import Lock
from datetime import datetime, timedelta
from database import init_db, add_scan_record, get_scan_records, search_records, get_db_connection, get_db, close_db
from werkzeug.security import generate_password_hash, check_password_hash
from utils import calculate_bmr, calculate_daily_calories, calculate_daily_protein
from ultralytics import YOLO
import numpy as np
# Load environment variables
load_dotenv()
print("[DEBUG] Environment variables loaded")
# Initialize Flask app
app = Flask(__name__, static_url_path='/static')
app.secret_key = os.getenv("SECRET_KEY", "your-secret-key")
print("[DEBUG] Flask app initialized")
# Register database close function
app.teardown_appcontext(close_db)
# Add min function to Jinja2 environment
app.jinja_env.globals.update(min=min)
print("[DEBUG] Added min function to Jinja2 environment")
# Add helper functions to Jinja2 environment
def get_user_achievements(user_id):
"""Get user achievements with unlock status."""
conn = get_db_connection()
cur = conn.cursor()
# Get user's scan count and other stats
cur.execute('''
SELECT COUNT(*) as scan_count,
COUNT(DISTINCT DATE(timestamp)) as active_days,
COUNT(DISTINCT food_name) as unique_foods
FROM scanned_items
WHERE user_id = %s
''', (user_id,))
stats = cur.fetchone()
conn.close()
# Define achievements
achievements = [
{
'title': 'First Scan',
'description': 'Scan your first food item',
'icon': 'bi-camera',
'unlocked': stats['scan_count'] > 0
},
{
'title': 'Health Explorer',
'description': 'Scan 10 different food items',
'icon': 'bi-search',
'unlocked': stats['unique_foods'] >= 10
},
{
'title': 'Consistency King',
'description': 'Track food for 7 consecutive days',
'icon': 'bi-calendar-check',
'unlocked': stats['active_days'] >= 7
},
{
'title': 'Nutrition Master',
'description': 'Complete 100 food scans',
'icon': 'bi-trophy',
'unlocked': stats['scan_count'] >= 100
}
]
return achievements
def get_activity_data(user_id):
"""Get user's activity data for the last year."""
conn = get_db_connection()
cur = conn.cursor()
# Get daily scan counts for the last year
cur.execute('''
SELECT DATE(timestamp) as date, COUNT(*) as count
FROM scanned_items
WHERE user_id = %s
AND timestamp >= CURRENT_DATE - INTERVAL '1 year'
GROUP BY DATE(timestamp)
ORDER BY date
''', (user_id,))
activity = cur.fetchall()
conn.close()
# Convert to week-based format
weeks = []
current_week = []
# Fill in missing dates with zero counts
date_counts = {row['date']: row['count'] for row in activity}
from datetime import datetime, timedelta
end_date = datetime.now()
start_date = end_date - timedelta(days=365)
current_date = start_date
while current_date <= end_date:
date_str = current_date.strftime('%Y-%m-%d')
current_week.append({
'date': date_str,
'count': date_counts.get(date_str, 0)
})
if len(current_week) == 7:
weeks.append(current_week)
current_week = []
current_date += timedelta(days=1)
if current_week:
weeks.append(current_week)
return weeks
def get_activity_color(count):
"""Get color for activity based on count."""
if count == 0:
return '#ebedf0'
elif count <= 2:
return '#9be9a8'
elif count <= 4:
return '#40c463'
elif count <= 6:
return '#30a14e'
else:
return '#216e39'
# Add functions to Jinja environment
app.jinja_env.globals.update(
get_user_achievements=get_user_achievements,
get_activity_data=get_activity_data,
get_activity_color=get_activity_color
)
# Initialize Flask-Login
login_manager = LoginManager()
login_manager.init_app(app)
login_manager.login_view = 'login'
# Initialize database
init_db()
print("[DEBUG] Database initialized")
# Initialize OpenAI clients
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
perplexity_client = OpenAI(
api_key=os.getenv("PERPLEXITY_API_KEY"),
base_url="https://api.perplexity.ai"
)
print("[DEBUG] API clients initialized")
# Global variables for video streaming
class VideoCamera:
def __init__(self):
print("[DEBUG] Initializing VideoCamera")
self.camera = cv2.VideoCapture(0)
self.lock = Lock()
self.frame = None
self.detections = None
self.stopped = False
# Initialize YOLO model
print("[DEBUG] Loading YOLO model")
try:
self.model = YOLO('yolo11n_food_2.pt') # Load the model
print("[DEBUG] YOLO model loaded successfully")
except Exception as e:
print(f"[DEBUG] Error loading YOLO model: {str(e)}")
self.model = None
# Start frame capture thread
self.thread = threading.Thread(target=self._capture_loop)
self.thread.daemon = True
self.thread.start()
print("[DEBUG] Frame capture thread started")
def _capture_loop(self):
print("[DEBUG] Starting capture loop")
while not self.stopped:
success, frame = self.camera.read()
if success:
# Run YOLO detection
[truncated — 24921 more characters]
```
[83 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]