# Project export: Sproutify - Table 244 This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 12.0 - Tagline: AR cat companion gamifies eating disorder recovery. Earn points for healthy behaviors, unlock toys for your pet. Fetch.ai agents track patterns and generate automated doctor reports. - Devpost: https://devpost.com/software/sproutify-7r4o1u - GitHub: https://github.com/irenkani/calhacks12.0 - Video: https://www.youtube.com/embed/-BbFmhjR_9I?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — Irenka Ni (8 commits), EvelynFung (4 commits) ## Devpost submission (written by the team) ### Inspiration 28 million Americans struggle with eating disorders, where every meal becomes a source of anxiety and fear. We discovered a powerful insight from clinical research: people with eating disorders often care for others before themselves. This sparked an idea—what if we could redirect that nurturing instinct to support their own recovery? We wanted to transform one of the most challenging aspects of recovery—eating meals—from a moment of stress into something rewarding and positive. By combining the emotional power of pet companionship with cutting-edge AR technology and AI agents, we created Sproutify to make recovery feel less like a battle and more like raising a friend. ### What it does Sproutify is a gamified AR companion system that transforms eating disorder recovery into a rewarding experience. For Patients: AR Cat Companion "Sprout" appears through Snap Spectacles during meals Real-time AI Detection: Uses depth sensing and Google Gemini to identify food items, track macros, and measure consumption Gamification System: Earn points for healthy eating behaviors: Eating speed and consistency, Meal completion rates, Nutritional balance (macros), Total time taken to eat Rewards & Customization: Spend points on toys and cosmetics for Sprout Intelligent Support: Sprout provides adaptive encouragement when hesitation is detected Predictive AI: Detects concerning patterns and provides gentle interventions before relapse For Doctors: Automated Progress Reports generated by Fetch.ai agents Long-term Behavior Analysis: Track eating patterns, nutritional progress, and consistency Early Warning System: AI flags concerning behaviors for clinical intervention Data-Driven Decisions: Objective metrics to inform treatment adjustments The Result: A complete support ecosystem that makes recovery feel rewarding instead of restrictive. ### How we built it We used Spectacles and Lens Studio as our main platform for AR interactivity with the outside world. Our algorithm accesses our custom Fetch.ai endpoint to access the long-term data of the user and returns a JSON object to our Lens client. We also use local object recognition algorithms to identity objects locally. ### Challenges we ran into One challenge is real-time Object Detection: Achieving accurate food recognition and portion estimation through Spectacles’ limited onboard compute required optimization and hybrid inference (local + cloud via Google Gemini). Another challenge is balancing gamification with clinical sensitivity was difficult—we had to ensure the app motivates recovery without triggering anxiety or unhealthy behaviors. ### Accomplishments we're proud of Functional AR Prototype: We successfully created an interactive AR companion that detects food items in real-time, reacts emotionally, and evolves with the user’s progress. We also built a full End-to-End AI Pipeline: from food detection to behavior analysis, our Fetch.ai agents and Gemini-powered models enable seamless monitoring and insights for clinicians. We are also proud of our clinically inspired design: We conducted psychological research on recovery motivation, turning empathy and care into measurable progress through gamified behavior reinforcement. ### What we learned Technical Skills AR Development: How to build immersive experiences for Snap Spectacles Depth Sensing: Understanding and leveraging 3D computer vision Multi-agent Systems: Orchestrating specialized AI agents with Fetch.ai Multimodal AI: Using Gemini's vision capabilities for food analysis Real-time Systems: Balancing latency, accuracy, and user experience Design Insights Gamification Psychology: How to motivate behavior change without triggering Healthcare UX: Building interfaces that work for both patients and clinicians AR Interaction: Creating intuitive controls in 3D space ### What's next for Sproutify Sproutify hopes to continue the transition o making good from a negative to a positive experience for all patients struggling with anorexia. We hope to fully integrate nutritional data in the form of subtle suggestions through Sprout into Sprout’s verbal and visual cues. We also hope to export our nutritional analysis in a format suitable to be used in medical reports to track and predict past and future recovery trends to assist medical professionals. ## README (from the GitHub repository) # Patient Nutrition Analysis System A comprehensive system for analyzing patient eating patterns and nutritional intake using Fetch.ai agents, Supabase storage, and Gemini Vision API. ## 🏥 System Overview This system provides healthcare professionals with detailed analysis of patient meal patterns, including: - **Eating Pattern Analysis**: Regular vs irregular eating schedules - **Nutritional Assessment**: Calorie intake, food categories, portion analysis - **Professional Recommendations**: Evidence-based suggestions for nutritionists/doctors - **Historical Data**: Analysis of all stored meal images ## 🏗️ Architecture ``` Spectacles AR → Storage Agent → Analysis Agent → Frontend Web App ↓ ↓ ↓ ↓ Image Capture Supabase Upload Gemini Analysis Doctor Dashboard ``` ### Components: 1. **Storage Agent** (`storage_agent.py`) - Handles image uploads to Supabase 2. **Analysis Agent** (`nutrition_analysis_agent.py`) - Performs comprehensive analysis 3. **Frontend Web App** (`nutrition_frontend.py`) - Professional dashboard 4. **Batch Upload** (`upload_assets_folder.py`) - Uploads multiple images ## 🚀 Quick Start ### 1. Prerequisites - Python 3.11+ - Supabase account and project - Gemini API key - Fetch.ai Agentverse account ### 2. Environment Setup Create `.env` file: ```bash SUPABASE_URL=https://your-project.supabase.co SUPABASE_KEY=your-anon-public-key GEMINI_API_KEY=your-gemini-api-key ``` ### 3. Install Dependencies ```bash cd backend pip install -r requirements.txt ``` ### 4. Supabase Setup Run this SQL in Supabase SQL Editor: ```sql -- Create meal_images table CREATE TABLE meal_images ( id SERIAL PRIMARY KEY, session_id TEXT NOT NULL, frame_id TEXT NOT NULL, file_path TEXT NOT NULL, url TEXT NOT NULL, uploaded_at BIGINT NOT NULL, created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW() ); -- Disable RLS for testing ALTER TABLE meal_images DISABLE ROW LEVEL SECURITY; -- Create storage bucket -- Go to Storage → Create bucket: "meals" (set to public) ``` ### 5. Run the System ```bash python setup_and_run.py ``` ## 📋 Usage Instructions ### Option 1: Complete System ```bash python setup_and_run.py # Choose option 4 to run everything ``` ### Option 2: Individual Components #### Upload Assets Folder ```bash python upload_assets_folder.py ``` - Uploads all images from `assets/` folder - 30-second intervals between uploads - Creates session tracking #### Start Analysis Agent ```bash python nutrition_analysis_agent.py ``` - Runs on port 8003 - Provides REST API for analysis - Integrates with Gemini Vision API #### Start Frontend Web App ```bash python nutrition_frontend.py ``` - Runs on port 5000 - Professional dashboard for healthcare workers - Access at: http://127.0.0.1:5000 ## 🔍 Analysis Features ### Eating Pattern Analysis - **Meal Frequency**: Number of meals per day - **Timing Patterns**: Regular vs irregular eating schedules - **Consumption Patterns**: Portion sizes and eating pace - **Interval Analysis**: Time between meals ### Nutritional Assessment - **Food Identification**: AI-powered food recognition - **Calorie Estimation**: Estimated caloric intake per meal - **Food Categories**: Protein, carbs, vegetables, etc. - **Portion Analysis**: Consumption percentage per meal ### Professional Recommendations - **Eating Schedule**: Suggestions for regular meal timing - **Portion Control**: Recommendations for appropriate serving sizes - **Nutritional Balance**: Advice on food variety and nutrients - **Behavioral Patterns**: Insights into eating habits ## 🌐 Frontend Dashboard ### Features: - **Patient Analysis**: Enter patient ID and date range - **Real-time Results**: Live analysis of stored images - **Professional Interface**: Designed for healthcare workers - **Comprehensive Reports**: Detailed eating pattern analysis ### Usage: 1. Open http://127.0.0.1:5000 2. Enter patient ID 3. Select date range (optional) 4. Choose analysis type 5. View comprehensive results ## 📊 API Endpoints ### Analysis Agent (Port 8003) - `POST /analyze` - Analyze patient data - `GET /health` - Health check ### Frontend (Port 5000) - `GET /` - Main dashboard - `POST /analyze_patient` - Patient analysis request - `GET /health` - System health check ## 🔧 Configuration ### Agent Configuration - **Storage Agent**: Port 8001 - **Analysis Agent**: Port 8003 - **Frontend**: Port 5000 ### Supabase Configuration - **Bucket**: "meals" (public access) - **Table**: "meal_images" - **RLS**: Disabled for testing ### Gemini Configuration - **Model**: gemini-2.0-flash-exp - **Analysis**: Food recognition and nutritional assessment ## 🧪 Testing ### Test Individual Components ```bash # Test Supabase upload python test_supabase_upload.py # Test custom image analysis python upload_custom_image.py # Query uploaded data python query_uploads.py ``` ### Test Complete Flow 1. Upload images using batch script 2. Start analysis agent 3. Start frontend 4. Access dashboard and run analysis ## 📁 File Structure ``` backend/ ├── storage_agent.py # Storage Agent ├── nutrition_analysis_agent.py # Analysis Agent ├── nutrition_frontend.py # Flask Frontend ├── upload_assets_folder.py # Batch Upload ├── test.py # Core Analysis Logic ├── templates/ │ └── nutrition_dashboard.html # Web Interface ├── assets/ # Image Storage ├── requirements.txt # Dependencies ├── setup_and_run.py # Setup Script └── .env # Environment Variables ``` ## 🚨 Troubleshooting ### Common Issues: 1. **Supabase Upload Fails** - Check bucket exists and is public - Verify SUPABASE_URL and SUPABASE_KEY - Disable RLS: `ALTER TABLE meal_images DISABLE ROW LEVEL SECURITY;` 2. **Gemini Analysis Fails** - Verify GEMINI_API_KEY in .env - Check API quota and limits - Ensure images are accessible via URL 3. **Agent Communication Issues** - Check agent addresses and ports - Verify all agents are running - Check network connectivity 4. **Frontend Not Loading** - Ensure Flask is installed - Check port 5000 is available - Verify analysis agent is running ## 🎯 Hackathon Requirements Met ✅ **Fetch.ai Agentverse Integration**: Agents registered and deployed ✅ **Chat Protocol**: All agents use Fetch.ai messaging ✅ **LLM Integration**: Gemini Vision API for food analysis ✅ **Real-world Actions**: File uploads, database operations, API calls ✅ **Professional Interface**: Web dashboard for healthcare workers ✅ **Multi-agent Architecture**: Specialized agents for different tasks ✅ **External API Integration**: Supabase, Gemini, Flask ## 📞 Support For issues or questions: 1. Check troubleshooting section 2. Verify all dependencies are installed 3. Ensure .env file is properly configured 4. Check Supabase and Gemini API access ## 🏆 Features Delivered - **Complete Fetch.ai Integration**: Pure chat protocols, no REST endpoints - **Professional Healthcare Interface**: Designed for nutritionists/doctors - **Comprehensive Analysis**: Eating patterns, nutritional assessment, recommendations - **Scalable Architecture**: Microservices with specialized agents - **Real-world Application**: Solving eating disorder support challenges ## Detected evidence (automated analysis) Indexed codebase: 38 recognized source files, 166 KB. - Flask (technology) — detected in the code - HTML (language) — detected in the code - Python (language) — detected in the code - Supabase (technology) — detected in the code - TypeScript (language) — detected in the code ## Codebase structure (from repository index) ### Files (120 of 130) ``` .DS_Store .gitignore backend/=0.12.0 backend/agent_addresses.txt backend/agent_config.py backend/AGENTVERSE_DEPLOYMENT_GUIDE.md backend/analysis_agent.py backend/check_agentverse_status.py backend/debug.py backend/deploy_to_agentverse.py backend/eating_disorder_chat_agent.py backend/FETCHAI_SETUP.md backend/nutrition_analysis_agent.py backend/nutrition_frontend.py backend/private_keys.json backend/proper_agentverse_deploy.py backend/query_uploads.py backend/requirements.txt backend/run_agents_for_agentverse.py backend/setup_and_run.py backend/storage_agent.py backend/templates/nutrition_dashboard.html backend/test_chat_protocol_components.py backend/test_chat_protocol.py backend/test.py backend/upload_assets_folder.py backend/upload_custom_image.py backend/upload_results_1761473408.json backend/upload_results_1761474474.json Depth Cache/.gitattributes Depth Cache/.gitignore Depth Cache/Assets/Images/loading.gif.meta Depth Cache/Assets/Images/MicIcon.png.meta Depth Cache/Assets/Materials/CaptureTexture.mat Depth Cache/Assets/Materials/CaptureTexture.mat.meta Depth Cache/Assets/Materials/CircleOutline.mat Depth Cache/Assets/Materials/CircleOutline.mat.meta Depth Cache/Assets/Materials/circleOutline.ss_graph Depth Cache/Assets/Materials/circleOutline.ss_graph.meta Depth Cache/Assets/Materials/label.ss_graph Depth Cache/Assets/Materials/label.ss_graph.meta Depth Cache/Assets/Materials/loadingGif.mat Depth Cache/Assets/Materials/loadingGif.mat.meta Depth Cache/Assets/Materials/loadingGif.ss_graph Depth Cache/Assets/Materials/loadingGif.ss_graph.meta Depth Cache/Assets/Materials/micIcon.mat Depth Cache/Assets/Materials/micIcon.mat.meta Depth Cache/Assets/Materials/MyAi_Voice.mat Depth Cache/Assets/Materials/MyAi_Voice.mat.meta Depth Cache/Assets/Materials/MyAi_Voice.ss_graph Depth Cache/Assets/Materials/MyAi_Voice.ss_graph.meta Depth Cache/Assets/Materials/PinchButton.mat Depth Cache/Assets/Materials/PinchButton.mat.meta Depth Cache/Assets/Materials/pointRedLabeled.mat Depth Cache/Assets/Materials/pointRedLabeled.mat.meta Depth Cache/Assets/Materials/SWIS_flat_color.ss_graph Depth Cache/Assets/Materials/SWIS_flat_color.ss_graph.meta Depth Cache/Assets/Materials/SWIS_tile.mat Depth Cache/Assets/Materials/SWIS_tile.mat.meta Depth Cache/Assets/Materials/unlitColor.ss_graph Depth Cache/Assets/Materials/unlitColor.ss_graph.meta Depth Cache/Assets/Materials/unlitTexture.ss_graph Depth Cache/Assets/Materials/unlitTexture.ss_graph.meta Depth Cache/Assets/Materials/unlitTextureColor.ss_graph Depth Cache/Assets/Materials/unlitTextureColor.ss_graph.meta Depth Cache/Assets/Materials/White.mat Depth Cache/Assets/Materials/White.mat.meta Depth Cache/Assets/Materials/WorldLabelFrame.mat Depth Cache/Assets/Materials/WorldLabelFrame.mat.meta Depth Cache/Assets/Mesh/arrow.mesh Depth Cache/Assets/Mesh/arrow.mesh.meta Depth Cache/Assets/Mesh/SWIS_tile_solid.mesh Depth Cache/Assets/Mesh/SWIS_tile_solid.mesh.meta Depth Cache/Assets/Mesh/Unit Box.mesh Depth Cache/Assets/Mesh/Unit Box.mesh.meta Depth Cache/Assets/Mesh/Unit Plane.mesh Depth Cache/Assets/Mesh/Unit Plane.mesh.meta Depth Cache/Assets/Mesh/Unit Sphere.mesh Depth Cache/Assets/Mesh/Unit Sphere.mesh.meta Depth Cache/Assets/Modules/Camera Module.cameraModule Depth Cache/Assets/Modules/Camera Module.cameraModule.meta Depth Cache/Assets/Prefabs/coloredPoint.prefab Depth Cache/Assets/Prefabs/coloredPoint.prefab.meta Depth Cache/Assets/Prefabs/WorldArrow.prefab Depth Cache/Assets/Prefabs/WorldArrow.prefab.meta Depth Cache/Assets/Prefabs/WorldLabel.prefab Depth Cache/Assets/Prefabs/WorldLabel.prefab.meta Depth Cache/Assets/Rendering/Device Camera Texture.deviceCameraTexture Depth Cache/Assets/Rendering/Device Camera Texture.deviceCameraTexture.meta Depth Cache/Assets/Rendering/Echopark.hdr Depth Cache/Assets/Rendering/Echopark.hdr.meta Depth Cache/Assets/Rendering/Render Target.renderTarget Depth Cache/Assets/Rendering/Render Target.renderTarget.meta Depth Cache/Assets/Scene.scene Depth Cache/Assets/Scene.scene.meta Depth Cache/Assets/Scripts/APIKeyHint.ts Depth Cache/Assets/Scripts/APIKeyHint.ts.meta Depth Cache/Assets/Scripts/ASRController.ts Depth Cache/Assets/Scripts/ASRController.ts.meta Depth Cache/Assets/Scripts/DebugVisualizer.ts Depth Cache/Assets/Scripts/DebugVisualizer.ts.meta Depth Cache/Assets/Scripts/DepthCache.ts Depth Cache/Assets/Scripts/DepthCache.ts.meta Depth Cache/Assets/Scripts/FaceCamera.ts Depth Cache/Assets/Scripts/FaceCamera.ts.meta Depth Cache/Assets/Scripts/FetchAIAgent.ts Depth Cache/Assets/Scripts/GeminiAPI.ts Depth Cache/Assets/Scripts/GeminiAPI.ts.meta Depth Cache/Assets/Scripts/Labels.ts Depth Cache/Assets/Scripts/Labels.ts.meta Depth Cache/Assets/Scripts/Loading.ts Depth Cache/Assets/Scripts/Loading.ts.meta Depth Cache/Assets/Scripts/ResponseUI.ts Depth Cache/Assets/Scripts/ResponseUI.ts.meta Depth Cache/Assets/Scripts/SceneController.ts Depth Cache/Assets/Scripts/SceneController.ts.meta Depth Cache/Assets/Scripts/spectacles_controller.ts Depth Cache/Assets/Scripts/SpeechUI.ts Depth Cache/Assets/Scripts/SpeechUI.ts.meta Depth Cache/Assets/Scripts/TouchDownButton.ts [10 more files omitted for size] ``` ### Dependencies - backend/requirements.txt: Flask, google-generativeai, Pillow, pydantic, python-dotenv, requests, supabase@>=2.0.0, uagents@>=0.12.0 ### Recent commits (newest first) - all updates past sunday morning - need to to delete many things - switching branches - .env - put key - moved .ts - Create spectacles_controller.ts - delete lens - Create lens_studio_test.ts - deleted - Create lens_studio_test.py - Update test.py - init ## Key source files (fetched from GitHub, selected and truncated for size) ### backend/AGENTVERSE_DEPLOYMENT_GUIDE.md ```markdown # Agentverse Deployment Guide ## 🚀 Deploy Eating Disorder Support Agent to Agentverse Based on the [Fetch.ai Innovation Lab documentation](https://innovationlab.fetch.ai/resources/docs/examples/chat-protocol/image-analysis-agent), here's how to properly deploy our agent: ## 📋 Prerequisites 1. **Agentverse Account**: Sign up at [agentverse.ai](https://agentverse.ai) 2. **Environment Variables**: Set up your `.env` file with: ``` SUPABASE_URL=https://your-project.supabase.co SUPABASE_KEY=your-anon-public-key GEMINI_API_KEY=your-gemini-api-key ``` ## 🔧 Step-by-Step Deployment ### 1. Create New Agent on Agentverse 1. Go to [agentverse.ai](https://agentverse.ai) 2. Click **"Create Agent"** 3. Name: `Eating Disorder Support Agent` 4. Description: `AI-powered meal analysis and eating disorder support agent that analyzes meal images and provides nutritional insights with gentle, supportive feedback.` ### 2. Upload Agent Files Create these files in your Agentverse agent: #### **agent.py** (Main agent file) ```python # Copy the entire content from eating_disorder_chat_agent.py ``` #### **requirements.txt** ``` uagents>=0.22.0 uagents-core>=0.3.0 supabase>=2.0.0 google-generativeai Pillow requests python-dotenv ``` ### 3. Set Environment Variables In Agentverse, go to **Settings** → **Environment Variables** and add: - `SUPABASE_URL` - `SUPABASE_KEY` - `GEMINI_API_KEY` ### 4. Configure Agent Settings - **Protocol**: Chat Protocol - **Storage**: Enable Agent Storage - **Attachments**: Enable (for image uploads) ### 5. Deploy and Test 1. Click **"Deploy Agent"** 2. Wait for deployment to complete 3. Go to **Overview** → **"Chat with Agent"** 4. Test by uploading a meal image ## 🧪 Testing the Agent ### Via Agentverse Chat Interface: 1. Upload a meal image using the **Attach** button 2. Ask: "Analyze this meal and tell me about the nutritional content" 3. The agent should: - Analyze the image with Gemini Vision - Upload to Supabase storage - Provide detailed nutritional insights - Give supportive feedback ### Expected Response Format: ``` 🍽️ **Meal Analysis** **Food Items Detected:** - Grilled chicken breast - Steamed broccoli - Brown rice **Nutritional Assessment:** - Good protein source - Healthy vegetables - Balanced meal **Portion Analysis:** - Appropriate serving sizes - Good macronutrient balance 💡 *This analysis has been saved to your meal tracking system.* ``` ## 🔗 Integration with Spectacles Once deployed, update your Spectacles `FetchAIAgent.ts`: ```typescript @input storageAgentAddress: string = "YOUR_AGENTVERSE_AGENT_ADDRESS"; // Get from Agentverse ``` ## 📊 Features ✅ **Image Analysis**: Uses Gemini Vision API for food detection ✅ **Supabase Integration**: Stores meal images and metadata ✅ **Chat Protocol**: Compatible with Agentverse Chat Interface ✅ **Supportive Responses**: Gentle, helpful feedback for eating disorder support ✅ **Meal Tracking**: Automatic storage of analyz [truncated — 573 more characters] ``` ### backend/FETCHAI_SETUP.md ```markdown # Fetch.ai Agentverse Deployment Guide ## Overview This guide walks you through deploying the eating support agents to Fetch.ai Agentverse with chat protocols enabled. ## Prerequisites - Supabase account and project set up - Gemini API key - Fetch.ai Agentverse account ## Step 1: Environment Setup Create `.env` file in `backend/` directory: ```bash SUPABASE_URL=https://your-project.supabase.co SUPABASE_KEY=your-anon-public-key GEMINI_API_KEY=your-gemini-api-key ``` ## Step 1: Environment Setup Create `.env` file in `backend/` directory: ```bash SUPABASE_URL=https://your-project.supabase.co SUPABASE_KEY=your-anon-public-key GEMINI_API_KEY=your-gemini-api-key ``` ## Step 2: Supabase Setup 1. **Create Supabase project** at supabase.com 2. **Create storage bucket**: `meal-images` (set to public) 3. **Run SQL setup**: Execute `supabase_setup.sql` in Supabase SQL Editor 4. **No depth-data bucket needed** - using hardcoded depth data ## Step 3: Install Dependencies ```bash cd backend pip install -r requirements.txt ``` ## Step 4: Run Agents Locally First Get agent addresses by running both agents: ### Terminal 1 - Storage Agent ```bash cd backend python storage_agent.py ``` Copy the agent address: `Agent Address: agent1q...` ### Terminal 2 - Analysis Agent ```bash cd backend python test.py ``` Copy the agent address: `Agent Address: agent1q...` ## Step 5: Update Agent Addresses Edit `backend/storage_agent.py` and update: ```python ANALYSIS_AGENT_ADDRESS = "agent1q..." # Paste Analysis Agent address here ``` ## Step 5: Register Storage Agent on Agentverse 1. Go to [agentverse.ai](https://agentverse.ai) 2. Create account / login 3. Click "Register Agent" 4. Paste Storage Agent address 5. Name: "storage_agent" 6. Get mailbox key from dashboard 7. Add to `.env`: `STORAGE_AGENT_MAILBOX_KEY=...` ## Step 6: Register Analysis Agent on Agentverse 1. Same process as Storage Agent 2. Paste Analysis Agent address 3. Name: "eating_support_agent" 4. Get mailbox key 5. Add to `.env`: `ANALYSIS_AGENT_MAILBOX_KEY=...` ## Step 7: Enable Chat Protocol on Both Agents ### Storage Agent 1. In Agentverse, select Storage Agent 2. Go to "Protocols" tab 3. Enable "Chat Protocol" 4. Protocol name: "StorageChat" ### Analysis Agent 1. In Agentverse, select Analysis Agent 2. Go to "Protocols" tab 3. Enable "Chat Protocol" 4. Protocol name: "MealTrackingChat" ## Step 8: Test Agents via Agentverse Chat ### Test Storage Agent 1. Go to Agentverse → Agent Chat 2. Find "storage_agent" 3. Send UploadRequest message: ```json { "image_base64": "iVBORw0KGgo...", "session_id": "test_session", "frame_id": "frame_001", "user_id": "test_user" } ``` ### Test Analysis Agent 1. Go to Agentverse → Agent Chat 2. Find "eating_support_agent" 3. Send CaptureRequest message: ```json { "session_id": "test_session", "user_id": "test_user", "image_url": "https://your-project.supabase.co/storage/v1/object/public/meal-images/test_session/frame_001.jpg", "times [truncated — 1726 more characters] ``` ### backend/requirements.txt ``` uagents>=0.12.0 supabase>=2.0.0 google-generativeai Pillow requests python-dotenv pydantic Flask ``` ### backend/agent_config.py ```python # agent_config.py import os from dotenv import load_dotenv load_dotenv() STORAGE_AGENT_SEED = "storage_agent_seed_phrase" ANALYSIS_AGENT_SEED = "eating_disorder_support_seed_phrase" SUPABASE_URL = os.getenv("SUPABASE_URL") SUPABASE_KEY = os.getenv("SUPABASE_KEY") GEMINI_API_KEY = os.getenv("GEMINI_API_KEY") # Fill after first run STORAGE_AGENT_ADDRESS = "" ANALYSIS_AGENT_ADDRESS = "" ``` ### backend/query_uploads.py ```python # query_uploads.py from supabase import create_client import os from dotenv import load_dotenv load_dotenv() supabase = create_client(os.getenv("SUPABASE_URL"), os.getenv("SUPABASE_KEY")) def query_recent_uploads(limit=10): """Query recent uploads from database""" print("📊 Querying recent uploads...") # Query meal images only meal_images = supabase.table('meal_images')\ .select('*')\ .order('uploaded_at', desc=True)\ .limit(limit)\ .execute() print(f"\n🖼️ Recent Meal Images ({len(meal_images.data)}):") for img in meal_images.data: print(f" Session: {img['session_id']}") print(f" Frame: {img['frame_id']}") print(f" Uploaded: {img['uploaded_at']}") print(f" URL: {img['url']}") print(f" Created: {img['created_at']}") print() def query_by_session(session_id): """Query uploads by session ID""" print(f"🔍 Querying uploads for session: {session_id}") meal_images = supabase.table('meal_images')\ .select('*')\ .eq('session_id', session_id)\ .order('uploaded_at', desc=True)\ .execute() print(f"📸 Images: {len(meal_images.data)}") return meal_images.data if __name__ == "__main__": # Query recent uploads query_recent_uploads(5) # Query specific session (replace with actual session ID) # query_by_session("test_session_1234567890") ``` ### backend/nutrition_frontend.py ```python # nutrition_frontend.py from flask import Flask, render_template, request, jsonify import requests import json app = Flask(__name__) # Agent endpoints ANALYSIS_AGENT_URL = "http://127.0.0.1:8003" @app.route('/') def index(): """Main dashboard for nutritionists/doctors""" return render_template('nutrition_dashboard.html') @app.route('/analyze_patient', methods=['POST']) def analyze_patient(): """Analyze patient data via analysis agent""" try: # Use default patient ID for simplicity patient_id = "patient_001" # Prepare request for analysis agent payload = { "patient_id": patient_id, "date_range_start": None, "date_range_end": None, "analysis_type": "comprehensive" } # Call analysis agent response = requests.post(f"{ANALYSIS_AGENT_URL}/analyze", json=payload) response.raise_for_status() result = response.json() return jsonify({ "success": True, "analysis": result }) except requests.RequestException as e: return jsonify({"error": f"Failed to connect to analysis agent: {str(e)}"}) except Exception as e: return jsonify({"error": f"Analysis failed: {str(e)}"}) @app.route('/health') def health_check(): """Check health of analysis agent""" try: response = requests.get(f"{ANALYSIS_AGENT_URL}/health", timeout=5) if response.status_code == 200: return jsonify({"status": "healthy", "agent": "nutrition_analysis_agent"}) else: return jsonify({"status": "unhealthy", "agent": "nutrition_analysis_agent"}) except: return jsonify({"status": "offline", "agent": "nutrition_analysis_agent"}) if __name__ == '__main__': app.run(debug=True, host='0.0.0.0', port=5001) ``` ### backend/test_chat_protocol_components.py ```python #!/usr/bin/env python3 """ Test Chat Protocol Components Verify that all chat protocol imports work correctly """ def test_chat_protocol_imports(): """Test if chat protocol components can be imported""" print("🧪 Testing Chat Protocol Components") print("=" * 40) try: from uagents_core.contrib.protocols.chat import ( chat_protocol_spec, ChatMessage, ChatAcknowledgement, TextContent, ResourceContent, StartSessionContent, MetadataContent, ) print("✅ All chat protocol components imported successfully!") # Test creating a message from datetime import datetime, timezone from uuid import uuid4 test_message = ChatMessage( timestamp=datetime.now(timezone.utc), msg_id=uuid4(), content=[TextContent(type="text", text="Hello, world!")], ) print("✅ ChatMessage created successfully!") print(f" Message ID: {test_message.msg_id}") print(f" Content: {test_message.content[0].text}") return True except ImportError as e: print(f"❌ Import error: {e}") print("💡 Make sure you have uagents-core installed:") print(" pip install uagents-core>=0.3.0") return False except Exception as e: print(f"❌ Error: {e}") return False def test_agent_creation(): """Test if agent can be created with chat protocol""" print("\n🧪 Testing Agent Creation") print("=" * 30) try: from uagents import Agent, Protocol from uagents_core.contrib.protocols.chat import chat_protocol_spec agent = Agent(name="test_chat_agent") chat_proto = Protocol(spec=chat_protocol_spec) print("✅ Agent and chat protocol created successfully!") print(f" Agent address: {agent.address}") return True except Exception as e: print(f"❌ Error: {e}") return False def main(): """Run all tests""" print("🚀 Chat Protocol Test Suite") print("=" * 50) success = True success &= test_chat_protocol_imports() success &= test_agent_creation() if success: print("\n🎉 All tests passed!") print("✅ Ready for Agentverse deployment!") else: print("\n❌ Some tests failed!") print("🔧 Fix the issues above before deploying") if __name__ == "__main__": main() ``` ### backend/check_agentverse_status.py ```python #!/usr/bin/env python3 """ Check Agentverse Agent Status This script checks if our agents are visible on Agentverse """ import requests import json from dotenv import load_dotenv load_dotenv() def check_agent_on_agentverse(agent_address): """Check if an agent is visible on Agentverse""" try: url = f"https://agentverse.ai/v1/agents/{agent_address}" response = requests.get(url, timeout=10) if response.status_code == 200: agent_data = response.json() return True, agent_data else: return False, f"HTTP {response.status_code}: {response.text}" except Exception as e: return False, str(e) def main(): """Check all agents on Agentverse""" print("🔍 Checking Agent Status on Agentverse") print("=" * 50) # Get agent addresses try: from storage_agent import storage_agent from test import analysis_agent from nutrition_analysis_agent import analysis_agent as nutrition_agent agents = [ ("Storage Agent", storage_agent.address), ("Analysis Agent", analysis_agent.address), ("Nutrition Analysis Agent", nutrition_agent.address) ] print("📋 Checking agent visibility...") for name, address in agents: print(f"\n🔍 Checking {name}...") print(f" Address: {address}") print(f" URL: https://agentverse.ai/agents/{address}") is_visible, result = check_agent_on_agentverse(address) if is_visible: print(f" ✅ {name} is visible on Agentverse!") if isinstance(result, dict): print(f" 📊 Status: {result.get('status', 'unknown')}") print(f" 📝 Name: {result.get('name', 'unknown')}") else: print(f" ❌ {name} not found on Agentverse") print(f" 📝 Error: {result}") print("\n📋 Summary:") print("If agents show as 'not found', they may need to be:") print("1. Running continuously (keep the agents running)") print("2. Properly registered with Agentverse") print("3. Have sufficient funding") print("\n🔗 Manual Check:") print("Visit https://agentverse.ai and search for:") print("- storage_agent") print("- eating_support_agent") print("- nutrition_analysis_agent") except Exception as e: print(f"❌ Error checking agents: {e}") import traceback traceback.print_exc() if __name__ == "__main__": main() ``` ### backend/debug.py ```python # debug_image_processing.py import requests from PIL import Image import io import os from supabase import create_client from dotenv import load_dotenv load_dotenv() # Initialize Supabase supabase = create_client(os.getenv("SUPABASE_URL"), os.getenv("SUPABASE_KEY")) def debug_image_processing(): """Debug image processing step by step""" print("🔍 Debugging image processing...") try: # Get first image response = supabase.table('meal_images').select('*').order('uploaded_at', desc=True).limit(1).execute() if not response.data: print("❌ No images found") return image_record = response.data[0] image_url = image_record['url'] print(f"📸 Testing image: {image_url}") # Step 1: Download image print("\n1️⃣ Downloading image...") download_response = requests.get(image_url, timeout=30) print(f" Status code: {download_response.status_code}") print(f" Content type: {download_response.headers.get('content-type', 'unknown')}") print(f" Content length: {len(download_response.content)} bytes") if download_response.status_code != 200: print("❌ Download failed") return # Step 2: Check content print("\n2️⃣ Checking content...") content = download_response.content print(f" First 20 bytes: {content[:20]}") print(f" Is JPEG header? {content.startswith(b'\\xff\\xd8\\xff')}") # Step 3: Try to open with PIL print("\n3️⃣ Opening with PIL...") try: # Create BytesIO object image_buffer = io.BytesIO(content) print(f" BytesIO object created: {type(image_buffer)}") # Try to open image image = Image.open(image_buffer) print(f" ✅ Image opened successfully!") print(f" Format: {image.format}") print(f" Mode: {image.mode}") print(f" Size: {image.size}") # Try to verify image.verify() print(f" ✅ Image verified successfully!") # Reopen for actual use image_buffer.seek(0) image = Image.open(image_buffer) print(f" ✅ Image ready for processing!") except Exception as e: print(f" ❌ PIL error: {e}") print(f" Error type: {type(e)}") # Try alternative approach print("\n4️⃣ Trying alternative approach...") try: # Save to temp file and open with open('temp_image.jpg', 'wb') as f: f.write(content) temp_image = Image.open('temp_image.jpg') print(f" ✅ Opened via temp file: {temp_image.size}") # Clean up os.remove('temp_image.jpg') except Exception as e2: print(f" ❌ Alternative approach failed: {e2}") except Exception as e: print(f"❌ Debug failed: {e}") if __name__ == "__main__": debug_image_processing() ``` ### backend/upload_assets_folder.py ```python # upload_assets_folder.py import base64 import json import time import os import glob from storage_agent import upload_image_to_supabase def image_to_base64(image_path: str) -> str: """Convert image file to base64 string""" try: with open(image_path, 'rb') as image_file: image_data = image_file.read() base64_string = base64.b64encode(image_data).decode('utf-8') return base64_string except Exception as e: print(f"❌ Error reading {image_path}: {e}") return "" def upload_image_with_metadata(image_path: str, session_id: str): """Upload single image with metadata""" print(f"\n📸 Uploading: {os.path.basename(image_path)}") # Convert to base64 image_base64 = image_to_base64(image_path) if not image_base64: return None # Upload to Supabase frame_id = f"frame_{int(time.time())}" image_url = upload_image_to_supabase(image_base64, session_id, frame_id) if image_url: print(f"✅ Uploaded: {image_url}") return { 'filename': os.path.basename(image_path), 'url': image_url, 'session_id': session_id, 'frame_id': frame_id, 'uploaded_at': int(time.time()) } else: print(f"❌ Failed to upload {os.path.basename(image_path)}") return None def batch_upload_assets_folder(interval_seconds: int = 30): """Upload all images from assets folder with intervals""" print("🚀 Starting batch upload from assets folder") print(f"⏱️ Interval: {interval_seconds} seconds between uploads") assets_folder = "assets" if not os.path.exists(assets_folder): print(f"❌ Assets folder not found: {assets_folder}") return # Find all image files image_extensions = ['*.jpg', '*.jpeg', '*.png', '*.gif', '*.bmp'] image_files = [] for ext in image_extensions: image_files.extend(glob.glob(os.path.join(assets_folder, ext))) image_files.extend(glob.glob(os.path.join(assets_folder, ext.upper()))) if not image_files: print(f"❌ No image files found in {assets_folder}") return print(f"📁 Found {len(image_files)} images to upload") # Create session ID session_id = f"assets_batch_{int(time.time())}" # Upload each image uploaded_images = [] for i, image_path in enumerate(image_files, 1): print(f"\n{'='*50}") print(f"📸 Uploading image {i}/{len(image_files)}") result = upload_image_with_metadata(image_path, session_id) if result: uploaded_images.append(result) # Wait before next image (except for the last one) if i < len(image_files): print(f"⏳ Waiting {interval_seconds} seconds before next image...") time.sleep(interval_seconds) # Summary print(f"\n{'='*50}") print(f"🎉 BATCH UPLOAD COMPLETE!") print(f"📊 Total images: {len(image_files)}") print(f"✅ Successfully uploaded: {len(uploaded_images)}") print(f"❌ Failed uploads: {len(image_files) - len(uploaded_images)}") # Save results results_file = f"upload_results_{int(time.time())}.json" with open(results_file, 'w') as f: json.dump(uploaded_images, f, indent=2) print(f"📄 Results saved to: {results_file}") if __name__ == "__main__": batch_upload_assets_folder(30) # 30 second intervals ``` [27 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]