# Project export: AgroMind

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: UC Berkeley AI Hackathon 2026
- Tagline: A simulation-first AI platform for early plant disease detection using scout robots, computer vision, and live crop-health modeling.
- Devpost: https://devpost.com/software/agromind
- GitHub: https://github.com/Jeffuz/agromind
- Team: 3 GitHub contributor(s) — Jeff Zhang (29 commits), Mathew Raju (7 commits), Mona Zhao (3 commits)

## Devpost submission (written by the team)

### Overview

🌱

### Inspiration

Greenhouse disease can spread quickly, and growers often do not know there is a problem until symptoms are already visible across multiple plants. By that point, treatment becomes less targeted, more expensive, and more wasteful. We wanted to build a system that makes greenhouse monitoring more proactive. Instead of waiting for disease to spread, AgroMind combines environmental conditions, autonomous scouting robots, computer vision, and crop-health modeling to help detect disease earlier and recommend where attention is needed. AgroMind was inspired by one question: what if a greenhouse could maintain a live understanding of crop health and use scouting robots to inspect the highest-risk plants first? 🍅

### What it does

AgroMind is an AI greenhouse intelligence platform for tomato disease monitoring. Our project combines a 3D greenhouse simulation with a real tomato leaf disease classifier. Users can generate a greenhouse, configure environmental conditions, run autonomous robot scouting, inspect tomato plants, classify leaf images, update a live belief map, and view recommendations for what the system should do next. The system includes: Configurable greenhouse conditions like humidity, temperature, light, and soil moisture A 3D greenhouse simulation with tomato plants and scouting robots Hidden disease pressure generated from environmental risk and spatial hotspots A real tomato disease classifier trained on PlantVillage tomato leaf images FastAPI endpoints for image prediction A crop-health belief map that updates after inspections Agent logs, scouting metrics, sensor panels, and intervention recommendations Instead of treating the whole greenhouse the same way, AgroMind helps identify where disease is likely forming and where targeted inspection or treatment should happen first. 🛠️

### How we built it

We built AgroMind as two connected parts: a greenhouse intelligence frontend and a computer vision backend. The frontend is built with Next.js, React, TypeScript, Tailwind CSS, Three.js, and React Three Fiber. It renders a 3D greenhouse with plant rows, tomato plants, risk overlays, and rover-style scouting robots. The dashboard shows the real simulated greenhouse next to AgroMind’s belief map, so users can see how the system’s understanding changes as inspections happen. The greenhouse simulation uses environmental inputs to generate hidden disease pressure. Humidity, soil moisture, low light, and temperature all contribute to risk. A simplified version of the environmental risk model is: [ EnvironmentRisk = 0.35H + 0.25M + 0.20L_{low} + 0.20T ] where (H) is humidity risk, (M) is soil moisture risk, (L_{low}) is low-light stress, and (T) is temperature-related disease pressure. Each plant receives a hidden disease state and an image associated with one of four tomato classes: Healthy Early blight Late blight Leaf mold For the computer vision backend, we trained a real tomato leaf classifier using TensorFlow/Keras and MobileNetV2. The model trains on four PlantVillage tomato classes and outputs a prediction, confidence score, and severity value. The backend exposes predictions through a FastAPI service so robot-captured images can be classified through an API. Example model response: The scouting agent chooses which plant to inspect using a scoring function based on nearby disease signals, distance, and greenhouse position. A simplified version is: [ TargetScore = 0.5N + 0.3C + 0.2D ] where (N) is nearby disease belief signal, (C) is center priority, and (D) is distance efficiency. When a robot inspects a plant, the system updates the plant’s prediction, confidence, belief risk, metrics, logs, and next-step recommendation. 🌧️

### Challenges we ran into

One of the biggest challenges was connecting multiple systems into one coherent product. AgroMind is not just a classifier and not just a simulation. We had to connect environmental risk modeling, robot scouting, image classification, belief-map updates, and dashboard recommendations into one workflow. Another challenge was scope. The full vision includes real greenhouse sensors, real robots, trained computer vision models, digital twins, and automated greenhouse actuators. For the hackathon, we focused on building the strongest demoable version: a 3D greenhouse simulation connected to a real tomato disease classification backend. Training the CV model also required careful setup. We built a standalone TensorFlow/Keras pipeline, prepared deterministic train/validation/test splits, trained MobileNetV2, generated model artifacts, and exposed inference through FastAPI. We also had to be careful not to overclaim model performance, because PlantVillage images have clean backgrounds and do not fully represent messy real greenhouse conditions. The 3D simulation was also challenging. Early versions looked too flat and grid-like, so we moved to a Three.js-based greenhouse with plant rows, robot movement, risk overlays, zoom controls, and a clearer split between physical ground truth and the system’s belief map. 🏆

### Accomplishments we're proud of

We are proud that AgroMind demonstrates a full AI workflow instead of only showing one isolated model. The project includes a real trained tomato disease classifier, a FastAPI inference service, a 3D greenhouse simulation, autonomous robot scouting, belief-map updates, metrics, and recommendations. Together, these pieces show how computer vision can become part of a larger greenhouse decision-support system. We are also proud that the simulation makes the AI loop easy to understand visually. Users can configure the greenhouse, run scouting, inspect plants, see classification results, watch the belief map update, and compare the system’s understanding against the hidden disease scenario. Most importantly, AgroMind is built around a real agricultural problem: detecting crop disease earlier and reducing unnecessary whole-greenhouse treatment. 🧠

### What we learned

We learned that the value of AI in agriculture is not only the model prediction. A classifier can identify disease in one image, but the real product value comes from connecting that prediction to plant location, environmental conditions, scouting behavior, uncertainty, and recommended action. We also learned how important simulation is for robotics and agriculture. Even before deploying real robot hardware, a simulation lets us test scouting strategies, disease-risk modeling, robot movement, and operator workflows. On the computer vision side, we learned how to build a practical model pipeline: dataset preparation, deterministic splits, transfer learning with MobileNetV2, model evaluation, artifact generation, and API deployment. We also learned to be honest about model limits. A model trained on PlantVillage can perform well on clean test images, but real greenhouse images may include occlusion, lighting changes, motion blur, overlapping leaves, and background noise. Future versions need greenhouse-specific validation before making real-world accuracy claims. 🚜

### What's next

Next, AgroMind will expand from a tomato-focused demo into a crop-agnostic greenhouse intelligence platform. The system is designed so any crop can be supported as long as there is a compatible disease dataset and trained model. The next major step is connecting the full greenhouse response loop. Instead of only detecting disease and showing recommendations, AgroMind should coordinate simulated or real greenhouse actuators such as humidifiers, ventilation systems, irrigation, medicine sprayers, and targeted treatment tools. This would move the platform from monitoring toward closed-loop intervention. Future versions can also add a model registry for multiple crops, real sensor adapters, stronger robot hardware integration, and more advanced predictive planning. Long term, AgroMind could become a full greenhouse operating system that combines sensors, autonomous scouting, computer vision, crop-health modeling, and automated intervention to detect disease earlier, reduce unnecessary pesticide use, and help growers manage outbreaks before they spread.

## README (from the GitHub repository)

# AgroMind

AI greenhouse intelligence platform. An autonomous scouting robot traverses a generated farm, classifies plant disease using a CV model, and updates a Bayesian belief map. A Markov Decision Process guides the robot toward disease clusters. A Fetch.ai agent reasons over the completed scan and delivers a structured farm health report.

---

## Architecture

```
frontend (Next.js :3000)
    ↕  REST
backend (FastAPI :8000)
    ↕  HTTP
Fetch.ai farm_analyst agent (:8001 uAgent protocol · :8002 REST companion)
    ↕  NVIDIA API
Nemotron (nvidia/llama-3.1-nemotron-nano-8b-v1)
```

---

## Prerequisites

- Windows 11 with WSL2 enabled (`wsl --install`)
- Node.js 20+ inside WSL2
- Python 3.12 inside WSL2
- NVIDIA API key (free tier at [build.nvidia.com](https://build.nvidia.com))

---

## 1 — Clone into WSL2

Always work from the WSL2 native filesystem to avoid Windows path-length issues with npm and pip.

```bash
# inside WSL2 terminal
git clone <repo-url> ~/agromind
cd ~/agromind
```

---

## 2 — Backend setup

```bash
cd ~/agromind/backend

# create venv
python3 -m venv ~/.venvs/agromind
source ~/.venvs/agromind/bin/activate

# install dependencies
pip install -r requirements.txt
```

Create `backend/.env`:

```env
NVIDIA_API_KEY=your_nvidia_api_key_here
FARM_ANALYST_URL=http://localhost:8002
```

---

## 3 — Frontend setup

```bash
cd ~/agromind/frontend
npm install
```

---

## Running

You need **three terminals** inside WSL2, all with the venv active.

### Terminal 1 — FastAPI backend

```bash
source ~/.venvs/agromind/bin/activate
cd ~/agromind/backend
uvicorn main:app --port 8000
```

### Terminal 2 — Fetch.ai farm analyst agent

```bash
source ~/.venvs/agromind/bin/activate
cd ~/agromind/backend
python -m agents.farm_analyst
```

The agent registers on Agentverse automatically. You will see its address logged:
```
farm_analyst online | address: agent1q...
```

### Terminal 3 — Frontend

```bash
cd ~/agromind/frontend
npm run dev
```

Open [http://localhost:3000](http://localhost:3000) in your browser.

---

## Usage

1. **Configure** — set greenhouse conditions (humidity, temperature, light, soil moisture) on the home screen and click **Launch Simulation**
2. **Scout** — on the dashboard, click **Run Agent Step** to move the robot one cell at a time, or **Auto Run** to let it traverse the full farm autonomously
3. **Analyse** — when all plants are inspected the Fetch.ai agent automatically reasons over the belief map and populates the **Recommendation** panel with a structured farm health report

The **Reveal Ground Truth** toggle shows the hidden disease distribution for comparison against the robot's belief map.

---

## API reference

| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/farm/agent/step` | Run MDP on frontend belief grid, return next cell |
| `POST` | `/farm/analyze` | Delegate to Fetch.ai agent → Nemotron analysis |
| `POST` | `/cv/predict` | Classify a leaf image with the MobileNetV2 model |
| `GET`  | `/cv/health` | Verify CV model is loaded |
| `GET`  | `/farm/grid` | Current observed belief grid |
| `POST` | `/farm/reset` | Reset all observations |

Full interactive docs at `http://localhost:8000/docs`.

---

## Environment variables

| Variable | Required | Description |
|----------|----------|-------------|
| `NVIDIA_API_KEY` | Yes | NVIDIA NIM API key for Nemotron |
| `FARM_ANALYST_URL` | No | URL of the farm analyst companion API (default `http://localhost:8002`) |
| `FARM_ANALYST_SEED` | No | Deterministic seed for the Fetch.ai agent identity |
| `AGROMIND_CV_MODEL_PATH` | No | Override path to the `.keras` model file |
| `AGROMIND_CV_CLASSES_PATH` | No | Override path to `class_names.json` |


## Detected evidence (automated analysis)

Indexed codebase: 56 recognized source files, 152 KB.
- CSS (language) — detected in the code
- FastAPI (technology) — detected in the code
- Next.js (technology) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TensorFlow (technology) — detected in the code
- TypeScript (language) — detected in the code
- PyTorch (technology) — claimed on Devpost, not found in the code
- AI coding agent: Claude Code — evidence: config files committed to the repository
- AI coding agent: Codex — evidence: config files committed to the repository

## Codebase structure (from repository index)

### Files (68 of 68)

```
.env.example
.gitignore
backend/agents/__init__.py
backend/agents/farm_analyst.py
backend/agents/models.py
backend/belief.py
backend/cv_service.py
backend/farm.py
backend/main.py
backend/mdp.py
backend/models/agromind_mobilenetv2.keras
backend/models/class_names.json
backend/models/metadata.json
backend/README.md
backend/requirements-gpu.txt
backend/requirements.txt
backend/ROBOT_HANDOFF.md
backend/schemas.py
backend/scripts/activate_wsl.sh
backend/scripts/benchmark_cv.py
backend/scripts/setup_wsl.sh
backend/scripts/smoke_robot_contract.py
backend/tests/test_belief.py
backend/tests/test_cv.py
frontend/AGENTS.md
frontend/CLAUDE.md
frontend/eslint.config.mjs
frontend/next-env.d.ts
frontend/next.config.mjs
frontend/next.config.ts
frontend/package.json
frontend/postcss.config.mjs
frontend/README.md
frontend/src/app/config/page.tsx
frontend/src/app/dashboard/page.tsx
frontend/src/app/globals.css
frontend/src/app/layout.tsx
frontend/src/app/page.tsx
frontend/src/components/config/ConditionAdjuster.tsx
frontend/src/components/config/ConfigurationPage.tsx
frontend/src/components/config/QuickStartScenarios.tsx
frontend/src/components/config/SimulationSettings.tsx
frontend/src/components/config/WhatHappensNext.tsx
frontend/src/components/dashboard/AgentLog.tsx
frontend/src/components/dashboard/DashboardPage.tsx
frontend/src/components/dashboard/InspectionPanel.tsx
frontend/src/components/dashboard/MetricsPanel.tsx
frontend/src/components/dashboard/RecommendationPanel.tsx
frontend/src/components/dashboard/SensorDataPanel.tsx
frontend/src/components/home/HomePage.tsx
frontend/src/components/layout/Card.tsx
frontend/src/components/layout/PageHeader.tsx
frontend/src/components/simulation/BeliefMap.tsx
frontend/src/components/simulation/FarmMap.tsx
frontend/src/components/simulation/GreenhouseScene3D.tsx
frontend/src/components/simulation/MapLegend.tsx
frontend/src/components/simulation/PlantCell.tsx
frontend/src/components/simulation/SimulationToolbar.tsx
frontend/src/lib/cvAdapter.ts
frontend/src/lib/diseaseMap.ts
frontend/src/lib/metrics.ts
frontend/src/lib/scenarios.ts
frontend/src/lib/simulation.ts
frontend/src/lib/tomatoImages.ts
frontend/src/lib/types.ts
frontend/src/store/simulationStore.ts
frontend/tsconfig.json
README.md
```

### Dependencies

- backend/requirements.txt: fastapi, httpx@>=0.27,<1, numpy, openai, pillow@>=10,<13, python-dotenv, python-multipart@>=0.0.9,<1, tensorflow@>=2.18,<2.22, uagents@>=0.13,<1, uvicorn[standard]
- frontend/package.json: @react-three/fiber@^9.6.1, @tailwindcss/postcss@^4, @types/node@^20, @types/react@^19, @types/react-dom@^19, @types/three@^0.184.1, eslint@^9, eslint-config-next@16.2.9, next@16.2.9, react@19.2.4, react-dom@19.2.4, react-icons@^5.6.0, tailwindcss@^4, three@^0.184.0, typescript@^5, zustand@^5.0.14

### Recent commits (newest first)

- new readme
- fix merge errors
- Merge branch 'main' of https://github.com/Jeffuz/agromind
- Fix belief heatmap rendering
- Merge pull request #4 from Jeffuz/mathew/robot
- Merge branch 'main' into mathew/robot
- final
- fix layout
- add home page
- Update SensorDataPanel.tsx
- improve belief heatmap color weighting
- Add belief heatmap overlay
- fix layout size
- scouting control - speed
- Refine auto scouting control
- more stuff for reasoning
- Add animated agent scouting loop
- Clarify disease pressure legend
- Add mock CV inspection flow
- implement image reveal for plant section

## Key source files (fetched from GitHub, selected and truncated for size)

### frontend/CLAUDE.md

```markdown
@AGENTS.md

```

### frontend/AGENTS.md

```markdown
<!-- BEGIN:nextjs-agent-rules -->
# This is NOT the Next.js you know

This version has breaking changes — APIs, conventions, and file structure may all differ from your training data. Read the relevant guide in `node_modules/next/dist/docs/` before writing any code. Heed deprecation notices.
<!-- END:nextjs-agent-rules -->

```

### backend/requirements.txt

```
fastapi
uvicorn[standard]
numpy
openai
python-dotenv
tensorflow>=2.18,<2.22
pillow>=10,<13
python-multipart>=0.0.9,<1
httpx>=0.27,<1
uagents>=0.13,<1

```

### frontend/package.json

```
{
  "name": "frontend",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "eslint"
  },
  "dependencies": {
    "@react-three/fiber": "^9.6.1",
    "next": "16.2.9",
    "react": "19.2.4",
    "react-dom": "19.2.4",
    "react-icons": "^5.6.0",
    "three": "^0.184.0",
    "zustand": "^5.0.14"
  },
  "devDependencies": {
    "@tailwindcss/postcss": "^4",
    "@types/node": "^20",
    "@types/react": "^19",
    "@types/react-dom": "^19",
    "@types/three": "^0.184.1",
    "eslint": "^9",
    "eslint-config-next": "16.2.9",
    "tailwindcss": "^4",
    "typescript": "^5"
  }
}

```

### backend/main.py

```python
import os
from typing import Annotated

from dotenv import load_dotenv
load_dotenv()

import httpx
import numpy as np
from fastapi import Body, FastAPI, File, Form, HTTPException, Query, UploadFile
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel

import farm
from belief import DEFAULT_PRIOR_RISK, update_belief
from cv_service import ModelUnavailableError, model_status, predict_image
from farm import GRID_SIZE, TRUE_GRID
from mdp import value_iteration, follow_policy, ACTIONS
from schemas import CVHealth, CVPrediction, ImageVisitResponse


class AgentStepRequest(BaseModel):
    belief_grid: list[list[float]]   # 1.1 sentinel for unvisited, beliefRisk for visited
    robot_row: int
    robot_col: int


class AnalyzeFarmBody(BaseModel):
    belief_grid: list[list[float | None]] | None = None  # None = unvisited

app = FastAPI(title="AgroMind API", version="0.1.0")

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_methods=["*"],
    allow_headers=["*"],
)

MAX_IMAGE_BYTES = 10 * 1024 * 1024


# ---------------------------------------------------------------------------
# Computer vision
# ---------------------------------------------------------------------------

@app.get("/cv/health", response_model=CVHealth)
def cv_health():
    """Verify that the packaged tomato classifier can be loaded."""
    try:
        return model_status()
    except ModelUnavailableError as error:
        raise HTTPException(status_code=503, detail=str(error)) from error


@app.post("/cv/predict", response_model=CVPrediction)
async def cv_predict(
    file: Annotated[UploadFile, File()],
    plantId: Annotated[str | None, Form()] = None,
):
    """Classify one tomato leaf image without changing simulation state."""
    image_bytes = await _read_image(file)
    try:
        return predict_image(image_bytes, plantId)
    except ModelUnavailableError as error:
        raise HTTPException(status_code=503, detail=str(error)) from error
    except ValueError as error:
        raise HTTPException(status_code=400, detail=str(error)) from error


# ---------------------------------------------------------------------------
# Grid state
# ---------------------------------------------------------------------------

@app.get("/farm/grid")
def get_grid():
    """Current observed grid. null = unvisited (CV model hasn't scored it yet)."""
    return {
        "size": GRID_SIZE,
        "grid": farm.observed,
    }


@app.get("/farm/grid/true")
def get_true_grid():
    """Full ground-truth infection grid (for debugging / visualisation only)."""
    return {
        "size": GRID_SIZE,
        "grid": TRUE_GRID.tolist(),
    }


@app.post("/farm/reset")
def reset_grid():
    """Reset the observed grid to empty (all unvisited)."""
    farm.reset()
    return {"message": "Grid reset. All cells unvisited."}


# ---------------------------------------------------------------------------
# Robot visit — simulates CV model scoring a plant
# ---------------------------------------------------------------------------

@app.post("/farm/visit")
def visit_plant(
    row: int = Query(..., ge=0, lt=GRID_SIZE),
    col: int = Query(..., ge=0, lt=GRID_SIZE),
):
    """
    Robot arrives at (row, col). CV model scores the plant.
    Returns the score, the updated MDP policy, and the recommended next cell.
    """
    already_visited = farm.observed[row][col] is not None
    score = farm.visit_plant(row, col)

    effective_grid = farm.get_effective_grid()
    V, policy = value_iteration(effective_grid)

    next_cell = _best_next(row, col, policy, V)

    return {
        "visited": (row, col),
        "score": score,
        "already_visited": already_visited,
        "next_recommended": next_cell,
        "all_done": farm.all_visited(),
    }


@app.post("/farm/visit/image", response_model=ImageVisitResponse)
async def visit_plant_with_image(
    file: Annotated[UploadFile, File()],
    row: int = Query(..., ge=0, lt=GRID_SIZE),
    col: int = Query(..., ge=0, lt=GRID_SIZE),
):
    """Classify a robot image, update belief state, and recommend the next cell."""
    image_bytes = await _read_image(file)
    plant_id = farm.plant_id_for(row, col)
    try:
        cv_result = predict_image(image_bytes, plant_id)
    except ModelUnavailableError as error:
        raise HTTPException(status_code=503, detail=str(error)) from error
    except ValueError as error:
        raise HTTPException(status_code=400, detail=str(error)) from error

    already_visited = farm.observed[row][col] is not None
    prior_risk = (
        float(farm.observed[row][col]) if already_visited else DEFAULT_PRIOR_RISK
    )
    belief_risk, uncertainty = update_belief(
        severity=float(cv_result["severity"]),
        confidence=float(cv_result["confidence"]),
        prior_risk=prior_risk,
    )
    farm.record_observation(row, col, belief_risk)
    effective_grid = farm.get_effective_grid()
    values, policy = value_iteration(effective_grid)
    return {
        "visited": (row, col),
        "plantId": plant_id,
        "cv": cv_result,
        "beliefRisk": belief_risk,
        "uncertainty": uncertainty,
        "priorRisk": prior_risk,
        "alreadyVisited": already_visited,
        "nextRecommended": _best_next(row, col, policy, values),
        "allDone": farm.all_visited(),
    }


# ---------------------------------------------------------------------------
# MDP endpoints
# ---------------------------------------------------------------------------

@app.get("/farm/mdp/policy")
def get_policy():
    """Optimal policy computed from the currently observed scores."""
    effective_grid = farm.get_effective_grid()
    _, policy = value_iteration(effective_grid)
    return {
        "policy": [[policy[r][c] for c in range(GRID_SIZE)] for r in range(GRID_SIZE)],
    }


@app.get("/farm/mdp/values")
def get_values():
    """Value function V(s) computed from the currently observed scores."""
    effective_g
[truncated — 9380 more characters]
```

### frontend/src/app/page.tsx

```typescript
import { HomePage as LandingPage } from "@/components/home/HomePage";

export default function HomePage() {
  return <LandingPage />;
}

```

### frontend/src/app/layout.tsx

```typescript
import type { Metadata } from "next";
import type { ReactNode } from "react";
import "./globals.css";

export const metadata: Metadata = {
  title: "AgroMind",
  description: "AgroMind farm simulation",
};

interface RootLayoutProps {
  children: ReactNode;
}

export default function RootLayout({ children }: RootLayoutProps) {
  return (
    <html lang="en">
      <body>{children}</body>
    </html>
  );
}

```

### frontend/src/app/dashboard/page.tsx

```typescript
import { DashboardPage } from "@/components/dashboard/DashboardPage";

export default function DashboardRoute() {
  return <DashboardPage />;
}

```

### frontend/src/app/config/page.tsx

```typescript
import { ConfigurationPage } from "@/components/config/ConfigurationPage";

export default function ConfigRoute() {
  return <ConfigurationPage />;
}

```

### frontend/next.config.ts

```typescript
import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  /* config options here */
};

export default nextConfig;

```

[44 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]