# Project export: BlackMarkIt

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2026
- Tagline: Predicting and marking global wildlife black markets
- Devpost: https://devpost.com/software/blackmarkit
- GitHub: https://github.com/pvelleleth/treehacks
- Video: https://www.youtube.com/embed/_TMLCT_ONxc?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 2 GitHub contributor(s) — sathvikr (19 commits), pvelleleth (12 commits)

## Devpost submission (written by the team)

### Inspiration

In the Gulf of California, the Totoaba’s swim bladder is known as "aquatic cocaine," fetching between $20,000 to $80,000 per kilogram. This astronomical value has drawn in the many Cartels, transforming local fishing villages like San Felipe into nodes of a global criminal network, and a similar story is being told for many other endangered species today. So we built BlackMarkIt, a predictive modeling program that allows police departments to find traffickers before they reach a consumer.

### What it does

BlackMarkIt is a global illicit-trade intelligence engine. It ingests seizure data (Origin, Transit, Destination, Commodity Type, etc) to map the hidden "connective tissue" of this $23B illegal wildlife market. Users click on regions our algorithm have determined to be likely spots for a black market to operate which starts a search algorithm for potential hubs (buildings, warehouses, stores, etc) and uses an agentic AI LLM agent to comb through online resources to check what companies are associated with what properties, whether these companies are shells, and their online presence. Through this, we determine which sites might be very likely within an area to be an operating base for these trafficking rings.

### How we built it

We utilized a global dataset of police seizures (intercepted packages containing ivory, rhino horns, etc.). This data provided not just a "point on a map," but the economic lineage: monetary value, origin point, and interception point. We then fed this information into a DBSCAN algorithm that allowed us to create clusters with centroids that could act as potential market areas. Then by drawing a sweep around all the points in a given cluster, we could determine an area we deemed likely to house an operating base for a trafficking ring. Then with this area, we would iterate through all of the buildings by utilizing OpenStreetMap. For each buildings, we would then use a ChatGPT based agent we created to operate on an OSINT based framework to find information about the buildings and their handlers with publicly available resources, This agent also determines the risk of a given building, determining it to be more or less likely of being apart of a ring.

### Challenges we ran into

One of the biggest challenges we faced was determining how we would cluster our points together. Initially, we were planning on creating a polygon based model where all the interception points act as a vertex of a given polygon area that we would deem as an area with higher probability of having a black market. This was later adapted to use a multidimensional k-means algorithm that weighs position, weight, and cost against one another to determine clusters from which their centroids would be the hotspots. We later had to pivot this idea once again as kmeans would have issues in how groupings were determined as well as how we created the area containing a possible black market. In the end, we decided to utilize a DBSCAN which would comb through the data and create groupings of closely knit points and have these clusters become the new searchable areas and had a lot higher accuracy.

### Accomplishments we're proud of

We were really proud of the fact that we were able to create a fun and unique idea product during the limited time of this hackathon. We wanted to incorporate a lot of technical and aesthetic features that would still streamline user interaction with the platform.

### What we learned

We learned about utilizing unique datasets with multiple non quantifiable metrics while still being able to train a prediction model upon it. We also learned how to leverage agentic AI to aid in searching that would have taken a person much longer to complete.

### What's next

Real time endangered species commodity package determination Transition into using models to find hubs for human trafficking and drug distribution or other hidden markets.

## README (from the GitHub repository)

# Development Setup

## Backend

1. Create and activate a Python virtual environment (recommended):  
   ```bash
   python -m venv .venv && source .venv/bin/activate
   ```
2. Install dependencies:
   ```bash
   pip install -r backend/requirements.txt
   ```
3. Start the FastAPI server (from the `backend` directory):
   ```bash
   cd backend
   fastapi dev main.py --reload --port 8000
   ```
   The API will be available at `http://127.0.0.1:8000`.

## Frontend

1. Install Node dependencies:
   ```bash
   cd frontend
   npm install
   ```
2. Run the development server:
   ```bash
   npm run dev -- --host 0.0.0.0 --port 5173
   ```
   The React app will be served on `http://127.0.0.1:5173`.

## Notes

- If both servers are running, ensure the frontend points to the backend API (check `src` for proxy or fetch endpoints).
- Use `npm run build` (frontend) and `uvicorn --reload` (backend) for production simulation.


## Detected evidence (automated analysis)

Indexed codebase: 108 recognized source files, 545 KB.
- CSS (language) — detected in the code
- FastAPI (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- AI coding agent: Codex — evidence: config files committed to the repository

## Codebase structure (from repository index)

### Files (120 of 138)

```
.gitignore
AGENTS.md
backend/agent-endpoints.md
backend/agent-workflow.md
backend/agent.py
backend/main.py
backend/preprocess_data.py
backend/requirements.txt
backend/start.sh
backend/test_data.py
backend/test_recording_viewer.html
backend/wildlife.db
BLACK_MARKET_INVESTIGATION.md
BROWSERBASE_CAPABILITIES.md
BUGFIX_422_ERROR.md
cluster_commodities_improved.py
cluster_commodities.py
clustered_commodities.csv
CLUSTERING_INTEGRATION.md
CORS_AND_500_ERROR_FIX.md
data/clusters/cluster_-0.4912_10.5822_356688.csv
data/clusters/cluster_-14.5141_34.0342_397066.csv
data/clusters/cluster_-33.7416_25.8504_385940.csv
data/clusters/cluster_17.4136_101.1630_2994287.csv
data/clusters/cluster_19.8662_102.4762_117302.csv
data/clusters/cluster_19.8852_102.2991_28640.csv
data/clusters/cluster_33.8893_74.2217_689485.csv
data/clusters/cluster_4.6885_101.8626_222428.csv
data/clusters/cluster_49.9542_22.1101_95431.csv
data/clusters/cluster_54.6717_25.2844_5043.csv
enrich_data.py
export_clusters_json.py
frontend/.gitignore
frontend/components.json
frontend/eslint.config.js
frontend/index.html
frontend/package.json
frontend/public/clusters.json
frontend/public/incident-data-3496.csv
frontend/public/incident-summary-and-locations-3497.csv
frontend/public/incident-summary-and-species-3496.csv
frontend/public/shipment-paths.json
frontend/README.md
frontend/src/App.css
frontend/src/App.tsx
frontend/src/components/black-market-markers.tsx
frontend/src/components/cluster-circles.tsx
frontend/src/components/globe-view.tsx
frontend/src/components/incident-card.tsx
frontend/src/components/incident-detail-panel.tsx
frontend/src/components/investigation-feed.tsx
frontend/src/components/investigation-panel.tsx
frontend/src/components/map-search.tsx
frontend/src/components/map-view.tsx
frontend/src/components/radar-sweep.tsx
frontend/src/components/shipment-map.tsx
frontend/src/components/ui/accordion.tsx
frontend/src/components/ui/alert-dialog.tsx
frontend/src/components/ui/alert.tsx
frontend/src/components/ui/aspect-ratio.tsx
frontend/src/components/ui/avatar.tsx
frontend/src/components/ui/badge.tsx
frontend/src/components/ui/breadcrumb.tsx
frontend/src/components/ui/button-group.tsx
frontend/src/components/ui/button.tsx
frontend/src/components/ui/calendar.tsx
frontend/src/components/ui/card.tsx
frontend/src/components/ui/carousel.tsx
frontend/src/components/ui/chart.tsx
frontend/src/components/ui/checkbox.tsx
frontend/src/components/ui/collapsible.tsx
frontend/src/components/ui/combobox.tsx
frontend/src/components/ui/command.tsx
frontend/src/components/ui/context-menu.tsx
frontend/src/components/ui/dialog.tsx
frontend/src/components/ui/direction.tsx
frontend/src/components/ui/drawer.tsx
frontend/src/components/ui/dropdown-menu.tsx
frontend/src/components/ui/empty.tsx
frontend/src/components/ui/field.tsx
frontend/src/components/ui/form.tsx
frontend/src/components/ui/hover-card.tsx
frontend/src/components/ui/input-group.tsx
frontend/src/components/ui/input-otp.tsx
frontend/src/components/ui/input.tsx
frontend/src/components/ui/item.tsx
frontend/src/components/ui/kbd.tsx
frontend/src/components/ui/label.tsx
frontend/src/components/ui/menubar.tsx
frontend/src/components/ui/native-select.tsx
frontend/src/components/ui/navigation-menu.tsx
frontend/src/components/ui/pagination.tsx
frontend/src/components/ui/popover.tsx
frontend/src/components/ui/progress.tsx
frontend/src/components/ui/radio-group.tsx
frontend/src/components/ui/resizable.tsx
frontend/src/components/ui/scroll-area.tsx
frontend/src/components/ui/select.tsx
frontend/src/components/ui/separator.tsx
frontend/src/components/ui/sheet.tsx
frontend/src/components/ui/sidebar.tsx
frontend/src/components/ui/skeleton.tsx
frontend/src/components/ui/slider.tsx
frontend/src/components/ui/sonner.tsx
frontend/src/components/ui/spinner.tsx
frontend/src/components/ui/switch.tsx
frontend/src/components/ui/table.tsx
frontend/src/components/ui/tabs.tsx
frontend/src/components/ui/textarea.tsx
frontend/src/components/ui/toggle-group.tsx
frontend/src/components/ui/toggle.tsx
frontend/src/components/ui/tooltip.tsx
frontend/src/hooks/use-mobile.ts
frontend/src/index.css
frontend/src/lib/agent-api.ts
frontend/src/lib/api.ts
frontend/src/lib/cluster-cache.ts
frontend/src/lib/utils.ts
frontend/src/main.tsx
frontend/src/pages/ClusterDetailPage.tsx
[18 more files omitted for size]
```

### Dependencies

- backend/requirements.txt: beautifulsoup4, browserbase, eval-type-backport, fastapi, httpx, openai, openai-agents, playwright, python-dotenv, requests, uvicorn[standard]
- frontend/package.json: @base-ui/react@^1.2.0, @eslint/js@^9.39.1, @hookform/resolvers@^5.2.2, @tailwindcss/vite@^4.1.18, @types/leaflet@^1.9.21, @types/node@^24.10.13, @types/react@^19.2.7, @types/react-dom@^19.2.3, @vitejs/plugin-react@^5.1.1, class-variance-authority@^0.7.1, clsx@^2.1.1, cmdk@^1.1.1, date-fns@^4.1.0, embla-carousel-react@^8.6.0, eslint@^9.39.1, eslint-plugin-react-hooks@^7.0.1, eslint-plugin-react-refresh@^0.4.24, globals@^16.5.0, input-otp@^1.4.2, leaflet@^1.9.4, lucide-react@^0.564.0, next-themes@^0.4.6, radix-ui@^1.4.3, react@^19.2.0, react-day-picker@^9.13.2, react-dom@^19.2.0, react-hook-form@^7.71.1, react-leaflet@^5.0.0, react-resizable-panels@^4.6.4, react-router-dom@^7.13.0, recharts@^2.15.4, shadcn@^3.8.4, sonner@^2.0.7, tailwind-merge@^3.4.0, tailwindcss@^4.1.18, tw-animate-css@^1.4.0, typescript@~5.9.3, typescript-eslint@^8.48.0, vaul@^1.1.2, vite@^7.3.1, zod@^4.3.6

### Recent commits (newest first)

- Updaerts in UI
- UI update
- Merge pull request #8 from pvelleleth/gptweb
- Merge remote-tracking branch 'origin/gptweb' into gptweb
- made the agent a lot better
- Merge pull request #7 from pvelleleth/ui-updates
- Radar updates
- Update with map view
- Update images
- Merge pull request #6 from pvelleleth/gptweb
- Merge branch 'main' into gptweb
- removed browserbase
- Images
- integrated api for building analysis
- Merge pull request #5 from pvelleleth/agent
- Merge pull request #4 from pvelleleth/model
- Update changes
- created the building radius endpoint
- built agent endpoint
- Update clusters

## Key source files (fetched from GitHub, selected and truncated for size)

### AGENTS.md

```markdown
# Project Guidelines & Structure

This file serves as the primary reference for AI agents working on this wildlife poaching treehacks project. It consolidates the backend structure, frontend guidelines, and tech stack.

**CRITICAL: Always reference `PRD.md` to ensure that all features and implementations are strictly aligned with the project's end goals and requirements.**

## Tech Stack

### Frontend
- **Framework:** React 19 (Vite)
- **Language:** TypeScript
- **Routing:** React Router 7
- **Authentication:** No auth
- **Styling:** ShadCN and Tailwind CSS
- **Icons:** Lucide React (proposed)
- **Animations:** Framer Motion (proposed for premium feel)

### Backend
- **Framework:** FastAPI
- **Language:** Python
- **Database:** No database, just a csv file

### Infrastructure
- **Hosting:** Vercel (Frontend), Google Cloud Run (Backend)
- **Package Management:** npm

---

## Backend Structure

The backend is built with Fastify and TypeScript, focused on security and scalability for AI agent operations. 
Make sure all code written is modular and maintainable. 
All backend logic API endpoints should be prefixed with /v1


---

## Frontend Guidelines

### Design Philosophy
- **Jobs/Ive Inspired:** Minimalist, intentional, and quiet.
- **Hierarchy first:** If everything is prominent, nothing is. Use whitespace to guide the eye.
- **Motion as feedback:** Subtle transitions for state changes. No gratuitous animation.

### Technical Standards
- **Component Structure:** Function-based components with TypeScript interfaces for props.
- **Styling:**
    - Use CSS Variables (Tokens) defined in `index.css`.
    - Component-specific styles in `*.css` files co-located with components.
    - No hardcoded hex codes; always reference the design tokens.
- **State Management:**
    - Use React Context for global state (Auth, Workspace).
    - Use `useState`/`useReducer` for local component state.
    - **Beyond any simple state management, use Zustand for state management.**
- **Performance:**
    - Memoize expensive computations with `useMemo`.
    - Use `React.memo` for pure presentational components.
- **Accessibility:**
    - Proper semantic HTML (`main`, `section`, `header`, `button`).
    - Aria labels for interactive elements.
    - Keyboard navigability for all actions.

---

## Agent Progress Tracking

**CRITICAL: All agents must maintain progress.txt to track development status and milestones.**

- Update `progress.txt` after completing significant tasks or milestones
- Include timestamps and clear descriptions of what was accomplished
- Track both completed work and upcoming priorities
- Use this file as a living document of project progress

### File Organization
```
src/
  components/  # Shared, reusable atoms/molecules
  pages/       # Page-level components (routes)
  hooks/       # Custom React hooks
  contexts/    # React Context providers
  assets/      # Static assets (images, fonts)
  lib/         # Utility functions and SDK clients
```

```

### MAP_STYLING_CHANGES.md

```markdown
# Map Styling Changes Summary

## Changes Made

### 1. **All Path Lines → Light Gray & Translucent**

**Before:**
- Blue lines (source to interception)
- Orange lines (interception to destination)
- Different opacity levels for active/inactive states

**After:**
- All lines are light gray (`#d1d5db`)
- Low opacity (0.3) with slightly higher on hover (0.5)
- Consistent styling across all paths

**Files Modified:**
- `frontend/src/components/shipment-map.tsx`
  - Updated `sourceToInterceptionStyle` to use gray color
  - Removed orange destination styling
  - Simplified hover states

### 2. **Removed Destination Points**

**What was removed:**
- Orange destination markers/circles
- Lines from interception to destination
- Arrows pointing to destination
- Destination information in tooltips

**Backend Changes:**
- `backend/main.py`
  - Removed `destination: MapNode` from `ShipmentPath` model (line 66)
  - Updated path generation logic to only collect source + interception (lines 154-207)
  - Removed destination node creation
  - Simplified path building to use only 2 points

**Frontend Changes:**
- `frontend/src/types/shipment-path.ts`
  - Removed `destination: MapNode` from interface

- `frontend/src/components/shipment-map.tsx`
  - Removed destination polylines and arrows (formerly lines 443-465)
  - Updated `allPoints` calculation to exclude destinations
  - Updated `activeVertexKeys` to exclude destinations
  - Removed destination from tooltip display
  - Updated `vertexStats` to not process destinations

### 3. **Arrow Styling**

**Before:**
- Blue arrows (source → interception)
- Orange arrows (interception → destination)

**After:**
- All arrows are light gray (`#d1d5db`)
- Match the line color for consistency

## Visual Changes

### Map Display

**Before:**
- Blue lines + orange lines creating complex route patterns
- Three-point paths (Origin → Seizure → Destination)
- Distinct visual hierarchy with color coding

**After:**
- Simple gray translucent lines
- Two-point paths (Origin → Seizure)
- Minimal visual footprint - emphasis on cluster circles
- Cleaner, less cluttered map view

### Tooltip

**Before:**
```
Origin: Location A
Seizure: Location B
Destination: Location C
```

**After:**
```
Origin: Location A
Seizure: Location B
```

## Testing

To test the changes:

1. **Restart Backend:**
```bash
cd backend
source .venv/bin/activate
python main.py
```

2. **Restart Frontend:**
```bash
cd frontend
npm run dev
```

3. **Check the map:**
   - All lines should be light gray
   - No orange elements visible
   - Only blue origin markers and orange interception markers
   - Hover over lines to see light gray arrows

## Rollback Instructions

If you need to revert these changes:

1. **Backend:** Restore `destination` field in ShipmentPath model
2. **Frontend:**
   - Restore destination in TypeScript types
   - Restore orange line styles and destination polylines in shipment-map.tsx
   - Restore destination in tooltip

## Files Changed

### Backe
[truncated — 445 more characters]
```

### backend/requirements.txt

```
fastapi
uvicorn[standard]
beautifulsoup4
requests
openai
openai-agents
playwright
browserbase
httpx
eval-type-backport
python-dotenv

```

### frontend/package.json

```
{
  "name": "frontend",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "tsc -b && vite build",
    "lint": "eslint .",
    "preview": "vite preview"
  },
  "dependencies": {
    "@base-ui/react": "^1.2.0",
    "@hookform/resolvers": "^5.2.2",
    "@tailwindcss/vite": "^4.1.18",
    "class-variance-authority": "^0.7.1",
    "clsx": "^2.1.1",
    "cmdk": "^1.1.1",
    "date-fns": "^4.1.0",
    "embla-carousel-react": "^8.6.0",
    "input-otp": "^1.4.2",
    "leaflet": "^1.9.4",
    "lucide-react": "^0.564.0",
    "next-themes": "^0.4.6",
    "radix-ui": "^1.4.3",
    "react": "^19.2.0",
    "react-day-picker": "^9.13.2",
    "react-dom": "^19.2.0",
    "react-hook-form": "^7.71.1",
    "react-leaflet": "^5.0.0",
    "react-resizable-panels": "^4.6.4",
    "react-router-dom": "^7.13.0",
    "recharts": "^2.15.4",
    "sonner": "^2.0.7",
    "tailwind-merge": "^3.4.0",
    "tailwindcss": "^4.1.18",
    "vaul": "^1.1.2",
    "zod": "^4.3.6"
  },
  "devDependencies": {
    "@eslint/js": "^9.39.1",
    "@types/leaflet": "^1.9.21",
    "@types/node": "^24.10.13",
    "@types/react": "^19.2.7",
    "@types/react-dom": "^19.2.3",
    "@vitejs/plugin-react": "^5.1.1",
    "eslint": "^9.39.1",
    "eslint-plugin-react-hooks": "^7.0.1",
    "eslint-plugin-react-refresh": "^0.4.24",
    "globals": "^16.5.0",
    "shadcn": "^3.8.4",
    "tw-animate-css": "^1.4.0",
    "typescript": "~5.9.3",
    "typescript-eslint": "^8.48.0",
    "vite": "^7.3.1"
  }
}

```

### frontend/src/main.tsx

```typescript
import { StrictMode } from 'react'
import { createRoot } from 'react-dom/client'
import { BrowserRouter } from 'react-router-dom'
import './index.css'
import 'leaflet/dist/leaflet.css'
import App from './App.tsx'

createRoot(document.getElementById('root')!).render(
  <StrictMode>
    <BrowserRouter>
      <App />
    </BrowserRouter>
  </StrictMode>,
)

```

### frontend/src/App.tsx

```typescript
import { Routes, Route } from 'react-router-dom'
import { HomePage } from '@/pages/HomePage'
import { IncidentDetailPage } from '@/pages/IncidentDetailPage'
import { ClusterDetailPage } from '@/pages/ClusterDetailPage'
import './App.css'

function App() {
  return (
    <Routes>
      <Route path="/" element={<HomePage />} />
      <Route path="/incident/:id" element={<IncidentDetailPage />} />
      <Route path="/cluster/:lat/:lng/:radius/:commodity" element={<ClusterDetailPage />} />
    </Routes>
  )
}

export default App

```

### backend/main.py

```python
import csv
from collections import defaultdict
from functools import lru_cache
from pathlib import Path
import re
from typing import List, Optional, Dict
from urllib.parse import urljoin, urlparse

from dotenv import load_dotenv

load_dotenv()

import requests
from bs4 import BeautifulSoup
from fastapi import FastAPI, Query
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel

from agent import router as agent_router

app = FastAPI()

# Include agent workflow router
app.include_router(agent_router)

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=False,
    allow_methods=["*"],
    allow_headers=["*"],
)

SPECIES_CSV_PATH = Path(__file__).resolve().parent.parent / "incident-summary-and-species-3496.csv"
LOCATIONS_CSV_PATH = Path(__file__).resolve().parent.parent / "incident-summary-and-locations-3497.csv"
INCIDENT_CSV_PATH = Path(__file__).resolve().parent.parent / "incident-data-3496.csv"
CLUSTER_DATA_DIR = Path(__file__).resolve().parent.parent / "data" / "clusters"

# Ensure cluster data directory exists
CLUSTER_DATA_DIR.mkdir(parents=True, exist_ok=True)


class SeizureLocation(BaseModel):
    lat: float
    lng: float
    name: str

class ShipmentRequest(BaseModel):
    transport_type: str
    seizure_location: SeizureLocation
    origin: str
    destination: str
    commodity_description: str
    weight: float
    package_type: str
    timestamp: str

class Reason(BaseModel):
    label: str
    detail: str
    confidence: float

class ScoreResponse(BaseModel):
    risk_score: int
    risk_tier: str
    reasons: List[Reason]


class MapNode(BaseModel):
    name: str
    lat: float
    lng: float


class ShipmentPath(BaseModel):
    id: int
    source: MapNode
    interception: MapNode
    # destination removed - no longer plotting
    timestamp: str
    commodity: str
    animal: str
    weight: float
    price: float
    package_type: str
    subject: str
    primary_source: str
    additional_sources: str


class ShipmentPathsResponse(BaseModel):
    paths: List[ShipmentPath]


class SourcePreviewResponse(BaseModel):
    url: str
    image_url: Optional[str]


class BrowserSession(BaseModel):
    id: str
    liveview_url: str


class BuildingData(BaseModel):
    lat: float
    lng: float
    risk_score: int
    explanation: str
    session_ids: Optional[List[str]] = None


class ClusterInvestigationData(BaseModel):
    cluster_id: str
    center_lat: float
    center_lng: float
    radius_meters: int
    commodity: Optional[str] = None
    timestamp: str
    buildings: List[BuildingData]
    total_buildings_checked: int


class ClusterSaveResponse(BaseModel):
    success: bool
    file_path: str
    cluster_id: str


def _safe_float(value: str, default: float = 0.0) -> float:
    """Safely parse a float value, returning default if invalid."""
    try:
        if not value or value.strip() == "":
            return default
        return float(value.strip())
    except (ValueError, AttributeError):
        return default


@lru_cache(maxsize=128)
def load_shipment_paths(max_rows: int = 1000) -> List[ShipmentPath]:
    """Load shipment paths from the wildlife trafficking incident data.

    Args:
        max_rows: Maximum number of rows to read from each CSV file (default 1000)
    """

    # First, load locations data grouped by Report ID
    locations_by_report: Dict[str, List[Dict[str, str]]] = defaultdict(list)

    with LOCATIONS_CSV_PATH.open(newline="", encoding="utf-8") as csv_file:
        reader = csv.DictReader(csv_file)
        for i, row in enumerate(reader):
            if i >= max_rows:
                break
            report_id = row["Report ID"].strip()
            locations_by_report[report_id].append(row)

    # Then load species data indexed by Report ID
    species_by_report: Dict[str, Dict[str, str]] = {}

    with SPECIES_CSV_PATH.open(newline="", encoding="utf-8") as csv_file:
        reader = csv.DictReader(csv_file)
        for i, row in enumerate(reader):
            if i >= max_rows:
                break
            report_id = row["Report ID"].strip()
            # Only keep the first entry for each report (since there can be multiple items per seizure)
            if report_id not in species_by_report:
                species_by_report[report_id] = row

    # Load incident data (with subject and source URLs) indexed by Report ID
    incident_by_report: Dict[str, Dict[str, str]] = {}

    with INCIDENT_CSV_PATH.open(newline="", encoding="utf-8") as csv_file:
        reader = csv.DictReader(csv_file)
        for i, row in enumerate(reader):
            if i >= max_rows:
                break
            report_id = row["Report ID"].strip()
            if report_id not in incident_by_report:
                incident_by_report[report_id] = row

    # Build shipment paths
    paths: List[ShipmentPath] = []
    path_id = 1

    for report_id, locations in locations_by_report.items():
        # Get species info for this report
        species_info = species_by_report.get(report_id)
        if not species_info:
            continue

        # Get incident info for this report (subject and sources)
        incident_info = incident_by_report.get(report_id)

        # Group locations by role - only collecting Origin and Discovery/Interception
        origin_loc: Optional[Dict[str, str]] = None
        transit_locs: List[Dict[str, str]] = []
        discovery_loc: Optional[Dict[str, str]] = None

        for loc in locations:
            role = loc.get("Role", "").strip()
            if role == "Origin Location":
                origin_loc = loc
            elif role == "Transit Location" or role == "Discovery Location":
                # Prefer Discovery Location as interception point
                if role == "Discovery Location":
                    discovery_loc = loc
                else:
                    transit_locs.append(loc)
            # Skip destination locations - we're not plott
[truncated — 17392 more characters]
```

### export_clusters_json.py

```python
#!/usr/bin/env python3
"""
Convert clustered CSV data to JSON format for frontend consumption.
"""

import pandas as pd
import numpy as np
import json

def export_clusters_to_json():
    """Export cluster data as JSON for the frontend."""

    # Load the clustered data
    df = pd.read_csv('clustered_commodities.csv')

    # Remove noise points (cluster = -1)
    df = df[df['cluster'] != -1]

    clusters = []

    # Group by commodity and cluster
    for (commodity, cluster_id), group in df.groupby(['commodity', 'cluster']):
        lats = group['Latitude'].values
        lons = group['Longitude'].values

        # Calculate cluster center
        center_lat = float(np.mean(lats))
        center_lon = float(np.mean(lons))

        # Calculate radius (max distance from center in meters)
        # Convert degrees to approximate meters (1 degree ≈ 111km)
        distances = np.sqrt((lats - center_lat)**2 + (lons - center_lon)**2) * 111000
        radius = float(np.max(distances))

        # Add some margin
        radius = radius * 1.3

        clusters.append({
            'commodity': commodity,
            'cluster_id': int(cluster_id),
            'center': {
                'lat': center_lat,
                'lng': center_lon
            },
            'radius': radius,  # in meters
            'incident_count': len(group),
            'incidents': group['Report ID'].tolist()
        })

    # Save as JSON
    output_data = {
        'clusters': clusters,
        'metadata': {
            'total_clusters': len(clusters),
            'total_incidents': len(df),
            'commodities': df['commodity'].unique().tolist(),
            'eps_km': 100,
            'min_samples': 5
        }
    }

    with open('frontend/public/clusters.json', 'w') as f:
        json.dump(output_data, f, indent=2)

    print(f"Exported {len(clusters)} clusters to frontend/public/clusters.json")

    # Print summary
    print("\nSummary by commodity:")
    for commodity in sorted(df['commodity'].unique()):
        commodity_clusters = [c for c in clusters if c['commodity'] == commodity]
        total_incidents = sum(c['incident_count'] for c in commodity_clusters)
        print(f"  {commodity:15s}: {len(commodity_clusters):3d} clusters, {total_incidents:5d} incidents")

if __name__ == '__main__':
    export_clusters_to_json()

```

### enrich_data.py

```python
import csv
import random

# Coordinates mapping (approximate lat, long)
coordinates = {
    # Interception Locations
    "Mombasa Port": (-4.0435, 39.6682),
    "Dubbi Warehouse": (25.276987, 55.296249), # Assuming Dubai
    "Hong Kong Airport": (22.3193, 114.1694),
    "Entebbe Staging": (0.0512, 32.4637),
    "Singapore Terminal": (1.3521, 103.8198),
    "Lagos Port": (6.5244, 3.3792),
    "Bangkok Transit": (13.7563, 100.5018),
    "Kampala Outskirts": (0.3476, 32.5825),
    "Dubai Free Zone": (25.276987, 55.296249),
    "Beira Port": (-19.8316, 34.8370),
    "Sihanoukville Port": (10.6253, 103.5234),
    "Dar es Salaam": (-6.7924, 39.2083),
    "Nairobi Airport": (-1.2921, 36.8219),
    "Hanoi Suburb": (21.0285, 105.8542),
    "Johannesburg Port": (-26.2041, 28.0473),
    
    # Destinations
    "Guangzhou": (23.1291, 113.2644),
    "Hanoi": (21.0285, 105.8542),
    "Shanghai": (31.2304, 121.4737),
    "Beijing": (39.9042, 116.4074),
    "Busan": (35.1796, 129.0756),
    "Shenzhen": (22.5431, 114.0579),
    "Chonburi": (13.3611, 100.9847),
    "Hong Kong": (22.3193, 114.1694),
    "Ho Chi Minh": (10.8231, 106.6297),
    "Nanning": (22.8170, 108.3665),
    "Phnom Penh": (11.5564, 104.9282),

    # Arbitrary Sources
    "Port of Santos": (-23.9618, -46.3322),
    "Port of Durban": (-29.8587, 31.0218),
    "Port of Rotterdam": (51.9244, 4.4777),
    "Port of Los Angeles": (33.7405, -118.2786),
    "Jebel Ali Port": (24.9857, 55.0273),
    "Qingdao Port": (36.0671, 120.3826),
    "Tianjin Port": (39.3434, 117.3632),
    "Port of Tokyo": (35.6895, 139.6917),
    "Port of Hamburg": (53.5511, 9.9937),
    "Port of Antwerp": (51.2194, 4.4025),
    "Port of Valencia": (39.4699, -0.3763),
    "Port of Algeciras": (36.1408, -5.4562),
    "Port of Colombo": (6.9271, 79.8612),
    "Port of Salalah": (17.0151, 54.0924),
    "Port of Tanjung Pelepas": (1.3667, 103.5500)
}

source_ports = [
    "Port of Santos", "Port of Durban", "Port of Rotterdam", "Port of Los Angeles",
    "Jebel Ali Port", "Qingdao Port", "Tianjin Port", "Port of Tokyo", 
    "Port of Hamburg", "Port of Antwerp", "Port of Valencia", "Port of Algeciras",
    "Port of Colombo", "Port of Salalah", "Port of Tanjung Pelepas"
]

def get_coords(location_name):
    if location_name not in coordinates:
        print(f"Warning: Coordinates not found for '{location_name}'")
    return coordinates.get(location_name, (0.0, 0.0))

input_file = 'samp_data.csv'
output_file = 'samp_data_enriched.csv'

with open(input_file, mode='r', newline='', encoding='utf-8') as infile:
    reader = csv.DictReader(infile)
    fieldnames = reader.fieldnames
    
    # Add new columns
    new_fieldnames = fieldnames + [
        'Source Lat', 'Source Long',
        'Interception Lat', 'Interception Long',
        'Destination Lat', 'Destination Long'
    ]
    
    rows = []
    for row in reader:
        # Fill Source if empty
        if not row['Source']:
            row['Source'] = random.choice(source_ports)
        
        # Get coordinates
        source_lat, source_long = get_coords(row['Source'])
        interception_lat, interception_long = get_coords(row['Interception Location'])
        destination_lat, destination_long = get_coords(row['Destination'])
        
        row['Source Lat'] = source_lat
        row['Source Long'] = source_long
        row['Interception Lat'] = interception_lat
        row['Interception Long'] = interception_long
        row['Destination Lat'] = destination_lat
        row['Destination Long'] = destination_long
        
        rows.append(row)

with open(output_file, mode='w', newline='', encoding='utf-8') as outfile:
    writer = csv.DictWriter(outfile, fieldnames=new_fieldnames)
    writer.writeheader()
    writer.writerows(rows)

print(f"Enriched data written to {output_file}")

```

### cluster_commodities.py

```python
#!/usr/bin/env python3
"""
Cluster geographically nearby incident points of the same commodity using DBSCAN.
Plot circles around each cluster.
"""

import pandas as pd
import numpy as np
from sklearn.cluster import DBSCAN
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
import re
from collections import defaultdict

def extract_commodity(subject):
    """Extract commodity type from the subject description."""
    subject_lower = str(subject).lower()

    # Define commodity keywords (order matters - more specific first)
    commodities = [
        ('elephant', ['elephant', 'tusk']),
        ('ivory', ['ivory']),
        ('rhino', ['rhino', 'rhinoceros']),
        ('pangolin', ['pangolin']),
        ('abalone', ['abalone', 'perlemoen']),
        ('tiger', ['tiger']),
        ('leopard', ['leopard']),
        ('tortoise', ['tortoise', 'turtle']),
        ('wood', ['sandalwood', 'timber', 'wood', 'logs', 'rosewood', 'teak']),
        ('hippo', ['hippo', 'hippopotamus']),
        ('lion', ['lion']),
        ('bear', ['bear']),
        ('snake', ['snake', 'python', 'cobra']),
        ('crocodile', ['crocodile', 'alligator']),
        ('bird', ['bird', 'parrot', 'eagle', 'owl', 'falcon']),
        ('fish', ['fish', 'shark', 'seahorse']),
        ('mammal', ['mammal', 'mongoose', 'otter']),
        ('reptile', ['reptile', 'lizard']),
    ]

    for commodity_name, keywords in commodities:
        for keyword in keywords:
            if keyword in subject_lower:
                return commodity_name

    return 'other'

def cluster_by_commodity(df, eps_km=50, min_samples=3):
    """
    Cluster incidents by commodity type and geographic proximity.

    Parameters:
    - eps_km: radius in kilometers for DBSCAN clustering
    - min_samples: minimum points to form a cluster
    """
    # Extract commodity for each row
    df['commodity'] = df['Subject'].apply(extract_commodity)

    # Remove 'other' and keep only valid coordinates
    df = df[df['commodity'] != 'other'].dropna(subset=['Latitude', 'Longitude'])

    print(f"\nTotal incidents with identified commodities: {len(df)}")
    print("\nCommodity distribution:")
    print(df['commodity'].value_counts())

    # Store clustering results
    all_clusters = []

    # Cluster each commodity separately
    for commodity in df['commodity'].unique():
        commodity_df = df[df['commodity'] == commodity].copy()

        if len(commodity_df) < min_samples:
            print(f"\nSkipping {commodity}: only {len(commodity_df)} incidents (< {min_samples})")
            continue

        print(f"\nClustering {commodity}: {len(commodity_df)} incidents")

        # Extract coordinates
        coords = commodity_df[['Latitude', 'Longitude']].values

        # Convert eps from km to approximate degrees
        # 1 degree latitude ≈ 111 km
        # For longitude, it varies by latitude, but we'll use 111 km as approximation
        eps_degrees = eps_km / 111.0

        # Apply DBSCAN
        clustering = DBSCAN(eps=eps_degrees, min_samples=min_samples, metric='euclidean')
        labels = clustering.fit_predict(coords)

        # Add labels to dataframe
        commodity_df['cluster'] = labels

        # Count clusters (excluding noise points with label -1)
        n_clusters = len(set(labels)) - (1 if -1 in labels else 0)
        n_noise = list(labels).count(-1)

        print(f"  Found {n_clusters} clusters, {n_noise} noise points")

        # Store results
        all_clusters.append(commodity_df)

    # Combine all results
    if all_clusters:
        result_df = pd.concat(all_clusters, ignore_index=True)
        return result_df
    else:
        return pd.DataFrame()

def plot_clusters(df, output_file='commodity_clusters.png'):
    """Plot clusters with circles around each group."""

    if df.empty:
        print("No clusters to plot!")
        return

    # Create figure
    fig, ax = plt.subplots(figsize=(20, 12))

    # Define colors for different commodities
    commodities = df['commodity'].unique()
    colors = plt.cm.tab20(np.linspace(0, 1, len(commodities)))
    commodity_colors = dict(zip(commodities, colors))

    # Plot each commodity
    for commodity in commodities:
        commodity_df = df[df['commodity'] == commodity]

        # Get unique clusters for this commodity (excluding noise)
        clusters = commodity_df[commodity_df['cluster'] != -1]['cluster'].unique()

        for cluster_id in clusters:
            cluster_points = commodity_df[commodity_df['cluster'] == cluster_id]

            # Get coordinates
            lats = cluster_points['Latitude'].values
            lons = cluster_points['Longitude'].values

            # Plot points
            ax.scatter(lons, lats,
                      c=[commodity_colors[commodity]],
                      s=50,
                      alpha=0.6,
                      label=f'{commodity} (cluster {cluster_id})' if cluster_id == clusters[0] else '',
                      edgecolors='black',
                      linewidth=0.5)

            # Calculate cluster center and radius
            center_lon = np.mean(lons)
            center_lat = np.mean(lats)

            # Calculate radius as max distance from center
            distances = np.sqrt((lons - center_lon)**2 + (lats - center_lat)**2)
            radius = np.max(distances) * 1.2  # Add 20% margin

            # Draw circle around cluster
            circle = plt.Circle((center_lon, center_lat),
                              radius,
                              color=commodity_colors[commodity],
                              fill=False,
                              linewidth=2,
                              linestyle='--',
                              alpha=0.7)
            ax.add_patch(circle)

            # Add label at cluster center
            ax.annotate(f'{commodity}\n({len(cluster_points)} pts)',
                       xy=(center_lon, center_lat),
                       fontsize=8,
             
[truncated — 2625 more characters]
```

[98 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]