# Project export: MotionCast This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: MotionCast turns any team's game film into pro-grade player + ball tracking — soccer, basketball, or football. Upload, pick your sport, and see the game mapped, accuracy proven on pro data. - Devpost: https://devpost.com/software/motioncast - GitHub: https://github.com/wheredawoodat949/AI-Hackathon - Team: 5 GitHub contributor(s) — Ashmeet Singh (22 commits), Claude Opus 4.8 (10 commits), Vincent Mendy (3 commits), Cursor (1 commits), wheredawoodat949 (1 commits) ## Devpost submission (written by the team) ### Overview Pro-grade game-film analysis across sports — for the teams that could never afford it. ### Inspiration Team sports are played everywhere, but tactical analysis is a luxury — pro clubs pay six figures for analytics platforms and analysts. Amateur, college, and youth teams film every game, then that footage just sits there. The teams with the most to learn have the least access to the tools that teach it. We set out to prove the foundational layer of sports analytics — knowing where every player is, every frame — could be built for free, shown to be correct, and work across sports. ### What it does MotionCast is an analyst's workspace for game film, built for multiple sports. You upload match video, pick your sport — soccer, basketball, or American football — and it automatically detects and tracks every player, plus the ball, casting them onto a 2D top-down view. It adds team differentiation, ball possession, and motion trails, then surfaces everything in a club dashboard (xG, win probability, pressing intensity, matchups). We've tracked real footage across all three sports with the same pipeline. Accuracy is rigorously validated on soccer: we score our tracking against professional ground-truth annotations with a real HOTA number, so the data feeding every view is measurably trustworthy. What's real today: the React frontend, multi-sport player + ball tracking (soccer, basketball, football), team/possession/trails, and HOTA-validated accuracy on soccer. The dashboard's predictive models (xG, win-prob) are the next layer on top of the tracking we've proven. ### How we built it Frontend: a React/Vite analyst app — landing → guided film upload (file → sport → game details → review) → dashboard, in a dark lime-accent theme. Sport selection drives the labels and models. Tracking: built on SAM 3.1 (HF Transformers) for segmentation/tracking, plus a Roboflow-sports-based pipeline for ball tracking, team differentiation (color clustering), possession, and motion trails. The same model tracks people and the ball regardless of sport — which is why basketball and football work without sport-specific rewrites. Swappable backend: everything depends only on an abstract SamBackend interface, with a hosted SAM path and a no-GPU GSR-replay backend that runs the whole pipeline on a laptop and acts as a "perfect tracker" upper bound ($\text{HOTA}\approx 1.0$) to sanity-check the eval. Data + detector validation: a self-contained reader for SoccerTrack v2's GSR annotations (COCO, ~2.6 GB/match) and a GSR → YOLOv5 converter + Colab run to prove the data was sound, with labels normalized as $x_c=\frac{x+w/2}{W},\ y_c=\frac{y+h/2}{H}$ and panorama-aware augmentation. Eval: we never reimplement metrics — we shell out to the official GSR-HOTA scorer, where $\text{HOTA}=\sqrt{\text{DetA}\cdot\text{AssA}}$ balances detection against identity consistency over time. ### Challenges we ran into A coordinate-space rabbit hole: GSR boxes floated in the sky above the pitch. The cause wasn't math — panorama_2nd is a substring of calibrated_panorama_2nd, so we were silently drawing correct boxes on the wrong (calibrated) video. Fixed with an explicit exclude filter. Drive download quotas on the multi-GB files — worked around by mounting our own Drive in Colab. No GPU, huge 4K video, ephemeral Colab — handled with deferred imports, strided sampling, a hosted backend, and the no-GPU replay path. ### Accomplishments we're proud of A polished React analyst UI; multi-sport tracking demonstrated across soccer, basketball, and football with one pipeline; ball tracking, team differentiation, possession, and trails; HOTA-validated accuracy with an eval harness that never fabricates a number; and a subtle data bug debugged honestly instead of papered over. ### What we learned Tracking is the hard, foundational part of analytics — and proving it with HOTA matters more than a shallow feature. A well-chosen tracking foundation generalizes: the same model that tracks a soccer pitch tracks a basketball court and a football field. Validate the data before the model — our overlay checks caught a coordinate bug that would've trained YOLO on grass and sky. And $\text{HOTA}=\sqrt{\text{DetA}\cdot\text{AssA}}$ forced us to care about identity over time, not just per-frame detection — the association problem every sport shares. ### What's next Wire the dashboard's predictive models (xG, win probability, pressing) onto the tracked coordinates we already produce; ground-truth accuracy eval for basketball and football (soccer has it today); a calibrated homography minimap; and event spotting for auto-highlights — all toward one goal: pro-grade film analysis any team, in any sport, can run on their own footage, free. ## README (from the GitHub repository) # AI-Hackathon — Soccer Game-State Analysis (SAM 3.1 → Minimap → HOTA) Full-pitch panoramic match video → **Meta SAM 3.1** segments + tracks every player/goalkeeper/referee → live **2D tactical minimap** → evaluated against **SoccerTrack v2** GSR ground truth (real HOTA, never fabricated). Instrumented for reliability (Sentry), eval/observability (Arize), and semantic search (Redis). > Mission framing: accessible tactical analysis for amateur/university teams who > can't afford pro analytics — the dataset is amateur matches. See [`CLAUDE.md`](CLAUDE.md) for the full architecture, dataset facts, phase ordering, and working agreement. See [`PROGRESS.md`](PROGRESS.md) for live status. ## Quickstart ```bash make setup # venv + editable install (.venv) with dev tools source .venv/bin/activate make test # ruff + pytest (no GPU/data needed) — should be green make gpu # CUDA check (fails loud if no GPU) make frame MATCH=117093 # download from Drive mirror + print a real GSR frame ``` Or the explicit commands: `pip install -r requirements.txt`, then `python -m src.data.download --match 117093 [--no-videos]` and `python -m src.data.inspect --match 117093`. Data comes from the link-public **Google Drive mirror** by default (no auth); use `--source hf` for the gated HF copy (`HF_TOKEN`). `notebooks/demo.ipynb` imports from `src/` and holds no logic. **Compute / GPUs:** see [docs/COMPUTE.md](docs/COMPUTE.md). **Team & branches:** see CLAUDE.md §10. ## Layout ``` src/ config.py # single source of truth (config.yaml + .env) data/ # download.py (one-match HF) + loader.py (GSR/BAS reader) + inspect.py utils/gpu.py # CUDA check, fails loud model/ # SAM 3.1 backend abstraction (local | api) — Phase 1 tracking/ pitch/ events/ eval/ obs/ store/ # per-phase modules pipeline.py # end-to-end orchestration entrypoint notebooks/demo.ipynb # the thing we run for judges outputs/ frontend/ # gitignored artifacts / web app (last phase) ``` All reusable logic lives in `src/`; notebooks and the frontend import it. Heavy artifacts (videos, weights, `data/`, `outputs/`) are gitignored and stay local. ## Dataset [SoccerTrack v2](https://github.com/AtomScott/SoccerTrack-v2) · [docs](https://atomscott.github.io/SoccerTrack-v2/) · [Hugging Face](https://huggingface.co/datasets/atomscott/soccertrack-v2) · [paper](https://arxiv.org/abs/2508.01802). - Dev match **117093**. Real mirror match IDs (verified by listing it; the docs' assumed 117091–117100 don't match the files): `117092, 117093, 118575, 118576, 118577, 118578, 128057, 128058, 132831, 132877`. Split 80/10/10: train (8) · eval `132831` · test `132877`. - Mirror carries `gsr/ bas/ raw/ videos/` (no `mot/`) → eval with GSR HOTA (`python -m src.evaluation.gs_hota …`); we do **not** reimplement metrics. **Attribution:** SoccerTrack v2 is licensed **CC BY 4.0** (A. Scott et al.). Dataset code is MIT. No player names — IDs are jersey-number based. ## Sponsors (each toggleable in `config.yaml`, none on the critical path) Sentry (reliability) · Arize (eval/observability) · Redis (vector search) · Anthropic / Claude Code (build layer). ## Detected evidence (automated analysis) Indexed codebase: 87 recognized source files, 374 KB. - HTML (language) — detected in the code - Hugging Face (technology) — detected in the code - Python (language) — detected in the code - PyTorch (technology) — detected in the code - Redis (technology) — detected in the code - Node.js (technology) — claimed on Devpost, not found in the code - React (technology) — claimed on Devpost, not found in the code - TypeScript (language) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers - AI coding agent: Cursor — evidence: commit authorship or trailers ## Codebase structure (from repository index) ### Files (107 of 107) ``` .env.example .gitignore AGENT_TASKS.md Basketball_1.ipynb Basketball_Training.ipynb CLAUDE.md config.yaml data.yaml docs/COMPUTE.md docs/DEFERRED.md docs/ML_DIRECTIONS.md docs/PIKA.md docs/TRACKING_HEALTH.md docs/TRAINING_BASKETBALL.md frontend/.gitkeep frontend/index.html Makefile notebooks/colab_sam_tracking.ipynb notebooks/demo.ipynb ONBOARDING.md PROGRESS.md pyproject.toml README.md requirements.txt segment_videos_with_segment_anything_3.ipynb Soccer_1.ipynb sports/.gitignore sports/examples/basketball/main.py sports/examples/basketball/README.md sports/examples/basketball/requirements.txt sports/examples/flag_football/main.py sports/examples/flag_football/README.md sports/examples/flag_football/requirements.txt sports/examples/soccer/main.py sports/examples/soccer/notebooks/train_ball_detector.ipynb sports/examples/soccer/notebooks/train_pitch_keypoint_detector.ipynb sports/examples/soccer/notebooks/train_player_detector.ipynb sports/examples/soccer/README.md sports/examples/soccer/requirements.txt sports/examples/soccer/setup.sh sports/LICENSE sports/README.md sports/setup.py sports/sports/__init__.py sports/sports/annotators/__init__.py sports/sports/annotators/soccer.py sports/sports/common/__init__.py sports/sports/common/ball.py sports/sports/common/possession.py sports/sports/common/team.py sports/sports/common/trace.py sports/sports/common/view.py sports/sports/configs/__init__.py sports/sports/configs/soccer.py src/__init__.py src/analysis/__init__.py src/analysis/tracking_health.py src/config.py src/data/__init__.py src/data/download.py src/data/inspect.py src/data/loader.py src/data/video.py src/eval/__init__.py src/eval/hota.py src/events/__init__.py src/events/bas.py src/integrations/__init__.py src/integrations/tracking_observer.py src/model/__init__.py src/model/replay.py src/model/sam_api.py src/model/sam_backend.py src/model/sam_local.py src/obs/__init__.py src/obs/arize.py src/obs/sentry.py src/pipeline.py src/pitch/__init__.py src/pitch/homography.py src/pitch/minimap.py src/store/__init__.py src/store/redis_store.py src/synthetic/__init__.py src/synthetic/pika.py src/tracking/__init__.py src/tracking/demo.py src/tracking/tracker.py src/tracking/visualize.py src/training/__init__.py src/training/basketball.py src/utils/__init__.py src/utils/gpu.py tests/__init__.py tests/test_arize.py tests/test_basketball_training.py tests/test_config.py tests/test_gpu.py tests/test_imports.py tests/test_loader.py tests/test_pika.py tests/test_redis_store.py tests/test_sports_analytics.py tests/test_tracking_health.py tests/test_tracking_observer.py tests/test_tracking.py UPDATE.md ``` ### Dependencies - pyproject.toml: gdown@>=5.0, huggingface_hub@>=0.24, numpy@>=1.26, pytest@>=8.0, python-dotenv@>=1.0, PyYAML@>=6.0, ruff@>=0.5 - requirements.txt: accelerate, arize@>=8.35,<9, fal-client@>=0.5, gdown@>=5.0, huggingface_hub@>=0.24, jupyterlab@>=4.0, numpy@>=1.26, opencv-python@>=4.9, pillow@>=10.0, python-dotenv@>=1.0, PyYAML@>=6.0, redis@>=5.0, requests@>=2.31, roboflow@>=1.3,<2, scipy@>=1.11, sentry-sdk@>=2.0, torch@>=2.2, transformers - sports/examples/basketball/requirements.txt: kagglehub, ultralytics - sports/examples/flag_football/requirements.txt: kagglehub, ultralytics - sports/examples/soccer/requirements.txt: gdown, ultralytics ### Recent commits (newest first) - Merge pull request #8 from wheredawoodat949/basketball - Flag football demo (verified, real footage) + minimal frontend - Add evidence-backed tracking health agent - Prepare reproducible basketball training - Add basketball possession demo mode - Wire basketball tracking to Redis and Arize - Merge pull request #7 from wheredawoodat949/basketball - Implement Pika synthetic media workflow - Implement Arize tracking telemetry - Implement Redis live track state - Log Codex handoff prompt finalization in UPDATE.md - Phase 1+2: basketball Path-A pipeline (verified) + Path-B data scaffold; multi-agent handoff - Merge pull request #6 from wheredawoodat949/basketball - Phase 0: orientation for basketball pivot — corrected model facts, plan - Add sports module (team differentiation) as standalone folder - Created using Colab - Created using Colab - Merge pull request #4 from wheredawoodat949/feat/tracking-ashmeet - Fix SAM3 'Can't load image processor': install transformers from git main - sam_local.py: switch to the official HF Transformers SAM3 path (not Ultralytics) ## Key source files (fetched from GitHub, selected and truncated for size) ### PROGRESS.md ```markdown # PROGRESS Running status of what works, what's stubbed, and the current demo command. Update at the end of every phase (CLAUDE.md §7). ## Repo health check (2026-06-20) `git fsck --full --strict` clean on both this repo and the `_reference_soccertrack` clone; no files >5MB ever committed; `.gitignore` correctly excludes `data/`, `outputs/`, `.venv/`, `__pycache__/`. **One real issue found and fixed:** `feat/sponsors-vincent`'s merge commit (`c98204c`) had left literal unresolved `<<<<<<<`/`=======`/`>>>>>>>` conflict markers committed into `src/model/{__init__,sam_api,sam_backend,sam_local}.py` (invalid Python — broke every import in that package). Vincent independently fixed it (`9b084c2`) before a parallel fix landed here; verified clean (no markers anywhere across all 5 refs, his resolution lints + tests green). All branches now import-clean. ## Current demo command (Drive mirror — no auth needed) ```bash python -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt python -m src.utils.gpu # CUDA check (fails loud if no GPU) python -m src.data.download --match 117093 --no-videos # Drive mirror, annotations only (fast) python -m src.data.inspect --match 117093 --half 1 # print one real GSR frame # Phase 1 (needs video): python -m src.data.download --match 117093 (adds the panorama mp4s) ``` ## Data sources (verified 2026-06-20 by listing the mirror) - **Drive mirror `1N2Qx2qkFgRtpbHitl2Vh6sLVYGgqkWwn`** — DEFAULT (`dataset.source: drive`). Official mirror (per repo docs), **link-public** → `gdown` lists+fetches it with no auth. Downloader filters to ONE match (skips videos with `--no-videos`). Contains `gsr/ bas/ raw/ videos/`. - **HF `atomscott/soccertrack-v2`** — canonical but **GATED** (401 without token). Kept as `--source hf` alternative; needs `hf auth login` / `HF_TOKEN`. - stc2025 challenge Drive (`1_o78gcL4…`) = competition subset, not used. ## Real match IDs (from the mirror — NOT the docs' assumed 117091–117100) `117092, 117093, 118575, 118576, 118577, 118578, 128057, 128058, 132831, 132877` (10 matches, each with gsr+bas+raw+videos). **No `mot/` in the mirror** → use GSR HOTA (`gs_hota`), not MOT HOTA. ## Phase 0 — Foundation ✅ scaffolded (verify on GPU box) - [x] Repo layout per CLAUDE.md §3 (`src/` packages, `notebooks/`, `outputs/`, `frontend/`). - [x] `.gitignore` (data/videos/weights/.env), `.env.example`, `config.yaml`, `requirements.txt`. - [x] `src/config.py` — single source of truth (paths, dev_match, 80/10/10 split, SAM backend, sponsor flags). - [x] `src/utils/gpu.py` — CUDA check, **fails loud** if no GPU (ALLOW_CPU=1 escape hatch for pure-Python smoke tests). - [x] `src/data/download.py` — one-match HF download (mirrors dataset `download.sh`, `--no-videos` fast path). - [x] `src/data/loader.py` — self-contained GSR/BAS reader matching the verified on-disk schema. - [x] `src/data/inspect.py` — prints one GSR frame's entities. - [ ] **Verif [truncated — 5311 more characters] ``` ### AGENT_TASKS.md ```markdown # AGENT TASKS — multi-agent coordination (Claude = lead, Codex 5.5 Max = secondary) **If you are Codex (or any agent) picking this up: read this file fully, then `CLAUDE.md` and `ONBOARDING.md`, then `UPDATE.md`'s most recent entries, in that order, before touching anything.** You're on branch `basketball`. `git pull` before you start and before every push — Claude (lead) and the user are also pushing to this branch. Phase list extended from CLAUDE.md's original 0–5 to 0–8 (the user asked for "all the way to phase 8"; 6–8 are a proposed extension, not yet confirmed — flag to the user if you think these are wrong): | # | Phase | Owner | Status | |---|---|---|---| | 0 | Orientation & ground truth | Claude | ✅ done | | 1 | Basketball demo video (Path A) | Claude | ✅ code done, GPU validation pending | | 2 | Basketball detection data (Path B prep) | Claude | ✅ scaffolded, download pending | | 3 | Train basketball model | **Codex** | 🟡 reproducible code/Colab ready; credentials + GPU run pending | | 4 | Sponsor wiring (Redis, Pika Labs, Arize) | **Codex** | 🟡 wired/tested offline; hosted validation pending | | 5 | Agentic layer + polish | **Codex** (+ Claude reviews) | 🟡 evidence-backed health agent ready; real-signal review pending | | 6 | Cross-sport polish (soccer + basketball demo parity) | **Codex** | 🔲 not started | | 7 | Devpost writeup + pitch prep | **Codex** (+ user content) | 🔲 not started | | 8 | Final QA + submission rehearsal | shared | 🔲 not started | --- ## What's already done (Phase 0–2 — read before redoing any of this) - **`sports/examples/basketball/main.py`** — Path A pipeline. One generic COCO detector (`yolo11n.pt`, ungated) + ByteTrack + `sports.common.team.TeamClassifier` (unmodified Roboflow code) + `sports.common.ball.BallTracker`. Modes: `PLAYER_DETECTION`, `BALL_DETECTION`, `PLAYER_TRACKING`, `TEAM_CLASSIFICATION`. See its own docstring + `README.md` next to it for exactly what's verified vs not. - **Verified locally (Mac, CPU, synthetic test clip):** `PLAYER_TRACKING`/`PLAYER_DETECTION`/ `BALL_DETECTION` run end-to-end, produce valid annotated video. `TEAM_CLASSIFICATION` hit a SIGSEGV in Roboflow's own `TeamClassifier.fit()` (UMAP/numba) on a **degenerate 1-frame-repeated test clip** — almost certainly a tiny-sample artifact, not a real bug, but **NOT YET confirmed on real multi-frame footage**. First thing to check if you get GPU/Colab access. - **`Basketball_1.ipynb`** (repo root) — the Colab notebook to actually run Phase 1 for real: GPU check → clone this repo → install → `kagglehub.dataset_download(...)` for Basketball-51 → pick a clip → run all 4 modes → render+preview. **Needs the user's Kaggle auth in Colab** — no Kaggle credentials exist outside Colab. If you have your own way to get Basketball-51 (or to run this notebook with GPU access), use it; otherwise this step may need the user to run it and report back the actual output (frame counts, any errors) in `UPDATE.md`. - ** [truncated — 6205 more characters] ``` ### pyproject.toml ``` [build-system] requires = ["setuptools>=68"] build-backend = "setuptools.build_meta" [project] name = "soccertrack-pipeline" version = "0.1.0" description = "SAM 3.1 soccer game-state analysis on SoccerTrack v2 (tracking -> minimap -> HOTA)" requires-python = ">=3.11" # Runtime deps live in requirements.txt (pinned for the GPU box). Keep the # editable-install lighter: just what's needed to import + run Phase 0 tooling. dependencies = [ "PyYAML>=6.0", "python-dotenv>=1.0", "huggingface_hub>=0.24", "gdown>=5.0", "numpy>=1.26", ] [project.optional-dependencies] dev = [ "pytest>=8.0", "ruff>=0.5", ] [tool.setuptools.packages.find] where = ["."] include = ["src*"] [tool.pytest.ini_options] testpaths = ["tests"] addopts = "-q" [tool.ruff] line-length = 100 target-version = "py311" [tool.ruff.lint] # Pragmatic for a hackathon: catch real errors, don't nag on style. select = ["E", "F", "I", "W"] ignore = ["E501"] # long lines are fine; docstrings explain a lot ``` ### requirements.txt ``` # Python 3.11+. Install in a venv: python -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt # Versions pinned with lower bounds; lock exact versions on the GPU box once verified. # --- Phase 0: foundation (data + config + GPU check) --- huggingface_hub>=0.24 # canonical (gated) source gdown>=5.0 # Google Drive mirror fallback (no HF gating) python-dotenv>=1.0 PyYAML>=6.0 numpy>=1.26 # Install the CUDA build of torch on the GPU box per https://pytorch.org/get-started/locally/ torch>=2.2 # --- Phase 1-2: tracking, video, pitch, minimap --- opencv-python>=4.9 pillow>=10.0 scipy>=1.11 # requests used by the hosted-SAM (api) backend requests>=2.31 # fal-client: deferred no-GPU SAM 3.1 path (config.yaml sam.backend: api), see # docs/DEFERRED.md. Needs FAL_KEY in .env (sign up at fal.ai). fal-client>=0.5 # transformers + accelerate: self-hosted SAM 3.1 (sam.backend: local), e.g. on a # Colab T4. Gated weights (facebook/sam3) auto-download via from_pretrained() # once HF access is approved + authenticated. # IMPORTANT: SAM3 is NOT in any stable transformers release yet (merged to main # 2025-11-19). The stable wheel is MISSING Sam3ImageProcessor -> "Can't load image # processor". Install from git main: # pip install -U "git+https://github.com/huggingface/transformers" # (left unpinned here so a normal `pip install -r` doesn't force a slow git build for # people not using the local backend; the Colab notebook installs main explicitly.) transformers accelerate # --- Phase 2: eval (uses the SoccerTrack-v2 package's HOTA — do NOT reimplement) --- # On the GPU box, also have the dataset repo importable: # pip install git+https://github.com/AtomScott/SoccerTrack-v2 # so `python -m src.evaluation.gs_hota ...` is available for HOTA/MOT/BAS scoring. # --- Phase 3: sponsors (optional; only needed when toggled on in config.yaml) --- sentry-sdk>=2.0 arize>=8.35,<9 redis>=5.0 # --- Basketball Path-B dataset download/training preparation --- roboflow>=1.3,<2 # --- Dev / notebook --- jupyterlab>=4.0 ``` ### sports/examples/soccer/requirements.txt ``` ultralytics gdown ``` ### sports/examples/basketball/requirements.txt ``` ultralytics kagglehub ``` ### sports/examples/flag_football/requirements.txt ``` ultralytics kagglehub ``` ### sports/examples/soccer/main.py ```python import argparse from enum import Enum from typing import Iterator, List import os import cv2 import numpy as np import supervision as sv from tqdm import tqdm from ultralytics import YOLO from sports.annotators.soccer import draw_pitch, draw_points_on_pitch from sports.common.ball import BallTracker, BallAnnotator from sports.common.team import TeamClassifier from sports.common.view import ViewTransformer from sports.configs.soccer import SoccerPitchConfiguration PARENT_DIR = os.path.dirname(os.path.abspath(__file__)) PLAYER_DETECTION_MODEL_PATH = os.path.join(PARENT_DIR, 'data/football-player-detection.pt') PITCH_DETECTION_MODEL_PATH = os.path.join(PARENT_DIR, 'data/football-pitch-detection.pt') BALL_DETECTION_MODEL_PATH = os.path.join(PARENT_DIR, 'data/football-ball-detection.pt') BALL_CLASS_ID = 0 GOALKEEPER_CLASS_ID = 1 PLAYER_CLASS_ID = 2 REFEREE_CLASS_ID = 3 STRIDE = 60 CONFIG = SoccerPitchConfiguration() COLORS = ['#FF1493', '#00BFFF', '#FF6347', '#FFD700'] VERTEX_LABEL_ANNOTATOR = sv.VertexLabelAnnotator( color=[sv.Color.from_hex(color) for color in CONFIG.colors], text_color=sv.Color.from_hex('#FFFFFF'), border_radius=5, text_thickness=1, text_scale=0.5, text_padding=5, ) EDGE_ANNOTATOR = sv.EdgeAnnotator( color=sv.Color.from_hex('#FF1493'), thickness=2, edges=CONFIG.edges, ) TRIANGLE_ANNOTATOR = sv.TriangleAnnotator( color=sv.Color.from_hex('#FF1493'), base=20, height=15, ) BOX_ANNOTATOR = sv.BoxAnnotator( color=sv.ColorPalette.from_hex(COLORS), thickness=2 ) ELLIPSE_ANNOTATOR = sv.EllipseAnnotator( color=sv.ColorPalette.from_hex(COLORS), thickness=2 ) BOX_LABEL_ANNOTATOR = sv.LabelAnnotator( color=sv.ColorPalette.from_hex(COLORS), text_color=sv.Color.from_hex('#FFFFFF'), text_padding=5, text_thickness=1, ) ELLIPSE_LABEL_ANNOTATOR = sv.LabelAnnotator( color=sv.ColorPalette.from_hex(COLORS), text_color=sv.Color.from_hex('#FFFFFF'), text_padding=5, text_thickness=1, text_position=sv.Position.BOTTOM_CENTER, ) class Mode(Enum): """ Enum class representing different modes of operation for Soccer AI video analysis. """ PITCH_DETECTION = 'PITCH_DETECTION' PLAYER_DETECTION = 'PLAYER_DETECTION' BALL_DETECTION = 'BALL_DETECTION' PLAYER_TRACKING = 'PLAYER_TRACKING' TEAM_CLASSIFICATION = 'TEAM_CLASSIFICATION' RADAR = 'RADAR' def get_crops(frame: np.ndarray, detections: sv.Detections) -> List[np.ndarray]: """ Extract crops from the frame based on detected bounding boxes. Args: frame (np.ndarray): The frame from which to extract crops. detections (sv.Detections): Detected objects with bounding boxes. Returns: List[np.ndarray]: List of cropped images. """ return [sv.crop_image(frame, xyxy) for xyxy in detections.xyxy] def resolve_goalkeepers_team_id( players: sv.Detections, players_team_id: np.array, goalkeepers: sv.Detections ) -> np.ndarray: """ Resolve the team IDs for detected goalkeepers based on the proximity to team centroids. Args: players (sv.Detections): Detections of all players. players_team_id (np.array): Array containing team IDs of detected players. goalkeepers (sv.Detections): Detections of goalkeepers. Returns: np.ndarray: Array containing team IDs for the detected goalkeepers. This function calculates the centroids of the two teams based on the positions of the players. Then, it assigns each goalkeeper to the nearest team's centroid by calculating the distance between each goalkeeper and the centroids of the two teams. """ goalkeepers_xy = goalkeepers.get_anchors_coordinates(sv.Position.BOTTOM_CENTER) players_xy = players.get_anchors_coordinates(sv.Position.BOTTOM_CENTER) team_0_centroid = players_xy[players_team_id == 0].mean(axis=0) team_1_centroid = players_xy[players_team_id == 1].mean(axis=0) goalkeepers_team_id = [] for goalkeeper_xy in goalkeepers_xy: dist_0 = np.linalg.norm(goalkeeper_xy - team_0_centroid) dist_1 = np.linalg.norm(goalkeeper_xy - team_1_centroid) goalkeepers_team_id.append(0 if dist_0 < dist_1 else 1) return np.array(goalkeepers_team_id) def render_radar( detections: sv.Detections, keypoints: sv.KeyPoints, color_lookup: np.ndarray ) -> np.ndarray: mask = (keypoints.xy[0][:, 0] > 1) & (keypoints.xy[0][:, 1] > 1) transformer = ViewTransformer( source=keypoints.xy[0][mask].astype(np.float32), target=np.array(CONFIG.vertices)[mask].astype(np.float32) ) xy = detections.get_anchors_coordinates(anchor=sv.Position.BOTTOM_CENTER) transformed_xy = transformer.transform_points(points=xy) radar = draw_pitch(config=CONFIG) radar = draw_points_on_pitch( config=CONFIG, xy=transformed_xy[color_lookup == 0], face_color=sv.Color.from_hex(COLORS[0]), radius=20, pitch=radar) radar = draw_points_on_pitch( config=CONFIG, xy=transformed_xy[color_lookup == 1], face_color=sv.Color.from_hex(COLORS[1]), radius=20, pitch=radar) radar = draw_points_on_pitch( config=CONFIG, xy=transformed_xy[color_lookup == 2], face_color=sv.Color.from_hex(COLORS[2]), radius=20, pitch=radar) radar = draw_points_on_pitch( config=CONFIG, xy=transformed_xy[color_lookup == 3], face_color=sv.Color.from_hex(COLORS[3]), radius=20, pitch=radar) return radar def run_pitch_detection(source_video_path: str, device: str) -> Iterator[np.ndarray]: """ Run pitch detection on a video and yield annotated frames. Args: source_video_path (str): Path to the source video. device (str): Device to run the model on (e.g., 'cpu', 'cuda'). Yields: Iterator[np.ndarray]: Iterator over annotated frames. """ pitch_detection_model = YOLO(PITCH_DETECTION_MODEL_PATH).to(device=device) frame_generator = sv.get_vi [truncated — 11050 more characters] ``` ### sports/examples/basketball/main.py ```python """Basketball player + ball tracking — Phase 1 (Path A), CLAUDE.md §4/§7/§8. Adapted from sports/examples/soccer/main.py, simplified for basketball: - Soccer uses 3 SEPARATE soccer-domain checkpoints (ball/player/pitch detection) with custom 4-class labels (ball/goalkeeper/player/referee). Basketball-51 has no detection labels at all, and there's no basketball-specific pretrained checkpoint in this repo — so per CLAUDE.md §0/§3, Path A uses ONE generic COCO-pretrained Ultralytics model (default yolo11n.pt, ungated, auto-downloads) detecting COCO class 0 (person) and 32 (sports ball) in a single pass. - No PITCH_DETECTION/RADAR modes — there is no basketball court keypoint model. (Soccer's pitch/radar modes are explicitly soccer-specific; CLAUDE.md §3 says not to expect them to transfer.) - Team classification reuses `sports.common.team.TeamClassifier` UNCHANGED (SigLIP + UMAP + KMeans on player crops) — it's sport-agnostic, just clusters crops by appearance. No goalkeeper/referee sub-roles to resolve (COCO `person` doesn't distinguish them) — every detected person is a TeamClassifier input. - Ball smoothing reuses `sports.common.ball.BallTracker`/`BallAnnotator` unchanged, fed from the SAME generic model's class-32 detections (no separate slicer/ball model needed since COCO already includes `sports ball`). Usage (mirrors the soccer script): python main.py --source_video_path clip.mp4 --target_video_path out.mp4 \ --device cuda --mode TEAM_CLASSIFICATION Modes: PLAYER_DETECTION, BALL_DETECTION, PLAYER_TRACKING, TEAM_CLASSIFICATION, POSSESSION. POSSESSION adds image-space trails and a proximity-based estimate; it is not possession ground truth because no basketball court homography exists. """ # Direct script execution needs both repository package roots before first-party imports. # ruff: noqa: E402 import argparse import os import sys from enum import Enum from pathlib import Path from typing import Iterator, List REPO_ROOT = Path(__file__).resolve().parents[3] for package_root in (REPO_ROOT, REPO_ROOT / "sports"): package_path = str(package_root) if package_path not in sys.path: sys.path.insert(0, package_path) import cv2 import numpy as np import supervision as sv from sports.common.ball import BallAnnotator, BallTracker from sports.common.possession import PossessionTracker from sports.common.team import TeamClassifier from sports.common.trace import TraceAnnotator from tqdm import tqdm from ultralytics import YOLO from src.config import load_config from src.integrations.tracking_observer import ObservedDetection, TrackingObserver # Generic COCO-pretrained weights — ungated, auto-downloads. Override with a # basketball-specific checkpoint here later (Phase 3) without touching the rest # of this file: just point DETECTION_MODEL_PATH elsewhere. DETECTION_MODEL_PATH = os.environ.get("BASKETBALL_DETECTION_MODEL", "yolo11n.pt") def parse_class_ids(name: str, default: str) -> tuple[int, ...]: try: values = tuple(dict.fromkeys(int(value.strip()) for value in os.environ.get( name, default).split(",") if value.strip())) except ValueError as exc: raise ValueError(f"{name} must be a comma-separated list of integer class ids") from exc if not values or any(value < 0 for value in values): raise ValueError(f"{name} must contain non-negative class ids") return values PERSON_CLASS_IDS = parse_class_ids("BASKETBALL_PERSON_CLASS_IDS", "0") # COCO person BALL_CLASS_IDS = parse_class_ids("BASKETBALL_BALL_CLASS_IDS", "32") # COCO sports ball DETECTION_CLASS_IDS = tuple(dict.fromkeys(PERSON_CLASS_IDS + BALL_CLASS_IDS)) COLORS = ['#FF1493', '#00BFFF', '#FFD700'] # team 0, team 1, unresolved COLOR_PALETTE = sv.ColorPalette.from_hex(COLORS) BOX_ANNOTATOR = sv.BoxAnnotator(color=COLOR_PALETTE, thickness=2) BOX_LABEL_ANNOTATOR = sv.LabelAnnotator( color=COLOR_PALETTE, text_color=sv.Color.from_hex('#FFFFFF'), text_padding=5, text_thickness=1, ) ELLIPSE_ANNOTATOR = sv.EllipseAnnotator(color=COLOR_PALETTE, thickness=2) ELLIPSE_LABEL_ANNOTATOR = sv.LabelAnnotator( color=COLOR_PALETTE, text_color=sv.Color.from_hex('#FFFFFF'), text_padding=5, text_thickness=1, text_position=sv.Position.BOTTOM_CENTER, ) STRIDE = 30 # crop-collection stride for fitting the team classifier (shorter clips than soccer's) POSSESSION_PIXEL_RADIUS = 80.0 POSSESSION_SWITCH_MARGIN = 20.0 class Mode(Enum): PLAYER_DETECTION = 'PLAYER_DETECTION' BALL_DETECTION = 'BALL_DETECTION' PLAYER_TRACKING = 'PLAYER_TRACKING' TEAM_CLASSIFICATION = 'TEAM_CLASSIFICATION' POSSESSION = 'POSSESSION' def get_crops(frame: np.ndarray, detections: sv.Detections) -> List[np.ndarray]: return [sv.crop_image(frame, xyxy) for xyxy in detections.xyxy] def to_observations( detections: sv.Detections, class_name: str, *, team_ids: np.ndarray | None = None, ) -> list[ObservedDetection]: """Translate Supervision detections without inventing missing confidence.""" if detections.confidence is None: return [] tracker_ids = detections.tracker_id anchors = ( detections.get_anchors_coordinates(sv.Position.BOTTOM_CENTER) if tracker_ids is not None else None ) result = [] for index, confidence in enumerate(detections.confidence): track_id = int(tracker_ids[index]) if tracker_ids is not None else None x = float(anchors[index][0]) if anchors is not None else None y = float(anchors[index][1]) if anchors is not None else None team = int(team_ids[index]) if team_ids is not None else None result.append(ObservedDetection( class_name=class_name, confidence=float(confidence), track_id=track_id, x=x, y=y, team=team, )) return result def tracking_observer(source_video_path: str) -> TrackingObserver: """Build t [truncated — 11761 more characters] ``` ### sports/examples/flag_football/main.py ```python """Flag football player + ball tracking — Path A (generic COCO detector + ByteTrack). Adapted from sports/examples/soccer/main.py, simplified for basketball: - Soccer uses 3 SEPARATE soccer-domain checkpoints (ball/player/pitch detection) with custom 4-class labels (ball/goalkeeper/player/referee). Basketball-51 has no detection labels at all, and there's no basketball-specific pretrained checkpoint in this repo — so per CLAUDE.md §0/§3, Path A uses ONE generic COCO-pretrained Ultralytics model (default yolo11n.pt, ungated, auto-downloads) detecting COCO class 0 (person) and 32 (sports ball) in a single pass. - No PITCH_DETECTION/RADAR modes — there is no basketball court keypoint model. (Soccer's pitch/radar modes are explicitly soccer-specific; CLAUDE.md §3 says not to expect them to transfer.) - Team classification reuses `sports.common.team.TeamClassifier` UNCHANGED (SigLIP + UMAP + KMeans on player crops) — it's sport-agnostic, just clusters crops by appearance. No goalkeeper/referee sub-roles to resolve (COCO `person` doesn't distinguish them) — every detected person is a TeamClassifier input. - Ball smoothing reuses `sports.common.ball.BallTracker`/`BallAnnotator` unchanged, fed from the SAME generic model's class-32 detections (no separate slicer/ball model needed since COCO already includes `sports ball`). Usage (mirrors the soccer script): python main.py --source_video_path clip.mp4 --target_video_path out.mp4 \ --device cuda --mode TEAM_CLASSIFICATION Modes: PLAYER_DETECTION, BALL_DETECTION, PLAYER_TRACKING, TEAM_CLASSIFICATION, POSSESSION. POSSESSION adds image-space trails and a proximity-based estimate; it is not possession ground truth because no basketball court homography exists. """ # Direct script execution needs both repository package roots before first-party imports. # ruff: noqa: E402 import argparse import os import sys from enum import Enum from pathlib import Path from typing import Iterator, List REPO_ROOT = Path(__file__).resolve().parents[3] for package_root in (REPO_ROOT, REPO_ROOT / "sports"): package_path = str(package_root) if package_path not in sys.path: sys.path.insert(0, package_path) import cv2 import numpy as np import supervision as sv from sports.common.ball import BallAnnotator, BallTracker from sports.common.possession import PossessionTracker from sports.common.team import TeamClassifier from sports.common.trace import TraceAnnotator from tqdm import tqdm from ultralytics import YOLO from src.config import load_config from src.integrations.tracking_observer import ObservedDetection, TrackingObserver # Generic COCO-pretrained weights — ungated, auto-downloads. Override with a # basketball-specific checkpoint here later (Phase 3) without touching the rest # of this file: just point DETECTION_MODEL_PATH elsewhere. DETECTION_MODEL_PATH = os.environ.get("FLAG_FOOTBALL_DETECTION_MODEL", "yolo11n.pt") def parse_class_ids(name: str, default: str) -> tuple[int, ...]: try: values = tuple(dict.fromkeys(int(value.strip()) for value in os.environ.get( name, default).split(",") if value.strip())) except ValueError as exc: raise ValueError(f"{name} must be a comma-separated list of integer class ids") from exc if not values or any(value < 0 for value in values): raise ValueError(f"{name} must contain non-negative class ids") return values PERSON_CLASS_IDS = parse_class_ids("FLAG_FOOTBALL_PERSON_CLASS_IDS", "0") # COCO person BALL_CLASS_IDS = parse_class_ids("FLAG_FOOTBALL_BALL_CLASS_IDS", "32") # COCO sports ball DETECTION_CLASS_IDS = tuple(dict.fromkeys(PERSON_CLASS_IDS + BALL_CLASS_IDS)) COLORS = ['#FF1493', '#00BFFF', '#FFD700'] # team 0, team 1, unresolved COLOR_PALETTE = sv.ColorPalette.from_hex(COLORS) BOX_ANNOTATOR = sv.BoxAnnotator(color=COLOR_PALETTE, thickness=2) BOX_LABEL_ANNOTATOR = sv.LabelAnnotator( color=COLOR_PALETTE, text_color=sv.Color.from_hex('#FFFFFF'), text_padding=5, text_thickness=1, ) ELLIPSE_ANNOTATOR = sv.EllipseAnnotator(color=COLOR_PALETTE, thickness=2) ELLIPSE_LABEL_ANNOTATOR = sv.LabelAnnotator( color=COLOR_PALETTE, text_color=sv.Color.from_hex('#FFFFFF'), text_padding=5, text_thickness=1, text_position=sv.Position.BOTTOM_CENTER, ) STRIDE = 30 # crop-collection stride for fitting the team classifier (shorter clips than soccer's) POSSESSION_PIXEL_RADIUS = 80.0 POSSESSION_SWITCH_MARGIN = 20.0 class Mode(Enum): PLAYER_DETECTION = 'PLAYER_DETECTION' BALL_DETECTION = 'BALL_DETECTION' PLAYER_TRACKING = 'PLAYER_TRACKING' TEAM_CLASSIFICATION = 'TEAM_CLASSIFICATION' POSSESSION = 'POSSESSION' def get_crops(frame: np.ndarray, detections: sv.Detections) -> List[np.ndarray]: return [sv.crop_image(frame, xyxy) for xyxy in detections.xyxy] def to_observations( detections: sv.Detections, class_name: str, *, team_ids: np.ndarray | None = None, ) -> list[ObservedDetection]: """Translate Supervision detections without inventing missing confidence.""" if detections.confidence is None: return [] tracker_ids = detections.tracker_id anchors = ( detections.get_anchors_coordinates(sv.Position.BOTTOM_CENTER) if tracker_ids is not None else None ) result = [] for index, confidence in enumerate(detections.confidence): track_id = int(tracker_ids[index]) if tracker_ids is not None else None x = float(anchors[index][0]) if anchors is not None else None y = float(anchors[index][1]) if anchors is not None else None team = int(team_ids[index]) if team_ids is not None else None result.append(ObservedDetection( class_name=class_name, confidence=float(confidence), track_id=track_id, x=x, y=y, team=team, )) return result def tracking_observer(source_video_path: str) -> TrackingObserv [truncated — 11785 more characters] ``` [72 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]