# Project export: Gambit

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 12.0
- Tagline: Voice-enabled AI for gaming accessibility.
- Devpost: https://devpost.com/software/gambit-kbp7je
- GitHub: https://github.com/nathaniellaurent/Gambit
- Video: https://www.youtube.com/embed/2oDkTXADV0A?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 4 GitHub contributor(s) — Nathaniel Laurent (27 commits), geedbee (4 commits), Sarahgit config --global user.email bananabasqueak@gmail.com (1 commits), scluba (1 commits)

## Devpost submission (written by the team)

### Inspiration

Gambit addresses a critical gap in the $300 billion global video game market by introducing a voice-enabled AI to enhance online game accessibility. Our solution serves the massive, yet underserved, population of gamers with disabilities.

### What it does

Gambit listens to a user's voice and then executes the in-game actions, such as moving a character or initiating a sequence of moves, all in real time. By converting voice commands into complex inputs and information, Gambit is positioned not only as a crucial social good but also as a powerful economic driver that unlocks a significant segment of the market.

### How we built it

Together, these components form a seamless voice-to-action system adaptable to nearly any game or platform. 🧠 Google Gemini – Bridges the gap between computer vision and spoken commands. Converts spoken instructions into intelligent, context-aware action sequences. 🗣️ Deepgram – Handles accurate, low-latency speech-to-text conversion, with the ability to conversational cues and pauses. 👁️ Computer Vision – Allows Gambit to “see” and interpret on-screen environments dynamically. 🐍 Python Automation Engine – Executes the translated commands by controlling keyboard and mouse inputs directly.

### Challenges we ran into

We ran into challenges with Omniparser, the technology we used for image segmentation. Downloading the process was smooth on MacOS, but took multiple hours for Windows.

### Accomplishments we're proud of

We’re most proud of Gambit’s adaptability. It can interpret and respond to any visual interface without needing game-specific integrations — a major step toward universal gaming accessibility.

### What we learned

We explored new domains across AI vision, AI speech to text technologies, and real-time system control. More importantly, we learned how these technologies can intersect to create meaningful, inclusive innovation that extends beyond entertainment.

### What's next

Next, we plan to: Expand Gambit’s compatibility to include a wider range of game genres (e.g., FPS, strategy, racing). Integrate customizable command mappings for different player needs. Partner with game studios and accessibility organizations to scale our impact. Explore mobile and VR adaptations, making hands-free play accessible across devices.

## README (from the GitHub repository)

# Voice-Controlled Board Game Player (Python)

## Prereqs
- Python 3.10+
- API Keys as env vars
  - `DEEPGRAM_API_KEY`
  - `ANTHROPIC_API_KEY`

## Install
```bash
pip install -r requirements.txt
```

## Run the backend server (API + Web UI)
```bash
python -m uvicorn server:app --host 127.0.0.1 --port 8000
```

- Open the web UI at: http://127.0.0.1:8000/
- Endpoints used by the UI:
  - GET `/api/connect4/state`
  - POST `/api/connect4/reset`
  - POST `/api/connect4/move/{column}` (1–7)
  - POST `/api/llm/parse` and `/api/llm/command` (for LLM parsing/apply)

Env vars (recommended):
- `ANTHROPIC_API_KEY` for LLM parsing on the server

## Run the voice controller (optional)
```bash
python main.py
```

- If `DEEPGRAM_API_KEY` is set, the app starts a background speech-to-text worker (Deepgram listen.v2) and enqueues transcripts.
- Each transcript is sent to the backend via `POST /api/llm/command`, so the web UI updates automatically.
- If the server isn’t reachable, it falls back to local parsing and engine execution (the web UI will not reflect those local-only changes).

Env vars (optional/required):
- `DEEPGRAM_API_KEY` for live voice input.
- `ANTHROPIC_API_KEY` for LLM parsing (used by both server and local fallback).

## Notes
- Game state is stored in-memory on the backend. Restarting the server resets the board.
- The frontend periodically refreshes state so speech-driven moves appear without clicking.


## Detected evidence (automated analysis)

Indexed codebase: 18 recognized source files, 67 KB.
- Anthropic (technology) — detected in the code
- FastAPI (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- Google Gemini (technology) — claimed on Devpost, not found in the code
- React (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (20 of 20)

```
.gitignore
annotated_viewer.py
chess_game.py
connect4.py
deepgram_test.py
game_engine.py
game_json.py
gemini_client.py
image_segmentation.py
llm_client.py
main.py
README.md
requirements.txt
schema.py
server.py
stt_client.py
ui_automation.py
unified_llm.py
web/app.jsx
web/index.html
```

### Dependencies

- requirements.txt: anthropic@>=0.30.0, deepgram-sdk@>=3.4.0, fastapi, google-generativeai@>=0.7.2, numpy, pillow, pyautogui, python-chess@>=1.999, python-dotenv@>=1.0.1, sounddevice@>=0.4.6, uvicorn[standard]@>=0.30.0

### Recent commits (newest first)

- Merge pull request #6 from nathaniellaurent/segmentation
- working interactive version
- working fast segmentation on windows
- class image segmentation
- segmentation startup
- started pyautogui stuff
- Merge pull request #5 from nathaniellaurent/win_condition
- fixed win condition problem
- Merge pull request #4 from nathaniellaurent/add_gemini
- Merge branch 'main' into add_gemini
- added gemini support and allow switch from claude and gemini
- updated req
- updated req
- updated readme to say how to run webapp
- Merge pull request #3 from nathaniellaurent/visuals
- working voice activated moved on connect 4 with visuals
- Merge branch 'main' of github.com:nathaniellaurent/Gambit into visuals
- Merge pull request #2 from nathaniellaurent/integrate_voice
- working queue added for game updates
- Merge branch 'deep-deepgram' of github.com:nathaniellaurent/Gambit into integrate_voice

## Key source files (fetched from GitHub, selected and truncated for size)

### requirements.txt

```
numpy
anthropic>=0.30.0
deepgram-sdk>=3.4.0
python-dotenv>=1.0.1
sounddevice>=0.4.6
python-chess>=1.999
uvicorn[standard]>=0.30.0
google-generativeai>=0.7.2
fastapi
pyautogui
pillow
```

### server.py

```python
from typing import Dict, Any
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from fastapi.staticfiles import StaticFiles
from fastapi.responses import FileResponse

from connect4 import Connect4Game

app = FastAPI(title="Gambit Connect4 Server")

# CORS for local dev
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Singleton Connect4 instance
CONNECT4 = Connect4Game()

# No server-side LLM; parsing happens client-side


def _serialize_state() -> Dict[str, Any]:
    return {
        "board": CONNECT4.board,
        "current_player": CONNECT4.current_player,
        "winner": getattr(CONNECT4, "winner", None),
    }


@app.get("/api/connect4/state")
def get_state() -> Dict[str, Any]:
    return _serialize_state()


@app.post("/api/connect4/reset")
def reset_game() -> Dict[str, Any]:
    CONNECT4.reset()
    return _serialize_state()


@app.post("/api/connect4/move/{column}")
def drop_disc(column: int) -> Dict[str, Any]:
    # Map to engine-style command
    cmd = {
        "game": "connect4",
        "action": "drop_disc",
        "parameters": {"column": column},
    }
    # Use the game's execute method directly
    # It prints to console; here we apply the move logic without printing via a small helper.
    # Re-implement minimal move from execute() for API context.
    if getattr(CONNECT4, "winner", None):
        raise HTTPException(status_code=400, detail="Game is over. Reset to play again.")
    if not (1 <= column <= CONNECT4.COLS):
        raise HTTPException(status_code=400, detail="Column must be between 1 and 7.")

    disc = CONNECT4.current_player
    if not CONNECT4._drop_in_column(column - 1, disc):  # type: ignore[attr-defined]
        raise HTTPException(status_code=400, detail="Column is full.")

    # Check winner
    if CONNECT4._check_winner(disc):  # type: ignore[attr-defined]
        # Persist winner on the game instance so future state reflects it
        setattr(CONNECT4, "winner", disc)
        return _serialize_state()

    # Toggle player
    CONNECT4.current_player = "Y" if CONNECT4.current_player == "R" else "R"
    return _serialize_state()


# Server provides only Connect4 endpoints; clients must parse and call these.


# Serve static React app from ./web
app.mount("/", StaticFiles(directory="web", html=True), name="static")


# Convenience root route to serve index.html explicitly (for some servers)
@app.get("/")
def root() -> FileResponse:
    return FileResponse("web/index.html")

```

### main.py

```python
import os
import json
import time
from typing import Optional, Dict, Any
import queue
import asyncio
import urllib.request
import urllib.error
import threading
from datetime import datetime

from stt_client import DeepgramSTTClient
from unified_llm import LLMClient
from game_engine import execute_game_command
from dotenv import load_dotenv
from image_segmentation import process_image
from game_json import create_game_json
from schema import GAME_CONTEXT_STRING
from ui_automation import UIAutomationExecutor
from annotated_viewer import start_annotated_viewer

try:
    import pyautogui  # type: ignore
except Exception:  # pragma: no cover
    pyautogui = None  # type: ignore

try:
    from PIL import Image, ImageTk  # type: ignore
except Exception:
    Image = None  # type: ignore
    ImageTk = None  # type: ignore


# ---- Screenshot Region Config (edit here) ----
# Defines how the screenshot region is computed from the current screen size.
# - base_x, base_y: top-left start coordinates
# - width_factor, height_factor: fraction of screen width/height to capture
# - offset_top, offset_bottom: additional vertical adjustments (pixels)
SCREENSHOT_CFG = {
    "base_x": 0,
    "base_y": 0,
    "width_factor": 0.33,     # quarter of screen width
    "height_factor": 0.25,    # quarter of screen height
    "offset_top": 230,        # vertical offset added to y
    "offset_bottom": 300,     # extra pixels added to the height
}


# Compute the screenshot region (x, y, width, height) from the current screen and config
def compute_screenshot_region(cfg: Dict[str, Any]) -> Dict[str, int]:
    if pyautogui is None:
        # fallback: assume 1920x1080 if pyautogui not present
        screen_w, screen_h = 1920, 1080
    else:
        screen_w, screen_h = pyautogui.size()
    x = int(cfg["base_x"]) 
    y = int(cfg["base_y"]) + int(cfg["offset_top"])
    cap_w = int(screen_w * float(cfg["width_factor"]))
    cap_h = int(screen_h * float(cfg["height_factor"])) + int(cfg["offset_bottom"]) 
    return {"x": x, "y": y, "w": cap_w, "h": cap_h}


# Convert fractional 0..1 coordinates in the plan into absolute pixels inside the screenshot region
def convert_plan_coordinates(plan: Dict[str, Any], region: Dict[str, int]) -> Dict[str, Any]:
    ops = list(plan.get("operations") or [])
    rx, ry, rw, rh = region["x"], region["y"], region["w"], region["h"]

    def to_abs(v: Any, size: int, offset: int) -> Any:
        try:
            fv = float(v)
        except Exception:
            return v
        # Heuristic: treat values in [0,1] as fractional
        if 0.0 <= fv <= 1.0:
            return int(round(offset + fv * size))
        return int(round(fv))

    def to_abs_delta(v: Any, size: int) -> Any:
        try:
            fv = float(v)
        except Exception:
            return v
        # Heuristic: treat values in [-1,1] as fractional deltas of the region
        if -1.0 <= fv <= 1.0:
            return int(round(fv * size))
        return int(round(fv))

    new_ops = []
    for op in ops:
        name = (op.get("op") or "").lower()
        o = dict(op)
        if name in {"move", "click", "double_click", "drag"}:
            if "x" in o:
                o["x"] = to_abs(o["x"], rw, rx)
            if "y" in o:
                o["y"] = to_abs(o["y"], rh, ry)
        if name in {"move_relative", "drag_relative"}:
            if "dx" in o:
                o["dx"] = to_abs_delta(o["dx"], rw)
            if "dy" in o:
                o["dy"] = to_abs_delta(o["dy"], rh)
        new_ops.append(o)

    new_plan = dict(plan)
    new_plan["operations"] = new_ops
    return new_plan


def _prune_dir_keep_last_n(dir_path: str, keep: int = 10) -> None:
    try:
        entries = [
            os.path.join(dir_path, f)
            for f in os.listdir(dir_path)
            if f.lower().endswith(".png") and os.path.isfile(os.path.join(dir_path, f))
        ]
        entries.sort(key=lambda p: os.path.getmtime(p), reverse=True)
        for p in entries[keep:]:
            try:
                os.remove(p)
            except Exception:
                pass
    except Exception:
        pass

# def listen_for_command() -> Optional[bytes]:
#     """
#     Placeholder audio capture function.
#     Return raw PCM/WAV bytes or None to stop.
#     For now, we simulate audio capture with text input for demo purposes.
#     """
#     try:
#         user = input("Say a command (or 'quit' to exit): ")
#     except EOFError:
#         return None

#     if not user or user.lower().strip() == "quit":
#         return None

#     # Simulate audio bytes by embedding the typed text for the STT stub to unwrap.
#     return f"__TEXT__:{user}".encode("utf-8")


def get_transcription(stt: DeepgramSTTClient) -> Optional[str]:
    try:
        return stt.transcribe_speech()
    except Exception as e:
        print(f"STT error: {e}")
        return None


def get_game_command(llm, text_command: str) -> Optional[Dict[str, Any]]:
    try:
        return llm.get_game_command(text_command)
    except Exception as e:
        print(f"LLM error: {e}")
        return None


def execute_game_command_wrapper(command_json: Dict[str, Any]) -> None:
    try:
        execute_game_command(command_json)
    except Exception as e:
        print(f"Engine error: {e}")


def apply_command_via_server(cmd: Dict[str, Any], base_url: str = "http://127.0.0.1:8000") -> Optional[Dict[str, Any]]:
    """Apply a parsed command to the FastAPI Connect4 endpoints so the web UI updates.
    Returns updated state on success, or None if the server call fails."""
    try:
        action = (cmd or {}).get("action")
        game = (cmd or {}).get("game") or "generic"
        params = (cmd or {}).get("parameters") or {}
        if action == "reset_game" and (game in {"connect4", "generic"}):
            url = f"{base_url}/api/connect4/reset"
            req = urllib.request.Request(url, data=b"{}", headers={"Content-Type": "application/json"}, method="POST")
            with url
[truncated — 8211 more characters]
```

### web/app.jsx

```javascript
const { useEffect, useState } = React;

async function api(path, opts) {
  const res = await fetch(path, { headers: { 'Content-Type': 'application/json' }, ...opts });
  if (!res.ok) {
    const text = await res.text();
    throw new Error(text || `${res.status}`);
  }
  return res.json();
}

function Cell({ value, onClick }) {
  return (
    <div className="cell" onClick={onClick}>
      {value === 'R' || value === 'Y' ? <div className={`disc ${value}`}></div> : null}
    </div>
  );
}

function Board() {
  const [board, setBoard] = useState(Array.from({ length: 6 }, () => Array(7).fill(' ')));
  const [current, setCurrent] = useState('R');
  const [winner, setWinner] = useState(null);
  const [loading, setLoading] = useState(false);
  

  const load = async () => {
    const s = await api('/api/connect4/state');
    setBoard(s.board);
    setCurrent(s.current_player);
    setWinner(s.winner || null);
  };

  useEffect(() => { load(); }, []);

  // Periodically refresh state so updates triggered outside the UI (e.g., speech) appear automatically
  useEffect(() => {
    const id = setInterval(() => {
      load().catch(() => {});
    }, 1000); // 1s refresh interval
    return () => clearInterval(id);
  }, []);

  const drop = async (colIndex) => {
    if (winner) return;
    setLoading(true);
    try {
      const s = await api(`/api/connect4/move/${colIndex + 1}`, { method: 'POST' });
      setBoard(s.board);
      setCurrent(s.current_player);
      setWinner(s.winner || null);
    } catch (e) {
      alert(e.message);
    } finally {
      setLoading(false);
    }
  };

  const reset = async () => {
    await api('/api/connect4/reset', { method: 'POST' });
    await load();
  };

  

  return (
    <div className="board">
      <div className="panel" style={{flexWrap:'wrap'}}>
        <button className="btn" onClick={reset} disabled={loading}>Reset</button>
        <div className="status">
          {winner ? (
            <span>Winner: <b>{winner}</b></span>
          ) : (
            <span>Current: <b>{current}</b></span>
          )}
        </div>
        
      </div>
      {/* Column number labels (clickable) */}
      <div
        className="col-labels"
        style={{
          display: 'grid',
          gridTemplateColumns: 'repeat(7, 64px)',
          gap: 12,
          justifyContent: 'center',
          padding: '12px',
          color: '#9aa3b2',
          fontWeight: 600,
          textAlign: 'center',
          userSelect: 'none',
        }}
      >
        {Array.from({ length: 7 }).map((_, i) => (
          <div
            key={`label-${i}`}
            onClick={() => !winner && drop(i)}
            style={{ cursor: winner ? 'not-allowed' : 'pointer' }}
            title={`Drop in column ${i + 1}`}
          >
            {i + 1}
          </div>
        ))}
      </div>
      <div className="grid">
        {board.map((row, rIdx) =>
          row.map((cell, cIdx) => (
            <Cell key={`${rIdx}-${cIdx}`} value={cell} onClick={() => drop(cIdx)} />
          ))
        )}
      </div>
      {winner && (
        <div
          style={{
            position: 'fixed',
            inset: 0,
            background: 'rgba(0,0,0,0.45)',
            display: 'flex',
            alignItems: 'center',
            justifyContent: 'center',
            zIndex: 1000,
          }}
          aria-modal="true"
          role="dialog"
        >
          <div
            style={{
              background: '#0d1b2a',
              borderRadius: 16,
              padding: 24,
              minWidth: 300,
              boxShadow: '0 10px 30px rgba(0,0,0,.35), inset 0 0 0 1px rgba(255,255,255,.06)'
            }}
          >
            <div style={{fontSize: 20, fontWeight: 800, marginBottom: 8}}>Game Over</div>
            <div style={{color:'#aab1cc', marginBottom: 16}}>Winner: <b>{winner}</b></div>
            <div style={{display:'flex', gap:8, justifyContent:'flex-end'}}>
              <button className="btn" onClick={reset}>Reset Game</button>
            </div>
          </div>
        </div>
      )}
      
    </div>
  );
}

function App() {
  return <Board />;
}

const root = ReactDOM.createRoot(document.getElementById('app'));
root.render(<App />);

```

### game_json.py

```python
import json


def create_game_json(ocr_data_list, game_context_string, prompt_string):
    """Combines OCR data, game context, and a prompt into a single JSON string.

    Args:
        ocr_data_list (list): A list of dictionaries (like the example provided).
        game_context_string (str): A string describing the game state or context.
        prompt_string (str): A string for the prompt.

    Returns:
        str: A JSON formatted string containing all the data.
    """
    combined_data = {
        "game_context": game_context_string,
        "prompt": prompt_string,
        "ocr_data": ocr_data_list,
    }
    json_output = json.dumps(combined_data, indent=2)
    return json_output

```

### unified_llm.py

```python
import os
from typing import Any, Dict, Optional

from llm_client import ClaudeLLMClient
from gemini_client import GeminiLLMClient


class LLMClient:
    """
    Unified LLM client wrapper. Choose provider at init and expose a single
    get_game_command API that matches existing clients.

    provider: 'claude' or 'gemini'
    keys are pulled from env by default unless explicitly provided.
    """

    def __init__(
        self,
        provider: str = "claude",
        anthropic_api_key: Optional[str] = None,
        google_api_key: Optional[str] = None,
        claude_model: str = "claude-haiku-4-5-20251001",
        gemini_model: str = "gemini-2.0-flash-lite",
    ) -> None:
        self.provider = provider.lower().strip()
        print("provider: ", self.provider)
        if self.provider == "gemini":
            key = google_api_key or os.getenv("GOOGLE_API_KEY")
            self._client = GeminiLLMClient(api_key=key, model=gemini_model)
        elif(self.provider == "claude"):
            key = anthropic_api_key or os.getenv("ANTHROPIC_API_KEY")
            self._client = ClaudeLLMClient(api_key=key, model=claude_model)
        else:
            raise ValueError(f"Unknown provider: {self.provider}")

    def get_game_command(self, text_command: str) -> Dict[str, Any]:
        return self._client.get_game_command(text_command)

```

### game_engine.py

```python
from typing import Dict, Any

from connect4 import Connect4Game
from chess_game import ChessGame

_games: Dict[str, Any] = {}

def _get_connect4() -> Connect4Game:
    game = _games.get("connect4")
    if game is None:
        game = Connect4Game()
        _games["connect4"] = game
    return game

def _get_chess() -> ChessGame:
    game = _games.get("chess")
    if game is None:
        game = ChessGame()
        _games["chess"] = game
    return game

def execute_game_command(command_json: Dict[str, Any]) -> None:
    action = command_json.get("action")
    params = command_json.get("parameters", {}) or {}
    game = command_json.get("game") or "generic"

    if game == "connect4" or action in {"drop_disc", "reset_game"}:
        _get_connect4().execute(command_json)
        return

    if game == "chess":
        _get_chess().execute(command_json)
        return

    if action == "move_piece":
        print(
            f"GAME: Moving {params.get('piece_name')} from {params.get('from_location')} to {params.get('to_location')}"
        )
    elif action == "draw_card":
        qty = params.get("quantity") or 1
        print(f"GAME: Drawing {qty} card(s)...")
    elif action == "end_turn":
        print("GAME: Ending turn...")
    elif action == "place_settlement":
        print(f"GAME: Placing settlement at {params.get('to_location')}")
    elif action == "buy_property":
        print(f"GAME: Buying property {params.get('to_location')} for {params.get('target_player') or 'current player'}")
    elif action == "unknown":
        print("GAME: I didn't understand that command. Please try again.")
    else:
        print(f"GAME: Unhandled action '{action}'.")

```

### image_segmentation.py

```python
import sys
import os
import base64

path_to_omniparser = r"C:\Users\natha\Documents\Calhacks\OmniParser"
if path_to_omniparser not in sys.path:
    sys.path.append(path_to_omniparser)

from PIL import Image
from util.utils import (
    get_yolo_model,
    get_caption_model_processor,
    check_ocr_box,
    get_som_labeled_img,
)

# 1) Init once
yolo = get_yolo_model(model_path=os.path.join(path_to_omniparser, path_to_omniparser+"/weights/icon_detect/model.pt"))
caption_proc = get_caption_model_processor(
    model_name="florence2", model_name_or_path=os.path.join(path_to_omniparser, path_to_omniparser+"/weights/icon_caption_florence"), device="cuda:0")

# 2) Given a PIL image
# pil_image = Image.open("path/to/image.png")

def process_image(pil_image: Image.Image):
    # 3) OCR (offline-safe: keep use_paddleocr=False)
    (ocr_text, ocr_bbox), _ = check_ocr_box(
        pil_image,
        display_img=False,
        output_bb_format="xyxy",
        goal_filtering=None,
        easyocr_args={"paragraph": False, "text_threshold": 0.9},
        use_paddleocr=False,
    )

    # 4) Run segmentation/annotation
    encoded_image, label_coords, items = get_som_labeled_img(
        pil_image,
        model=yolo,
        BOX_TRESHOLD=0.25,          # tune
        iou_threshold=0.7,          # tune
        output_coord_in_ratio=True,
        ocr_bbox=ocr_bbox,
        draw_bbox_config={"text_scale": 0.6, "text_thickness": 2, "text_padding": 3, "thickness": 3},
        caption_model_processor=caption_proc,
        ocr_text=ocr_text,
        scale_img=False,            # set True to use imgsz below
        imgsz=1280,                 # tune
        batch_size=128,             # tune
    )

    # 5) Use outputs
    annotated_bytes = base64.b64decode(encoded_image)  # PNG bytes
    return annotated_bytes, label_coords, items

if __name__ == "__main__":
    if len(sys.argv) < 2:
        print("Usage: python image_segmentation.py <image_path>")
        sys.exit(1)
    in_path = sys.argv[1]
    pil_image = Image.open(in_path)
    annotated_bytes, label_coords, items = process_image(pil_image)
    with open("annotated.png", "wb") as f:
        f.write(annotated_bytes)
    print("annotated.png")

```

### chess_game.py

```python
from typing import Dict, Any, List, Optional
import chess


class ChessGame:
    def __init__(self) -> None:
        self.board = chess.Board()

    def reset(self) -> None:
        self.board = chess.Board()

    def serialize(self) -> Dict[str, Any]:
        return {
            "fen": self.board.fen(),
            "turn": "white" if self.board.turn == chess.WHITE else "black",
            "is_game_over": self.board.is_game_over(),
            "result": self.board.result(claim_draw=True) if self.board.is_game_over() else None,
            "legal_moves": [self._uci_to_dict(m.uci()) for m in self.board.legal_moves],
            "board": self._board_matrix(),
            "check": self.board.is_check(),
        }

    def _uci_to_dict(self, uci: str) -> Dict[str, str]:
        return {"from": uci[:2], "to": uci[2:4]}

    def _board_matrix(self) -> List[List[Optional[str]]]:
        # 8x8 from rank 8 to 1, file a to h
        matrix: List[List[Optional[str]]] = []
        for rank in range(7, -1, -1):
            row: List[Optional[str]] = []
            for file in range(8):
                square = chess.square(file, rank)
                piece = self.board.piece_at(square)
                row.append(piece.symbol() if piece else None)
            matrix.append(row)
        return matrix

    def execute(self, command_json: Dict[str, Any]) -> None:
        action = command_json.get("action")
        params = command_json.get("parameters", {}) or {}

        if action == "reset_game":
            self.reset()
            print("CHESS: Game reset. White to move.")
            return

        if action == "move_piece":
            from_sq = params.get("from_location")
            to_sq = params.get("to_location")
            if not from_sq or not to_sq:
                print("CHESS: Missing 'from_location' or 'to_location'.")
                return
            try:
                move = chess.Move.from_uci((from_sq + to_sq).lower())
            except Exception:
                print("CHESS: Invalid coordinates.")
                return
            if move not in self.board.legal_moves:
                print("CHESS: Illegal move.")
                return
            self.board.push(move)
            print(f"CHESS: {from_sq}->{to_sq} {'#' if self.board.is_checkmate() else ''}")
            return

        print(f"CHESS: Unknown action '{action}'.")

```

### connect4.py

```python
from typing import List, Optional, Dict, Any


class Connect4Game:
    ROWS = 6
    COLS = 7

    def __init__(self) -> None:
        self.board: List[List[str]] = [[" "] * self.COLS for _ in range(self.ROWS)]
        self.current_player: str = "R"  # R and Y
        self.winner: Optional[str] = None

    def reset(self) -> None:
        self.board = [[" "] * self.COLS for _ in range(self.ROWS)]
        self.current_player = "R"
        self.winner = None

    def _drop_in_column(self, col: int, disc: str) -> bool:
        # Drop from bottom up
        for r in range(self.ROWS - 1, -1, -1):
            if self.board[r][col] == " ":
                self.board[r][col] = disc
                return True
        return False

    def _check_winner(self, disc: str) -> bool:
        # Horizontal
        for r in range(self.ROWS):
            for c in range(self.COLS - 3):
                if all(self.board[r][c + i] == disc for i in range(4)):
                    return True
        # Vertical
        for c in range(self.COLS):
            for r in range(self.ROWS - 3):
                if all(self.board[r + i][c] == disc for i in range(4)):
                    return True
        # Diagonal down-right
        for r in range(self.ROWS - 3):
            for c in range(self.COLS - 3):
                if all(self.board[r + i][c + i] == disc for i in range(4)):
                    return True
        # Diagonal up-right
        for r in range(3, self.ROWS):
            for c in range(self.COLS - 3):
                if all(self.board[r - i][c + i] == disc for i in range(4)):
                    return True
        return False

    def _print_board(self) -> None:
        print("CONNECT4:")
        for row in self.board:
            print("|" + "|".join(cell if cell != " " else "." for cell in row) + "|")
        print(" " + " ".join(str(i + 1) for i in range(self.COLS)))

    def execute(self, command_json: Dict[str, Any]) -> None:
        action = command_json.get("action")
        params = command_json.get("parameters", {}) or {}

        if action == "reset_game":
            self.reset()
            print("CONNECT4: Game reset. R to move.")
            self._print_board()
            return

        if action == "drop_disc":
            if self.winner:
                print("CONNECT4: Game is over. Use 'reset_game' to start a new game.")
                return
            col_param: Optional[int] = params.get("column")
            if col_param is None:
                print("CONNECT4: Missing 'column' parameter (1-7).")
                return
            # Accept 1-7 from users/LLM; convert to 0-based
            col = int(col_param) - 1
            if not (0 <= col < self.COLS):
                print("CONNECT4: Column must be between 1 and 7.")
                return

            disc = self.current_player
            if not self._drop_in_column(col, disc):
                print("CONNECT4: Column is full.")
                return

            print(f"CONNECT4: Player {disc} drops in column {col + 1}.")
            self._print_board()

            if self._check_winner(disc):
                self.winner = disc
                print(f"CONNECT4: Player {disc} wins! Use 'reset_game' to start over.")
                return

            # Toggle player
            self.current_player = "Y" if self.current_player == "R" else "R"
            print(f"CONNECT4: Next to move: {self.current_player}")
            return

        print(f"CONNECT4: Unknown action '{action}'.")

```

[8 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]