# Project export: Pirates This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: Yo ho, yo ho, it’s a lexicon life for us - Devpost: https://devpost.com/software/pirates-w09x2f - GitHub: https://github.com/rosie-m-banks/pirates.git - Video: https://www.youtube.com/embed/rbFwrpA2R1E?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 3 GitHub contributor(s) — Sabrina Yen-Ko (36 commits), malti (28 commits), Preston Seay (12 commits) ## Devpost submission (written by the team) No Devpost description available. ## README (from the GitHub repository) # pirates Pirates Game Analysis via CV ## Detected evidence (automated analysis) Indexed codebase: 65 recognized source files, 351 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - Express (technology) — detected in the code - HTML (language) — detected in the code - Hugging Face (technology) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - PyTorch (technology) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - TensorFlow (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (87 of 87) ``` .gitignore backend/data/definitions.json backend/data/too-many-words.txt backend/data/word_frequencies.json backend/data/words.txt backend/dataFusion.js backend/gameState.js backend/logs/.gitkeep backend/logs/player_vocabulary.jsonl backend/logs/vocabulary_aggregate.json backend/package.json backend/README.md backend/RECOMMENDATION_SCORING.md backend/recommendationScorer.js backend/scripts/generate_word_frequencies.py backend/scripts/get_definitions.py backend/scripts/requirements.txt backend/server.js backend/stateLogger.js backend/test backend/test-recommendations.js backend/test-state-logging.js backend/worker.js backend/worker.test.js frontend/.gitignore frontend/eslint.config.js frontend/index.html frontend/package.json frontend/README.md frontend/src/App.css frontend/src/App.tsx frontend/src/assets/fonts/FatPix-SVG.otf frontend/src/components/BeachTile.tsx frontend/src/components/MoveLog.tsx frontend/src/components/MoveLogEntry.tsx frontend/src/components/PlayerCard.tsx frontend/src/components/PlayerStatistics.tsx frontend/src/components/ScrollHint.tsx frontend/src/components/WavePattern.tsx frontend/src/components/WordTile.tsx frontend/src/hooks/useGameImage.ts frontend/src/hooks/useMoveLog.ts frontend/src/hooks/usePersistedMoveLog.ts frontend/src/index.css frontend/src/main.tsx frontend/src/StudentView.tsx frontend/src/TeacherView.tsx frontend/src/types/stats.ts frontend/src/utils/definition.ts frontend/src/utils/vocabularyLevels.ts frontend/src/ValidationView.tsx frontend/tsconfig.app.json frontend/tsconfig.json frontend/tsconfig.node.json frontend/vite.config.ts README.md requirements.txt vision/.gitignore vision/augment_synthetic_data.py vision/build_templates.py vision/camera.yaml vision/copy_crops_1_7_to_letter_data.py vision/copy_crops_8_13_to_letter_data.py vision/extract_tiles.py vision/generic_camera.py vision/hand_detector.py vision/lazy_relabel vision/lenet_letter.py vision/letter_cnn.py vision/letter_data/README.md vision/letter_model.py vision/live_tile_viewer.py vision/model.py vision/move_templates_to_letter_data.py vision/oak.py vision/populate_letter_data.py vision/process_image.py vision/requirements_vlm.txt vision/take_photo.py vision/template_recognizer.py vision/templates/README.md vision/tile_character_extractor.py vision/tile_cnn.pth vision/tile_extractor_vlm.py vision/tile_frame_pub.py vision/train_lenet_letter.py vision/train_letter_model.py ``` ### Dependencies - backend/package.json: cors@^2.8.6, express@^4.21.0, socket.io@^4.8.0 - backend/scripts/requirements.txt: wordfreq@>=3.0.0 - frontend/package.json: @eslint/js@^9.39.1, @tailwindcss/vite@^4.1.18, @types/node@^24.10.1, @types/react@^19.2.7, @types/react-dom@^19.2.3, @vitejs/plugin-react@^5.1.1, eslint@^9.39.1, eslint-plugin-react-hooks@^7.0.1, eslint-plugin-react-refresh@^0.4.24, globals@^16.5.0, react@^19.2.0, react-dom@^19.2.0, socket.io-client@^4.8.3, tailwindcss@^4.1.18, typescript@~5.9.3, typescript-eslint@^8.48.0, vite@^7.3.1 - requirements.txt: anthropic, depthai, matplotlib, mediapipe, opencv-python, protobuf, pytesseract, sentencepiece, torch, torchvision, transformers ### Recent commits (newest first) - Fix scroll so it's the right height - Merge pull request #18 from rosie-m-banks/malti/definition-api - Merge remote-tracking branch 'refs/remotes/origin/malti/definition-api' into malti/definition-api - added utils - Merge pull request #17 from rosie-m-banks/frontend/aesthetic - Add page indicator - Add logo + thumbnail - Switch to query params for demo - Change levels to be grade appropriate - aesthetic students - Improve teacher view - Make teacherview less terrible - changed model and tweaked prompt for better results - Merge pull request #16 from rosie-m-banks/malti/definition-api - Merge branch 'main' into malti/definition-api - added definitions to frontend - Merge pull request #15 from rosie-m-banks/frontend/image - fixed link - fixed .gitignore - need to test definition endpoint ## Key source files (fetched from GitHub, selected and truncated for size) ### backend/RECOMMENDATION_SCORING.md ```markdown # Recommendation Scoring System This document explains how to customize the word recommendation scoring system. ## Overview The scoring system filters and ranks recommended words based on: 1. **Word Frequency** (zipf scale 0-8, from corpus data) 2. **Word Length** (number of characters) ## Quick Start ### Default Behavior - **Filters out** words with frequency < 1.0 (extremely rare words) - **Prioritizes** longer words that are relatively common - Formula: `score = (frequency / 8) * 1.0 + length * 2.0` ### Test the System ```bash node test-recommendations.js ``` ## Customization Guide All customization happens in `backend/recommendationScorer.js`. ### 1. Adjust Filtering Threshold **Where:** `ScoringConfig.minFrequency` ```javascript // More permissive (include rare words) minFrequency: 0.5, // Include uncommon words // More restrictive (only common words) minFrequency: 3.0, // Only fairly common words minFrequency: 5.0, // Only common words ``` **Frequency Reference:** - 0-1: Extremely rare (technical terms, typos) - 1-3: Uncommon (xylophone, aardvark) - 3-5: Fairly common (elephant, running) - 5-7: Common (cat, dog, run) - 7-8: Very common (the, and, is) ### 2. Change Score Weights **Where:** `ScoringConfig.weights` ```javascript // Prioritize length over frequency (longer words first) weights: { frequency: 0.5, length: 3.0, } // Prioritize frequency over length (common words first) weights: { frequency: 3.0, length: 0.5, } // Balanced (default) weights: { frequency: 1.0, length: 2.0, } ``` ### 3. Use Pre-Built Strategies **Where:** Import from `ScoringStrategies` ```javascript // In worker.js, replace ScoringConfig with: import { ScoringStrategies } from './recommendationScorer.js'; // Then in the message handler: result.recommended_words = sortRecommendations( result.recommended_words, frequencies, ScoringStrategies.longestFirst // or .mostCommonFirst, .balanced, .commonAndLong ); ``` **Available Strategies:** - `balanced` - Good mix of common and long words (default) - `longestFirst` - Prioritize longest words - `mostCommonFirst` - Prioritize most frequent words - `commonAndLong` - Only very common words (freq ≥ 5), prioritize length ### 4. Change Scoring Strategy **Where:** `ScoringConfig.strategy` ```javascript // Additive (default): weighted sum strategy: 'additive', // score = freq_weight * freq + len_weight * len // Multiplicative: emphasize both factors strategy: 'multiplicative', // score = freq^freq_weight * len^len_weight // Custom: your own logic strategy: 'custom', // Edit customScoringFunction() in recommendationScorer.js ``` ### 5. Enable Normalization **Where:** `ScoringConfig.normalize` ```javascript normalize: { frequency: true, // Scale frequency (0-8) to (0-1) length: false, // Keep length as raw value (3-15) } ``` Normalization helps when using multiplicative strategy or comparing different scales. ## Examples ### Example 1: Only suggest common, long words [truncated — 2473 more characters] ``` ### requirements.txt ``` opencv-python pytesseract matplotlib depthai # Letter CNN (train + inference) torch torchvision # TrOCR (optional) transformers protobuf sentencepiece mediapipe anthropic ``` ### backend/package.json ``` { "name": "pirates-backend", "version": "1.0.0", "description": "Multithreaded Express server with game state and WebSocket relay for Pirates", "main": "server.js", "type": "module", "scripts": { "start": "node server.js", "dev": "node --watch server.js", "test": "node --test worker.test.js" }, "dependencies": { "cors": "^2.8.6", "express": "^4.21.0", "socket.io": "^4.8.0" } } ``` ### frontend/package.json ``` { "name": "frontend", "private": true, "version": "0.0.0", "type": "module", "scripts": { "dev": "vite", "build": "tsc -b && vite build", "lint": "eslint .", "preview": "vite preview" }, "dependencies": { "@tailwindcss/vite": "^4.1.18", "react": "^19.2.0", "react-dom": "^19.2.0", "socket.io-client": "^4.8.3", "tailwindcss": "^4.1.18" }, "devDependencies": { "@eslint/js": "^9.39.1", "@types/node": "^24.10.1", "@types/react": "^19.2.7", "@types/react-dom": "^19.2.3", "@vitejs/plugin-react": "^5.1.1", "eslint": "^9.39.1", "eslint-plugin-react-hooks": "^7.0.1", "eslint-plugin-react-refresh": "^0.4.24", "globals": "^16.5.0", "typescript": "~5.9.3", "typescript-eslint": "^8.48.0", "vite": "^7.3.1" } } ``` ### backend/scripts/requirements.txt ``` wordfreq>=3.0.0 ``` ### backend/server.js ```javascript /** * Backend server: Express HTTP API + Socket.IO for game state and image updates. * - POST /update-data -> run anagram worker on game state, broadcast result to all WS clients. * - POST /update-image -> run image worker, broadcast to WS clients. * - WebSocket path /receive-data -> clients receive broadcasted 'data' events. */ import express from "express"; import { createServer } from "http"; import { Server } from "socket.io"; import { Worker } from "worker_threads"; import { fileURLToPath } from "url"; import { dirname, join } from "path"; import cors from "cors"; import { readFile } from "fs/promises"; const __dirname = dirname(fileURLToPath(import.meta.url)); const PORT = process.env.PORT || 3000; const app = express(); app.use(cors()); app.use(express.json({ limit: "10mb" })); app.use(express.raw({ type: "application/octet-stream", limit: "10mb" })); const httpServer = createServer(app); // Socket.IO on path /receive-data — clients connect at ws://host/receive-data and get 'data' events const io = new Server(httpServer, { path: "/receive-data", cors: { origin: "*", }, }); /** Send a message to every connected WebSocket client. */ function broadcast(message) { io.emit("data", message); } /** Broadcast move log entries to all connected clients via dedicated event. */ function broadcastMoveLog(entries) { io.emit("move-log", { entries }); } /** Broadcast image updates to all connected clients via dedicated event. */ function broadcastImage(imageData) { io.emit("image", imageData); } const workerPath = join(__dirname, "worker.js"); let sharedWorker = null; /** @type {{ kind: string, payload: unknown, resolve: (r: unknown) => void, reject: (e: Error) => void }[] */ const workerQueue = []; function onWorkerMessage(msg) { const next = workerQueue.shift(); if (next) { if (msg.ok) next.resolve(msg); else next.reject(new Error(msg.error)); } const pending = workerQueue[0]; if (pending) sharedWorker.postMessage({ kind: pending.kind, payload: pending.payload, }); } /** * Run worker with the given message kind and payload. Uses a single long-lived worker and a queue. * Resolves with worker result or rejects on error. Worker state (e.g. subset cache) is reused across requests. */ function runWorker(kind, payload) { return new Promise((resolve, reject) => { workerQueue.push({ kind, payload, resolve, reject }); if (workerQueue.length === 1) { if (!sharedWorker) { sharedWorker = new Worker(workerPath, { workerData: {} }); sharedWorker.on("message", onWorkerMessage); sharedWorker.on("error", (err) => { const p = workerQueue.shift(); if (p) p.reject(err); sharedWorker = null; }); } sharedWorker.postMessage({ kind, payload }); } }); } io.on("connection", () => { // Clients connected on /receive-data; they receive via io.emit('data', ...) }); // Load definitions file (lazy load, cached in memory) let definitionsCache = null; const definitionsPath = join(__dirname, "data", "definitions.json"); async function loadDefinitions() { if (definitionsCache === null) { try { const data = await readFile(definitionsPath, "utf-8"); definitionsCache = JSON.parse(data); console.log(`Loaded ${Object.keys(definitionsCache).length} definitions`); } catch (err) { console.error("Failed to load definitions:", err.message); definitionsCache = {}; // Empty cache on error } } return definitionsCache; } // GET /definition/:word — get definition for a word app.get("/definition/:word", async (req, res) => { try { const word = req.params.word.toLowerCase().trim(); if (!word) { return res.status(400).json({ ok: false, error: "Word parameter required" }); } const definitions = await loadDefinitions(); const definition = definitions[word]; if (definition) { res.json({ ok: true, word, definition }); } else { res.json({ ok: true, word, definition: null }); } } catch (err) { res.status(500).json({ ok: false, error: err.message }); } }); // POST /update-data — receive game state (JSON), process in worker, broadcast result to all WS clients app.post("/update-data", async (req, res) => { try { const payload = req.body; const workerResponse = await runWorker("game-state", payload); const { result, logEntries } = workerResponse; // Broadcast game state to all clients broadcast(result); // Broadcast log entries via dedicated event if any were created if (logEntries && logEntries.length > 0) { broadcastMoveLog(logEntries); } res.json({ ok: true, broadcast: io.engine.clientsCount }); } catch (err) { res.status(500).json({ ok: false, error: err.message }); } }); // POST /update-image — accept JSON (metadata/base64) or raw binary; broadcast via dedicated 'image' event app.post("/update-image", async (req, res) => { try { const isJson = req.is("application/json"); const payload = isJson ? req.body : { binary: true, length: req.body?.length ?? 0, base64: req.body?.length ? req.body.toString("base64") : undefined, }; if ( isJson && (!payload || (typeof payload === "object" && !Object.keys(payload).length)) ) { return res .status(400) .json({ ok: false, error: "Expected JSON body or image data" }); } const workerResponse = a [truncated — 2484 more characters] ``` ### frontend/src/main.tsx ```typescript import { StrictMode } from 'react' import { createRoot } from 'react-dom/client' import './index.css' import App from './App.tsx' createRoot(document.getElementById('root')!).render( , ) ``` ### frontend/src/App.tsx ```typescript import { useEffect, useState } from "react"; import { io, Socket } from "socket.io-client"; import StudentView from "./StudentView"; import ValidationView from "./ValidationView"; import TeacherView from "./TeacherView"; import type { TeacherGameData } from "./types/stats"; type ViewMode = "student" | "validation" | "teacher"; interface Player { words: string[]; } // Backend data structure (snake_case from API) interface BackendGameData { players: Player[]; recommended_words: Record; availableLetters: string; } // Frontend data structure (camelCase for consistency) export interface GameData { players: Player[]; recommendedWords: Record; availableLetters: string; } // Connect to backend WebSocket const socket: Socket = io("http://localhost:3000", { path: "/receive-data", autoConnect: false, }); const temporaryGameData: GameData = { players: [ { words: ["app", "banana", "cherry"] }, { words: ["dog", "cat", "bird"] }, ], availableLetters: "le", recommendedWords: { apple: ["app", "l", "e"], }, }; function App() { const [gameData, setGameData] = useState( temporaryGameData, ); const [teacherGameData, setTeacherGameData] = useState(null); const [isConnected, setIsConnected] = useState(false); const viewMode: ViewMode = (() => { const params = new URLSearchParams(window.location.search); const view = params.get("view"); if (view === "validation" || view === "teacher") return view; return "student"; })(); useEffect(() => { const titles: Record = { student: "Pirates - Student", validation: "Pirates - Data", teacher: "Pirates - Teacher", }; document.title = titles[viewMode]; }, [viewMode]); useEffect(() => { socket.connect(); function onConnect() { setIsConnected(true); console.log("Connected to socket server"); } function onDisconnect() { setIsConnected(false); console.log("Disconnected from socket server"); } function onGameState(data: any) { console.log("Received game state:", data); // Transform backend snake_case to frontend camelCase for basic views const transformedData: GameData = { players: data.players, availableLetters: data.availableLetters, recommendedWords: data.recommended_words, }; setGameData(transformedData); // Store full data including move and analytics for teacher view const teacherData: TeacherGameData = { ...transformedData, move: data.move, _analytics: data._analytics, }; setTeacherGameData(teacherData); } socket.on("connect", onConnect); socket.on("disconnect", onDisconnect); socket.on("data", onGameState); return () => { socket.off("connect", onConnect); socket.off("disconnect", onDisconnect); socket.off("data", onGameState); socket.disconnect(); }; }, []); return ( <>
{isConnected ? "Connected" : "Disconnected"}
Pirates logo

{/* Shadow layer */} Pirates {/* Main text */} Pirates

{viewMode == "student" && "Play"} {viewMode == "teacher" && "Teach"} {viewMode == "validation" && "Learn"}
{gameData && viewMode === "student" && ( )} {gameData && viewMode === "validation" && ( frontend
``` [52 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]