# Project export: Ghost Fighter This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: Turn human movements into robot-ready skill cards. Analyze, improve, and battle custom robot moves in a creator platform for the future of robot sports. - Devpost: https://devpost.com/software/ghost-fighter - GitHub: https://github.com/lyeric2022/aaa - Video: https://www.youtube.com/embed/k9QzWbXGmCU?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 6 GitHub contributor(s) — lyeric2022 (18 commits), Sparsh Bansal (17 commits), Amanda Chen (6 commits), Cursor (5 commits), Claude Opus 4.8 (1M context) (4 commits), Natasha (2 commits) ## Devpost submission (written by the team) ### Inspiration Humanoid robot sports are coming. Projects like Ultimate Bots and Ghost Trials make it possible to capture human movement and deploy it onto humanoid robots. But we noticed a missing step in the workflow: How do creators know whether a move is actually robot-ready before it reaches hardware? A punch combo might look amazing on camera but be unstable, inefficient, or difficult for a humanoid robot to execute safely. Today, most motion-capture workflows stop at visualization. We wanted to build the layer that evaluates, coaches, and stress-tests human-created movements before deployment. That's how Ghost Fighter was born. Our vision is simple: treat human movement like a draft pick for a robot athlete. Every move should be scored, analyzed, improved, and battle-tested before it ever reaches a real robot. ### What it does Ghost Fighter transforms human-created motions into structured robot skills. A creator can export a motion from Ultimate Bots Studio, and Ghost Fighter automatically: Ingests SONIC trajectory data Maps joints from IsaacLab ordering to MuJoCo / Unitree G1 ordering Generates a structured move card Scores robot readiness and deployability Provides AI coaching feedback Replays the motion on a simulated Unitree G1 Tests the move inside a robot-sports arena Instead of viewing motion as a video, we treat it as a reusable robot skill. Each move receives attributes such as: Speed Power Smoothness Balance Risk Recovery Deployability We also generate coach-style feedback such as: "Looks powerful, but widening your stance before commitment would improve balance and recovery." During the hackathon we shipped two fully functioning skills: Ghost Jab Combo Block (including full SONIC defensive animation replay) ### How we built it We built a complete end-to-end Physical AI pipeline: Human Motion → SONIC Export → Move Analysis → G1 Replay → Robot Sports Arena Motion Processing We ingest SONIC motion trajectories exported from UFB Studio and convert them into a format compatible with the Unitree G1 humanoid model. To achieve this, we implemented: Joint remapping between IsaacLab and MuJoCo conventions Motion playback systems Move-card generation and scoring Skill metadata extraction AI Agents We introduced specialized agents that each serve a distinct role: Coach Agent Provides actionable feedback Suggests improvements for robot safety and execution quality Judge Agent Evaluates whether a move is deployable Produces readiness assessments Announcer Agent Generates live fight commentary Creates the feeling of a robot-sports broadcast For voice commentary, we integrated Deepgram to bring matches to life. Arena Simulation We built a lightweight robot-sports arena where players can test moves in real gameplay scenarios. The arena includes: Health Stamina Balance Positioning Range checking Blocking mechanics Recovery windows Moves can be evaluated not just visually, but competitively. We also implemented multiplayer support so teammates can join the same arena and test move cards against one another. ### Challenges we ran into The hardest challenge was building a reliable bridge between motion-capture data and humanoid robot playback. Different robotics systems use different joint ordering conventions, so motion data could not simply be imported and replayed directly. We had to carefully map and validate trajectories between SONIC exports, simulation environments, and the Unitree G1 model. Another challenge was deciding where AI should actually add value. Rather than using LLMs everywhere, we focused on specialized agents that provide meaningful coaching, judging, and commentary while keeping gameplay deterministic and responsive. Finally, we had to balance two very different experiences: A serious Physical AI evaluation platform A fun robot-sports game Designing a system that could be both useful and entertaining within 48 hours was one of the most rewarding parts of the project. ### Accomplishments we're proud of Built a complete human-motion-to-robot pipeline Replayed real SONIC trajectories on a Unitree G1 model Generated structured robot skill cards Developed AI coaching and judging systems Created a multiplayer robot-sports arena Integrated live AI-powered commentary Delivered an end-to-end working demo in under 48 hours Most importantly, we transformed raw human motion into something measurable, coachable, and competitive. ### What we learned Building Ghost Fighter taught us that the future of Physical AI is not just about making robots move. It's about translating messy human creativity into structured robot behavior. A useful robot-sports ecosystem needs more than robots. It needs creators, coaches, evaluation tools, testing environments, and feedback loops. We believe Ghost Fighter represents an early version of that creator ecosystem. ### What's next This hackathon gave us the foundation for a much larger platform. Next steps include: Direct Coach and Judge integration into the move-builder workflow Custom fighter loadouts Persistent player progression and leaderboards Redis-powered move memory and analytics Automated highlight reel generation Larger-scale multiplayer tournaments Hardware validation workflows for real humanoid robots Long term, we envision Ghost Fighter becoming a creator platform for robot athletes. Anyone should be able to design a move, receive AI coaching, test it in competition, and eventually watch it perform on real humanoid hardware. One-line summary Ghost Fighter turns human movement into ranked, coachable, battle-tested robot skills—the layer between human creativity and the future of humanoid robot sports. ## README (from the GitHub repository) # Ghost Fighter Ghost Fighter is a Physical AI / robot-sports platform that turns human movement into structured robot skills. The demo path is: ```text human motion → UFB Studio / SONIC export → scored move card → 3D G1 replay → robot-sports arena ``` We score whether a human-created move is clean, expressive, and robot-executable enough to become a reusable robot-athlete skill. ## Run the app ```bash cd web npm install npm run dev ``` Open: - **App:** http://localhost:3000 - **Move card / Skill Lab:** http://localhost:3000/moves/ghost_jab_combo_sonic - **3D Arena:** http://localhost:3000/arena ### Run Fetch.ai Judge/Coach agents The live Judge panel calls a local bridge, which sends move stats to the Judge uAgent and asks the Coach uAgent for fixes when needed. Before using the live Judge panel, run these in separate terminals: ```bash cd agents conda activate ghost-fighter-agents python coach_agent.py ``` ```bash cd agents conda activate ghost-fighter-agents python judge_agent.py ``` ```bash cd agents conda activate ghost-fighter-agents uvicorn web_bridge:app --port 8010 --reload ``` Then start the web app with the bridge URL: ```bash cd web JUDGE_BRIDGE_URL=http://localhost:8010 npm run dev ``` The agents read `agents/.env`. Make sure it includes `COACH_ADDRESS`, `JUDGE_ADDRESS`, `COACH_ENDPOINT`, `JUDGE_ENDPOINT`, and `BRIDGE_MODE=agent`. ### Optional: arena announcer voice The arena can call fights with Deepgram TTS. Copy the example env file and add your key: ```bash cp .env.local.example .env.local # edit web/.env.local and set DEEPGRAM_API_KEY=... ``` Without a key, the app builds and runs normally — you just see “Announcer off” in the arena. ### Production build ```bash cd web npm install npm run build npm start ``` If you see `Can't resolve '@deepgram/sdk'`, run `npm install` in `web/` first. The dependency is already listed in `package.json`. ## What to demo 1. Open `/moves/ghost_jab_combo_sonic`. 2. Show the move card: speed, power, smoothness, balance risk, recovery, deployability, and coach feedback (hover the **i** icons for definitions). 3. Show the 3D SONIC replay: G1 URDF driven by remapped `joint_pos.csv` trajectories. 4. Open `/arena`. 5. Use each player’s move buttons to trigger a 3D robot duel with HP bars, balance bars, hit effects, and knockback. 6. Toggle the announcer if `DEEPGRAM_API_KEY` is configured. 7. Mention the Unitree G1 assets loaded from `web/public/models/g1_description` and `web/public/models/unitree_g1`. ## Routes - `/` — dashboard and move library - `/ingest` — upload a SONIC `.zip` or source video - `/moves/[id]` — skill card, 3D replay, verification ladder - `/fighters/build` — create a fighter loadout from move cards - `/arena` — 3D robot-sports duel with health bars and move playback - `/leaderboard` — top moves and fighters by deployability ## AI opponent ("the enemy") The arena ships with a deterministic, seeded AI opponent. It satisfies the same per-frame controller contract as the human input handler, reuses the existing Move Card stats and arena loop/physics, and runs **no LLM in the frame loop**. It thinks at three internal rates (mapped onto the arena tick by an adapter): - **Strategist (~3 Hz)** — game-plan + online player-modeling (move-frequency, anti-spam counter-bias, intent: `pressure` / `zone` / `counter` / `reset`). - **Tactician (~15 Hz)** — Utility-AI scoring of every Move Card *now* from the existing stats (range fit, whiff-punish/interrupt timing, safety, intent match, payoff − stamina − balance_risk − recovery) with seeded noise. - **Executor (per frame)** — footwork toward preferred range, else commit the move via the arena's move-commit API. Difficulty knobs (`reaction_delay`, `optimal_prob`, `mistake_rate`, `adaptation`, `noise`) are orthogonal to style (intent). Code lives in `web/lib/enemy/`. ### Personas `Rusher`, `Zoner`, `Counter-Puncher`, `Adapter-Boss` (`web/lib/enemy/personas.ts`). **Select one in the UI:** open `/arena` and use the **Player 2 AI** dropdown. Choose a persona to let it pilot Player 2; "Manual (human)" returns control. **Headless:** `POST /api/arena/fight` with `{ fighter_a, fighter_b, persona_b, seed }` — when `persona_b` is set, that persona drives fighter_b. `seed` makes the match replayable. ### Evaluation harness (competition-readiness) Run a fighter's deck against the full persona pool through the **real** arena and get a win-rate + why-it-loses profile (and a readiness score blended with the existing Deployability): ```bash curl -X POST http://localhost:3000/api/arena/evaluate \ -H 'Content-Type: application/json' \ -d '{ "fighter": "", "matches": 5, "seed": 1 }' ``` ### Tests ```bash cd web npm test # vitest, fully offline (seeded unit + real-arena integration) ``` ## Motion replay notes SONIC zip exports store joints in **IsaacLab/internal order** (`joint_0`…`joint_28`). The web replay and trajectory API remap that to **MuJoCo / Unitree SDK order** before driving the G1 URDF (`web/lib/g1Motion.ts`). Lafan CSV exports use a different layout (`XYZ` + `QX QY QZ QW` + 29 joints at 30fps). Use those for training pipelines like mjlab; the in-app replay expects the SONIC zip CSVs. ## CLI scoring ```bash python3 scripts/analyze_motion.py ``` This reads the extracted SONIC CSVs and writes: ```text move_cards/ghost_jab_combo.json ``` ## Assets - `assets/motions/ghost_jab_combo_sonic.zip` — Studio SONIC export for G1 deploy - `assets/motions/ghost_jab_combo_extracted/` — extracted SONIC CSVs - `assets/motions/block_sonic.zip` — defensive block SONIC export - `assets/motions/block_extracted/` — extracted block motion CSVs - `web/public/models/unitree_g1/` — MuJoCo Menagerie Unitree G1 reference assets - `web/public/models/g1_description/` — Unitree ROS G1 URDF + meshes for browser rendering - `scripts/verify_with_gear_sonic.sh` — prepares the motion folder for official GEAR-SONIC/MuJoCo verification ## Stack - Next.js 15 + React 19 - Three.js + URDFLoader for 3D replay and arena - TypeScript scoring engine ported from `scripts/analyze_motion.py` - JSON file store in `web/data/` during local dev - Optional Deepgram TTS for arena announcer (`/api/tts`) - SONIC / G1 motion assets from [UFB Studio](https://studio.ultimatebots.com/editor) ## Pitch Ghost Fighter turns human moves into ranked, coachable, deployable robot skills. Studio retargets the motion; Ghost Fighter scores it, visualizes it on a G1 model, turns it into a move card, and lets it fight in a robot-sports arena. ## Detected evidence (automated analysis) Indexed codebase: 98 recognized source files, 690 KB. - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - JavaScript (language) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: commit authorship or trailers - AI coding agent: Cursor — evidence: commit authorship or trailers ## Codebase structure (from repository index) ### Files (120 of 374) ``` .gitignore agents/.env.example agents/agent1qvhpl2pxgu_data.json agents/coach_agent.py agents/core.py agents/judge_agent.py agents/protocols.py agents/README.md agents/requirements.txt agents/sample_caller.py agents/web_bridge.py assets/demo/.gitkeep assets/motions/block_extracted/body_ang_vel.csv assets/motions/block_extracted/body_lin_vel.csv assets/motions/block_extracted/body_pos.csv assets/motions/block_extracted/body_quat.csv assets/motions/block_extracted/info.txt assets/motions/block_extracted/joint_pos.csv assets/motions/block_extracted/joint_vel.csv assets/motions/block_extracted/metadata.txt assets/motions/ghost_jab_combo_extracted/body_ang_vel.csv assets/motions/ghost_jab_combo_extracted/body_lin_vel.csv assets/motions/ghost_jab_combo_extracted/body_pos.csv assets/motions/ghost_jab_combo_extracted/body_quat.csv assets/motions/ghost_jab_combo_extracted/info.txt assets/motions/ghost_jab_combo_extracted/joint_pos.csv assets/motions/ghost_jab_combo_extracted/joint_vel.csv assets/motions/ghost_jab_combo_extracted/metadata.txt demo/move_card.html docs/demo_story.md docs/enemy_integration_plan.md move_cards/block.json move_cards/ghost_counter_cross.json move_cards/ghost_jab_combo.json README.md scripts/analyze_motion.py scripts/plaza_checklist.md scripts/verify_with_gear_sonic.sh web/.env.local.example web/.gitignore web/app/api/arena/evaluate/route.ts web/app/api/arena/fight/route.ts web/app/api/arena/room/[id]/events/route.ts web/app/api/arena/room/[id]/route.ts web/app/api/arena/room/route.ts web/app/api/fighters/route.ts web/app/api/ingest/route.ts web/app/api/leaderboard/route.ts web/app/api/moves/[id]/judge/route.ts web/app/api/moves/[id]/plaza/route.ts web/app/api/moves/[id]/route.ts web/app/api/moves/[id]/trajectory/route.ts web/app/api/moves/[id]/verification/route.ts web/app/api/moves/route.ts web/app/api/tts/route.ts web/app/arena/page.tsx web/app/globals.css web/app/ingest/page.tsx web/app/layout.tsx web/app/leaderboard/page.tsx web/app/moves/[id]/page.tsx web/app/page.tsx web/components/Arena3D.tsx web/components/CameraDebugPanel.tsx web/components/LiveJudgePanel.tsx web/components/MoveAnnouncerButton.tsx web/components/MoveCardView.tsx web/components/Nav.tsx web/components/PlazaUpload.tsx web/components/RobotReplay3D.tsx web/components/StatBar.tsx web/components/VerificationPanel.tsx web/eslint.config.mjs web/lib/analyze.ts web/lib/announcer.ts web/lib/arena-physics.ts web/lib/arena.ts web/lib/arenaCombat.ts web/lib/arenaEnvironment.ts web/lib/arenaRoomStore.ts web/lib/cameraFrame.ts web/lib/deepgram.ts web/lib/defaultMoves.ts web/lib/enemy/__tests__/arena-integration.test.ts web/lib/enemy/__tests__/fixtures.ts web/lib/enemy/__tests__/strategist.test.ts web/lib/enemy/__tests__/tactician.test.ts web/lib/enemy/controller.ts web/lib/enemy/evaluate.ts web/lib/enemy/executor.ts web/lib/enemy/personas.ts web/lib/enemy/rng.ts web/lib/enemy/strategist.ts web/lib/enemy/tactician.ts web/lib/enemy/types.ts web/lib/g1Motion.ts web/lib/judge.ts web/lib/judgeBridge.ts web/lib/storage.ts web/lib/types.ts web/lib/useArenaMultiplayer.ts web/lib/useCameraDebug.ts web/next.config.ts web/package.json web/postcss.config.mjs web/public/models/g1_description/g1_23dof_mode_10.urdf web/public/models/g1_description/g1_23dof_rev_1_0.urdf web/public/models/g1_description/g1_23dof_rev_1_0.xml web/public/models/g1_description/g1_23dof.urdf web/public/models/g1_description/g1_23dof.xml web/public/models/g1_description/g1_29dof_lock_waist_rev_1_0.urdf web/public/models/g1_description/g1_29dof_lock_waist_rev_1_0.xml web/public/models/g1_description/g1_29dof_lock_waist_with_hand_rev_1_0.urdf web/public/models/g1_description/g1_29dof_lock_waist_with_hand_rev_1_0.xml web/public/models/g1_description/g1_29dof_lock_waist.urdf web/public/models/g1_description/g1_29dof_lock_waist.xml web/public/models/g1_description/g1_29dof_mode_11.urdf web/public/models/g1_description/g1_29dof_mode_12.urdf web/public/models/g1_description/g1_29dof_mode_13.urdf web/public/models/g1_description/g1_29dof_mode_14.urdf [254 more files omitted for size] ``` ### Dependencies - agents/requirements.txt: fastapi@>=0.110.0, openai@>=1.0.0, python-dotenv@>=1.0.0, uagents@==0.23.6, uagents-core@==0.4.0, uvicorn[standard]@>=0.29.0 - web/package.json: @deepgram/sdk@^5.4.0, @eslint/eslintrc@^3, @tailwindcss/postcss@^4, @types/adm-zip@^0.5.8, @types/node@^20, @types/react@^19, @types/react-dom@^19, @types/three@^0.184.1, @types/uuid@^10.0.0, adm-zip@^0.5.17, eslint@^9, eslint-config-next@15.5.19, next@15.5.19, react@19.1.0, react-dom@19.1.0, tailwindcss@^4, three@^0.184.0, typescript@^5, urdf-loader@^0.13.0, uuid@^14.0.1, vite@^6.4.3, vitest@^3.2.6 ### Recent commits (newest first) - Document Judge and Coach agent startup - Merge remote main into agent bridge work - Call Judge and Coach agents from web bridge - Improve block defense, G1 floor alignment, and fullscreen arena sizing. - Wire Fetch.ai live judge into ingest and move pages. - Merge branch 'amanda/agent-fetch-ai' - Run live Judge evaluation from move cards. - Merge pull request #4 from lyeric2022/opponent - Merge main into opponent: footwork + block trajectory arena - Improve arena viewport controls and keyboard move bindings. - Fix online movement: reinstall room sim loop on dev HMR reload - Add block move and polish arena combat presentation. - Add server-authoritative footwork so players move in online play - Fix broken keyboard movement: make online multiplayer opt-in - Integrate arenaCombat module and online multiplayer into opponent branch - Fix AI reach mismatch, arrow-key focus guard, lunge clamp, rope normals - Merge pull request #5 from lyeric2022/amanda/agent-fetch-ai - Add Fetch.ai judge and coach agents into web - Merge remote-tracking branch 'origin/main' into amanda/agent-fetch-ai - Add real-time synced arena rooms for remote 1v1 over ngrok. ## Key source files (fetched from GitHub, selected and truncated for size) ### scripts/plaza_checklist.md ```markdown # Plaza deploy checklist ## File to bring - `assets/motions/ghost_jab_combo_sonic.zip` (or `~/Downloads/untitled_project_sonic.zip`) ## At Lower Sproul Plaza 1. Find Ultimate Bots + Unitree G1 station. 2. Say: **"I have a SONIC zip from Studio, ready to deploy."** 3. Hand them the zip (AirDrop / USB / Slack if they prefer). 4. Film wide + tight shots of the robot run. 5. Save the clip to `assets/demo/plaza_g1_run.mp4` when you get back. ## Demo narration (15 sec) > We recorded a fight combo, retargeted it in Ultimate Bots Studio, scored the SONIC trajectory for balance and smoothness, and deployed the validated zip on a real G1. ``` ### docs/demo_story.md ```markdown # Ghost Fighter Demo Story ## What to show 1. Open `/moves/ghost_jab_combo`. 2. Start with the **3D SONIC Replay**. - Say: "This is driven by the exported 29-DOF SONIC joint trajectory, not a video." 3. Show the **Robotics Verification** ladder. - Studio SONIC: retarget/export passed. - 3D Replay: browser visualization from `joint_pos.csv`. - MuJoCo / GEAR: official verification slot. - G1 Plaza: hardware proof slot. 4. Scroll to the scorecard. - Explain speed, smoothness, balance risk, recovery, deployability. 5. Show the arena as the creator/game layer. - Say: "The arena ranks robot skills; serious moves graduate into MuJoCo and G1 deployment." ## What not to claim - Do not claim the browser 3D replay is full MuJoCo physics. - Do not claim the arena is contact-rich humanoid simulation. - Do not claim the scoring is trained ML. ## Winning claim Ghost Fighter is a robot-skill scouting platform: > creators record human motion, Studio retargets it to SONIC, Ghost Fighter scores and ranks it, top moves get verified in 3D/MuJoCo, then deployed to a real G1. ## One-liner We turn human fight motions into ranked, coachable, deployable robot skills. ``` ### agents/requirements.txt ``` uagents==0.23.6 uagents-core==0.4.0 openai>=1.0.0 python-dotenv>=1.0.0 fastapi>=0.110.0 uvicorn[standard]>=0.29.0 ``` ### web/package.json ``` { "name": "web", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev --turbopack", "build": "next build --turbopack", "start": "next start", "lint": "eslint", "test": "vitest run", "test:deepgram": "node --env-file=.env.local scripts/test-deepgram.mjs" }, "dependencies": { "@deepgram/sdk": "^5.4.0", "adm-zip": "^0.5.17", "next": "15.5.19", "react": "19.1.0", "react-dom": "19.1.0", "three": "^0.184.0", "urdf-loader": "^0.13.0", "uuid": "^14.0.1" }, "devDependencies": { "@eslint/eslintrc": "^3", "@tailwindcss/postcss": "^4", "@types/adm-zip": "^0.5.8", "@types/node": "^20", "@types/react": "^19", "@types/react-dom": "^19", "@types/three": "^0.184.1", "@types/uuid": "^10.0.0", "eslint": "^9", "eslint-config-next": "15.5.19", "tailwindcss": "^4", "typescript": "^5", "vite": "^6.4.3", "vitest": "^3.2.6" } } ``` ### web/app/layout.tsx ```typescript import type { Metadata } from "next"; import { Geist, Geist_Mono } from "next/font/google"; import { Nav } from "@/components/Nav"; import "./globals.css"; const geistSans = Geist({ variable: "--font-geist-sans", subsets: ["latin"], }); const geistMono = Geist_Mono({ variable: "--font-geist-mono", subsets: ["latin"], }); export const metadata: Metadata = { title: "Ghost Fighter — Robot Skill Lab", description: "Human motion → evaluable robot skills → 1v1 arena", }; export default function RootLayout({ children, }: Readonly<{ children: React.ReactNode }>) { return (