Project Info

BAYMAX

Inspiration

Humanoid robots have two arms, two legs, and a torso but almost every demo treats the whole body as one unit controlled by one script. We wanted to build a robot where every body part has its own AI agent, its own reasoning, and its own ability to act independently at the same time. A nervous system for a robot, not a remote control. We chose assistive guidance for the blind as our first application because it perfectly demonstrates the core idea: one arm guides a person while the other scans the environment completely independently. ##

What it does

Baymax is a multi-agent architecture for humanoid robots where every part of the robot thinks and acts independently at the same time. As a first application, the robot guides visually impaired people by holding their hand with one arm and communicating direction through touch, while the other arm sweeps for obstacles, raises as a barrier, or halts palm-out in an emergency. Eight specialized agents each control a different part of the robot in parallel over Band's multi-agent platform. ##

How we built it

We modeled the system after the human nervous system. A Vision Agent reads the camera via LiveKit and generates scene descriptions with Gemini 2.5 Flash. A Threat Agent watches for hazards in parallel and fires an emergency reflex that bypasses the normal decision loop. A Conductor makes navigation decisions and dispatches tasks to three motor agents simultaneously controlling each arm and the legs independently. A Safety Agent vetoes unsafe plans before any command reaches the robot. A Spine Agent can halt everything in under 100ms. All agents communicate over Band and drive the robot via the Booster SDK. ##

Challenges we ran into

Coordinating eight AI agents without deadlocking or desynchronizing was the core challenge. LangGraph's recursion limit crashed the conductor mid-cycle. The reflex path had to stay fast enough to be useful. Booster Studio required a licensed account we didn't have so we pivoted to open source Webots. Band silently rejected messages without an @mention which broke command delivery for hours. ##

Accomplishments we're proud of

The reflex arc actually works, halting the robot in under 100ms before the conscious decision loop finishes. The two arms genuinely operate independently with separate agents doing separate reasoning simultaneously. That is not a demo trick, it is the actual architecture. The end to end pipeline closes the full loop: camera sees the world, agents think, robot moves, repeat.

What we learned

Distributed AI systems fail in ways monolithic scripts never do. Timing, ordering, and deduplication matter enormously. The human nervous system turned out to be a surprisingly accurate model for robotics. We also learned that the gap between sim and hardware is mostly just a network address. The hard part is the intelligence layer, not the hardware interface. ##

What's next

Baymax is a foundation, not a finished product. The same architecture that lets one arm guide a person while the other sweeps for obstacles can let a robot cook while monitoring a patient or carry objects while navigating a crowd. Any task that benefits from a body doing two things at once with independent reasoning is a candidate. Near term: integrating GR00T N1.7 for natural language understanding and training a proper locomotion policy via Isaac Lab. The codebase is ready, it just needs GPU time.

Baymax — A Nervous System for a Humanoid Guide Robot

Baymax is a humanoid guide robot that safely walks a blindfolded person through the world — perceiving hazards, planning a path, and steering with gentle hand signals (left / right / stop / forward).

Its control system is built as a biologically-inspired multi-agent nervous system: each AI agent maps to a region of the human brain, and the agents coordinate over Band the way neurons exchange signals. A camera feed enters as perception, flows through cortical planning and a fast-path reflex circuit, and exits as velocity commands that drive the robot.

The flagship demo runs the full pipeline end to end against the Booster T1 humanoid in a Webots physics simulation — camera → agents → Band → robot SDK → simulator — from a single command.

Architecture

Eight agents, each modeled on a brain region, communicate through a shared Band room. Perception fans out to specialized agents that run in parallel; their outputs are arbitrated into a single safe velocity command.

Agent	Brain region	Responsibility
Vision	Sensory Cortex	Reads the camera feed, describes the scene (obstacles, people, terrain)
Conductor	Prefrontal Cortex	Plans the route and dispatches tasks to the limb agents
UpperRight	Motor Cortex	Drives the guide arm that signals the person
UpperLeft	Motor Cortex	Drives the free arm that scans the environment
Lower	Cerebellum	Manages walking pace; slows and stops at curbs, drops, and obstacles
Threat	Amygdala	Detects sudden danger and fires the fast-path reflex
Spine	Spinal Cord	Reflex coordinator — halts the limb agents the instant Threat fires
Safety	Brainstem	Vetoes any unsafe command and issues the final stop

 camera ─▶ Vision ─▶ Band room ─┬─▶ Conductor ─▶ UpperLeft / UpperRight / Lower ─┐
                                │                                                 ├─▶ FINAL_COMMAND
                                └─▶ Threat ─▶ Spine ─(reflex halt)────────────────┘        │
                                                       Safety (veto / stop) ◀──────────────┘
                                                                                            ▼
                                                            command bridge ─▶ robot SDK ─▶ Booster T1 (Webots)

Two paths run concurrently: a cortical path (Conductor plans, limbs act) and a faster reflex path (Threat → Spine) that can halt motion without waiting on the planner. The command bridge arbitrates incoming commands — emergency stops always win — enforces a no-command STOP failsafe, and maps the winning command to a {vx, vy, vyaw} velocity for the robot.

Quick start

The demo runs on Ubuntu 22.04 (WSL2 is supported). It drives the Booster T1 humanoid in a Webots simulation, so a desktop with OpenGL is required.

Prerequisites (run once)

# 1. Booster Robotics SDK (clone it, run its install.sh, then build the binding)
bash scripts/build_sdk_22.sh

# 2. Python 3.11 venv for the robot-side listener (Band + SDK)
bash scripts/setup_bridge_311.sh

# 3. Download the Booster T1 Webots world and control runner (~1.3 GB)
bash scripts/setup_t1_sim.sh

# 4. Add the agent + camera dependencies to the same venv
bash scripts/setup_demo_venv.sh

Then copy .env.example to .env and fill in your Gemini and Band credentials.

Run the demo

bash scripts/run_demo.sh

This single command brings up the whole pipeline in order — a fresh Band room, Webots with the T1 world, the control runner, the command listener, and the eight agents plus the camera — and tears everything down cleanly on Ctrl-C. Tune the gait speed with BAYMAX_SPEED=1.2 bash scripts/run_demo.sh.

Repository layout

agents/            The eight Band agents + shared config and LLM setup
  shared/          AGENT_CONFIGS, Band URLs, LLM provider selection
robot/             Robot-side I/O
  command_bridge.py  Band FINAL_COMMAND -> arbitration/failsafe -> velocity -> SDK
  sim_camera.py      Synthetic camera publisher (LiveKit)
  b1_loco_client_sink.py  Booster SDK motion sink
scripts/           Setup scripts + the one-command demo (run_demo.sh)
clean_and_reset.py Creates a fresh Band room before each run

Tech stack

Agents: Band multi-agent framework · LangGraph · LangChain
LLM: Google Gemini 2.5 Flash (default) or Nebius AI Studio (open models)
Perception transport: LiveKit · OpenCV
Robot: Booster Robotics SDK · Booster T1 humanoid · Webots simulation

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

LangChainIn code
PythonIn code
FastAPIClaimed

2 of 3 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

Claude CodeConfig · Commits

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

89 KB

Source files

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.