# Project export: Daemon This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: Teaching AI how to understand its body and act. - Devpost: https://devpost.com/software/daemon-qwebpd - GitHub: https://github.com/Sachin-dot-py/Daemon - Video: https://www.youtube.com/embed/iPFMnFJEYhA?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — Sachin Ramanathan (41 commits), Ved Panse (10 commits) ## Devpost submission (written by the team) ### Overview We started with something simple: an RC car. It could move forward, backward, turn left, turn right. That was it. It worked. We were happy. Then we thought — what if we add a robotic arm? So we bought one. From a completely different manufacturer. Different firmware. Different assumptions. Different everything. And that’s when we hit the wall. You don’t just plug in an arm and magically get a smarter robot. You end up rewriting firmware, changing control logic, debugging serial communication, fixing timing issues, and basically rebuilding half your system just to make two pieces of hardware cooperate. Every time you add something new, you start over. That didn’t feel right. The AI was capable. The hardware worked. But the AI had no idea what its own body was. So we asked a simple question: What if the robot could figure that out on its own? That’s why we built Daemon. Daemon lets AI learn what its body parts are and how to use them. You attach a new arm. Instead of hardcoding support for it, the AI begins exploring: “Oh, this rotates.” “This joint moves up and down.” “This closes.” “If I close this around something, I can grab it.” It builds an internal model of what it can do. Not because we wrote special case logic. But because it tried, failed, adjusted, and learned. Let’s say you tell the robot: “Go pick up that banana.” The AI understands the goal. But it doesn’t automatically know how to use the arm perfectly. So it experiments. It drives forward. Too far. Adjusts. It lowers the arm. Misses. Tries again. It grips too early. Drops it. Learns. You point your laptop camera at the robot while it practices. That visual feedback becomes the signal that tells it whether it succeeded or failed. It keeps iterating. And here’s the part we love: You can leave. Go grab dinner. Come back a few hours later. Now the robot can pick up the banana. You didn’t write new motor logic. You didn’t rewrite the firmware stack. You didn’t manually calibrate every joint. It learned how to use its new arm. Normally, hardware integration is rigid. Static. Painful. With Daemon: Adding a new part isn’t a rewrite. It’s a capability to be discovered. We separate the system into layers: The AI decides what it wants to do. Daemon handles safe execution and learning. The hardware simply exposes what it can physically do. That separation is what makes adaptation possible. Right now, robots are custom-built systems. Every upgrade is engineering overhead. Time and money are lost! If AI can understand its own body, hardware becomes modular. You could swap wheels for legs. Add a gripper. Attach a drill. And instead of rewriting everything, the intelligence adapts. We didn’t just build a robot that moves. We built a system where AI learns how to move. Firmware to DAEMON interface (daemon-cli) Manufacturers annotate existing firmware APIs, and daemon-cli generates a standardized DAEMON manifest + runtime wrapper. Firmware to DAEMON interface (daemon-cli) Manufacturers annotate existing firmware APIs, and daemon-cli generates a standardized DAEMON manifest + runtime wrapper. Node discovery and capability exposure Each hardware module (base, arm, camera, etc.) exposes its commands and telemetry as a DAEMON node over serial/TCP. Node discovery and capability exposure Each hardware module (base, arm, camera, etc.) exposes its commands and telemetry as a DAEMON node over serial/TCP. Natural-language control from desktop-app Users connect devices and type goals in plain English. They do not need to manually map commands to components. Natural-language control from desktop-app Users connect devices and type goals in plain English. They do not need to manually map commands to components. Planning + orchestration The planner converts user intent into structured action steps, and orchestrator validates, routes, and executes those steps across one or more nodes safely. Planning + orchestration The planner converts user intent into structured action steps, and orchestrator validates, routes, and executes those steps across one or more nodes safely. Closed-loop autonomy (autonomy-engine) The autonomy loop uses live camera frames plus an OpenAI-based critic to run an execute → evaluate → adjust cycle. It updates control parameters between attempts and stops when success is stable (for example, 2 consecutive successful iterations). Closed-loop autonomy (autonomy-engine) The autonomy loop uses live camera frames plus an OpenAI-based critic to run an execute → evaluate → adjust cycle. It updates control parameters between attempts and stops when success is stable (for example, 2 consecutive successful iterations). End result DAEMON turns heterogeneous firmware and hardware into one AI-operable system, enabling multi-device behavior without custom per-device orchestration logic. End result DAEMON turns heterogeneous firmware and hardware into one AI-operable system, enabling multi-device behavior without custom per-device orchestration logic. ## README (from the GitHub repository) # Daemon # Inspiration We started with something simple: an RC car. It could move forward, backward, turn left, turn right. That was it. It worked. We were happy. Then we thought — what if we add a robotic arm? So we bought one. From a completely different manufacturer. Different firmware. Different assumptions. Different everything. And that’s when we hit the wall. You don’t just plug in an arm and magically get a smarter robot. You end up rewriting firmware, changing control logic, debugging serial communication, fixing timing issues, and basically rebuilding half your system just to make two pieces of hardware cooperate. Every time you add something new, you start over. That didn’t feel right. The AI was capable. The hardware worked. But the AI had no idea what its own body was. So we asked a simple question: What if the robot could figure that out on its own? That’s why we built **Daemon**. --- # The Core Idea Daemon lets AI learn what its body parts are and how to use them. You attach a new arm. Instead of hardcoding support for it, the AI begins exploring: “Oh, this rotates.” “This joint moves up and down.” “This closes.” “If I close this around something, I can grab it.” It builds an internal model of what it can do. Not because we wrote special case logic. But because it tried, failed, adjusted, and learned. --- # How It Works (In Real Life) Let’s say you tell the robot: > “Go pick up that banana.” The AI understands the goal. But it doesn’t automatically know how to use the arm perfectly. So it experiments. It drives forward. Too far. Adjusts. It lowers the arm. Misses. Tries again. It grips too early. Drops it. Learns. You point your laptop camera at the robot while it practices. That visual feedback becomes the signal that tells it whether it succeeded or failed. It keeps iterating. And here’s the part we love: You can leave. Go grab dinner. Come back a few hours later. Now the robot can pick up the banana. You didn’t write new motor logic. You didn’t rewrite the firmware stack. You didn’t manually calibrate every joint. It learned how to use its new arm. --- # What Makes This Different Normally, hardware integration is rigid. Static. Painful. With Daemon: Adding a new part isn’t a rewrite. It’s a capability to be discovered. We separate the system into layers: The AI decides what it wants to do. Daemon handles safe execution and learning. The hardware simply exposes what it can physically do. That separation is what makes adaptation possible. --- # Why This Matters Right now, robots are custom-built systems. Every upgrade is engineering overhead. Time and money are lost! If AI can understand its own body, hardware becomes modular. You could swap wheels for legs. Add a gripper. Attach a drill. And instead of rewriting everything, the intelligence adapts. We didn’t just build a robot that moves. We built a system where AI learns how to move. --- # How We Did It 1. **Firmware to DAEMON interface (`daemon-cli`)** Manufacturers annotate existing firmware APIs, and `daemon-cli` generates a standardized DAEMON manifest + runtime wrapper. 2. **Node discovery and capability exposure** Each hardware module (base, arm, camera, etc.) exposes its commands and telemetry as a DAEMON node over serial/TCP. 3. **Natural-language control from `desktop-app`** Users connect devices and type goals in plain English. They do not need to manually map commands to components. 4. **Planning + orchestration** The planner converts user intent into structured action steps, and `orchestrator` validates, routes, and executes those steps across one or more nodes safely. 5. **Closed-loop autonomy (`autonomy-engine`)** The autonomy loop uses live camera frames plus an OpenAI-based critic to run an execute → evaluate → adjust cycle. It updates control parameters between attempts and stops when success is stable (for example, 2 consecutive successful iterations). 6. **End result** DAEMON turns heterogeneous firmware and hardware into one AI-operable system, enabling multi-device behavior without custom per-device orchestration logic. ## Detected evidence (automated analysis) Indexed codebase: 175 recognized source files, 750 KB. - C (language) — detected in the code - CSS (language) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Rust (language) — detected in the code - TypeScript (language) — detected in the code - Vercel (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (120 of 222) ``` .gitignore autonomy-engine/autonomy_engine/__init__.py autonomy-engine/autonomy_engine/camera.py autonomy-engine/autonomy_engine/judge.py autonomy-engine/autonomy_engine/logging.py autonomy-engine/autonomy_engine/manifest.py autonomy-engine/autonomy_engine/openai_client.py autonomy-engine/autonomy_engine/orchestrator_client.py autonomy-engine/autonomy_engine/perception.py autonomy-engine/autonomy_engine/plan_utils.py autonomy-engine/autonomy_engine/policy.py autonomy-engine/autonomy_engine/run.py autonomy-engine/autonomy_engine/semantics.py autonomy-engine/autonomy_engine/shield.py autonomy-engine/autonomy_engine/taskspec.py autonomy-engine/autonomy_engine/tracker.py autonomy-engine/pyproject.toml autonomy-engine/README.md autonomy-engine/requirements.txt autonomy-engine/run_engine.py autonomy-engine/tasks/example.taskspec.json autonomy-engine/tests/test_semantics.py autonomy-engine/tests/test_taskspec.py daemon-cli/daemon daemon-cli/daemon_cli/__init__.py daemon-cli/daemon_cli/build.py daemon-cli/daemon_cli/cli.py daemon-cli/daemon_cli/generators/__init__.py daemon-cli/daemon_cli/generators/cgen.py daemon-cli/daemon_cli/generators/integration.py daemon-cli/daemon_cli/generators/manifest.py daemon-cli/daemon_cli/main.py daemon-cli/daemon_cli/models.py daemon-cli/daemon_cli/parsers/__init__.py daemon-cli/daemon_cli/parsers/annotation.py daemon-cli/daemon_cli/schema.py daemon-cli/docs/arduino_integration.md daemon-cli/docs/cmake_make_integration.md daemon-cli/docs/desktop-reference.md daemon-cli/docs/developer_quickstart.md daemon-cli/docs/manifest_schema.md daemon-cli/docs/manufacturers.md daemon-cli/docs/protocol.md daemon-cli/docs/sleepy_runbook.md daemon-cli/examples/annotated_firmware/generated/daemon_entry.c daemon-cli/examples/annotated_firmware/generated/DAEMON_INTEGRATION.md daemon-cli/examples/annotated_firmware/generated/daemon_runtime.c daemon-cli/examples/annotated_firmware/generated/daemon_runtime.h daemon-cli/examples/annotated_firmware/generated/DAEMON.yml daemon-cli/examples/annotated_firmware/main.c daemon-cli/examples/annotated_firmware/Makefile daemon-cli/examples/firmware_manufacturers/gripworks_gripper/generated/daemon_entry.c daemon-cli/examples/firmware_manufacturers/gripworks_gripper/generated/DAEMON_INTEGRATION.md daemon-cli/examples/firmware_manufacturers/gripworks_gripper/generated/daemon_runtime.c daemon-cli/examples/firmware_manufacturers/gripworks_gripper/generated/daemon_runtime.h daemon-cli/examples/firmware_manufacturers/gripworks_gripper/generated/DAEMON.yml daemon-cli/examples/firmware_manufacturers/gripworks_gripper/main.c daemon-cli/examples/firmware_manufacturers/gripworks_gripper/README.md daemon-cli/examples/firmware_manufacturers/linetrace_sensor/generated/daemon_entry.c daemon-cli/examples/firmware_manufacturers/linetrace_sensor/generated/DAEMON_INTEGRATION.md daemon-cli/examples/firmware_manufacturers/linetrace_sensor/generated/daemon_runtime.c daemon-cli/examples/firmware_manufacturers/linetrace_sensor/generated/daemon_runtime.h daemon-cli/examples/firmware_manufacturers/linetrace_sensor/generated/DAEMON.yml daemon-cli/examples/firmware_manufacturers/linetrace_sensor/main.c daemon-cli/examples/firmware_manufacturers/linetrace_sensor/README.md daemon-cli/examples/firmware_manufacturers/skylift_drone/generated/daemon_entry.c daemon-cli/examples/firmware_manufacturers/skylift_drone/generated/DAEMON_INTEGRATION.md daemon-cli/examples/firmware_manufacturers/skylift_drone/generated/daemon_runtime.c daemon-cli/examples/firmware_manufacturers/skylift_drone/generated/daemon_runtime.h daemon-cli/examples/firmware_manufacturers/skylift_drone/generated/DAEMON.yml daemon-cli/examples/firmware_manufacturers/skylift_drone/main.c daemon-cli/examples/firmware_manufacturers/skylift_drone/README.md daemon-cli/examples/manifests/arm.yml daemon-cli/examples/manifests/base.yml daemon-cli/examples/manifests/collision_grip.yml daemon-cli/examples/manifests/collision_turn.yml daemon-cli/examples/manifests/collision.yml daemon-cli/examples/manifests/gripworks_gripper.yml daemon-cli/examples/manifests/linetrace_sensor.yml daemon-cli/examples/manifests/skylift_drone.yml daemon-cli/examples/node-emulator/emulator.py daemon-cli/examples/node-emulator/README.md daemon-cli/examples/real_firmware_arduino/firmware.ino daemon-cli/examples/real_firmware_arduino/README.md daemon-cli/examples/run_two_nodes.sh daemon-cli/firmware-code/configs/arm_manipulator-20260214t065054z-2d6b47/daemon_entry.c daemon-cli/firmware-code/configs/arm_manipulator-20260214t065054z-2d6b47/DAEMON.yaml daemon-cli/firmware-code/configs/arm_manipulator-20260214t065054z-2d6b47/manifest.json daemon-cli/firmware-code/configs/arm_manipulator-20260214t070742z-711dc3/daemon_entry.c daemon-cli/firmware-code/configs/arm_manipulator-20260214t070742z-711dc3/DAEMON.yaml daemon-cli/firmware-code/configs/arm_manipulator-20260214t070742z-711dc3/manifest.json daemon-cli/firmware-code/configs/greenhouse_node-20260214t065054z-fb021b/daemon_entry.c daemon-cli/firmware-code/configs/greenhouse_node-20260214t065054z-fb021b/DAEMON.yaml daemon-cli/firmware-code/configs/greenhouse_node-20260214t065054z-fb021b/manifest.json daemon-cli/firmware-code/configs/greenhouse_node-20260214t070201z-e672ab/daemon_entry.c daemon-cli/firmware-code/configs/greenhouse_node-20260214t070201z-e672ab/DAEMON.yaml daemon-cli/firmware-code/configs/greenhouse_node-20260214t070201z-e672ab/manifest.json daemon-cli/firmware-code/configs/greenhouse_node-20260214t070742z-72d9d5/daemon_entry.c daemon-cli/firmware-code/configs/greenhouse_node-20260214t070742z-72d9d5/DAEMON.yaml daemon-cli/firmware-code/configs/greenhouse_node-20260214t070742z-72d9d5/manifest.json daemon-cli/firmware-code/configs/rc_car_pi_arduino-20260214T065023Z-d251ce/daemon_entry.c daemon-cli/firmware-code/configs/rc_car_pi_arduino-20260214T065023Z-d251ce/DAEMON.yaml daemon-cli/firmware-code/configs/rc_car_pi_arduino-20260214T065023Z-d251ce/manifest.json daemon-cli/firmware-code/configs/rc_car_pi_arduino-20260214t065054z-4804ce/daemon_entry.c daemon-cli/firmware-code/configs/rc_car_pi_arduino-20260214t065054z-4804ce/DAEMON.yaml daemon-cli/firmware-code/configs/rc_car_pi_arduino-20260214t065054z-4804ce/manifest.json daemon-cli/firmware-code/configs/rc_car_pi_arduino-20260214t070201z-a3f41e/daemon_entry.c daemon-cli/firmware-code/configs/rc_car_pi_arduino-20260214t070201z-a3f41e/DAEMON.yaml daemon-cli/firmware-code/configs/rc_car_pi_arduino-20260214t070201z-a3f41e/manifest.json daemon-cli/firmware-code/profiles/arm_manipulator/protocol/commands.md daemon-cli/firmware-code/profiles/arm_manipulator/README.md daemon-cli/firmware-code/profiles/arm_manipulator/src/joint_limits.c daemon-cli/firmware-code/profiles/arm_manipulator/src/joint_limits.h daemon-cli/firmware-code/profiles/arm_manipulator/src/motion_queue.c daemon-cli/firmware-code/profiles/greenhouse_node/protocol/messages.yaml daemon-cli/firmware-code/profiles/greenhouse_node/README.md daemon-cli/firmware-code/profiles/greenhouse_node/src/board_map.h daemon-cli/firmware-code/profiles/greenhouse_node/src/greenhouse_control.c daemon-cli/firmware-code/profiles/rc_car_pi_arduino/arduino/generated/daemon_entry.c daemon-cli/firmware-code/profiles/rc_car_pi_arduino/arduino/generated/DAEMON_INTEGRATION.md [102 more files omitted for size] ``` ### Dependencies - autonomy-engine/pyproject.toml: numpy@>=1.26, opencv-python@>=4.9 - autonomy-engine/requirements.txt: numpy@>=1.26, opencv-python@>=4.9 - daemon-cli/pyproject.toml: jsonschema@>=4.0, openai@>=1.0.0, PyYAML@>=6.0 - daemon-cli/requirements.txt: jsonschema@>=4.0, PyYAML@>=6.0 - desktop-app/package.json: @tauri-apps/api@^2, @tauri-apps/cli@^2, @tauri-apps/plugin-opener@^2, @vitejs/plugin-react@^4.6.0, react@^19.1.0, react-dom@^19.1.0, vite@^7.0.4 - desktop-app/src-tauri/Cargo.toml: base64@0.22, image@0.25, reqwest@0.12, serde@1, serde_json@1, serialport@4.6, tauri@2, tauri-build@2, tauri-plugin-opener@2 - tools/command_model/requirements.txt: scikit-learn@==1.5.2 - vercel-api/package.json: @types/node@20.17.57, @types/react@18.3.18, @types/react-dom@18.3.5, @vercel/blob@0.27.0, jpeg-js@0.4.4, next@14.2.32, react@18.3.1, react-dom@18.3.1, tsx@^4.21.0, typescript@5.6.3 ### Recent commits (newest first) - Use NVIDIA DGX Spark - Revise README to enhance project description - Fix detection of ethernet issues - Refactor code structure and remove redundant sections - Streamlined UI - Complete RL loop - [temp] rl - Add vision step requests and responses for various instructions, including error handling for network issues - pi brain: bind dual-stack for .local reachability - Enhance orchestrator with vision step integration and error handling - feat: increase node connection timeout and enhance instruction parsing - feat: enhance orchestrator and vision step functionality (fix camera) - Enhance logging and error handling in vision and bootstrap processes - Enhance service manifest handling in orchestrator and update vision task metrics (implement camera support) - Create Troubleshoot.md - Add orchestrator request and response logs for status checks - Add additional orchestrator request and response logs for status checks - Add vision reinforcement - Add functions for local host normalization and TCP port management in orchestrator - Add vision request handling and update error reporting in App component ## Key source files (fetched from GitHub, selected and truncated for size) ### NEXT_CONTEXT_HANDOFF.md ```markdown # Handoff / Debug Summary ## Current status - `move in a square` now executes end-to-end from `Send` and is no longer stuck on first step. - Remaining failures: `move left`, `move right`, `move back` (intermittent/phrase-dependent). - SSH direct serial test (`mecanum_test.py`) works, so firmware/serial path is mostly fine. - This points to **NL parse / plan generation / token mapping** layer, not raw motor control. ## Most likely gap - `vision_step` parser/planner currently has explicit motion-pattern support for `forward/backward/circle/square/triangle`. - “left/right” natural language is likely not mapped to a motion macro (`STRAFE` or `TURN`) consistently. - “move back” may still fall into non-motion parse in some phrasings. ## Files to inspect first (root-cause) 1. `logs/vision_trace.jsonl` Check `parsed_instruction.task_type`, `pattern`, `policy_branch`, generated `plan`. 2. `logs/orchestrator_trace.jsonl` Check what plan was actually sent to orchestrator (`RUN` tokens + args + duration). 3. `logs/backend_audit.jsonl` Ground truth HTTP payloads to `/execute_plan`. 4. `daemon-cli/firmware-code/profiles/rc_car_pi_arduino/raspberry_pi/.daemon_logs/orchestrator.log` Execution-side errors/timeouts/reconnects. 5. `daemon-cli/firmware-code/profiles/rc_car_pi_arduino/raspberry_pi/.daemon_logs/vercel_api.log` API runtime mismatches / stale build symptoms. 6. On Pi itself (important): `~/mecanum_node.log` or `systemctl status daemon-mecanum.service` Confirms node received `RUN` tokens and mapped them. ## Code files to patch for left/right/back 1. `vercel-api/src/lib/visionPolicy.ts` Extend `parseInstruction` synonyms for: - `move left/right` (decide strafe vs turn) - all back variants (`back`, `backward`, `reverse`, `go back`) 2. `vercel-api/src/app/api/vision_step/route.ts` In `buildMotionSteps`, map: - left/right intents to `STRAFE L/R` or `TURN +/-deg` - backward intent to `BWD` reliably 3. `daemon-cli/firmware-code/profiles/rc_car_pi_arduino/raspberry_pi/mecanum_daemon_node.py` Verify handlers for `BWD`, `STRAFE`, `TURN` parse args exactly as planner emits. 4. `vercel-api/tests/vision-policy.test.ts` + `vercel-api/tests/vision-step.test.ts` Add regression tests for phrases: - “move left” - “move right” - “move back” - “go backward a bit” ## Fast log queries to run ```bash cd /Users/vedpanse/Daemon rg -n "applied_instruction|parsed_instruction|policy_branch|planLength|\"token\"" logs/vision_trace.jsonl -S | tail -n 80 rg -n "orchestrator.execute_plan.request|\"token\"" logs/orchestrator_trace.jsonl -S | tail -n 80 rg -n "/execute_plan|\"plan\"" logs/backend_audit.jsonl -S | tail -n 80 ``` If those show wrong parse (e.g., `task_type: "unknown"`), fix is in `visionPolicy.ts`. If parse is correct but token wrong, fix is in `buildMotionSteps` in `route.ts`. If token is correct but robot doesn’t move, fix is in Pi node mapping/logs. ``` ### EXECUTIVE.md ```markdown # EXECUTIVE.md # DAEMON ### AI-Native Firmware-to-Agent Bridge --- # 1. What Daemon Is Daemon is a distributed embodiment protocol that enables AI systems to control microcontroller-based hardware **without custom per-device integration**. Each hardware component (robot base, robotic arm, etc.) runs its own firmware. Daemon generates a structured, safe control interface on top of that firmware. At runtime: * Devices expose a **Manifest** * An **Orchestrator** fuses multiple devices into a unified “body” * A **Planner (Vercel-hosted)** converts natural language into structured execution plans * The Orchestrator validates and executes those plans safely Daemon does **not** merge firmware. It composes devices via capability manifests. --- # 2. Repository Structure (Monorepo) ``` / ├── daemon-cli/ # Developer tool: ./daemon build + emulator + schema ├── orchestrator/ # Runtime system: multi-node composition + plan execution ├── vercel-api/ # Planner API (hosted on Vercel) ├── desktop-app/ # Thin client (UI layer) – NOT core runtime └── EXECUTIVE.md ``` ### Ownership * Ved → `daemon-cli/` + `orchestrator/` * Sachin → `vercel-api/` * `desktop-app/` is thin UI and not core architecture --- # 3. System Architecture ## 3.1 Components ### A) Daemon Node (runs on hardware) Each firmware repo runs: ``` ./daemon build ``` Which generates: * `DAEMON.yml` (manifest) * `daemon_entry.c` (command dispatcher) * `daemon_runtime.c/h` (protocol + safety layer) Each device becomes a **Daemon Node**. --- ### B) Wire Protocol (serial-line-v1) Host → Device: ``` HELLO READ_MANIFEST RUN STOP SUB TELEMETRY UNSUB TELEMETRY ``` Device → Host: ``` MANIFEST OK ERR TELEMETRY key=value ... ``` Telemetry is opt-in (no unsolicited output before SUB). --- ### C) Orchestrator * Connects to multiple nodes * Fetches MANIFEST from each * Builds a unified capability graph * Routes commands to correct node * Validates planner output * Enforces safety (timeouts, STOP, panic stop) Supports: * Multi-node composition * Planner fallback if remote fails * Emergency STOP on Ctrl+C --- ### D) Planner (Vercel API) Endpoint: ``` POST /api/plan ``` Input: ``` { instruction, system_manifest, telemetry_snapshot } ``` Output: ``` { plan: [ { type: "RUN", target, token, args, duration_ms? }, { type: "STOP" } ], explanation } ``` Planner never touches hardware directly. --- # 4. Manifest Schema (v0.1) Each node exposes: ``` daemon_version: "0.1" device: name version node_id commands: - token - description - args: - name - type (int|float|bool|string) - min - max - required - safety: rate_limit_hz watchdog_ms clamp - nlp: synonyms examples telemetry: keys: - name - type - unit (optional) transport: type: "serial-line-v1" ``` --- # 5. Current Working Demo ## Nodes * Base robot [truncated — 2889 more characters] ``` ### daemon-cli/requirements.txt ``` PyYAML>=6.0 jsonschema>=4.0 ``` ### autonomy-engine/requirements.txt ``` numpy>=1.26 opencv-python>=4.9 ``` ### orchestrator/requirements.txt ``` # No external dependencies required. ``` ### autonomy-engine/pyproject.toml ``` [build-system] requires = ["setuptools>=68"] build-backend = "setuptools.build_meta" [project] name = "daemon-autonomy-engine" version = "0.0.1" description = "Laptop-local closed-loop autonomy runner for DAEMON robots" readme = "README.md" requires-python = ">=3.10" dependencies = [ "numpy>=1.26", "opencv-python>=4.9", ] [tool.setuptools] package-dir = {"" = "."} [tool.setuptools.packages.find] where = ["."] include = ["autonomy_engine*"] ``` ### daemon-cli/pyproject.toml ``` [build-system] requires = ["setuptools>=68", "wheel"] build-backend = "setuptools.build_meta" [project] name = "daemon-cli" version = "0.1.0" description = "Daemon firmware generation CLI" readme = "README.md" requires-python = ">=3.10" dependencies = [ "PyYAML>=6.0", "jsonschema>=4.0" ] [project.optional-dependencies] ai = ["openai>=1.0.0"] [project.scripts] daemon = "daemon_cli.cli:main" [tool.setuptools.packages.find] include = ["daemon_cli*"] ``` ### desktop-app/package.json ``` { "name": "desktop-app", "private": true, "version": "0.1.0", "type": "module", "scripts": { "dev": "vite", "build": "vite build", "preview": "vite preview", "tauri": "tauri" }, "dependencies": { "@tauri-apps/api": "^2", "@tauri-apps/plugin-opener": "^2", "react": "^19.1.0", "react-dom": "^19.1.0" }, "devDependencies": { "@tauri-apps/cli": "^2", "@vitejs/plugin-react": "^4.6.0", "vite": "^7.0.4" } } ``` ### vercel-api/package.json ``` { "name": "daemon-vercel-api", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "typecheck": "tsc --noEmit", "health:keys": "node scripts/api-health-check.mjs", "test:vision": "tsx tests/vision-policy.test.ts && tsx tests/vision-step.test.ts" }, "dependencies": { "@vercel/blob": "0.27.0", "jpeg-js": "0.4.4", "next": "14.2.32", "react": "18.3.1", "react-dom": "18.3.1" }, "devDependencies": { "@types/node": "20.17.57", "@types/react": "18.3.18", "@types/react-dom": "18.3.5", "tsx": "^4.21.0", "typescript": "5.6.3" } } ``` ### tools/command_model/requirements.txt ``` scikit-learn==1.5.2 ``` [155 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]