Project Info
Abstract: This project aims to assist visually impaired individuals, including those who are blind, by leveraging the Boston Dynamics Spot robot and AI technology to help them navigate complex environments. Through voice commands and intelligent decision-making, the robot autonomously moves in response to user inputs. The system utilizes Deepgram's speech-to-text API to process voice commands and Groq’s AI API to determine appropriate movement actions. The commands are then executed using the Boston Dynamics Spot SDK. Although network connectivity challenges limited testing, the project lays a solid foundation for further development into a practical assistive technology. Features: Voice Command Input: Users provide voice commands to control the robot’s movements. Speech-to-Text Processing: Audio is converted to text using the Deepgram API. AI-Powered Decision Making: Groq's AI API processes text commands to determine movement sequences. Robot Movement Execution: Commands are sent to Spot using the Boston Dynamics Spot SDK. Docker Integration: The application is containerized using Docker and runs on Spot’s onboard computer. Tech Stack: Hardware: Boston Dynamics Spot robot APIs: Deepgram: Speech-to-text conversion Groq: AI-based decision making for movement Boston Dynamics Spot SDK: For robot control Deepgram: Speech-to-text conversion Groq: AI-based decision making for movement Boston Dynamics Spot SDK: For robot control Programming Language: Python Containerization: Docker Platform: Runs on Spot’s onboard computer About the Project:
Inspiration
: Our team was inspired by the challenges faced by visually impaired individuals in navigating unfamiliar or complex environments. We aimed to use cutting-edge robotics and AI technology to create a guide dog-like experience that could improve independence and safety for blind and visually impaired users. What We Learned: We learned the complexities of working with robotics hardware in real-world conditions, especially when integrating multiple APIs for voice control and AI-based decision-making. Handling real-time input and making decisions autonomously based on that data was both a rewarding and educational experience. How We Built the Project: We developed a Python-based application that runs through a Docker image on the Spot robot’s onboard computer. The system takes voice command input, converts it to text using Deepgram, and processes the command using Groq’s AI to decide on the best movement action. The Boston Dynamics Spot SDK then executes these movements. Challenges We Faced: Our main challenge was network connectivity with the Spot robot. Due to these issues, we were unable to test our program on the actual robot until late on the second day of the Hackathon. This delayed our progress and limited our ability to fine-tune and optimize the robot’s real-world interactions. Despite this setback, we successfully implemented a functional voice-controlled movement system. Things to Consider: Current version only includes basic voice commands and robot movement. Further development could involve more complex navigation logic, multi-modal feedback (audio/tactile), and integration with Fetch.ai for real-time path planning. Future Work: Enhanced Command Recognition: Add sentiment detection or context for better alignment of verbal inputs with actions. Multi-modal Feedback: Implement audio and tactile feedback to improve user interaction. Path Planning: Integrate Fetch.ai for dynamic navigation and obstacle avoidance.
AI-Powered Voice-Controlled Spot Robot (Cal-Hacks-11.0)
Abstract:
This project explores the use of Boston Dynamics’ Spot robot, integrating AI and voice control to assist in navigating complex environments. The Python-based application processes voice commands through Deepgram's speech-to-text API and utilizes Groq’s AI API to determine the robot's movement actions. The robot executes these actions using the Boston Dynamics Spot SDK. The current implementation showcases the ability to control Spot via voice commands, laying a foundation for future development as an assistive technology.
Features:
- Voice Command Input: Users provide voice commands to control the robot’s movements.
- Speech-to-Text Processing: Audio is converted to text using the Deepgram API.
- AI-Powered Decision Making: Groq's AI API, provided with context, processes text commands to determine movement sequences.
- Robot Movement Execution: Commands are sent to Spot using the Boston Dynamics Spot SDK.
- Docker Integration: The application is containerized using Docker and runs on Spot’s onboard computer.
Tech Stack:
- Hardware: Boston Dynamics Spot robot
- Boston Dynamics Spot SDK: For robot control
- APIs:
- Deepgram: Speech-to-text conversion
- Groq: AI-semantic decision making for movement
- Programming Language: Python
- Containerization: Docker
- Platform: Runs on Spot’s onboard computer
Things to Consider:
- Current version only includes voice commands and robot movement.
- Further development could involve more complex navigation logic, multi-modal feedback (audio/tactile), and integration with Fetch.ai for real-time path planning.
Future Work:
- Enhanced Command Recognition: Add sentiment detection or context for better alignment of verbal inputs with actions.
- Multi-modal Feedback: Implement audio and tactile feedback to improve user interaction.
- Path Planning: Integrate Fetch.ai agents for dynamic navigation and obstacle avoidance on a large scale (i.e. the city or surrounding blocks).
Analysis
View
Metric
- 47
- 19
- 2
- 1
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- PythonIn code
1 of 1 appear in the indexed code.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
24 KB
Source files
14
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
purelyKai/AI-Powered-Voice-Controlled-Spot-Robot
27 files · 2.2 MB · @ 31a2d39
Structure
API & routing
5 files · 19%Request entry points: routes, handlers and controllers.
Application logic
8 files · 30%Domain rules, services and shared utilities.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- Python80%
- Markdown15%
- YAML4%
Share of indexed source by file size. Binary and vendored files are excluded.
Feature verification
AI-powered decision making with GroqVerified
Groq's AI API processes text commands to determine movement sequences
Claimed on readmehigh confidenceSpot/groq_ai.py:4— get_commands sends the transcribed text to the Groq chat completions API with a system prompt mapping speech to movement commandsSpot/commands.py:82— getCommands calls get_commands(text) and splits the returned string into a command list
Docker containerization for onboard deploymentVerified
The application is containerized using Docker and runs on Spot's onboard computer
Claimed on readmehigh confidenceSpot/Dockerfile:22— Dockerfile installs requirements.txt and sets CMD to run main.py, containerizing the Spot control application
Robot movement execution via Boston Dynamics Spot SDKVerified
Commands are sent to Spot using the Boston Dynamics Spot SDK
Claimed on readmehigh confidenceSpot/spot_controller.py:2— Imports bosdyn.client and implements SpotController wrapping RobotCommandClient for movement, standing, and estop controlSpot/commands.py:12— SpotCommands methods (forward, back, turnLeft, bow, circleDance, sidestep, patrol, lieDown) call SpotController methods that issue real Spot SDK commandsSpot/main.py:23— main loop dispatches recognized voice commands to the corresponding SpotCommands movement methods
Voice command input and speech-to-text processingVerified
Users provide voice commands, converted to text via Deepgram's speech-to-text API
Claimed on readmehigh confidenceSpot/commands.py:76— getCommands records audio with arecord and calls getText() to transcribe itSpot/stt.py:25— getText uses the Deepgram client's transcription.prerecorded to convert the recorded audio buffer to text
Alternate Flask API server for transcription and Fetch.ai processingCode-supported
Not explicitly in README features list, but a server component exists suggesting a networked command relay
Claimed on Devpostlow confidenceServer/server.py:9— Flask app defines /transcribe and /spot/command endpoints, but the actual Spot execution call is stubbed out (result = 0, SpotInterface import commented out), and this server is not referenced by Spot/main.py which is the Dockerfile's entrypoint
Full voice-to-movement pipeline wired end-to-endCode-supported
Voice command triggers autonomous robot movement via Deepgram, Groq, and Spot SDK in sequence
Claimed on Devpostmedium confidenceSpot/commands.py:74— getCommands chains arecord audio capture, Deepgram transcription, and Groq command extraction, then main.py dispatches the results to SpotController actionsSpot/stt.py:40— getText is async but is called without await/asyncio.run in commands.py, and the __main__ test block is commented out, suggesting this pipeline was not confirmed working end-to-end (matches the project's stated network/testing issues)
Fetch.ai integration for path planningClaimed only
Integration with Fetch.ai for real-time path planning and dynamic navigation
Claimed on readmehigh confidence
An AI agent derived these features from the project’s Devpost page and readme, then searched the code for each one. Verified features are backed by cited code; claimed-only features had no supporting code, which is not by itself proof a feature is missing.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.