HackStack

Project Info

Neurodog

Devpost

Inspiration

For individuals suffering from conditions like mental health or physical disabilities, service animals can offer immense improvement to a person’s quality of life. Yet, tragically, many are unable to own a service animal due to the limitations imposed by their conditions. In 2023, 74% of pet owners believed their mental health improved after getting a pet; however, less than 1% of Americans with disabilities own service dogs due to cognitive/physical incapacities that may result in animal neglect and differences in the attributes of dogs. We began to think: How can people with disabilities receive emotional support and physical assistance from service animals, without needing to shoulder the responsibility of caring for the animals' needs?

What it does

Designed as a self-functioning robotic service animal, Neurodog has the capability to recognize familiar brainwave patterns, including emotional states like "sadness" in its user, and is equipped to automatically execute corresponding commands based on those emotions, such as approaching the user for comforting interactions through play or engaging in entertaining tricks. Neurodog is a service dog that offers therapy and a range of essential medical services. By merging AI with voice commands we were able to implement into Neurodog to recognize natural language voice prompts— providing a more realistic feel while extending the range of voice functionalities available.

How we built it

Frontend was built using Next.js (React) with Shadcn UI for the components. We included a Github OAuth for dashboard login, and Neurosity OAuth to gain access to the tab containing brainwave scans and functionalities. The backend consisted of serverless functions using NextJS’s app router. This app router allowed us to develop with SSR that improved the performance of the application. We used Drizzle as the ORM and lucia-auth for the session management. Through Lucia’s documentation, we were able to spin up our oauth quickly. In addition to those tools, we used Kirimase which bootstraps NextJS applications with all the tools above with one command. For natural language comprehension, we used OpenAI’s whisper to understand speech. We then processed the response into ChatGPT’s function calls to allow us to derive a command and run it on the frontend or backend. This enabled us to do Voice Actions on the frontend. With Spot, we used Elevenlabs to generate the speech and ChatGPT’s whisper to understand recorded audio. We also used Neurosity’s SDK to allow Spot to receive brain waves from our EEG.

Challenges we ran into

This project encountered a significant number of integration challenges. Here are some interesting tidbits: Kinesis, the Motor Imagery Brain-Computer Interaction (BCI) method that allows us to trigger actions through trained brainwave patterns, needs to be consistently trained to maintain the accuracy of the brain scans and work on command. Sharing a singular Boston Dynamic SPOT dog between numerous teams, significantly reduced the time we had to execute our programs. Figuring out the Neurosity SDK, and connecting it with the SPOT SDK proved to be difficult because of the complex intricacies in both documentations.

What's next

Unlike competitor products such as Tombot which is currently only touch-activated and real live dogs, Neurodog will interact and safeguard users through voice and based on their brain activity using an electroencephalogram (EEG) allowing immediate care reliably. Neurodog will include precautionary SOS features to ensure it is protecting its user in case of emergencies We plan to target households, hospitals, and nursing homes to assist with caretaking. Neurodog will have computer vision capabilities so it will act as the eyes for its user. Neurodog will be able to detect these dangerous situations and analyze traffic signals allowing them to alert the user while protecting them.

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only
  • CSSIn code
  • JavaScriptIn code
  • Next.jsIn code
  • OpenAIIn code
  • PythonIn code
  • ReactIn code
  • SQLIn code
  • Tailwind CSSIn code
  • TypeScriptIn code
  • DockerClaimed
  • VercelClaimed

9 of 11 appear in the indexed code. 2 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

146 KB

Source files

83

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.

0 stars