HackStack

Project Info

Winner

Fetch.ai: Agentic Track Prize

Autonomous AI Society

Devpost

Inspiration

I got this idea because of the current hurricane Milton causing devastation across Florida. The inspiration behind Autonomous AI Society stems from the need for faster, more efficient, and autonomous systems that can make critical decisions during disaster situations. With multiple sponsors like Fetch.ai, Groq, Deepgram, Hyperbolic, and Vapi providing powerful tools, I envisioned an intelligent system of AI agents capable of handling a disaster response chain—from analyzing distress calls to dispatching drones and contacting rescue teams. The goal was to build an AI-driven solution that can streamline emergency responses, save lives, and minimize risks.

What it does

Autonomous AI Society is a fully autonomous multi-agent system that performs disaster response tasks in the following workflow: Distress Call Analysis: The system first analyzes distress calls using Deepgram for speech-to-text and Hume AI to score distress levels. Based on the analysis, the agent identifies the most urgent calls and the city. Distress Call Analysis: The system first analyzes distress calls using Deepgram for speech-to-text and Hume AI to score distress levels. Based on the analysis, the agent identifies the most urgent calls and the city. Drone Dispatch: The distress analyzer agent communicates with the drone agent (built using Fetch.ai) to dispatch drones to specific locations, assisting with flood and rescue operations. Drone Dispatch: The distress analyzer agent communicates with the drone agent (built using Fetch.ai) to dispatch drones to specific locations, assisting with flood and rescue operations. Human Detection: Drones capture aerial images, which are analyzed by the human detection agent using Hyperbolic's LLaMA Vision model to detect humans in distress. The agent provides a description and coordinates. Human Detection: Drones capture aerial images, which are analyzed by the human detection agent using Hyperbolic's LLaMA Vision model to detect humans in distress. The agent provides a description and coordinates. Priority-Based Action: The drone results are displayed on a dashboard, ranked based on priority using Groq. Higher priority areas receive faster dispatches, and this is determined dynamically. Priority-Based Action: The drone results are displayed on a dashboard, ranked based on priority using Groq. Higher priority areas receive faster dispatches, and this is determined dynamically. Rescue Call: The final agent, built using Vapi, places an emergency call to the rescue team. It uses instructions generated by Hyperbolic’s text model to give precise directions based on the detected individuals and their location. Rescue Call: The final agent, built using Vapi, places an emergency call to the rescue team. It uses instructions generated by Hyperbolic’s text model to give precise directions based on the detected individuals and their location. How I built it The system consists of five agents, all built using Fetch.ai’s framework, allowing them to interact autonomously and make real-time decisions: Request-sender agent sends the initial requests. Distress analyzer agent uses Hume AI to analyze calls and Groq to generate dramatic messages. Drone agent dispatches drones to designated areas based on the distress score. Human detection agent uses Hyperbolic’s LLaMA Vision to process images and detect humans in danger. Call rescue agent sends audio instructions using Deepgram’s TTS and Vapi for automated phone calls. Challenges I ran into Simulating a drone movement on florida map: The lat_lon_to_pixel function converts latitude and longitude coordinates to pixel positions on the screen. The drone starts at the center of Florida. Its movement is calculated using trigonometry. The angle to the target city is calculated using math.atan2. The drone moves towards the target using sin and cos functions.This allows placing cities and the drone accurately on the map. Simulating a drone movement on florida map: The lat_lon_to_pixel function converts latitude and longitude coordinates to pixel positions on the screen. The drone starts at the center of Florida. Its movement is calculated using trigonometry. The angle to the target city is calculated using math.atan2. The drone moves towards the target using sin and cos functions.This allows placing cities and the drone accurately on the map. Callibrating the map to right coordinates: I had manually experiment with increasing and decreasing the coordinates to fit them at right spots on the florida map. Callibrating the map to right coordinates: I had manually experiment with increasing and decreasing the coordinates to fit them at right spots on the florida map. Coordinating AI agents: Getting agents to communicate effectively while working autonomously was a challenge. Coordinating AI agents: Getting agents to communicate effectively while working autonomously was a challenge. Handling dynamic priorities: Ensuring real-time analysis and updating the priority of drone dispatch based on Groq's risk assessment was tricky. Handling dynamic priorities: Ensuring real-time analysis and updating the priority of drone dispatch based on Groq's risk assessment was tricky. Integration of multiple APIs: Each sponsor's tools had specific nuances, and integrating all of them smoothly, especially with Fetch.ai, required careful handling. Integration of multiple APIs: Each sponsor's tools had specific nuances, and integrating all of them smoothly, especially with Fetch.ai, required careful handling. Accomplishments that I am proud of Successfully built an end-to-end autonomous system where AI agents can make intelligent decisions during a disaster, from distress call analysis to rescue actions. Integrated cutting-edge technologies like Fetch.ai, Groq, Hyperbolic, Deepgram, and Vapi in a single project to create a highly functional and real-time response system. What I learned AI for disaster response: Building systems that leverage multimodal AI agents can significantly improve response times and decision-making in life-critical scenarios. Cross-platform integration: We learned how to seamlessly integrate various tools, from vision AI to TTS to drone dispatch, using Fetch.ai and sponsor technologies. Working with real-time data: Developing an autonomous system that processes data in real-time provided insights into handling complex workflows.

What's next

Scaling to more disasters: Expanding the system to handle other types of natural disasters like wildfires or earthquakes. Edge deployment: Enabling drones and agents to run on the edge to reduce response times further. Improved human detection: Enhancing human detection with more precise models to handle low-light or difficult visual conditions. Expanded rescue communication: Integrating real-time communication with the victims themselves using Deepgram’s speech technology.

Analysis

Compare with all teams

View

Metric

No commits on this project resolved to a GitHub account.

Technology

Found in codeClaimed only
  • CSSIn code
  • ExpressIn code
  • HTMLIn code
  • JavaScriptIn code
  • PythonIn code
  • ReactIn code
  • Node.jsClaimed

6 of 7 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

56 KB

Source files

23

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.

0 stars