# Project export: DAGr - Dynamic Agent Generator This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 12.0 - Tagline: A Voice AI Journal: Talk, reflect, and let it detect when to spawn AI agents, or ask it directly to create them. It listens, reasons, and creates custom agents to solve your problems. - Devpost: https://devpost.com/software/dagr-dynamic-agent-generator - GitHub: https://github.com/maxzfan/dag - Video: https://www.youtube.com/embed/DXhnyYmyHjg?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 3 GitHub contributor(s) — maxfan070601 (42 commits), Josh Pham (21 commits), Parth Sheth (14 commits) ## Devpost submission (written by the team) ### Overview What Inspired Us We imagine a future where you could just talk to an AI about what’s slowing you down — bugs, cluttered data, messy workflows — and instead of giving you another list of “steps to try,” it would actually build the tools to fix them for you. That’s where DAGr began. We wanted to merge the intuitiveness of conversation with the autonomy of agents so that a few spoken words could spark an entire AI workforce into action. We drew inspiration from: The human desire to delegate — to have something that truly helps, not just advises. The evolution of AI — from chat to reasoning, from reasoning to acting. And from our own frustrations — long hours debugging, cleaning data, or managing systems when we knew AI could do more. DAGr is that missing bridge — between voice and action, between thought and execution. What It Does DAGr is a voice-powered AI platform that turns conversation into automation. You talk to it like a human: “I’ve been struggling to organize my schedule lately.” It listens, transcribes, and sends your words to a reasoning layer, which interprets your goals. If the problem is actionable, Claude drafts a specialized agent definition. DAGr then deploys that agent using the Agentverse network — instantly creating a helper that works for you. With DAGr, a simple log into your journal transforms into conversation that creates capability. Two Modes: Journal Mode – A reflective, conversational mode where you share thoughts or ideas. DAGR listens and offers intelligent, memory-aware responses, drawing on past conversations and stepping in to create an assistive agent when necessary. Agent Mode – When DAGR identifies a concrete task, it automatically creates or activates a dedicated agent to solve it (e.g., scheduling, cleaning data, managing logs). Every dialogue has the potential to birth a new digital teammate. How We Built It DAGR is built on the intersection of voice interaction, reasoning LLMs, and autonomous agents: Speech-to-Text & Text-to-Speech (Fish.AI) Converts natural conversation into text and back to speech for a seamless two-way dialogue. In journal mode, this enables the AI assistant to have a conversation with you. Speech-to-Text & Text-to-Speech (Fish.AI) Converts natural conversation into text and back to speech for a seamless two-way dialogue. In journal mode, this enables the AI assistant to have a conversation with you. Reasoning Engine (Claude) Interprets user intent, determines whether to respond conversationally or generate an agent plan. Outputs structured JSON/YAML defining which agents to call or how to create a new one. Reasoning Engine (Claude) Interprets user intent, determines whether to respond conversationally or generate an agent plan. Outputs structured JSON/YAML defining which agents to call or how to create a new one. Agent Execution (Fetch.ai) Each agent is a uAgent with a specific purpose — like DataCleaner, Scheduler, or LogAnalyzer. The Orchestrator Agent manages creation, coordination, and communication between agents. Agents are deployed to AgentVerse.ai, where users can go in and make manual changes as necessary. Agent Execution (Fetch.ai) Each agent is a uAgent with a specific purpose — like DataCleaner, Scheduler, or LogAnalyzer. The Orchestrator Agent manages creation, coordination, and communication between agents. Agents are deployed to AgentVerse.ai, where users can go in and make manual changes as necessary. Dynamic Agent Generation (Python Backend) When Claude outputs a “create_agent” instruction, DAGr compiles it into code, deploys it as a new Fetch.ai uAgent, and registers it for future use. Dynamic Agent Generation (Python Backend) When Claude outputs a “create_agent” instruction, DAGr compiles it into code, deploys it as a new Fetch.ai uAgent, and registers it for future use. Frontend (Web App) Clean React + Tailwind interface that allows switching between Journal and Agent modes and deploying your active agents. Frontend (Web App) Clean React + Tailwind interface that allows switching between Journal and Agent modes and deploying your active agents. Challenges We Ran Into Bridging AI reasoning and real-world execution: Getting Claude to produce usable, secure agent definitions required tight prompt engineering and validation layers. We ultimately wrote a proprietary algorithm that loops over natural text, extracting as much information as possible into a more parsable YAML file. Bridging AI reasoning and real-world execution: Getting Claude to produce usable, secure agent definitions required tight prompt engineering and validation layers. We ultimately wrote a proprietary algorithm that loops over natural text, extracting as much information as possible into a more parsable YAML file. Managing live agent lifecycles: Creating, running, and communicating with dynamic Fetch.ai agents in real time while maintaining state coherence wasn’t trivial, especially automating the Fetch.ai agent generation from a single Claude prompt. We decided on YAML to pass in the most detailed information we could. Managing live agent lifecycles: Creating, running, and communicating with dynamic Fetch.ai agents in real time while maintaining state coherence wasn’t trivial, especially automating the Fetch.ai agent generation from a single Claude prompt. We decided on YAML to pass in the most detailed information we could. Seamless voice integration: Synchronizing speech recognition (STT), reasoning latency, and speech synthesis (TTS) so that the user experience felt natural took careful orchestration, especially in journal form. We wanted an experience where the AI felt helpful, not intrusive. Seamless voice integration: Synchronizing speech recognition (STT), reasoning latency, and speech synthesis (TTS) so that the user experience felt natural took careful orchestration, especially in journal form. We wanted an experience where the AI felt helpful, not intrusive. Accomplishments That We’re Proud Of We built a full-stack system that connects voice, reasoning, and autonomous agents — live. Our LLM can not only chat, but actually design and deploy new AI helpers on the fly. DAGr can create Fetch.ai Agentverse agents from natural conversation — a glimpse into a true agentic AI swarm. The system works across domains: DevOps, data organization, personal productivity, and more. What’s Next for DAGR We see DAGR as more than a hackathon project; it’s the beginning of a new AI paradigm. Coming soon: Enterprise Integrations – Connecting DAGR to APIs like Slack, Notion, Jira, GitHub, and Google Workspace. Mobile App – A portable voice-first AI workspace that travels with you. Desktop App – Overlays on your computer so you can watch the agents complete tasks in live time, not just through logs but with your own eyes. Our vision: To create an ecosystem where anyone can speak their needs and watch a custom AI team assemble itself to help. Team DAGR Built by Max Fan, Akshat Kannan, Josh Pham, and Parth Sheth at CalHacks 2025. “From conversation to creation, DAGr AI builds custom AI solutions for you.” ## README (from the GitHub repository) # Dynamic Agent Generator (DAG) ## Detected evidence (automated analysis) Indexed codebase: 61 recognized source files, 344 KB. - CSS (language) — detected in the code - Flask (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - Node.js (technology) — claimed on Devpost, not found in the code - AI coding agent: GitHub Copilot — evidence: config files committed to the repository ## Codebase structure (from repository index) ### Files (80 of 80) ``` .editorconfig .env .env.example .gitattributes .github/copilot-instructions.md .gitignore .idea/.gitignore .idea/misc.xml .idea/modules.xml .idea/vcs.xml .vite/deps_temp_847f4296/package.json .vite/deps_temp_c1771fe7/package.json .vite/deps_temp_e139d595/package.json ARCHITECTURE.md backend/agents/agent_20251026T064008Z/generated_agent.py backend/agents/agent_20251026T064758Z/generated_agent.py backend/agents/agent_6a99f505-c4ca-4ee2-8bf0-613f27bf1401/generated_agent.py backend/agents/agent_8f6040c9-c72e-4d07-a3b0-63e063eb918f/generated_agent.py backend/agents/agent_f563d763-3416-45e5-9554-809452ced597/generated_agent.py backend/agents/agent_weather-app/agent1qdwjuay02a_data.json backend/agents/agent_weather-app/fetch-agent.service backend/agents/agent_weather-app/requirements.txt backend/agents/agent_weather-app/weather_email_agent.py backend/detail.py backend/journal.py backend/prompt_detail.md backend/prompt_journal.md backend/prompt_yaml.md backend/requirements.txt backend/sysprompt.py backend/systemprompt.md backend/systemprompt2.md backend/systemprompt3.md backend/voice_server.py backend/yaml_helper.py backend/yaml/agent-20251026T064008Z.yaml backend/yaml/agent-20251026T064758Z.yaml backend/yaml/agent-20251026T065454Z.yaml backend/yaml/README.md backend/yaml/test-agent-20231215T143022Z.yaml backend/yaml/test-agent-fixed.yaml dag/deploy.py dag/yamlToFetch.py docs/README.md docs/SETUP.md frontend/components.json frontend/index.html frontend/package.json frontend/postcss.config.js frontend/src/App.tsx frontend/src/components/AgentDashboard.tsx frontend/src/components/ui/badge.tsx frontend/src/components/ui/breadcrumb.tsx frontend/src/components/ui/button.tsx frontend/src/components/ui/card.tsx frontend/src/components/ui/collapsible.tsx frontend/src/components/ui/input.tsx frontend/src/components/ui/scroll-area.tsx frontend/src/components/ui/separator.tsx frontend/src/components/ui/sheet.tsx frontend/src/components/ui/sidebar.tsx frontend/src/components/ui/skeleton.tsx frontend/src/components/ui/textarea.tsx frontend/src/components/ui/tooltip.tsx frontend/src/hooks/use-mobile.tsx frontend/src/index.css frontend/src/lib/utils.ts frontend/src/main.tsx frontend/tailwind.config.js frontend/tsconfig.json frontend/tsconfig.node.json frontend/vite.config.ts LICENSE.txt prompt.md README.md start.bat start.sh SYSTEM_ARCHITECTURE.md tempCodeRunnerFile.py yaml_template.md ``` ### Dependencies - backend/agents/agent_weather-app/requirements.txt: openai@>=2.0.0, python-dotenv, requests, uagents - backend/requirements.txt: fish-audio-sdk, flask, flask-cors, openrouter, python-dotenv, PyYAML, requests - frontend/package.json: @radix-ui/react-collapsible@^1.1.12, @radix-ui/react-dialog@^1.1.15, @radix-ui/react-scroll-area@^1.2.10, @radix-ui/react-separator@^1.1.7, @radix-ui/react-slot@^1.2.3, @radix-ui/react-tooltip@^1.2.8, @types/node@^24.9.1, @types/react@^18.2.0, @types/react-dom@^18.2.0, @vitejs/plugin-react@^4.0.0, autoprefixer@^10.4.14, class-variance-authority@^0.7.1, clsx@^2.1.1, lucide-react@^0.548.0, postcss@^8.4.24, react@^18.2.0, react-dom@^18.2.0, tailwind-merge@^3.3.1, tailwindcss@^3.3.0, tailwindcss-animate@^1.0.7, typescript@^5.0.0, vite@^4.4.0 ### Recent commits (newest first) - Update README.md - Update README.md - Update README.md - Merge pull request #7 from maxzfan/dev - ui fixes - a little bit of prompt movment - small ui polishing - Merge branch 'dev' of github.com:maxzfan/dag into dev - weather noti agent - refix noted bug - Merge pull request #6 from maxzfan/dev - agent dashboard with controls on ui v0 - Add complete dark mode support to AgentDashboard - Add agent generation functionality to frontend - Fix agent details and logs endpoints for yaml_only agents - Add agent loading from YAML files on startup - New Agent Dashboard - Merge pull request #5 from maxzfan/dev - Merge branch 'main' into dev - full end-to-en ## Key source files (fetched from GitHub, selected and truncated for size) ### ARCHITECTURE.md ```markdown # Project Architecture ## Overview This application is a voice-first AI assistant that captures user input (voice or text), analyzes it for journaling vs. automation opportunities, and when appropriate, generates and prepares a Fetch.ai agent from AI-produced YAML. The backend is a Flask server that orchestrates conversation state, integrates with external AI and TTS/STT services, and manages generated artifacts. Complementary doc: see SYSTEM_ARCHITECTURE.md for an end-to-end, higher-level view. This document focuses on the code in this repository and concrete module interactions. ## Key Components - **Flask Backend** (`backend/voice_server.py`) - HTTP API endpoints for conversation, TTS, STT, journal entry management, and agent lifecycle operations. - Orchestrator state machine controlling phases: `detail` and `yaml`, with problem detection triggering transitions. - Integrates with journal/detail/yaml helpers and external services. - **Journal Processing** (`backend/journal.py`) - Calls LLM with `prompt_journal.md` to classify/summarize input. - Detects structured ProblemBrief JSON from fenced code blocks. - Applies `_is_problem_heuristic` to gate problem mode. - When a problem is detected, routes to YAML generation path. - **Detail Gathering** (`backend/detail.py`) - Uses `prompt_detail.md` to ask targeted questions and synthesize a `detail_spec` for YAML generation. - **YAML Generation** (`backend/yaml_helper.py`) - Uses `prompt_yaml.md` to produce an agent YAML configuration. - Returns a `missing_info` prompt when information is insufficient. - **Prompts** (`backend/prompt_*.md`) - `prompt_journal.md`: classify journal input vs. problem. - `prompt_detail.md`: gather missing requirements. - `prompt_yaml.md`: produce final agent YAML. - **Data & Artifacts** (`backend/data/`) - Conversations stored as JSON: `conversation-.json`. - Generated artifacts: `backend/data/generated/*.yaml`. - **Voice Integration** - Fish Audio SDK for STT and TTS via `/speech-to-text` and `/text-to-speech` endpoints. ## Orchestration State Machine (voice_server) - **State** - `phase`: `None | detail | yaml` - `pending_questions`: list of outstanding questions or next prompts - `problem_brief`: structured detection result from journal step - `detail_spec`: aggregate of user-provided details - `ready_yaml`: produced YAML when complete - **Typical Flow** 1. User message hits `/conversation`. 2. `run_journal(...)` analyzes input. - If only journaling: returns a concise summary. - If `ProblemBrief` detected and heuristic passes: proceed to detail. 3. `run_detail(...)` refines requirements; may queue follow-up questions. 4. `run_yaml(...)` generates YAML or returns a `missing_info` question. 5. When YAML is ready, backend can proceed to create/deploy an agent (via DAG tooling). ## Primary Endpoints (selected) - `POST /conversation` - Input: `{ text: string, conversation_id?: string }` - B [truncated — 2570 more characters] ``` ### SYSTEM_ARCHITECTURE.md ```markdown # Nexus System Architecture & Process Flow ## 🏗️ System Overview Nexus is a sophisticated voice AI conversation platform that combines real-time voice interaction with intelligent agent generation and deployment. The system operates in two distinct modes: **Journal Mode** (passive note-taking) and **Agent Mode** (active problem-solving and automation). ## 🔄 Complete Process Flow ### 1. **User Interface Layer** ``` Frontend (React + TypeScript) ├── Voice Recording Interface ├── Real-time Speech-to-Text ├── AI Response Display ├── Text-to-Speech Playback └── Mode Switching (Journal ↔ Agent) ``` ### 2. **Backend Processing Layer** ``` Flask Server (Python) ├── Speech-to-Text Processing (Fish Audio) ├── AI Conversation Engine (OpenRouter + Claude) ├── Text-to-Speech Generation (Fish Audio) ├── Multi-Agent Orchestration └── Agent Generation & Deployment ``` ### 3. **Agent Generation Pipeline** ``` DAG System (Dynamic Agent Generator) ├── YAML Configuration Generation ├── Fetch.ai Agent Code Generation ├── Deployment Configuration └── Agent Lifecycle Management ``` --- ## 🎯 Detailed Component Connections ### **Frontend → Backend Communication** #### **Voice Recording Flow:** 1. **User clicks record** → Browser microphone access 2. **Speech captured** → Real-time transcription via Web Speech API 3. **Recording stops** → Audio sent to backend for processing 4. **Backend processes** → Fish Audio STT + OpenRouter AI 5. **Response generated** → Text-to-speech via Fish Audio TTS 6. **Audio played** → User hears AI response #### **API Endpoints Used:** ```typescript // Frontend API calls (App.tsx) const apiBaseUrl = 'http://localhost:5001' // Speech-to-Text POST /speech-to-text Content-Type: multipart/form-data Body: audio file // AI Conversation POST /conversation Content-Type: application/json Body: { text: "user message" } // Text-to-Speech POST /text-to-speech Content-Type: application/json Body: { text: "AI response" } ``` #### **Vite Proxy Configuration:** ```typescript // vite.config.ts server: { proxy: { '/api': { target: 'http://localhost:5001', changeOrigin: true, rewrite: (path) => path.replace(/^\/api/, '') } } } ``` --- ### **Backend Processing Architecture** #### **Multi-Agent Orchestration System:** ```python # voice_server.py - Core orchestration logic orchestrator_state = { "phase": None, # None | "detail" | "yaml" "pending_questions": None, # list[str] | None "problem_brief": None, # dict | None "detail_spec": None, # dict | None } ``` #### **Agent Mode Detection:** 1. **Journal Analysis** → `_run_journal(user_text)` - Uses `PROMPT_JOURNAL` to analyze user input - Detects problems using `_is_problem_heuristic()` - Returns either summary or `ProblemBrief` JSON 2. **Detail Gathering** → `_run_detail(problem_brief, user_text, current_spec)` - Uses `PROMPT_DETAIL` to ask clarifying questions - Builds detailed specification [truncated — 6654 more characters] ``` ### backend/requirements.txt ``` fish-audio-sdk flask flask-cors openrouter python-dotenv requests PyYAML ``` ### frontend/package.json ``` { "name": "voice-ai-conversation", "version": "1.0.0", "description": "Real-time voice conversation with AI using Fish Audio and OpenRouter APIs", "private": true, "keywords": [ "voice", "ai", "conversation", "speech-to-text", "text-to-speech", "fish-audio", "openrouter" ], "license": "MIT", "author": "Max Fan", "scripts": { "dev": "vite", "build": "tsc && vite build", "preview": "vite preview", "test": "echo \"Error: no test specified\" && exit 1" }, "dependencies": { "@radix-ui/react-collapsible": "^1.1.12", "@radix-ui/react-dialog": "^1.1.15", "@radix-ui/react-scroll-area": "^1.2.10", "@radix-ui/react-separator": "^1.1.7", "@radix-ui/react-slot": "^1.2.3", "@radix-ui/react-tooltip": "^1.2.8", "@types/node": "^24.9.1", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "lucide-react": "^0.548.0", "react": "^18.2.0", "react-dom": "^18.2.0", "tailwind-merge": "^3.3.1", "tailwindcss-animate": "^1.0.7" }, "devDependencies": { "@types/react": "^18.2.0", "@types/react-dom": "^18.2.0", "@vitejs/plugin-react": "^4.0.0", "autoprefixer": "^10.4.14", "postcss": "^8.4.24", "tailwindcss": "^3.3.0", "typescript": "^5.0.0", "vite": "^4.4.0" } } ``` ### .vite/deps_temp_847f4296/package.json ``` { "type": "module" } ``` ### .vite/deps_temp_c1771fe7/package.json ``` { "type": "module" } ``` ### .vite/deps_temp_e139d595/package.json ``` { "type": "module" } ``` ### backend/agents/agent_weather-app/requirements.txt ``` uagents openai>=2.0.0 python-dotenv requests ``` ### frontend/src/main.tsx ```typescript import React from 'react' import ReactDOM from 'react-dom/client' import App from './App.tsx' import './index.css' ReactDOM.createRoot(document.getElementById('root')!).render( , ) ``` ### frontend/src/App.tsx ```typescript import { useState, useRef, useEffect } from 'react' // Web Speech API types interface SpeechRecognitionEvent extends Event { resultIndex: number results: SpeechRecognitionResultList } interface SpeechRecognitionErrorEvent extends Event { error: string message: string } interface SpeechRecognition extends EventTarget { continuous: boolean interimResults: boolean lang: string start(): void stop(): void onstart: ((this: SpeechRecognition, ev: Event) => any) | null onresult: ((this: SpeechRecognition, ev: SpeechRecognitionEvent) => any) | null onerror: ((this: SpeechRecognition, ev: SpeechRecognitionErrorEvent) => any) | null onend: ((this: SpeechRecognition, ev: Event) => any) | null } declare global { interface Window { SpeechRecognition: { new(): SpeechRecognition } webkitSpeechRecognition: { new(): SpeechRecognition } } } import { Button } from '@/components/ui/button' import { Badge } from '@/components/ui/badge' import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card' import { ScrollArea } from '@/components/ui/scroll-area' import { Mic, MessageSquare, History, Trash2, Bot, RefreshCw } from 'lucide-react' import { AgentDashboard } from '@/components/AgentDashboard' interface Message { id: string sender: 'user' | 'ai' text: string timestamp: Date } interface JournalEntry { id: string title: string timestamp: string messages: Message[] summary?: string } function App() { const [mode, setMode] = useState<'agent' | 'journal'>('journal') const [selectedAgent, setSelectedAgent] = useState(null) const [showAgentDashboard, setShowAgentDashboard] = useState(false) const [isRecording, setIsRecording] = useState(false) const [status, setStatus] = useState('Click to start recording') const [messages, setMessages] = useState([]) const [agentMessages, setAgentMessages] = useState([]) const [isProcessing, setIsProcessing] = useState(false) const [liveTranscript, setLiveTranscript] = useState('') const [journalEntries, setJournalEntries] = useState([]) const [selectedEntry, setSelectedEntry] = useState(null) const [isSaving, setIsSaving] = useState(false) const [showSummary, setShowSummary] = useState(false) const [currentSummary, setCurrentSummary] = useState('') const [showAgentChatHistory, setShowAgentChatHistory] = useState(false) const audioPlaybackRef = useRef(null) const conversationRef = useRef(null) const recognitionRef = useRef(null) const apiBaseUrl = 'http://localhost:5001' // Available agents for the dashboard - will be populated by agent mode system const [availableAgents, setAvailableAgents] = useState>([]) useEffect(() => { checkMicrophonePermission() loadJournalEntries() loadAgents() initializeSpeechRecognition() }, []) const initializeSpeechRecognition = () => { if ('webkitSpeechRecognition' in window || 'SpeechRecognition' in window) { const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition const recognition = new SpeechRecognition() recognition.continuous = true recognition.interimResults = true recognition.lang = 'en-US' recognition.onstart = () => { console.log('Speech recognition started') } recognition.onresult = (event: SpeechRecognitionEvent) => { let currentTranscript = '' for (let i = event.resultIndex; i < event.results.length; i++) { const transcript = event.results[i][0].transcript currentTranscript += transcript } // Update live transcript with current results (not accumulating) setLiveTranscript(currentTranscript) } recognition.onerror = (event: SpeechRecognitionErrorEvent) => { console.error('Speech recognition error:', event.error) setStatus(`Speech recognition error: ${event.error}`) } recognition.onend = () => { console.log('Speech recognition ended') if (isRecording) { // Restart recognition if we're still recording setTimeout(() => { if (isRecording && recognitionRef.current) { try { recognitionRef.current.start() } catch (error) { console.warn('Failed to restart speech recognition:', error) } } }, 100) } } recognitionRef.current = recognition } else { console.warn('Speech recognition not supported in this browser') } } const loadJournalEntries = async () => { try { const response = await fetch(`${apiBaseUrl}/journal-entries`) if (response.ok) { const data = await response.json() // Ensure timestamps in messages are Date objects const entriesWithDates = data.entries.map((entry: JournalEntry) => ({ ...entry, messages: entry.messages.map((msg: Message) => ({ ...msg, timestamp: new Date(msg.timestamp) })) })) setJournalEntries(entriesWithDates) } } catch (error) { console.error('Error loading journal entries:', error) } } const loadAgents = async () => { try { const response = await fetch(`${apiBaseUrl}/agents`) if (response.ok) { const data = await response.json() setAvailableAgents(data.agents) } } catch (error) { console.error('Error loading agents:', error) } } const generateAgent = async (agentId: string) => { try { c [truncated — 41583 more characters] ``` [52 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]