# Project export: Bob: The Agent Builder This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2025 - Tagline: Forget drag-and-drop AI. One prompt, complete agentic workflow. Automating automation, finally. - Devpost: https://devpost.com/software/bob-the-agent-builder - GitHub: https://github.com/pringlesinghal/bob-the-builder - Video: https://www.youtube.com/embed/wkK4DVwOGac?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — Pranava Singhal (13 commits), rishma1999 (8 commits) ## Devpost submission (written by the team) ### Inspiration First, we had foundation models. Then we had AI agents built on top of these. Now we have agentic workflows composed of several agents with different specialized abilities working to solve a complex problem. Existing tools for creating agentic workflows, despite being low/no-code and having modular drag-and-drop designs, are hard to use. The more complex the problem, the more specialized components we need to manually add and connect. These tools have a high learning curve and you need to bend your task to fit the tool set available in these applications. We want to change this paradigm of agentic workflow creation. ### What it does Bob, the agent builder, takes a single text prompt as input and then intelligently breaks it down into the sub-tasks that need to be solved to complete the whole solution. The interface is familiar with blocks and connecting edges but now the entire graph is automatically generated and the blocks and edges can be any agentic workflow, as simple as code generation or LLM reasoning or as complex as computer use. A global planning agent chooses the best tools for each subtask by factoring in the demands of efficiency, nature of the subtask and input and output requirements. The entire agentic workflow is generated hierarchically with checking at each stage to ensure consistency and logical soundness. You just tell it what to do and it designs the best way to do it. ### How we built it We designed generalized JSON schemas to represent tasks and links for inter-task communication. A planning agent transforms the user prompt into a task object according to this schema. Then we iteratively pick the best tool for a task and if required break it down into simpler tasks. At each stage we keep track of the inputs and outputs connecting between these tasks. The entire graph of agentic blocks is tracked with LangGraph. This way we can substitute the planning model for anything of our choice: Perplexity, OpenAI, Gemini, or even custom fine-tuned models for your domain. The agents currently support tools like computer use with Scrapybara, LLM based search and planning, and code generation and deterministic execution. The tasks at leaf nodes immediately start running and wait on the outputs from other subtasks to be ready before they take these as inputs for their own processing maximizing parallelism of tasks. ### Challenges we ran into Architecting the design of an agentic workflow : Making a consistent schema for tasks, tools, and links was challenging. We wanted to build something that could decompose any task, being universal and compact at the same time. We iterated through several designs before landing on something that made sense and could accurately and comprehensively capture our problem structure. The problem of splitting tasks into subtasks : We employed different heuristics, prompting engineering and chain of thought techniques to ensure that the task decomposition was just right. Initially we ran into situations where the subtasks were either too many and too simple or on the other extreme, too similar to the parent task. Managing inter-task communication : We used asyncio to ensure that we could spawn tasks as soon as we reached the leaf nodes of a subtask while also waiting for its dependencies to be ready. Moreover, we designed the links so that the whole agentic workflow can be reused by just resetting the link values. Handling error checking for smaller tasks to prevent single points of failure: We run error checking for JSON schema validation and other kinds of structured output along with repeated LLM prompting until a valid output is generated. This way the model doesn't fail when a single subtask fails. Building a flexible front-end: We wanted to build a familiar block interface while also giving users the power of an entire agent builder inside each block. We can modify the contents of a task block to regenerate the agentic workflow with this new component while the parents in the tree are unaffected. The users can also manually connect blocks that they have constructed and blocks can be made to solve any problem, simple or complex, with the desired I/O format. ### Accomplishments we're proud of We are blown away by the things this model is capable of. We exceeded our expectations by putting together several new technologies including Computer Use Agents (Scrapybara), Langchain, Structured output and reasoning with LLMs, and building and managing interprocess communication in an agentic workflow. We strongly believe that automating agentic workflow creation is the future. ### What's next There are several features we want to add to our agentic workflow: Eigenlayer verification for agent computations on Web3 Streaming user input during task computation and dynamically creating user input fields during evaluation Integrating existing vertical AI SaaS tools to strengthen the capabilities of our agentic system to pick specialized components reducing the number of subtask decompositions needed Integrating this with user edits from the front-end for dynamically regenerating the computation graph ## README (from the GitHub repository) # bob-the-builder ## Detected evidence (automated analysis) Indexed codebase: 29 recognized source files, 80 KB. - CSS (language) — detected in the code - Flask (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - LangChain (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Google Gemini (technology) — claimed on Devpost, not found in the code - OpenAI (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (40 of 40) ``` .gitignore all_code.txt client/.gitignore client/package.json client/public/index.html client/public/manifest.json client/public/robots.txt client/src/App.css client/src/App.js client/src/App.test.js client/src/components/BranchedEdge.js client/src/components/CustomEdge.js client/src/components/Header.js client/src/components/RightSidebar.js client/src/components/SubtaskNode.js client/src/dummydata.json client/src/index.css client/src/index.js client/src/reportWebVitals.js client/src/setupTests.js client/src/store.js client/tailwind.config.js code_compilation.py config.py env evaluation.py llm_interaction.py main.py new_main.py orchestration.py out.txt README.md requirements.txt schemas/__init__.py schemas/Link.py schemas/Task.py task_execution.py task_manager.py task.txt tree_utils.py ``` ### Dependencies - client/package.json: @testing-library/dom@^10.4.0, @testing-library/jest-dom@^6.6.3, @testing-library/react@^16.2.0, @testing-library/user-event@^13.5.0, @xyflow/react@^12.4.3, axios@^1.7.9, react@^19.0.0, react-dom@^19.0.0, react-icons@^5.4.0, react-scripts@5.0.1, web-vitals@^2.1.4, zustand@^5.0.3 - requirements.txt: flask@==2.3.2, flask-cors@==4.0.0, flask-socketio@==5.3.4, jsonschema@==4.19.1, langchain@==0.0.267, langchain_google_genai, langchain-community@==0.0.6, langsmith@==0.0.43, networkx@==3.1, pydantic@==2.4.2, python-dotenv@==1.0.0, scikit-learn@==1.3.0 ### Recent commits (newest first) - frontend and backend integration with real time input handling - Merge pull request #2 from pringlesinghal/merged_frontend_backend - Merge pull request #1 from pringlesinghal/task_execution - Updated - frontend changes - Merge remote-tracking branch 'origin/task_execution' into merged_frontend_backend - bakcend and frotnend changes - Adding Scrapybara - added requirements - Merge branch 'task_execution' into merged_frontend_backend - Updated files - merge with websockets - Merge branch 'task_execution' into merged_frontend_backend - frontend changes - added connection handles and app.js changes - Fixed circular reference issue - initial task exec commit - circular ref error - fixing tree: - fixed tool prompt ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` flask==2.3.2 flask-cors==4.0.0 flask-socketio==5.3.4 python-dotenv==1.0.0 langchain==0.0.267 langchain-community==0.0.6 langsmith==0.0.43 pydantic==2.4.2 jsonschema==4.19.1 networkx==3.1 scikit-learn==1.3.0 langchain_google_genai ``` ### client/package.json ``` { "name": "client", "version": "0.1.0", "private": true, "dependencies": { "@testing-library/dom": "^10.4.0", "@testing-library/jest-dom": "^6.6.3", "@testing-library/react": "^16.2.0", "@testing-library/user-event": "^13.5.0", "@xyflow/react": "^12.4.3", "axios": "^1.7.9", "react": "^19.0.0", "react-dom": "^19.0.0", "react-icons": "^5.4.0", "react-scripts": "5.0.1", "web-vitals": "^2.1.4", "zustand": "^5.0.3" }, "scripts": { "start": "react-scripts start", "build": "react-scripts build", "test": "react-scripts test", "eject": "react-scripts eject" }, "eslintConfig": { "extends": [ "react-app", "react-app/jest" ] }, "browserslist": { "production": [ ">0.2%", "not dead", "not op_mini all" ], "development": [ "last 1 chrome version", "last 1 firefox version", "last 1 safari version" ] } } ``` ### main.py ```python # app.py from flask import Flask, request, jsonify import asyncio import os from schemas import Task from langchain.prompts import ChatPromptTemplate from langchain.schema import HumanMessage, SystemMessage from config import LANGCHAIN_TRACING_V2, chat_model from task_manager import TaskManager from orchestration import a_generate_task_tree from tree_utils import print_task_tree from evaluation import evaluate_task_decomposition from langchain_core.tracers.context import tracing_v2_enabled import json from typing import Dict from jsonschema import validate, ValidationError from flask import Flask, request, jsonify from flask_cors import CORS app = Flask(__name__) CORS(app) # Enable CORS for all routes and origins async def generate_task_tree(prompt: str): task_manager = TaskManager() #Creating task manager object here with tracing_v2_enabled(project_name="Task Decomposition") if LANGCHAIN_TRACING_V2 else open(os.devnull, "w") as f: # only trace if the relevant flag is turned on # TODO: How can I transform the user prompt to be more specific and actionable for the LLM? # system_message = SystemMessage( # content="You are a computer use agent capable of doing anything. Rephrase the user's task prompt to highlight the key action verbs in the user's request and identify what needs to be done.") # human_message = HumanMessage( # content=f"Output a very concise task prompt to help an LLM understand the user's task prompt: {prompt}.\n\nEmphasize what action verbs are specified by the user. Only output the prompt and nothing else.") # chat_prompt = ChatPromptTemplate.from_messages([system_message, human_message]) # response = await chat_model.ainvoke(chat_prompt.format_messages()) # response_content = response.content # print(f"{response_content=}") response_content = prompt full_task, _ = await a_generate_task_tree(response_content, Task.model_json_schema(), task_manager) # Passing task_manager object if full_task and validate_task(full_task, Task.model_json_schema()): # Save full_task to out.txt with open("out.txt", 'w') as file: json.dump(full_task, file, indent=4) return full_task else: return {"error": "Task generation or validation failed."} def validate_task(task: Dict, schema: Dict): #Keeping this function here since it is tiny try: validate(instance=task, schema=schema) return True except ValidationError as e: print(f"Task validation error: {e}") return False # Your routes here @app.route('/generate_task_tree', methods=['POST']) def generate_task_tree_api(): data = request.get_json() if 'prompt' not in data: return jsonify({"error": "Missing 'prompt' in request"}), 400 prompt = data['prompt'] prompt = prompt + ".Decompose the main task into atleast 2 subtasks and 1 ingest and produces." print(prompt) # Run the asynchronous function in a synchronous context loop = asyncio.new_event_loop() asyncio.set_event_loop(loop) result = loop.run_until_complete(generate_task_tree(prompt)) loop.close() return jsonify(result) if __name__ == "__main__": app.run(host="127.0.0.1", port=5001) ``` ### client/src/index.js ```javascript import React from 'react'; import ReactDOM from 'react-dom/client'; import './index.css'; import App from './App'; import reportWebVitals from './reportWebVitals'; const root = ReactDOM.createRoot(document.getElementById('root')); root.render( ); // If you want to start measuring performance in your app, pass a function // to log results (for example: reportWebVitals(console.log)) // or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals reportWebVitals(); ``` ### client/src/App.js ```javascript import React, { useState, useCallback, useMemo, useRef } from 'react'; import { ReactFlow, addEdge, useNodesState, useEdgesState, Controls, Background, Handle, Position, Connection, } from '@xyflow/react'; import { AiFillPlusCircle } from 'react-icons/ai'; import { IoSend } from "react-icons/io5"; import '@xyflow/react/dist/style.css'; import './App.css'; import Header from './components/Header'; import CustomEdge from './components/CustomEdge'; import BranchedEdge from './components/BranchedEdge'; import SubtaskNode from './components/SubtaskNode'; import RightSidebar from './components/RightSidebar'; import dummyTasks from './dummydata.json'; console.log('Loaded dummy tasks:', dummyTasks); let id = 0; const getId = () => `node_${id++}`; // Custom Node Component with multiple handles const CustomNode = ({ data, isSelected }) => { const [isHovered, setIsHovered] = useState(false); // Determine border color based on hover and selection state const getBorderColor = () => { if (isSelected) return 'border-blue-700'; if (isHovered) { return data.completed ? 'border-green-500' : 'border-blue-500'; } return 'border-gray-200'; // Neutral border when not hovered }; // Generate handles based on ingests and produces const renderIngestHandles = () => { const ingests = data.ingests || []; return ingests.map((ingest, index) => { const offset = ((index + 1) * 150) + 200; // Start from 200px down and space by 150px return ( ); }); }; const renderProducesHandles = () => { const produces = data.produces || []; return produces.map((output, index) => { const offset = ((index + 1) * 150) + 200; // Start from 200px down and space by 150px return ( ); }); }; return (
setIsHovered(true)} onMouseLeave={() => setIsHovered(false)}> {/* Task name at the top */}
{data.task_name}
{/* Divider line */}
{/* Task description in the middle */}
{data.task_description}
{/* Dynamic handles based on ingests and produces */} {renderIngestHandles()} {renderProducesHandles()} {/* Top handle for hierarchy */} {data.isChild && ( )} {/* Bottom handle for hierarchy */} {data.hasChildren && ( )} {/* Bottom handle - for all nodes */} {/* Left handle - for child nodes */} {data.isChild && ( )} {/* Right handle - for child nodes */} {data.isChild && ( )}
); }; const nodeTypes = { customNode: CustomNode, }; const edgeTypes = { custom: CustomEdge, // Curved edges for parent-to-parent and subtask-to-subtask branched: BranchedEdge, // Straight dotted edges for parent-to-child }; console.log('Available edge types:', edgeTypes); console.log('Available edge types:', Object.keys(edgeTypes)); function App() { // Add CSS to ensure the app takes full viewport height React.useEffect(() => { document.body.style.margin = '0'; document.body.style.height = '100vh'; document.documentElement.style.height = '100vh'; }, []); const [nodes, setNodes, onNodesChange] = useNodesState([]); const [edges, setEdges, onEdgesChange] = useEdgesState([]); const [prompt, setPrompt] = useState(''); const [selectedNode, setSelectedNode] = useState(null); const [isSidebarOpen, setIsSidebarOpen] = useState(false); const reactFlowInstance = useRef(null); // Function to create nodes and edges based on task data const createNodesAndEdges = (taskData) => { console.log('Creating nodes and edges from d [truncated — 19248 more characters] ``` ### tree_utils.py ```python def print_task_tree(task, indent=""): selected_tool = task.get('selected_tool', 'N/A') print(f"{indent}Task: {task['task_name']} (Tool: {selected_tool})") if 'subtasks' in task and task['subtasks']: for subtask in task['subtasks']: print_task_tree(subtask, indent + " ") elif 'result' in task: print(f"{indent} Result: {task['result']}") ``` ### task_manager.py ```python # task_manager.py from config import MAX_TASKS class TaskManager: def __init__(self, max_tasks=MAX_TASKS): self.tasks = [] self.max_tasks = max_tasks def add_task(self, task): if len(self.tasks) < self.max_tasks: self.tasks.append(task) return True return False def get_task_count(self): return len(self.tasks) ``` ### evaluation.py ```python from langchain.evaluation import load_evaluator from config import chat_model #Import chat model import json def evaluate_task_decomposition(task): evaluator = load_evaluator("criteria", criteria={ "completeness": "Does the decomposition cover all aspects of the task?", "actionability": "Are the subtasks concrete and actionable?", "independence": "Are the subtasks sufficiently independent?" }, llm=chat_model # Use the ChatPerplexity model for evaluation ) evaluation = evaluator.evaluate_strings( prediction=json.dumps(task, indent=2), input=task['task_description'] ) return evaluation ``` ### code_compilation.py ```python import os def concatenate_py_files(root_dir, output_file, exclude_file): with open(output_file, 'w', encoding='utf-8') as outfile: for dirpath, dirnames, filenames in os.walk(root_dir): for filename in filenames: if (filename.endswith('.py') or filename.endswith('.js')) and filename != exclude_file: relative_path = os.path.relpath(os.path.join(dirpath, filename), root_dir) outfile.write(f"# File: {relative_path}\n\n") with open(os.path.join(dirpath, filename), 'r', encoding='utf-8') as infile: outfile.write(infile.read()) outfile.write("\n\n") # Usage root_directory = '.' # Current directory, change this to your project's root directory if needed output_file = 'all_code.txt' script_filename = os.path.basename(__file__) # Get the name of this script concatenate_py_files(root_directory, output_file, script_filename) print(f"All Python code (except this script) has been concatenated into {output_file}") ``` ### orchestration.py ```python from typing import Dict, List from config import MAX_TASKS, MAX_DEPTH from task_manager import TaskManager from llm_interaction import a_transform_prompt, a_decompose_subtasks, a_select_tool from task_execution import execute_task async def a_generate_task_tree(prompt: str, schema: Dict, task_manager: TaskManager, max_depth: int = MAX_DEPTH): # print(f"{prompt=}, {schema=}") # print(f"{schema['$defs']['Task'].keys()=}") # TODO: clean up task = await a_transform_prompt(prompt, schema, "") task["ingests"] = [] if not task: raise Exception("Failed to generate task from user prompt") task_queue = [(task, 0, None, "")] root_task = None tasks_by_depth = {} while task_queue: current_task, current_depth, parent_task, parent_context = task_queue.pop(0) if task_manager.get_task_count() >= task_manager.max_tasks: break selected_tool = await a_select_tool(current_task, schema, current_depth, max_depth) if not selected_tool: continue current_task['selected_tool'] = selected_tool current_task['depth'] = current_depth if not task_manager.add_task(current_task): break if root_task is None: root_task = current_task if parent_task: if 'subtasks' not in parent_task: parent_task['subtasks'] = [] parent_task['subtasks'].append(current_task) if current_depth not in tasks_by_depth: tasks_by_depth[current_depth] = [] tasks_by_depth[current_depth].append(current_task) print(current_task) if selected_tool == 'D': # Only decompose if "Mix of Tools" is selected subtasks = await a_decompose_subtasks(current_task, schema, parent_context) if subtasks: new_parent_context = f"{parent_context}\nParent task: {current_task['task_description']}" for subtask in subtasks: task_queue.append((subtask, current_depth + 1, current_task, new_parent_context)) else: current_task['result'] = await execute_task(current_task) return root_task, tasks_by_depth ``` [20 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]