# Project export: FlowPilot: Your Agentic Co-Pilot

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: FlowPilot listens, understands, and executes, transforming Zoom meetings into Slide decks and Notion workflows. No more lost action items - just approve and watch it execute.
- Devpost: https://devpost.com/software/flowpilot-zoom-agent-that-automates-your-workflows
- GitHub: https://github.com/cyu60/dain-agent
- Demo: https://github.com/CarterCote/flow-pilot
- Video: https://www.youtube.com/embed/d9pkeKJIGRg?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner (DAIN Labs: AI Agent Excellence & Innovation Awards ($5k Cash [1st] & $2.5K Cash [2nd] & $500 Cash [3rd] + Agent Launchpad Invitation); Zoom: Best Use of Zoom APIs ($250 Git Card + Herschel Duffle Bags [1st] & $100 Git Card + Hoodies [2nd] & $50 Git Card + Bottles [3rd]); Perplexity: Hacking With Perplexity Award ($50 Gift Card per team member))
- Team: 1 GitHub contributor(s) — cyu60 (4 commits)

## Devpost submission (written by the team)

### Inspiration

FlowPilot listens, understands, and executes, transforming Zoom meetings into Slide decks and Notion workflows. No more lost action items - just approve and watch it execute. FlowPilot provides amazing collaboration opportunities for sales, engineering, and product teams alike. Having been apart of workflows at fast-growing startups and large corporations, our team grew increasingly frustrated with the significant friction caused by meetings. This includes the amount of action items discussed in meetings that often get forgotten, tediously written, or executed. We noticed that while tools like Notion excel at documentation and Google Workspace handles scheduling, there wasn't a seamless way to transform meeting conversations into automated workflows. FlowPilot bridges this gap by creating an intelligent layer that turns verbal commitments into automated actions.

### What it does

FlowPilot transforms Zoom meetings into an autonomous productivity engine. By leveraging DAIN's advanced agent framework, our system: Creates comprehensive Notion documentation including meeting notes, workflow templates, and task assignments Automatically generates follow-up slide decks for educational sessions and team presentations Handles calendar scheduling and email communications through Google Workspace Maintains contextual awareness across multiple meetings to track long-term projects and commitments Technical Architecture 1. Meeting Intelligence Layer Zoom RTMS Integration: Implements WebSocket connections for real-time audio stream processing Verifies local server client for the Zoom application via Ngrok Implements WebSocket connections for real-time audio stream processing Verifies local server client for the Zoom application via Ngrok 2. DAIN Agent Framework Implementation Agent Orchestration: Custom-built context router that manages state between multiple specialized agents Implements DAIN's memory system for maintaining conversation context and user preferences Uses structured JSON formats for inter-agent communication and task delegation Agent Orchestration: Custom-built context router that manages state between multiple specialized agents Implements DAIN's memory system for maintaining conversation context and user preferences Uses structured JSON formats for inter-agent communication and task delegation Specialized Agents: Documentation Agent: Handles Notion to add tasks and todo Calendar Agent: Manages Google Calendar scheduling and reminders Communication Agent: Handles Gmail operations and meeting follow-ups, and supports audio email attachments via Eleven Labs text-to-speech Presentation Agent: Automates slide deck creation and updates using a custom renderer from v0.dev Outbound Phone calls: Make outbound phone calls via DAIN client Specialized Agents: Documentation Agent: Handles Notion to add tasks and todo Calendar Agent: Manages Google Calendar scheduling and reminders Communication Agent: Handles Gmail operations and meeting follow-ups, and supports audio email attachments via Eleven Labs text-to-speech Presentation Agent: Automates slide deck creation and updates using a custom renderer from v0.dev Outbound Phone calls: Make outbound phone calls via DAIN client 3. Custom Agent Integration through Orchestrator Data Processing Pipeline: Orchestrator agent built using Magic Loops' schema system Real-time event processing for immediate action execution Orchestrator agent built using Magic Loops' schema system Real-time event processing for immediate action execution 4. Execution Framework with DAIN Tunneling Service Architecture: Secure WebSocket tunneling for real-time bi-directional communication Load-balanced request routing for scalable action execution OAuth token management for secure service authentication Tunneling Service Architecture: Secure WebSocket tunneling for real-time bi-directional communication Load-balanced request routing for scalable action execution OAuth token management for secure service authentication Service Integration Layer: Notion API integration for document/task creation and updates Google Workspace API integration for calendar and email management Custom templating engine for slide deck generation with v0.dev interface Send email with audio transcript generated with ElevenLabs Service Integration Layer: Notion API integration for document/task creation and updates Google Workspace API integration for calendar and email management Custom templating engine for slide deck generation with v0.dev interface Send email with audio transcript generated with ElevenLabs Implementation Challenges Solved Our journey in building FlowPilot presented several complex technical hurdles that required innovative solutions. The first major challenge was optimizing real-time processing of audio streams. We solved this by implementing a sophisticated buffering system that uses chunked transfer encoding, allowing us to process meeting data with minimal latency while maintaining data integrity. Context management proved to be another significant challenge, particularly when handling multiple concurrent meetings and maintaining conversation state. By leveraging DAIN's memory system, we developed a robust solution that maintains contextual awareness across sessions while efficiently managing system resources. Action reliability was perhaps our most critical challenge, given the importance of executing tasks accurately. We addressed this by building a resilient tunneling service with intelligent retry mechanisms and transaction management, ensuring that every action - from creating Notion pages to scheduling calendar events - completes successfully or fails gracefully with proper error handling. The final hurdle was integrating multiple third-party services seamlessly. We overcame this by developing a unified authentication layer that efficiently manages various OAuth flows while maintaining secure token storage and rotation. Another major hurdle was building out a custom presentation renderer. Developing it took a long time and was difficult to get running properly. With v0.dev, however, we were able to quickly build a custom frontend that reliably renders our presentations. Finally, integrating multiple third-party services seamlessly was challenging, so we developed a unified authentication layer that efficiently manages various OAuth flows while maintaining secure token storage and rotation. Future Technical Roadmap Our vision for FlowPilot's technical evolution focuses on two key areas: advancing our agent intelligence and scaling the platform infrastructure. We're working on implementing DAIN's advanced memory models to significantly enhance our agents' context retention capabilities, allowing for more nuanced understanding of long-term projects and recurring meetings. This will enable our agents to make more intelligent decisions about task creation and workflow automation while maintaining context across multiple sessions. On the infrastructure side, we're architecting a distributed processing system that will handle concurrent meetings more efficiently, implementing an enhanced caching layer for improved performance, and moving towards a microservices architecture for our agent components. This architectural evolution will allow us to scale horizontally while maintaining the reliability and speed our users expect. We're particularly excited about the potential to expand our agent ecosystem, creating specialized agents that cater to specific industries and use cases while maintaining our core focus on seamless automation and intelligent workflow management.

## README (from the GitHub repository)

# Productivity Tools DAIN Service

A DAIN protocol service providing various productivity tools for task management, meetings, and calendar events.

## Features

- Query databases
- Extract action items from meetings
- Add tasks to Notion
- Create calendar events
- Get Zoom meeting transcripts
- Generate presentations from meeting transcripts

## Prerequisites

- Node.js (v16 or higher)
- DAIN Protocol API key

## Installation

1. Clone the repository
2. Install dependencies:
3. Create a `.env` file in the root directory
4. Add your DAIN API key to the `.env` file:
   ```
   DAIN_API_KEY=your_api_key_here
   ```
5. Build the project:
   ```bash
   npm run build
   ```
6. Start the service:
   ```bash
   npm start
   ```

## Detected evidence (automated analysis)

Indexed codebase: 2 recognized source files, 13 KB.
- TypeScript (language) — detected in the code
- Docker (technology) — claimed on Devpost, not found in the code
- Express (technology) — claimed on Devpost, not found in the code
- Node.js (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (6 of 6)

```
.gitignore
dain.json
package.json
README.md
src/index.ts
tsconfig.json
```

### Dependencies

- package.json: @dainprotocol/cli@^1.0.31, @dainprotocol/service-sdk@^1.0.93, @dainprotocol/utils@^0.0.48, @types/express@^4.17.13, @types/node@^22.5.4, axios@^1.7.5, hono@^4.6.3, ts-node@^10.4.0, typescript@^5.5.4, zod@^3.23.8

### Recent commits (newest first)

- add audio generation
- add audio generation
- init commit
- first commit

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "name": "new-dain",
  "version": "1.0.0",
  "description": "A Dain Protocol project",
  "main": "src/index.ts",
  "scripts": {
    "start": "ts-node src/index.ts",
    "dev": "dain dev",
    "build": "dain build",
    "deploy": "dain deploy"
  },
  "dependencies": {
    "@dainprotocol/service-sdk": "^1.0.93",
    "@dainprotocol/utils": "^0.0.48",
    "zod": "^3.23.8",
    "hono": "^4.6.3",
    "ts-node": "^10.4.0",
    "typescript": "^5.5.4",
    "@types/express": "^4.17.13",
    "@types/node": "^22.5.4",
    "axios": "^1.7.5",
    "@dainprotocol/cli": "^1.0.31"
  }
}

```

### src/index.ts

```typescript
//File: example/example-node.ts

import { z } from "zod";
import axios from "axios";

import { defineDAINService, ToolConfig } from "@dainprotocol/service-sdk";

import {
  CardUIBuilder,
  TableUIBuilder,
  MapUIBuilder,
} from "@dainprotocol/utils";

const queryDatabaseConfig: ToolConfig = {
  id: "query-database",
  name: "Query Database",
  description: "Performs a database query operation",
  input: z
    .object({
      query: z.string().describe("The database query to execute"),
    })
    .describe("Input parameters for the database query"),
  output: z.string().describe("Query result message"),
  pricing: { pricePerUse: 0, currency: "USD" },
  handler: async ({ query }, agentInfo, context) => {
    console.log(
      `User / Agent ${agentInfo.id} requested database query: ${query}`
    );

    return {
      text: "Querying the database...",
      data: "Querying the database...",
      ui: new CardUIBuilder()
        .setRenderMode("page")
        .title("Database Query")
        .content("Querying the database...")
        .build(),
    };
  },
};

const getMeetingActionItemsConfig: ToolConfig = {
  id: "get-meeting-action-items",
  name: "Get Meeting Action Items",
  description: "Get action items from my most recent meeting",
  input: z.object({}).describe("No input required"),
  output: z
    .object({
      actionItems: z.string().describe("Description of meeting action items"),
    })
    .describe("Meeting action items and follow-up details"),
  pricing: { pricePerUse: 0, currency: "USD" },
  handler: async ({ meetingTitle }, agentInfo, context) => {
    console.log(
      `User / Agent ${agentInfo.id} requested action items for meeting`
    );

    const response = await fetch(
      "https://magicloops.dev/api/loop/e0243b5d-0f4a-4341-a57e-33ce9dddf1ac/run",
      {
        method: "POST",
        body: JSON.stringify({}),
      }
    );

    const responseJson = await response.json();
    const actionItems = responseJson.instructions || "";

    return {
      text: `Retrieved action items for ${meetingTitle}`,
      data: {
        actionItems,
      },
      ui: new CardUIBuilder()
        .setRenderMode("page")
        .title(`👨‍💻 Orchestrator Agent:`)
        .content(`## Instructions\n${actionItems}`)
        .build(),
    };
  },
};

const addToNotionConfig: ToolConfig = {
  id: "add-to-notion",
  name: "Add to Notion",
  description: "Adds a new task to Notion database",
  input: z
    .object({
      task: z.string().describe("Task description"),
      status: z.string().describe("Task status (e.g., doing, todo, done)"),
      date: z.string().describe("Due date for the task"),
    })
    .describe("Input parameters for creating a Notion task"),
  output: z
    .object({
      pageId: z.string().describe("ID of the created Notion page"),
      url: z.string().describe("URL of the created Notion page"),
    })
    .describe("Created Notion page details"),
  pricing: { pricePerUse: 0, currency: "USD" },
  handler: async ({ task, status, date }, agentInfo, context) => {
    console.log(
      `User / Agent ${agentInfo.id} requested to add task to Notion: ${task}`
    );

    const response = await fetch(
      "https://magicloops.dev/api/loop/0fae0e40-97cf-42d5-bd8c-40bc1a696c32/run",
      {
        method: "POST",
        body: JSON.stringify({
          task,
          status,
          date,
        }),
      }
    );

    const responseJson = await response.json();

    return {
      text: `Added task "${task}" to Notion`,
      data: {
        pageId: responseJson.id,
        url: responseJson.url,
      },
      ui: new CardUIBuilder()
        .setRenderMode("page")
        .title("Task Added to Notion")
        .content(
          `Successfully added task to Notion:\n\n- Task: ${task}\n- Status: ${status}\n- Due Date: ${date}\n\n[View in Notion](${responseJson.url})`
        )
        .build(),
    };
  },
};

const createCalendarEventConfig: ToolConfig = {
  id: "create-calendar-event",
  name: "Create Calendar Event",
  description: "Creates a new calendar event",
  input: z
    .object({
      task: z.string().describe("Event description or title"),
    })
    .describe("Input parameters for creating a calendar event"),
  output: z
    .object({
      calendarURL: z.string().describe("URL to the created calendar event"),
    })
    .describe("Calendar event details"),
  pricing: { pricePerUse: 0, currency: "USD" },
  handler: async ({ task }, agentInfo, context) => {
    console.log(
      `User / Agent ${agentInfo.id} requested to create calendar event: ${task}`
    );

    const response = await fetch(
      "https://magicloops.dev/api/loop/a02fc62a-a664-4e44-a2d8-79c2ac7c7779/run",
      {
        method: "POST",
        body: JSON.stringify({
          task,
        }),
      }
    );

    const responseJson = await response.json();

    return {
      text: `Created calendar event for "${task}"`,
      data: {
        calendarURL: responseJson.calendarURL,
      },
      ui: new CardUIBuilder()
        .setRenderMode("page")
        .title("Calendar Event Created")
        .content(
          `Successfully created calendar event:\n\n- Event: ${task}\n\n[Add to Calendar](${responseJson.calendarURL})`
        )
        .build(),
    };
  },
};

const getTranscriptConfig: ToolConfig = {
  id: "get-zoom-meeting",
  name: "Get Zoom Meeting",
  description: "Gets most recent zoom meeting",
  input: z.object({}).describe(""),
  output: z
    .object({
      transcript: z.string().describe("Zoom meeting transcript"),
    })
    .describe("Zoom meeting transcript"),
  pricing: { pricePerUse: 0, currency: "USD" },
  handler: async ({ input }, agentInfo, context) => {
    console.log(
      `User / Agent ${agentInfo.id} requested zoom meeting for: ${input}`
    );

    const response = await fetch(
      "https://magicloops.dev/api/loop/3d16aa92-7b25-415a-ad37-b5f9a33815a6/run",
      {
        method: "POST",
        body: JSON.stringify({
          input
[truncated — 6076 more characters]
```