# Project export: Klip AI This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: You build the product, we’ll build the hype. Helping early tech founders and builders promote their products with instant AI-generated social media reels. - Devpost: https://devpost.com/software/velocity-studio - GitHub: https://github.com/immedha/treehacks2026_heygen - Video: https://www.youtube.com/embed/3K5KgKxsR9c?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — immedha (13 commits), SaipranavSripathi (9 commits) ## Devpost submission (written by the team) ### Inspiration Early-stage tech founders and builders know their product deeply, but most do not have the time, budget, or knowledge to advertise. As sudents and builders, we faced this issue ourselves; we coded many personal projects but didn't know where to find users. In this era, social media is one of the best methods for promotion. Klip AI started from one question: can a founder turn a website/application URL into social media reels in minutes? ### What it does Klip AI converts a startup’s public website URL into multiple short-form reels in under 10 minutes. Flow: Founder pastes a URL. Our agent scrapes all pages in the website and extracts product messaging and key visuals. Our agent does deep research to understand the context behind the product - market research, competitor analysis, and other information about the website on the internet. It also researches patterns in trending reels on social media so it can emulate those styles. The agent generates one reel based on this research context. The user has a session with an AI Marketing Director avatar which will ask questions about the user's style preferences for the reels. The agent creates a plan for 3-5 more reels and generates them with this additional information. Founder exports MP4 files and posts. Speed-to-first-reel: No editing timeline, no design tools, no blank page. ### How we built it Scraping + capture: Browserbase StageHand for page discovery, screenshots, and text extraction. Trend analysis: Engagement-weighted template scoring from current short-form format signals. AI layer: Perplexity Sonar-based deep-researched content and LLM-generated hook/copy options mapped to product features and audience intent. Video pipeline: HeyGen Video Generation API, Remotion + Google Veo for programmatic composition, transitions, and rendering. Backend: FastAPI orchestration, Node Frontend: React ### Challenges we ran into It was difficult for HeyGen's realtime Avatar API to converse when there was background noise, because of its auto VAD. So, we decided to optimize the auto VAD by implementing Kalman filters in the frontend to remove background noise before passing it into the avatar stream. It was difficult to make the videos include accurate images of the website because LLMs/image models often hallucinate. So we used BrowserBase's Stagehand to dynamically take the best screenshots and images in the website when scraping it, then passed them into HeyGen. Video generation is expensive. We parallelized jobs and generated reels before-and-after the session with the Marketing Director so users can see reels quickly. Accomplishments we’re proud of End-to-end pipeline from URL to downloadable reels. Sub 6-minute generation for multiple reels. We created reels for the founders and hackathon participants and posted them on Instagram: https://www.instagram.com/klip.techfounders?igsh=MzRlODBiNWFlZA==. Being able to have a realtime/low-latency speech conversation with the AI avatar, which asks high-quality questions. The content and visuals in the reels are accurate - no hallucinations. ### What we learned We learned that generating many reels is better than generating a single reel because users prefer multiple variations that they can post and A/B test quickly. We gained technical skills in creating agents that do multi-turn conversations, web automation, video editing, research, and putting that all together in a pipeline that includes both sequential and parallel steps. ### What's next Multi-platform exports (Shorts, Reels, TikTok formats). Deeper personalization from prior performance history. Allowing editing ## README (from the GitHub repository) # React + TypeScript + Vite This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules. Currently, two official plugins are available: - [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Babel](https://babeljs.io/) (or [oxc](https://oxc.rs) when used in [rolldown-vite](https://vite.dev/guide/rolldown)) for Fast Refresh - [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/) for Fast Refresh ## React Compiler The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation). ## Expanding the ESLint configuration If you are developing a production application, we recommend updating the configuration to enable type-aware lint rules: ```js export default defineConfig([ globalIgnores(['dist']), { files: ['**/*.{ts,tsx}'], extends: [ // Other configs... // Remove tseslint.configs.recommended and replace with this tseslint.configs.recommendedTypeChecked, // Alternatively, use this for stricter rules tseslint.configs.strictTypeChecked, // Optionally, add this for stylistic rules tseslint.configs.stylisticTypeChecked, // Other configs... ], languageOptions: { parserOptions: { project: ['./tsconfig.node.json', './tsconfig.app.json'], tsconfigRootDir: import.meta.dirname, }, // other options... }, }, ]) ``` You can also install [eslint-plugin-react-x](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-x) and [eslint-plugin-react-dom](https://github.com/Rel1cx/eslint-react/tree/main/packages/plugins/eslint-plugin-react-dom) for React-specific lint rules: ```js // eslint.config.js import reactX from 'eslint-plugin-react-x' import reactDom from 'eslint-plugin-react-dom' export default defineConfig([ globalIgnores(['dist']), { files: ['**/*.{ts,tsx}'], extends: [ // Other configs... // Enable lint rules for React reactX.configs['recommended-typescript'], // Enable lint rules for React DOM reactDom.configs.recommended, ], languageOptions: { parserOptions: { project: ['./tsconfig.node.json', './tsconfig.app.json'], tsconfigRootDir: import.meta.dirname, }, // other options... }, }, ]) ``` ## Detected evidence (automated analysis) Indexed codebase: 74 recognized source files, 346 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - Express (technology) — detected in the code - FastAPI (technology) — detected in the code - Firebase (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - TypeScript (language) — detected in the code - Node.js (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (88 of 88) ``` .env.example .gitignore eslint.config.js index.html package.json README.md research_logs.jsonl server/index.ts src/App.tsx src/components/AvatarApp.tsx src/components/AvatarSession.tsx src/components/ProtectedRoute.tsx src/components/Toast.tsx src/index.css src/lib/avatarConfigs.ts src/lib/avatarSystemPrompt.ts src/lib/dbQueries.ts src/lib/dualResearch.ts src/lib/firebase.ts src/lib/perplexity.ts src/lib/productResearch.ts src/lib/reelGenerationMessages.ts src/lib/scrape.ts src/lib/storeResearch.ts src/lib/videoGenerationIntegration.ts src/main.tsx src/pages/History.tsx src/pages/Landing.tsx src/pages/Login.tsx src/pages/Research.tsx src/pages/Signup.tsx src/store/global/globalSaga.ts src/store/global/globalSlice.ts src/store/rootSaga.ts src/store/store.ts src/store/storeStates.ts src/store/user/userSaga.ts src/store/user/userSlice.ts tsconfig.app.json tsconfig.json tsconfig.node.json video_generation_server/.env.example video_generation_server/.gitignore video_generation_server/agents/__init__.py video_generation_server/agents/audio_agent.py video_generation_server/agents/avatar_agent.py video_generation_server/agents/compositor_agent.py video_generation_server/agents/orchestrator.py video_generation_server/agents/research_agent.py video_generation_server/agents/script_agent.py video_generation_server/agents/video_agent_pipeline.py video_generation_server/agents/visual_agent.py video_generation_server/API_DOCS.md video_generation_server/config.py video_generation_server/context.txt video_generation_server/main.py video_generation_server/models/__init__.py video_generation_server/models/schemas.py video_generation_server/README.md video_generation_server/remotion/package.json video_generation_server/remotion/remotion.config.ts video_generation_server/remotion/render.mjs video_generation_server/remotion/src/components/AnimatedCaption.tsx video_generation_server/remotion/src/components/GradientBackground.tsx video_generation_server/remotion/src/components/index.ts video_generation_server/remotion/src/components/ProgressBar.tsx video_generation_server/remotion/src/components/TextOverlay.tsx video_generation_server/remotion/src/index.ts video_generation_server/remotion/src/ReelComposition.tsx video_generation_server/remotion/src/Root.tsx video_generation_server/remotion/src/scenes/AvatarScene.tsx video_generation_server/remotion/src/scenes/index.ts video_generation_server/remotion/src/scenes/TextOverlayScene.tsx video_generation_server/remotion/src/scenes/VisualScene.tsx video_generation_server/remotion/src/transitions/index.ts video_generation_server/remotion/src/types.ts video_generation_server/remotion/tsconfig.json video_generation_server/requirements.txt video_generation_server/templates/__init__.py video_generation_server/templates/viral_templates.py video_generation_server/utils/__init__.py video_generation_server/utils/email_sender.py video_generation_server/utils/ffmpeg_utils.py video_generation_server/utils/firebase_client.py video_generation_server/utils/heygen_client.py video_generation_server/utils/perplexity_client.py video_generation_server/utils/veo_client.py vite.config.ts ``` ### Dependencies - package.json: @browserbasehq/stagehand@^3.0.0, @eslint/js@^9.39.1, @heygen/liveavatar-web-sdk@^0.0.10, @heygen/streaming-avatar@^2.1.0, @reduxjs/toolkit@^2.11.2, @tailwindcss/vite@^4.1.18, @types/cors@^2.8.19, @types/express@^5.0.6, @types/node@^24.10.1, @types/react@^19.2.14, @types/react-dom@^19.2.3, @vitejs/plugin-react@^5.1.1, cors@^2.8.6, dotenv@^16.6.1, eslint@^9.39.1, eslint-plugin-react-hooks@^7.0.1, eslint-plugin-react-refresh@^0.4.24, express@^5.2.1, firebase@^12.9.0, framer-motion@^12.34.0, globals@^16.5.0, livekit-client@^2.17.1, lucide-react@^0.564.0, nodemon@^3.1.11, react@^19.2.0, react-dom@^19.2.0, react-markdown@^10.1.0, react-redux@^9.2.0, react-router-dom@^7.13.0, redux-saga@^1.4.2, tailwind@^4.0.0, tsx@^4.21.0, typescript@~5.9.3, typescript-eslint@^8.48.0, vite@^7.3.1, zod@^3.23.0 - video_generation_server/remotion/package.json: @remotion/bundler@^4.0.0, @remotion/cli@^4.0.0, @remotion/media@^4.0.0, @remotion/media-utils@^4.0.0, @remotion/renderer@^4.0.0, @remotion/transitions@^4.0.0, @types/react@^19.0.0, react@^19.0.0, react-dom@^19.0.0, remotion@^4.0.0, typescript@^5.7.0 - video_generation_server/requirements.txt: aiofiles, anthropic, certifi, fastapi, firebase-admin, google-genai, httpx, openai, pydantic, python-dotenv, uvicorn[standard], websockets ### Recent commits (newest first) - Updated the prompt for research - updated logo and name to Klip - New video for landing page - Debugging statements - fixed bug about avatar id - fixed bug about avatar id - added notif - Beautification of landing page - changed alignment during session - Added API call to HeyGen soon after crawling the web - updated all styling - had frontend call api to gneerate video - moved video backend from branch to main - Added Kalman Filters to avoid interuptions due to background noises - updated .env - Revert "Added Noise Filters" - Added Noise Filters - added stagehand and screenshots - Modified the prompt for a better interaction - added scraping screenshots and images ## Key source files (fetched from GitHub, selected and truncated for size) ### video_generation_server/API_DOCS.md ```markdown # Viral Reel Generator — API Documentation **Base URL:** `http://localhost:8000` --- ## Quick Start (for other teams) Your team researches a product and collects context. Then you call **one endpoint** to generate a video: ```bash # 1. Submit generation request (returns job_id immediately) curl -X POST http://localhost:8000/generate-reel-agent \ -H "Content-Type: application/json" \ -d '{ "product_name": "Claude AI", "product_url": "https://claude.ai", "product_description": "Advanced AI assistant by Anthropic", "product_features": ["Projects & document upload", "Artifacts code panel", "Custom instructions"], "target_audience": "developers and knowledge workers", "tone": "energetic", "template_type": "problem_solution", "brand_colors": ["#da7756", "#1a1a2e"], "user_images": [ "https://example.com/screenshot1.png", "https://example.com/screenshot2.png" ] }' # → {"job_id": "fc2e4ed42157", "status": "pending", "mode": "video_agent"} # 2. Poll until status is "completed" (~5-8 min) curl http://localhost:8000/reel-status/fc2e4ed42157 # → {"job_id": "fc2e4ed42157", "status": "generating", "progress": "Sending to HeyGen Video Agent..."} # 3. Download the finished video curl -o reel_output.mp4 http://localhost:8000/reel/fc2e4ed42157/download ``` --- ## Endpoints ### `POST /generate-reel-agent` — HeyGen Video Agent (Recommended) **This is the primary endpoint for other teams.** It takes your research context and product screenshots, then generates a complete marketing reel via HeyGen's Video Agent (avatar + b-roll + transitions + editing — all handled automatically). **curl command:** ```bash curl -X POST http://localhost:8000/generate-reel-agent \ -H "Content-Type: application/json" \ -d '{ "product_name": "Claude AI", "product_url": "https://claude.ai", "product_description": "Advanced AI assistant by Anthropic with projects, artifacts, and custom instructions", "product_features": [ "Projects & document upload", "Artifacts code panel", "Custom instructions" ], "target_audience": "developers and knowledge workers", "tone": "energetic", "template_type": "meme_remix", "brand_colors": ["#da7756", "#1a1a2e"], "user_images": [ "https://example.com/screenshot1.png", "https://example.com/screenshot2.png", "https://example.com/screenshot3.png" ], "key_selling_points": [ "10x faster than competitors", "Used by 1M+ developers" ], "competitor_info": "Competes with ChatGPT and Gemini" }' ``` **Field Reference:** | Field | Type | Required | Description | |-------|------|----------|-------------| | `product_name` | `string` | Yes | Name of the product | | `product_url` | `string` | Yes | Product website URL (used for automated research) | | `product_description` | `string` | Yes | 1-3 sentence description of what the product does | | `product_features` | `string[]` | Yes | List of key fe [truncated — 19372 more characters] ``` ### package.json ``` { "name": "treehacks2026_heygen", "private": true, "version": "0.0.0", "type": "module", "scripts": { "dev": "vite", "build": "tsc -b && vite build", "lint": "eslint .", "preview": "vite preview", "server": "nodemon --exec tsx server/index.ts", "dev:all": "npm run server & npm run dev" }, "dependencies": { "@browserbasehq/stagehand": "^3.0.0", "@heygen/liveavatar-web-sdk": "^0.0.10", "@heygen/streaming-avatar": "^2.1.0", "@reduxjs/toolkit": "^2.11.2", "@tailwindcss/vite": "^4.1.18", "cors": "^2.8.6", "dotenv": "^16.6.1", "express": "^5.2.1", "firebase": "^12.9.0", "framer-motion": "^12.34.0", "livekit-client": "^2.17.1", "lucide-react": "^0.564.0", "react": "^19.2.0", "react-dom": "^19.2.0", "react-markdown": "^10.1.0", "react-redux": "^9.2.0", "react-router-dom": "^7.13.0", "redux-saga": "^1.4.2", "tailwind": "^4.0.0", "zod": "^3.23.0" }, "overrides": { "p-retry": "4.6.2" }, "devDependencies": { "@eslint/js": "^9.39.1", "@types/cors": "^2.8.19", "@types/express": "^5.0.6", "@types/node": "^24.10.1", "@types/react": "^19.2.14", "@types/react-dom": "^19.2.3", "@vitejs/plugin-react": "^5.1.1", "eslint": "^9.39.1", "eslint-plugin-react-hooks": "^7.0.1", "eslint-plugin-react-refresh": "^0.4.24", "globals": "^16.5.0", "nodemon": "^3.1.11", "tsx": "^4.21.0", "typescript": "~5.9.3", "typescript-eslint": "^8.48.0", "vite": "^7.3.1" } } ``` ### video_generation_server/requirements.txt ``` fastapi uvicorn[standard] python-dotenv httpx certifi anthropic google-genai pydantic websockets aiofiles openai firebase-admin ``` ### video_generation_server/remotion/package.json ``` { "name": "viral-reel-remotion", "version": "1.0.0", "private": true, "type": "module", "scripts": { "dev": "remotion studio", "build": "remotion render MarketingReel out/reel.mp4", "render": "node render.mjs" }, "dependencies": { "remotion": "^4.0.0", "@remotion/cli": "^4.0.0", "@remotion/bundler": "^4.0.0", "@remotion/renderer": "^4.0.0", "@remotion/transitions": "^4.0.0", "@remotion/media": "^4.0.0", "@remotion/media-utils": "^4.0.0", "react": "^19.0.0", "react-dom": "^19.0.0" }, "devDependencies": { "typescript": "^5.7.0", "@types/react": "^19.0.0" } } ``` ### src/main.tsx ```typescript import React from "react"; import ReactDOM from "react-dom/client"; import { Provider } from "react-redux"; import { BrowserRouter } from "react-router-dom"; import App from "./App"; import "./index.css"; import store from "./store/store"; ReactDOM.createRoot(document.getElementById("root")!).render( ); ``` ### src/App.tsx ```typescript import { useEffect, useRef } from "react"; import { Link, Route, Routes } from "react-router-dom"; import { onAuthStateChanged } from "firebase/auth"; import { doc, onSnapshot } from "firebase/firestore"; import Login from "./pages/Login"; import Signup from "./pages/Signup"; import History from "./pages/History"; import Research from "./pages/Research"; import Landing from "./pages/Landing"; import ProtectedRoute from "./components/ProtectedRoute"; import Toast from "./components/Toast"; import { useDispatch, useSelector } from "react-redux"; import { type AppDispatch, type RootState } from "./store/store"; import { logoutRequest, listenToUserUpdatesRequest, stopListenToUserUpdatesRequest, clearUser, } from "./store/user/userSlice"; import { setAuthStatus, setHaveUserDoc } from "./store/global/globalSlice"; import { auth, db } from "./lib/firebase"; import { User } from "lucide-react"; function NavBar() { const dispatch = useDispatch(); const uid = useSelector((s: RootState) => s.user.uid); const email = useSelector((s: RootState) => s.user.email); return ( ); } export default function App() { const dispatch = useDispatch(); const docUnsubRef = useRef<(() => void) | null>(null); useEffect(() => { dispatch(setAuthStatus("checking")); const unsubAuth = onAuthStateChanged(auth, (user) => { if (user) { dispatch(setAuthStatus("authenticated")); const userDocRef = doc(db, "users", user.uid); const unsubDoc = onSnapshot(userDocRef, (docSnap) => { if (docSnap.exists()) { dispatch(listenToUserUpdatesRequest({ userId: user.uid })); unsubDoc(); docUnsubRef.current = null; } }); docUnsubRef.current = unsubDoc; } else { if (docUnsubRef.current) { docUnsubRef.current(); docUnsubRef.current = null; } dispatch(stopListenToUserUpdatesRequest()); dispatch(clearUser()); dispatch(setAuthStatus("unauthenticated")); dispatch(setHaveUserDoc(false)); } }); return () => unsubAuth(); }, [dispatch]); return ( <> } /> } /> } /> } /> } /> ); } ``` ### server/index.ts ```typescript import express from 'express'; import cors from 'cors'; import dotenv from 'dotenv'; dotenv.config(); const app = express(); const PORT = 3001; app.use(cors()); app.use(express.json()); const LIVE_AVATAR_API_KEY = process.env.VITE_LIVE_AVATAR_API_KEY; const HEYGEN_API_KEY = process.env.VITE_HEYGEN_API_KEY; const PERPLEXITY_API_KEY = process.env.VITE_PERPLEXITY_API_KEY; if (!LIVE_AVATAR_API_KEY) { console.warn('Warning: VITE_LIVE_AVATAR_API_KEY is not set in environment variables'); } if (!HEYGEN_API_KEY) { console.warn('Warning: HEYGEN_API_KEY is not set in environment variables'); } // ==================== LiveAvatar API Endpoints ==================== // Create session token endpoint app.post('/api/create-session', async (req, res) => { try { if (!LIVE_AVATAR_API_KEY) { return res.status(500).json({ error: 'VITE_LIVE_AVATAR_API_KEY is not configured on server' }); } const { avatarId, voiceId, systemPrompt, language = 'en', interactivityType = 'CONVERSATIONAL' } = req.body; if (!avatarId || !voiceId) { return res.status(400).json({ error: 'avatarId and voiceId are required' }); } const prompt = systemPrompt || 'You are a helpful AI assistant.'; // Step 1: Create a context with the system prompt console.log('Creating context with prompt:', prompt); const contextResponse = await fetch('https://api.liveavatar.com/v1/contexts', { method: 'POST', headers: { 'X-API-KEY': LIVE_AVATAR_API_KEY, 'accept': 'application/json', 'content-type': 'application/json' }, body: JSON.stringify({ name: `Session Context ${Date.now()}`, prompt: prompt, opening_text: "Hello! I'm your AI assistant. How can I help you today?", links: [] }) }); const contextData = await contextResponse.json(); if (!contextResponse.ok) { console.error('Context creation error:', contextData); return res.status(contextResponse.status).json({ error: contextData.message || 'Failed to create context' }); } const contextId = contextData.data.id; console.log('Context created with ID:', contextId); console.log('Context details:', JSON.stringify(contextData.data, null, 2)); // Step 2: Create session token with the context_id console.log('Creating session token with config:', JSON.stringify({ mode: 'FULL', avatar_id: avatarId, interactivity_type: interactivityType, avatar_persona: { voice_id: voiceId, context_id: contextId, language: language } }, null, 2)); const response = await fetch('https://api.liveavatar.com/v1/sessions/token', { method: 'POST', headers: { 'X-API-KEY': LIVE_AVATAR_API_KEY, 'accept': 'application/json', 'content-type': 'application/json' }, body: JSON.stringify({ mode: 'FULL', avatar_id: avatarId, interactivity_type: interactivityType, avatar_persona: { voice_id: voiceId, context_id: contextId, language: language } }) }); const data = await response.json(); if (!response.ok) { console.error('LiveAvatar API error:', data); return res.status(response.status).json({ error: data.message || 'Failed to create session' }); } console.log('Session token created successfully'); console.log('Session response:', JSON.stringify(data, null, 2)); res.json({ sessionId: data.data.session_id, sessionToken: data.data.session_token, contextId: contextId }); } catch (error) { console.error('Error creating session:', error); res.status(500).json({ error: 'Internal server error' }); } }); // Start session endpoint (kept for backward compatibility) app.post('/api/start-session', async (req, res) => { try { const { sessionToken } = req.body; if (!sessionToken) { return res.status(400).json({ error: 'sessionToken is required' }); } // Start the session const response = await fetch('https://api.liveavatar.com/v1/sessions/start', { method: 'POST', headers: { 'accept': 'application/json', 'authorization': `Bearer ${sessionToken}` } }); const data = await response.json(); if (!response.ok) { console.error('LiveAvatar API error:', data); return res.status(response.status).json({ error: data.message || 'Failed to start session' }); } res.json(data); } catch (error) { console.error('Error starting session:', error); res.status(500).json({ error: 'Internal server error' }); } }); // ==================== Streaming Avatar SDK Endpoints ==================== // Create access token for Streaming Avatar SDK app.post('/api/streaming/token', async (req, res) => { try { if (!HEYGEN_API_KEY) { return res.status(500).json({ error: 'HEYGEN_API_KEY is not configured on server' }); } console.log('Creating Streaming Avatar access token...'); const response = await fetch('https://api.heygen.com/v1/streaming.create_token', { method: 'POST', headers: { 'x-api-key': HEYGEN_API_KEY, 'Content-Type': 'application/json' } }); const data = await response.json(); if (!response.ok) { console.error('Streaming Avatar token error:', data); return res.status(response.status).json({ error: data.message || 'Failed to create access token' }); } console.log('Access token created successfully'); res.json({ token: data.data.token }); } catch (error) { console.error('Error creating streaming token:', error); res.status(500).json({ error: 'Internal server error' }); } }); // Create and start avatar session for Streaming SDK app.post('/api/streaming/start-avatar', async (req, res) => { try { if (!HEYGEN_API_KEY) { return res.status(500).json({ error: 'HEYGEN_API_KE [truncated — 5331 more characters] ``` ### video_generation_server/main.py ```python import asyncio import json import os import uuid from typing import Optional from dotenv import load_dotenv # Load .env from the same directory as this file so it works regardless of cwd _load_env_dir = os.path.dirname(os.path.abspath(__file__)) load_dotenv(os.path.join(_load_env_dir, ".env")) from fastapi import FastAPI, WebSocket, WebSocketDisconnect, HTTPException from fastapi.responses import FileResponse from fastapi.middleware.cors import CORSMiddleware from models.schemas import ReelGenerationRequest, ReelJob, VideoAgentRequest, AvatarVideoRequest from agents.orchestrator import ReelOrchestrator from agents.video_agent_pipeline import VideoAgentPipeline from utils.heygen_client import HeyGenClient from templates.viral_templates import get_available_templates from utils.firebase_client import ( is_firebase_configured, set_job_pending, update_job, upload_video_and_get_link, ) from utils.email_sender import is_email_configured, send_video_ready_email app = FastAPI( title="Viral Reel Generator", description="Multi-agent AI marketing reel generation pipeline", version="1.0.0", ) app.add_middleware( CORSMiddleware, allow_origins=["*"], allow_credentials=True, allow_methods=["*"], allow_headers=["*"], ) # In-memory job storage jobs: dict[str, ReelJob] = {} # WebSocket connections per job ws_connections: dict[str, list[WebSocket]] = {} @app.get("/") async def root(): return { "service": "Viral Reel Generator", "version": "2.0.0", "endpoints": { "POST /generate-reel": "Multi-agent pipeline (avatar + Remotion composition)", "POST /generate-reel-broll": "Multi-agent pipeline with Video Agent b-roll", "POST /generate-reel-agent": "One-shot Video Agent pipeline (prompt → full video)", "POST /generate-avatar": "Generate a single avatar (a-roll) video clip", "GET /reel-status/{job_id}": "Check generation status", "GET /reel/{job_id}/download": "Download finished reel", "WS /ws/reel/{job_id}": "Real-time progress stream", "GET /templates": "List available viral templates", }, } @app.get("/templates") async def list_templates(): """List all available viral reel templates.""" return {"templates": get_available_templates()} @app.post("/generate-reel") async def generate_reel(request: ReelGenerationRequest): """Start reel generation. Returns a job_id to track progress.""" job_id = uuid.uuid4().hex[:12] job = ReelJob(job_id=job_id) jobs[job_id] = job # Run generation in background asyncio.create_task(_run_generation(job_id, request)) return {"job_id": job_id, "status": "pending", "user_id": request.user_id} async def _run_generation(job_id: str, request: ReelGenerationRequest, use_broll: bool = False): """Background task that runs the full generation pipeline.""" user_id = request.user_id if is_firebase_configured(): try: await asyncio.to_thread(set_job_pending, user_id, job_id) except Exception as e: import traceback traceback.print_exc() print(f"Firebase set_job_pending failed: {e}") else: print(f"Firebase not configured, skipping set_job_pending for job_id={job_id!r}") orchestrator = ReelOrchestrator() async def progress_callback(status: str, message): """Update job and broadcast to WebSocket connections.""" job = jobs.get(job_id) if job: job.status = status job.progress = str(message) # Broadcast to WebSocket clients msg = json.dumps({"status": status, "message": str(message)}) connections = ws_connections.get(job_id, []) for ws in connections[:]: try: await ws.send_text(msg) except Exception: connections.remove(ws) try: result = await orchestrator.generate_reel(request, progress_callback, use_broll=use_broll) jobs[job_id] = result jobs[job_id].job_id = job_id if is_firebase_configured() and result.output_path and os.path.exists(result.output_path): try: link = await asyncio.to_thread( upload_video_and_get_link, user_id, job_id, result.output_path ) await asyncio.to_thread(update_job, user_id, job_id, "completed", link=link) if link and is_email_configured(): try: await asyncio.to_thread( send_video_ready_email, request.email, link ) except Exception as mail_e: print(f"Email send failed: {mail_e}") except Exception as e: import traceback traceback.print_exc() print(f"Firebase upload/update failed: {e}") await asyncio.to_thread(update_job, user_id, job_id, "completed", link=None) elif is_firebase_configured(): await asyncio.to_thread(update_job, user_id, job_id, "completed", link=None) except Exception as e: import traceback error_detail = f"{type(e).__name__}: {e}" traceback.print_exc() job = jobs.get(job_id) if job: job.status = "failed" job.error = error_detail job.progress = f"Error: {error_detail}" if is_firebase_configured(): try: await asyncio.to_thread(update_job, user_id, job_id, "failed", link=None) except Exception as fb_e: print(f"Firebase update_job failed: {fb_e}") @app.post("/generate-reel-broll") async def generate_reel_with_broll(request: ReelGenerationRequest): """Start reel generation with Video Agent b-roll for visual scenes. Same multi-agent pipeline as /generate-reel but uses HeyGen Video Agent [truncated — 9174 more characters] ``` ### video_generation_server/remotion/src/index.ts ```typescript import { registerRoot } from "remotion"; import { RemotionRoot } from "./Root"; registerRoot(RemotionRoot); ``` ### video_generation_server/remotion/src/scenes/index.ts ```typescript export { TextOverlayScene } from "./TextOverlayScene"; export { AvatarScene } from "./AvatarScene"; export { VisualScene } from "./VisualScene"; ``` [61 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]