Project Info
[Runpod] Best use of Flash (1st Place: 200 Runpod credits per team member (up to 4). 2nd Place: 100 Runpod credits per team member (up to 4). 3rd Place: 50 Runpod credits per team member (up to 4).)
ADapt
Inspiration
Every brand wants personalized ads, but creating unique creatives for every audience segment is expensive and slow. We watched marketing teams manually re-edit the same video dozens of times, tweaking pacing, color, and captions for different demographics. We thought: what if AI could understand both the video and the audience, and automatically generate targeted variants from a single, manually developed ad?
What it does
ADapt is an AI-powered video ad localization platform. Upload one master video and a CSV of audience profiles, and it: Analyzes your video by extracting scenes, transcribing audio, and generating per-second action descriptions Clusters your audience into meaningful segments using embeddings, visualized on an interactive 3D map Researches each segment with real-time market insights via Perplexity Sonar Generates targeted variants with segment-specific speed adjustments, color grading, text overlays, vertical reframing, and more, all driven by a constraint-aware AI planner The result: multiple production-ready ad variants from a single upload, each tailored to a specific audience.
How we built it
Backend: FastAPI with a multi-agent architecture. An Orchestrator routes requests, a Transform Planner uses GPT with a constraint-checking review loop, a Market Research Agent queries Perplexity Sonar, and a Group Ads Generator coordinates the full pipeline. Video Processing: FFmpeg handles speed changes (bounded at ±6%), 7 color grading presets, text overlays with impact-scored phrase placement, film grain, backdrop blur, and vertical reframing. Audience Intelligence: Elasticsearch embeddings for clustering with a heuristic fallback, projected into 3D via SVD for interactive visualization. Frontend: Next.js 16, React 19, Tailwind CSS 4, and TypeScript. Features a campaign dashboard, 3-step upload modal, timeline scrubber, 3D embeddings map, and variant gallery. MCP Server: Exposes video editing tools via the Model Context Protocol so external AI agents can use our platform programmatically.
Challenges we ran into
Getting the LLM to reliably choose the right video transforms. We solved this with a multi-round planner/reviewer loop that enforces explicit constraints over up to 3 revision rounds. Deciding where to place text overlays required building an impact scoring algorithm that considers keywords, punctuation, position in the video, and audio vs. visual source. Cross-platform font handling with FFmpeg's drawtext filter behaves differently across OS, so we built a fallback chain with PIL-based rendering. Making audience clustering work without Elasticsearch by building a heuristic vectorization fallback from raw profile data.
Accomplishments we're proud of
Full end-to-end automation: from a raw video and a CSV to multiple targeted ad variants with zero manual editing. A constraint system that produces thoughtful, context-aware edit decisions (moody grades for urban professionals, bright grades for teen audiences). An interactive 3D audience visualization that makes segment groupings immediately intuitive. MCP integration that makes ADapt composable with any AI agent workflow.
What we learned
Structured constraint loops beat elaborate prompting for reliable LLM output. Small bounded transforms (like ±6% speed) compound into meaningful personalization without jarring artifacts. Deterministic randomization (MD5-based stable rolls) is critical for debugging pipelines with many moving parts. Robust fallbacks (heuristic clustering, font rendering, hardware acceleration detection) aren't just safety nets, they make the product actually usable across environments.
What's next
Generative transforms: connecting stubbed-out background replacement, object erasure, and text replacement to cloud GPUs. A/B testing integration: feeding variant performance data back so ADapt learns which transforms work best per segment. Multi-language voice synthesis: re-voicing ads in different languages while preserving tone and cadence. Production scale: moving to a production database, adding job queues for parallel processing, and deploying on GPU infrastructure.
treehacks starter
Structure
frontend/Next.js (minimal UI)backend/FastAPI + OpenAI tool-calling agent + FFmpeg + action timeline
Backend setup (Python)
cd backend- Create a venv and install deps:
python -m venv venvsource venv/bin/activatepip install -r requirements.txt
- Copy env:
cp .env.example .envand setOPENAI_API_KEY - Set
JWT_SECRETin.env(for auth) - Ensure
ffmpegis installed and on your PATH - Run:
./venv/bin/python -m uvicorn app:app --reload --port 8000 --env-file .env
Endpoints:
POST /api/transform(multipart form, field namevideo)POST /api/market-research(JSON body withdescription, optionalproduct,region,goal)GET /media/original/*andGET /media/processed/*GET /media/analysis/*(JSON action timeline)POST /api/auth/registerPOST /api/auth/loginGET /api/meGET /api/videosGET /api/videos/{id}
Frontend setup (Next.js)
cd frontendnpm installnpm run dev
Optional: set NEXT_PUBLIC_API_BASE in frontend/.env (default http://localhost:8000).
Notes
- The OpenAI tool-calling agent lives in
backend/ai_agents/agent.pyand always callsspeed_up_video. - The video processing logic is isolated in
backend/ai_agents/video.py. - Action timeline extraction is in
backend/ai_agents/action_timeline.pyand uses a VLM + optional audio transcription. - If you want timestamped audio segments, set
OPENAI_ASR_MODEL=whisper-1(it supportsverbose_jsonsegments). - The backend now also generates a couple of random edit variants (combos) and returns them in
variants. - For faster video processing on macOS, set
VIDEO_HWACCEL=videotoolboxandVIDEO_ENCODER=h264_videotoolbox. - You can reduce analysis cost with
ACTION_FPSandACTION_FRAME_SCALE. - GPU-heavy generative workflows live in
backend/ai_agents/generative/(background replace, object erase, text replace) and are triggered via an agent that writes job specs. - GPU dependencies for those workflows are listed in
backend/ai_agents/generative/requirements-gpu.txt. - Text overlays require an ffmpeg build with the
drawtextfilter (libfreetype).
Analysis
View
Metric
- 12
- 12
- 11
- 10
- 3
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- CSSIn code
- FastAPIIn code
- JavaScriptIn code
- Next.jsIn code
- OpenAIIn code
- PythonIn code
- ReactIn code
- Tailwind CSSIn code
- TypeScriptIn code
9 of 9 appear in the indexed code.
AI coding agents
- CursorCommits
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
484 KB
Source files
69
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
ssiddhantsood/treehacks
125 files · 267.3 MB · @ a8469b1
Structure
Interface
19 files · 15%Screens, components and styles rendered to the user.
Application logic
33 files · 26%Domain rules, services and shared utilities.
Background jobs
7 files · 6%Work run outside a request: tasks, workers and schedules.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- Python63%
- TypeScript31%
- Markdown5%
- CSS0%
- YAML0%
- JavaScript0%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
backend/ai_agents/generative/lucy_video_to_video/requirements.txt
pypi · 95- aiodns
- aiohappyeyeballs
- aiohttp
- aiohttp-retry
- aiosignal
- annotated-doc
- annotated-types
- anyio
- attrs
- backoff
- backports-zstd
- bcrypt
- boto3
- botocore
- brotli
- certifi
- cffi
- charset-normalizer
- +77 more
frontend/package.json
npm · 12- lucide-react
- next
- react
- react-dom
- +8 more
backend/requirements.txt
pypi · 10- fastapi
- mcp[cli]
- numpy
- openai
- passlib
- Pillow
- PyJWT
- python-dotenv
- python-multipart
- uvicorn[standard]
backend/ai_agents/generative/lucy_video_to_video/pyproject.toml
pypi · 7- fastapi
- numpy
- pillow
- python-multipart
- runpod-flash
- structlog
- uvicorn
backend/ai_agents/generative/lucy_video_to_video/lucy_run/pyproject.toml
pypi · 4- fastapi
- python-dotenv
- runpod-flash
- uvicorn
backend/ai_agents/generative/lucy_video_to_video/lucy_run/requirements.txt
pypi · 4- fastapi
- python-dotenv
- runpod-flash
- uvicorn
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.