Project Info

AdSmart

Inspiration

We were inspired by the Applovin Ad Intelligence Challenge!

What it does

Our project is an app where users can upload video/image advertisements, and it returns all the important features and insights from that ad that might inform an ad recommendation engine.

How we built it

We built the backend using python and various computer vision / audio processing frameworks including OpenCV, Gemini API, FFmpeg, TensorFlow, etc. We collaborated using a streamlined Git workflow.

Challenges we ran into

The biggest challenge we ran into that probably many other hackers shared was the persistent WiFi problem. We had to relocate to an area with better connection to work.

Accomplishments we're proud of

We're really proud of the efficiency of our code! The image/video processing for 12 unique features took on average 1 minute, and the audio processing for 4 other features took on average only 30 seconds! Plus most of our feature detection was highly accurate. We're also very proud about the unique features we came up with: for images, our best feature was a measure of negative space, and for videos, we examined the number of sound peaks that were in the audio.

What we learned

We learned that it is very important to plan ahead and reroute when necessary, especially due to connection issues.

What's next

In the future, we will make the website fully functional and polish it! Furthermore we plan to examine some more difficult signals that we came up with, but didn't have enough time to implement. These include: sychronization of audio/video, overall presentation method of video using audio transcription, etc.

📊 AdSmart - Intelligent Advertisement Feature Extraction

Calhacks 12.0 Submission

AdSmart is a comprehensive web application that leverages computer vision, audio analysis, and generative AI to provide actionable insights on advertisement performance. Upload your image or video ads and receive instant, data-driven feedback on key metrics that influence audience engagement.

🎯 Problem Statement

Modern advertisers struggle to quantify what makes an ad effective. Traditional A/B testing is expensive and time-consuming. AdMetrics AI solves this by providing instant, objective analysis of ad creative elements before launch, helping marketers optimize their content for maximum impact.

📊 Signals Explained

Image Signals

Metric	Range	Description
Color Contrast	0.0 - 1.0	Measures visual distinction and readability
Product Size Ratio	0.0 - 1.0	Product prominence (0=small, 1=large)
Text Number	Int	Answers the question: How many words appear on the ad?
Negative Space Ratio	0.0 - 1.0	White space for visual breathing room
Product Name	String	Answers the question: What product is being sold?
External Buttons	List	Detected CTA text elements
Human Features	List	Detected human characteristics
Appeal	String	Explains the appeal of product

Video Signals

Metric	Range	Description
Scene Count	Integer	Number of distinct scenes/cuts
Dimensions	Float	Ratio of width/length
Product Name	String	Answers the question: What is the name of the product?
Human Features	List	Detected human characteristics
Sound Peaks	dB/10s	Number of significant peaks
Speech Coverage	0.0 - 100%	Voice-over percentage
Music Coverage	0.0 - 100%	Music-over percentage
Time Until First Significant Peak	seconds	When the first significant sound change is created (whether music or speech)

🎨 User Experience

Batch Upload Support - Analyze multiple ads simultaneously
Persistent Media Library - View and re-analyze historical uploads
Real-time Processing - Instant feedback with progress indicators
Responsive Design - Works seamlessly on desktop and mobile
Exportable Results - Download full analysis as JSON

🛠️ Tech Stack

Frontend

Streamlit - Rapid web app development with Python
Custom CSS - Polished, professional UI/UX

Computer Vision & Image Processing

OpenCV (cv2) - Advanced image and video analysis
PIL/Pillow - Image manipulation and format conversion
NumPy - Efficient numerical computations

AI & Machine Learning

Google Gemini 2.5 Flash - State-of-the-art multimodal AI for product detection and feature recognition
Pydantic - Structured data validation and schema enforcement

Audio Processing

FFmpeg - Audio extraction and conversion
Custom Audio Analysis Pipeline - Peak detection, voice analysis, and music classification

Backend

Python 3.8+ - Core application logic
python-dotenv - Secure environment variable management
Tempfile - Safe file handling and cleanup

🚀 Installation & Setup

Prerequisites

- Python 3.8 or higher
- FFmpeg (for video/audio processing)
- Google Gemini API Key

Installation Steps

Clone the Repository

git clone https://github.com/PolnareffTurtle/ad-intelligence
cd ad-intelligence

Create Virtual Environment

python -m venv .venv
source .venv/bin/activate  # On Windows: venv\Scripts\activate

Install Dependencies

pip install -r requirements.txt

Install FFmpeg
- macOS: brew install ffmpeg
- Ubuntu/Debian: sudo apt-get install ffmpeg
- Windows: Download from ffmpeg.org
Configure Environment Variables

# Create .env file
touch .env

# Add your Gemini API key (add this to the .env file)
echo "GEMINI_API_KEY=your_gemini_api_key_here" >> .env

Run the Application

streamlit run calhacks.py

Alternatives to Website

python3 main.py # this runs the image/video processing, shows run time
python3 audio.py # this runs the audio processing

The app will open automatically in your browser at http://localhost:8501

📦 Dependencies

streamlit>=1.28.0
opencv-python>=4.8.0
numpy>=1.24.0
Pillow>=10.0.0
google-generativeai>=0.3.0
pydantic>=2.0.0
python-dotenv>=1.0.0

🎮 Usage Guide

Analyzing an Image Ad

Click "📷 Upload Image Ad" on the homepage
Select one or more image files (PNG, JPG, JPEG)
Preview your upload and click "🔍 Analyze Ad(s)"
Review metrics in the analysis dashboard
Download results as JSON for reporting

Analyzing a Video Ad

Click "🎥 Upload Video Ad" on the homepage
Select a video file (MP4, MOV, AVI)
Preview your upload and click "🔍 Analyze Ad(s)"
Wait for video and audio processing (may take 30-60 seconds)
Review comprehensive metrics across visual and audio dimensions

Batch Analysis

Upload multiple images at once for comparative analysis
View timing metrics for each file processed
Access all results from the media library

Managing History

View all uploaded images via "🖼️ All Images"
View all uploaded videos via "🎬 All Videos"
Clear specific media types or entire history via "🗑️ Clear History"

🔬 Methodology

Image Analysis Pipeline

Preprocessing - Resize and normalize images for consistent analysis
Edge Detection - Canny algorithm for negative space calculation
Local Contrast - Kernel-based intensity variance measurement
AI Vision - Gemini 2.5 Flash for semantic understanding
Metric Normalization - Scale all values to [0, 1] for comparability

Video Analysis Pipeline

Frame Extraction - Sample key frames (start, end, and midpoints)
Motion Analysis - Laplacian variance for blur detection
Scene Detection - Frame differencing for transition identification
Audio Extraction - FFmpeg conversion to WAV format
Audio Analysis - Peak detection, voice isolation, music classification
Temporal Metrics - First 3-second hook and loudness progression

AI Integration

Gemini 2.5 Flash provides structured JSON responses with thinking budgets
Pydantic Models ensure type safety and schema validation
Fallback Handling for graceful degradation when AI is unavailable

🔮 Future Enhancements

Engagement Prediction Model - ML model trained on historical ad performance
Competitor Benchmarking - Compare your ads to industry standards
Platform-Specific Optimization - Instagram vs. YouTube vs. TikTok recommendations
A/B Test Suggestions - AI-generated variation recommendations
Real-time Feedback - Live metrics during ad creation
Team Collaboration - Shared workspaces and commenting
API Access - Programmatic analysis for CI/CD pipelines
Advanced Audio NLP - Sentiment and message clarity analysis

👥 Team

Calhacks 12.0 Team

Kshitij Tomar - Team Member #1
Yechan Park - Team Member #2
Theo Dela Cruz - Team Member #3

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

PythonIn code
StreamlitIn code

2 of 2 appear in the indexed code.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

81 KB

Source files

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.