# Project export: Salient Labs This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 12.0 - Tagline: Sharper Where it Matters - Devpost: https://devpost.com/software/salient-9ohv4q - GitHub: https://github.com/k-kochhar/CalHacks - Demo: https://www.trysalient.tech/ - Video: https://www.youtube.com/embed/bSFhoFY6hIY?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Result: winner (BaseTen: Best Use of Open Source Models) - Team: 4 GitHub contributor(s) — Taimur Shaikh (20 commits), Kshitij Kochhar (7 commits), Anuraag Pandhi (4 commits), devyani (4 commits) ## Devpost submission (written by the team) ### Inspiration SOCIAL IMPACT Internet Cost by Country 2025 Machine learning–based video compression can shrink file sizes and cut bandwidth needs by up to 50%, directly lowering streaming infrastructure costs by 30–40%. By reducing the data required to deliver high-quality video, this technology makes online education, healthcare, and communication accessible to the 2.6 billion people still offline due to costly and limited internet access. We are motivated to actively work to bridge this digital divide in order to bring connection and opportunity to millions worldwide. DEMAND Why Buffering is Every Video Providers Worst Nightmare Twitch Shuts Down in South Korea Streaming Services Cutting Bitrates to Save Money Even before COVID-19, video providers were already losing massive engagement — a mere 1% increase in buffering time translated into about 2.9 billion hours of lost viewing in a single quarter. With the global video streaming market valued at roughly US$674 billion in 2024, even a slight reduction in viewer hours or increase in cost can equate to hundreds of millions in lost revenue or extra expense. At the same time, streaming platforms are taking desperate measures — slashing bitrate, reducing quality, and squeezing compression — to cut bandwidth and delivery costs that run into the billions annually across the industry. Mounting engagement risk plus soaring delivery cost creates a compelling demand for a solution that both preserves viewer experience and lowers data usage. ### What it does Our system rethinks video compression by mirroring how the human eye perceives importance in a scene. Rather than preserving every pixel equally, it identifies the regions that naturally capture attention—faces, motion, or areas of high contrast—and keeps those sections sharp, while less noticeable regions are transmitted at lower resolution. On the viewer’s side, these regions are simply scaled back up, creating a smooth but visibly adaptive level of detail across the frame. How we're unique What makes this approach unique is that it doesn’t just compress data: it compresses perception. Traditional codecs work uniformly across the screen, but this is the first model that embraces resolution variation as part of the design, keeping the experience realistic while using a fraction of the data. The effect is subtle yet powerful: scenes look natural, but the underlying file is dramatically smaller. The method has clear potential in both pre-processed content (YouTube and Netflix), where bandwidth directly affects cost, and live video (Zoom and FaceTime), where fluctuating network speeds are prevalent. ### How we built it We designed our system to mirror how the human eye selectively processes visual information, balancing computational efficiency with perceptual realism. Neural Saliency Detection: Implemented using ViNet, a PyTorch-based video saliency prediction model trained on DHF1K, Hollywood-2, and UCF-Sports datasets for visual attention, with additional fine-tuning on DIEM, AVAD, Coutrot-1/2, SumMe, and ETMD for visual saliency cues. The model outputs a spatiotemporal saliency heatmap for each frame, predicting where human gaze is most likely to focus based on motion, contrast, and semantic context. Preprocessing Pipeline: Frames are extracted, normalized, and batched using OpenCV and NumPy, converted into tensors, and passed through ViNet in float16 precision for GPU-optimized inference. Heatmap Postprocessing: Each saliency map is normalized and discretized into percentile bins (e.g., top 20%, 50%, 80%) to classify regions by visual importance. Server-Side Compression: A GPU-accelerated OpenGL fragment shader processes the frame using the saliency mask, preserving high-saliency pixels at full resolution while adaptively downsampling lower-saliency regions. The shader executes in parallel on the GPU using GLSL texelFetch operations for direct texture access, ensuring real-time throughput. Encoding Pipeline: The mixed-resolution output frame is encoded via FFmpeg (H.264/H.265), with saliency metadata embedded as sidecar data to guide client-side scaling. Client-Side Scaling: On playback, the client reads the saliency metadata and uses a lightweight OpenGL upscaling shader to scale low-resolution areas back up to the original frame size, maintaining smooth transitions between resolution zones. ### Challenges we ran into Real-Time Performance: Achieving frame-level saliency detection and GPU compression in real time was difficult. The PyTorch model alone could process only a few frames per second initially, so we experimented with quantizing it, optimizing batch loading, and pipelining the inference with shader execution. Shader Synchronization: Getting the OpenGL shader and PyTorch inference to share memory efficiently without stalling the baseten API was challenging, and we had to pivot approaches multiple times. Compression Artifacts: Early versions of the shader created visible seams and inaccurate pixels where resolution zones met. We had to fine-tune the saliency thresholds and interpolation filters to make transitions appear smoother while keeping data savings significant. Bandwidth Variability: Simulating unstable network conditions to test adaptive thresholding was harder than expected. We tested this by changing the resolution of the source video's frames randomly. Model Generalization: ViNet’s saliency predictions worked well for cinematic and YouTube-style videos but struggled with static or low-motion footage. We experimented with temporal smoothing and custom fine-tuning to improve stability across diverse content. Integration Overhead: Coordinating the PyTorch inference server, shader renderer, and FFmpeg encoder into one containerized pipeline required careful dependency management. Despite these challenges, each bottleneck led to a better understanding of how to balance neural inference, rendering, and compression in one real-time system. ### Accomplishments we're proud of and What We Learned Despite these challenges, we’ve achieved something remarkable: Functional Prototype: We built a fully operational system that dynamically adjusts video resolution in real time based on predicted viewer attention. Perceptual Compression: Our approach maintains sharpness where it matters most, proving that compression doesn’t have to be uniform to feel natural. Massive Efficiency Gains: Testing shows file sizes can be reduced by 2×–3× while retaining strong perceptual quality, dramatically lowering bandwidth and storage costs. GPU-Accelerated Pipeline: We achieved real-time performance by fusing a PyTorch saliency model with an OpenGL shader pipeline—compressing and transmitting frames at streaming speeds. Cross-Platform Applicability: The system runs efficiently for both pre-processed video (like streaming platforms) and live feeds, adapting dynamically to network fluctuations. End-to-End Integration: We combined deep learning, GPU rendering, and adaptive encoding into a single pipeline that can slot directly into existing video delivery workflows. ### What's next Salient Labs is committed to pushing the boundaries of machine learning–driven compression through rigorous experimentation and user testing, aiming for stable 50%+ efficiency and 95%+ satisfaction. As a research-oriented team, we plan to extend our work to other bandwidth and infrastructure challenges that limit global connectivity. ## README (from the GitHub repository) No README available. ## Detected evidence (automated analysis) Indexed codebase: 35 recognized source files, 233 KB. - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - JavaScript (language) — detected in the code - MongoDB (technology) — detected in the code - Next.js (technology) — detected in the code - Python (language) — detected in the code - PyTorch (technology) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - AWS (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (55 of 55) ``` .DS_Store .gitignore backend/.gitignore backend/app.py backend/heatmap_generation/CPP_INTERFACE.md backend/heatmap_generation/model/__init__.py backend/heatmap_generation/model/checkpoints/vinet_s_mvva_randomsplit.pt backend/heatmap_generation/model/model_utils.py backend/heatmap_generation/model/Vinet_S_model.py backend/heatmap_generation/video-saliency/config.yaml backend/heatmap_generation/video-saliency/data/checkpoints/vinet_s_mvva_randomsplit.pt backend/heatmap_generation/video-saliency/model/__init__.py backend/heatmap_generation/video-saliency/model/model.py backend/heatmap_generation/video-saliency/model/shaders/foveated_render_multi.glsl backend/heatmap_generation/video-saliency/model/shaders/vertex_shader.glsl backend/heatmap_generation/video-saliency/packages.txt backend/heatmap_generation/video-saliency/packages/vinet_model/__init__.py backend/heatmap_generation/video-saliency/packages/vinet_model/model_utils.py backend/heatmap_generation/video-saliency/packages/vinet_model/Vinet_S_model.py backend/heatmap_generation/video-saliency/quick_test.py backend/heatmap_generation/video-saliency/README.md backend/heatmap_generation/video-saliency/requirements.txt backend/Rendering/FoveatedShading/config.py backend/Rendering/FoveatedShading/example_usage.py backend/Rendering/FoveatedShading/foveated_demo.py backend/Rendering/FoveatedShading/foveated_pipeline.py backend/Rendering/FoveatedShading/foveated_renderer.py backend/Rendering/FoveatedShading/shaders/foveated_render_multi.glsl backend/Rendering/FoveatedShading/shaders/vertex_shader.glsl backend/Rendering/ReconstructionPython/config.py backend/Rendering/ReconstructionPython/reconstruction_renderer.py backend/Rendering/ReconstructionPython/reconstruction_video_demo.py backend/Rendering/ReconstructionPython/shaders/reconstruction_shader.glsl backend/Rendering/ReconstructionPython/shaders/vertex_shader.glsl backend/requirements.txt frontend/.gitignore frontend/eslint.config.mjs frontend/jsconfig.json frontend/next.config.mjs frontend/package.json frontend/postcss.config.mjs frontend/README.md frontend/src/app/api/upload/route.js frontend/src/app/api/videos/[id]/complete/route.js frontend/src/app/api/videos/[id]/fail/route.js frontend/src/app/api/videos/[id]/finalize/route.js frontend/src/app/api/videos/[id]/process/route.js frontend/src/app/api/videos/[id]/route.js frontend/src/app/demo/page.js frontend/src/app/globals.css frontend/src/app/layout.js frontend/src/app/page.js frontend/src/app/team/page.js frontend/src/app/upload/page.js frontend/src/lib/mongo.js ``` ### Dependencies - backend/heatmap_generation/video-saliency/requirements.txt: boto3@>=1.28.0, moderngl@>=5.8.0, numpy@>=1.24.0, opencv-python-headless@>=4.8.0, requests@>=2.31.0, scipy@>=1.10.0, torch@>=2.0.0, torchvision@>=0.15.0, tqdm@>=4.65.0 - backend/requirements.txt: boto3@>=1.26.0, fastapi@>=0.104.0, moderngl@>=5.8.0, numpy@>=1.24.0, opencv-python-headless@>=4.8.0, python-dotenv@>=1.0.0, requests@>=2.31.0, scipy@>=1.10.0, torch@>=2.0.0, torchvision@>=0.15.0, tqdm@>=4.65.0, uvicorn[standard]@>=0.24.0 - frontend/package.json: @aws-sdk/client-s3@^3.917.0, @aws-sdk/s3-request-presigner@^3.917.0, @tailwindcss/postcss@^4, babel-plugin-react-compiler@1.0.0, eslint@^9, eslint-config-next@16.0.0, mongodb@^6.20.0, next@16.0.0, react@19.2.0, react-dom@19.2.0, tailwindcss@^4 ### Recent commits (newest first) - autoplay - Polish - fixed requirements.txt - Hook up frontend and backend - fix - Merge pull request #1 from k-kochhar/feature/foveated-rendering-integration - Merge branch 'main' into feature/foveated-rendering-integration - Add foveated step of pipeline (takes in JSON) and updated env + boto - Fixing flicker - WIP: Integrate foveated rendering into saliency model - Merge branch 'main' of https://github.com/k-kochhar/CalHacks - Updated UI - req - upload pipeline hits salience - multisource babyyyyy - merge - Add streaming of frames to baseten - Merge branch 'main' of https://github.com/k-kochhar/CalHacks - [feat] mongo + aws - Centralize requirements.txt ## Key source files (fetched from GitHub, selected and truncated for size) ### backend/heatmap_generation/CPP_INTERFACE.md ```markdown # C++ Interface Documentation ## Overview This document describes the interface between the Python saliency detection module and the C++ shader pipeline for foveated rendering. ## Data Flow ``` Python (Saliency) → JSON File → C++ (Drop Shader) → Processed Frames → C++ (Fill Shader) → Client ``` ## JSON Schema The Python module outputs focal points in the following JSON format: ### Root Structure ```json { "video_info": { "width": 1920, "height": 1080, "fps": 30.0, "total_frames": 150 }, "focal_points": [ // Array of per-frame focal points ] } ``` ### Focal Points Array Each element in the `focal_points` array represents one frame: ```json { "frame_index": 0, "points": [ { "x": 960.5, // X coordinate of focal point center (pixels) "y": 540.25, // Y coordinate of focal point center (pixels) "radius": 180.75, // Radius of high-saliency region (pixels) "intensity": 0.95 // Normalized saliency intensity [0.0, 1.0] } // Additional focal points (up to MAX_FOCAL_POINTS per frame) ] } ``` ### Field Descriptions #### video_info - `width`: Video frame width in pixels (integer) - `height`: Video frame height in pixels (integer) - `fps`: Frames per second (float) - `total_frames`: Total number of frames (integer) #### focal_points - `frame_index`: Zero-based frame index (integer) - `points`: Array of focal point objects (may be empty if no salient regions detected) #### point object - `x`: X coordinate of the focal point center in pixels (float, range: [0, width]) - `y`: Y coordinate of the focal point center in pixels (float, range: [0, height]) - `radius`: Effective radius of the salient region in pixels (float, range: [30, 300]) - `intensity`: Normalized peak saliency value (float, range: [0.0, 1.0]) ## C++ Parsing Example Here's an example using the popular `nlohmann/json` library: ```cpp #include #include #include using json = nlohmann::json; struct FocalPoint { float x; float y; float radius; float intensity; }; struct FrameFocalPoints { int frame_index; std::vector points; }; struct VideoFocalData { int width; int height; float fps; int total_frames; std::vector focal_points; }; VideoFocalData loadFocalPoints(const std::string& json_path) { // Read JSON file std::ifstream file(json_path); json j; file >> j; VideoFocalData data; // Parse video info data.width = j["video_info"]["width"]; data.height = j["video_info"]["height"]; data.fps = j["video_info"]["fps"]; data.total_frames = j["video_info"]["total_frames"]; // Parse focal points for (const auto& frame_data : j["focal_points"]) { FrameFocalPoints frame_focal; frame_focal.frame_index = frame_data["frame_index"]; for (const auto& point : frame_data["points"]) { FocalPoint fp; fp.x = point["x"]; [truncated — 6038 more characters] ``` ### backend/requirements.txt ``` torch>=2.0.0 torchvision>=0.15.0 opencv-python-headless>=4.8.0 numpy>=1.24.0 tqdm>=4.65.0 scipy>=1.10.0 requests>=2.31.0 moderngl>=5.8.0 boto3>=1.26.0 fastapi>=0.104.0 uvicorn[standard]>=0.24.0 python-dotenv>=1.0.0 ``` ### frontend/package.json ``` { "name": "frontend", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev --webpack", "build": "next build --webpack", "start": "next start", "lint": "eslint" }, "dependencies": { "@aws-sdk/client-s3": "^3.917.0", "@aws-sdk/s3-request-presigner": "^3.917.0", "mongodb": "^6.20.0", "next": "16.0.0", "react": "19.2.0", "react-dom": "19.2.0" }, "devDependencies": { "@tailwindcss/postcss": "^4", "babel-plugin-react-compiler": "1.0.0", "eslint": "^9", "eslint-config-next": "16.0.0", "tailwindcss": "^4" } } ``` ### backend/heatmap_generation/video-saliency/requirements.txt ``` torch>=2.0.0 torchvision>=0.15.0 opencv-python-headless>=4.8.0 numpy>=1.24.0 tqdm>=4.65.0 scipy>=1.10.0 requests>=2.31.0 moderngl>=5.8.0 boto3>=1.28.0 ``` ### backend/app.py ```python #!/usr/bin/env python3 """ FastAPI Orchestration Server for Video Processing Pipeline =========================================================== Pipeline flow: 1. Receive video_id from frontend 2. Construct S3 URL from video_id 3. Call Baseten saliency model to get focal points 4. Download video from S3 5. Apply foveated rendering using focal points 6. Upload processed video to S3 7. Return dropped video URL """ import os import sys import tempfile import logging import time from pathlib import Path from typing import Optional import numpy as np import cv2 import torch from fastapi import FastAPI, HTTPException from fastapi.middleware.cors import CORSMiddleware from pydantic import BaseModel import requests import boto3 from botocore.exceptions import ClientError from dotenv import load_dotenv # Add Rendering directory to path for imports sys.path.append(str(Path(__file__).parent / "Rendering" / "FoveatedShading")) from foveated_renderer import FoveatedRenderer # type: ignore from config import FOVEATED_DEFAULTS # type: ignore # Load environment variables load_dotenv() # Configure logging logging.basicConfig( level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s", datefmt="%H:%M:%S", ) logger = logging.getLogger(__name__) # Initialize FastAPI app app = FastAPI(title="Video Processing Pipeline", version="1.0.0") # CORS middleware for frontend app.add_middleware( CORSMiddleware, allow_origins=["*"], allow_credentials=True, allow_methods=["*"], allow_headers=["*"], ) # Global clients s3_client = None foveated_renderer = None # Environment variables AWS_REGION = os.getenv("AWS_REGION") AWS_S3_BUCKET = os.getenv("AWS_S3_BUCKET") AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID") AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY") BASETEN_API_URL = os.getenv("BASETEN_API_URL") BASETEN_API_KEY = os.getenv("BASETEN_API_KEY") class ProcessVideoRequest(BaseModel): video_id: str s3_url: str class ProcessVideoResponse(BaseModel): video_id: str dropped_url: str focal_points_summary: dict performance_metrics: dict @app.on_event("startup") async def startup_event(): """Initialize connections and renderer on startup""" global s3_client, foveated_renderer logger.info("Starting up FastAPI server...") # Validate environment variables required_env_vars = [ "AWS_REGION", "AWS_S3_BUCKET", "AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "BASETEN_API_URL", "BASETEN_API_KEY", ] missing = [var for var in required_env_vars if not os.getenv(var)] if missing: raise RuntimeError(f"Missing required environment variables: {missing}") # Initialize S3 client logger.info("Initializing S3 client...") s3_client = boto3.client( "s3", region_name=AWS_REGION, aws_access_key_id=AWS_ACCESS_KEY_ID, aws_secret_access_key=AWS_SECRET_ACCESS_KEY, ) logger.info("✅ S3 client initialized") # Initialize FoveatedRenderer logger.info("Initializing FoveatedRenderer...") shader_dir = Path(__file__).parent / "Rendering" / "FoveatedShading" / "shaders" frag_path = shader_dir / "foveated_render_multi.glsl" vert_path = shader_dir / "vertex_shader.glsl" if not frag_path.exists() or not vert_path.exists(): raise RuntimeError(f"Shader files not found at {shader_dir}") foveated_renderer = FoveatedRenderer( frag_path=str(frag_path), vert_path=str(vert_path), params=FOVEATED_DEFAULTS.copy(), ) # Check GPU availability if torch.cuda.is_available(): logger.info("✅ CUDA GPU available for acceleration") elif torch.backends.mps.is_available(): logger.info("✅ MPS (Apple Silicon) GPU available for acceleration") else: logger.info("⚠️ No GPU detected, using CPU") logger.info("✅ FoveatedRenderer initialized") logger.info("🚀 Server ready to process videos") @app.on_event("shutdown") async def shutdown_event(): """Clean up connections on shutdown""" logger.info("Shutting down server...") @app.get("/") async def root(): """Health check endpoint""" return {"status": "healthy", "service": "video-processing-pipeline"} @app.post("/api/process-video", response_model=ProcessVideoResponse) async def process_video(request: ProcessVideoRequest): """ Main pipeline endpoint: processes video through saliency detection and foveated rendering, returning the processed video URL. """ video_id = request.video_id s3_url = request.s3_url logger.info( f"📥 Received processing request for video_id: {video_id}, s3_url: {s3_url}" ) try: result = await process_pipeline(video_id, s3_url) logger.info(f"✅ Successfully processed video {video_id}") return result except HTTPException: raise except Exception as e: logger.exception(f"❌ Unexpected error processing video {video_id}") raise HTTPException(status_code=500, detail=f"Pipeline error: {str(e)}") async def process_pipeline(video_id: str, s3_url: str) -> ProcessVideoResponse: """ Orchestrate the full video processing pipeline. """ pipeline_start = time.time() temp_files = [] try: # Step 1: Parse S3 URL to extract bucket and key logger.info(f"Step 1: Parsing S3 URL: {s3_url}") # Extract bucket and key from S3 URL # Expected format: https://bucket-name.s3.amazonaws.com/key/path # or https://s3.amazonaws.com/bucket-name/key/path if not s3_url.startswith("https://"): raise HTTPException( status_code=400, detail=f"Invalid S3 URL format: {s3_url}. Must start with https://", ) url_parts = s3_url.replace("https://", "").split("/", 1) if len(url_parts) != 2: raise HTTPException( status_code=400, [truncated — 9374 more characters] ``` ### frontend/src/app/layout.js ```javascript import { Geist, Geist_Mono } from "next/font/google"; import "./globals.css"; const geistSans = Geist({ variable: "--font-geist-sans", subsets: ["latin"], }); const geistMono = Geist_Mono({ variable: "--font-geist-mono", subsets: ["latin"], }); export const metadata = { title: "Salient Labs", description: "Focus on What Matters", icons: { icon: "/logo.svg", }, }; export default function RootLayout({ children }) { return ( {children} ); } ``` ### frontend/src/app/page.js ```javascript 'use client'; import Link from "next/link"; import { useEffect, useState, useRef } from "react"; export default function Home() { const [scrolled, setScrolled] = useState(false); const [isOriginalPlaying, setIsOriginalPlaying] = useState(false); const [isOptimizedPlaying, setIsOptimizedPlaying] = useState(false); const [isHeroPlaying, setIsHeroPlaying] = useState(false); // Refs for video elements const heroVideoRef = useRef(null); const originalVideoRef = useRef(null); const optimizedVideoRef = useRef(null); // Track which videos have been autoplayed const [hasAutoplayedHero, setHasAutoplayedHero] = useState(false); const [hasAutoplayedOriginal, setHasAutoplayedOriginal] = useState(false); const [hasAutoplayedOptimized, setHasAutoplayedOptimized] = useState(false); useEffect(() => { const handleScroll = () => { setScrolled(window.scrollY > 100); }; window.addEventListener('scroll', handleScroll); return () => window.removeEventListener('scroll', handleScroll); }, []); // Intersection Observer for autoplay on scroll useEffect(() => { const observerOptions = { threshold: 0.3, // Trigger when 30% of video is visible rootMargin: '0px 0px -100px 0px' // Start animation slightly before video is fully visible }; const handleIntersection = (entries) => { entries.forEach((entry) => { if (entry.isIntersecting) { const videoId = entry.target.id; // Add animation class to container const container = entry.target.closest('.video-container'); if (container) { container.classList.add('animate-in'); } if (videoId === 'hero-video' && !hasAutoplayedHero) { // Small delay for smooth animation setTimeout(() => { entry.target.play().catch(err => console.log('Autoplay prevented:', err)); setIsHeroPlaying(true); setHasAutoplayedHero(true); }, 300); } else if (videoId === 'original-video' && !hasAutoplayedOriginal) { setTimeout(() => { entry.target.play().catch(err => console.log('Autoplay prevented:', err)); setIsOriginalPlaying(true); setHasAutoplayedOriginal(true); }, 300); } else if (videoId === 'optimized-video' && !hasAutoplayedOptimized) { setTimeout(() => { entry.target.play().catch(err => console.log('Autoplay prevented:', err)); setIsOptimizedPlaying(true); setHasAutoplayedOptimized(true); }, 300); } } }); }; const observer = new IntersectionObserver(handleIntersection, observerOptions); // Observe all video elements if (heroVideoRef.current) observer.observe(heroVideoRef.current); if (originalVideoRef.current) observer.observe(originalVideoRef.current); if (optimizedVideoRef.current) observer.observe(optimizedVideoRef.current); return () => { observer.disconnect(); }; }, [hasAutoplayedHero, hasAutoplayedOriginal, hasAutoplayedOptimized]); const handleHeroVideoClick = () => { const video = document.getElementById('hero-video'); if (video.paused) { video.play(); setIsHeroPlaying(true); } else { video.pause(); setIsHeroPlaying(false); } }; const handleOriginalVideoClick = () => { const video = document.getElementById('original-video'); if (video.paused) { video.play(); setIsOriginalPlaying(true); } else { video.pause(); setIsOriginalPlaying(false); } }; const handleOptimizedVideoClick = () => { const video = document.getElementById('optimized-video'); if (video.paused) { video.play(); setIsOptimizedPlaying(true); } else { video.pause(); setIsOptimizedPlaying(false); } }; return (
{/* Background Gradient */}
{/* Navigation */} {/* Hero Section */}

Sharper {" "}where
it Matte [truncated — 12792 more characters] ``` ### frontend/src/app/team/page.js ```javascript import Link from "next/link"; import Image from "next/image"; export default function TeamPage() { return (
{/* Optional subtle noise / bg blobs */}
{/* NAV */} {/* MAIN CONTENT */}
{/* Header */}

Meet the Team

We make streaming actually efficient.

{/* Team Section */}
{/* Kshitij */}
Kshitij Kochhar
Kshitij Kochhar

CS @ UMD

{/* Anuraag */}
Anuraag Pandhi
Anuraag Pandhi

Financial Engineering @ Columbia

{/* Taimur */}
Taimur Shaikh
[truncated — 3640 more characters] ``` ### frontend/src/app/upload/page.js ```javascript 'use client'; import { useState } from 'react'; import Link from 'next/link'; export default function UploadPage() { const [isDragOver, setIsDragOver] = useState(false); const [isUploading, setIsUploading] = useState(false); const [uploadProgress, setUploadProgress] = useState(0); const [uploadComplete, setUploadComplete] = useState(false); const [uploadStatus, setUploadStatus] = useState('Preparing upload...'); const handleDragOver = (e) => { e.preventDefault(); setIsDragOver(true); }; const handleDragLeave = (e) => { e.preventDefault(); setIsDragOver(false); }; const handleDrop = (e) => { e.preventDefault(); setIsDragOver(false); handleFileUpload(e.dataTransfer.files[0]); }; const handleFileSelect = (e) => { handleFileUpload(e.target.files[0]); }; const handleFileUpload = async (file) => { if (!file) return; setIsUploading(true); setUploadProgress(0); setUploadStatus('Preparing upload...'); try { // Step 1: Request presigned URL and create MongoDB record setUploadStatus('Requesting upload URL...'); setUploadProgress(10); const res = await fetch("/api/upload", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ contentType: file.type || "video/mp4" }), }); if (!res.ok) { throw new Error('Failed to get upload URL'); } const { uploadUrl, origUrl, id } = await res.json(); setUploadProgress(25); // Step 2: Upload to S3 setUploadStatus('Uploading video to cloud...'); setUploadProgress(40); const put = await fetch(uploadUrl, { method: "PUT", headers: { "Content-Type": file.type || "video/mp4" }, body: file, }); if (!put.ok) { throw new Error('Upload failed'); } setUploadProgress(65); // Step 3: Finalize upload (set status to processing) setUploadStatus('Finalizing upload...'); const finalizeRes = await fetch(`/api/videos/${id}/finalize`, { method: "POST", }); if (!finalizeRes.ok) { throw new Error('Failed to finalize upload'); } setUploadProgress(75); // Step 4: Start processing (call saliency model) - with simulated progress setUploadStatus('Analyzing video with AI model...'); // Simulate gradual progress during ML processing const progressInterval = setInterval(() => { setUploadProgress(prev => { if (prev < 95) { return prev + 1; } return prev; }); }, 300); // Increment every 300ms const processRes = await fetch(`/api/videos/${id}/process`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ origUrl: origUrl }), }); clearInterval(progressInterval); if (!processRes.ok) { throw new Error('Failed to start processing'); } setUploadStatus('Complete!'); setUploadProgress(100); setIsUploading(false); setUploadComplete(true); // Store the video data for display window.uploadedVideoUrl = origUrl; window.uploadedVideoId = id; } catch (error) { console.error('Upload error:', error); setIsUploading(false); setUploadProgress(0); setUploadStatus('Upload failed'); alert('Upload failed. Please try again.'); } }; return (
{/* Navigation */} {/* Main Content */}
{/* Header */}

Upload your video to see Salient Labs in action

Experience AI-driven video optimization in real-time

{/* Upload Card */}
{!uploadComplete ? ( <> {/* Upload Area */}
📁

Drag and drop or click to upload

{/* Navigation */}

{/* Main Content */}
{/* Header */}

Experience Salient Labs in Action

Toggle between original and optimized versions to see the difference

{/* Toggle Switch */}
{/* Video Player Area */}
{isOriginal ? ( <>