# Project export: Lumen
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: UC Berkeley AI Hackathon 2026
- Tagline: Makes marketing more accessible for small businesses
- Devpost: https://devpost.com/software/lumen-q0yw9j
- GitHub: https://github.com/imraghavojha/uc_berkely_ai_hackathon
- Video: https://player.vimeo.com/video/1203256379?byline=0&portrait=0&title=0#t=
- Team: 2 GitHub contributor(s) — RaghavOjha (8 commits), Claude Opus 4.8 (5 commits)
## Devpost submission (written by the team)
### Overview
##
### Inspiration
64% of people on free, ad-supported video actively avoid ads. The ad break literally asks you to stop caring about what you came to watch. Product placement doesn't do that — it's a $33B market — but it's locked behind studio deals, long lead times, and custom VFX. So roughly 36M U.S. small businesses are shut out. I wanted to flip the price of entry down to one product photo and one scene. ##Short description Most people skip ads. So instead of interrupting the show, Lumen drops real, brand-faithful products into it — on the right surface, with the right light and motion. One product photo plus one scene becomes a set of ready-to-stream placement cuts. All the heavy AI runs once, upfront, so playback costs the same as normal video. That's what finally makes product placement affordable for small businesses that could never get on a set. ##
### What it does
Upload a phone photo of a product and Lumen rebuilds it as a clean studio render. Upload a clip and it finds the stable, believable surfaces to place on. Then it generates a small set of temporally stable variants — the same scene with Pepsi, or Red Bull, or a tracked billboard on a skyscraper — and a custom player swaps between them mid-playback. Render once, reuse the same scene for many brands. ##How I built it Pika MCP for generation — nano-banana-pro (Gemini 3 Pro) for brand-faithful product renders, and Kling v3 omni in base-edit mode for the actual placement (source clip as base, product as reference, original audio kept). OpenCV handles shot detection and surface tracking, with a model council (Claude Opus 4.8, Gemini 3.1 Pro, GPT-5.5) ranking placements. ffmpeg stitches the swapped shots back with lip-synced audio. The front end is a self-contained Creator Studio where the product-render step runs live. ##Challenges Local FLUX on my Mac kernel-panicked and had no temporal consistency, so I scrapped it for hosted models. Hand-tracking with OpenCV homography shook badly — the fix was to stop hand-tracking and let the video model glue the placement in. Kling also faithfully reproduced a shot-cut hiding in my source clip, so I learned to trim down to one clean continuous shot (about 3 seconds). And it kept rendering cans too tall until I anchored their size to the glasses in frame. ##What I'm proud of Real, stable video placement with zero local compute: a Pepsi can holding steady on a glass table while people move and the original audio plays. A billboard tracked onto a skyscraper in a drone shot. And one scene serving four interchangeable brands — proof of the "render once, serve many" economics. ##What I learned Push the expensive intelligence upstream and playback stays cheap — that's the whole business. Don't hand-track; let the video model do the placement, and only ever feed it one clean shot. Brand fidelity and physical scale are the hard part, and prompts matter as much as the model. ##
### What's next
Self-serve targeting by geography and content niche, a marketplace where content owners list approved scenes as renewable ad space, and automated surface detection across full catalogs. ##Built with Pika MCP · nano-banana-pro (Gemini 3 Pro) · Kling v3 omni · Claude Opus 4.8 · OpenCV · ffmpeg · Python · JavaScript · TokenRouter
## README (from the GitHub repository)
# Lumen
Lumen is a hackathon concept for placing branded products naturally inside
streaming video instead of interrupting viewers with traditional ad breaks.
The runnable project lives entirely in `demo-interface/`.
## Run
```bash
/usr/bin/python3 demo-interface/serve_demo.py
```
Open:
```text
http://127.0.0.1:4173/demo-interface/
```
See `demo-interface/README.md` for the live-versus-pre-rendered boundary and
Pika authentication instructions.
## Detected evidence (automated analysis)
Indexed codebase: 15 recognized source files, 207 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- AI coding agent: Claude Code — evidence: commit authorship or trailers
## Codebase structure (from repository index)
### Files (17 of 17)
```
.gitignore
demo-interface/app.js
demo-interface/generated/.gitignore
demo-interface/index.html
demo-interface/model-council.css
demo-interface/model-council.html
demo-interface/model-council.js
demo-interface/README.md
demo-interface/serve_demo.py
demo-interface/styles.css
demo-interface/test_creator_studio.py
demo-interface/test_tokenrouter_analysis.py
demo-interface/tokenrouter_analysis.py
presentation/app.js
presentation/index.html
presentation/styles.css
README.md
```
### Dependencies
No dependency index available.
### Recent commits (newest first)
- Refine pitch: drop surrogate-ad framing, lead with small-business relevance
- Redesign UI as polished Bauhaus light theme; add TokenRouter model council
- Add campaign demo assets and enhance creator studio UI
- Streamline repository to runnable Lumen demo
- Improve scene intelligence analysis UI
- Redesign demo as a minimalist AI ad-placement platform
- Build Lumen streaming product demo
- Lumen: AI product-placement demo (Pika video swaps) + MPS FLUX port
## Key source files (fetched from GitHub, selected and truncated for size)
### presentation/app.js
```javascript
const slides = [...document.querySelectorAll(".slide")];
const notes = JSON.parse(document.querySelector("#speaker-notes").textContent);
const currentLabel = document.querySelector("[data-current]");
const totalLabel = document.querySelector("[data-total]");
const progress = document.querySelector("[data-progress]");
const speakerCard = document.querySelector("[data-speaker-card]");
const sourcesCard = document.querySelector("[data-sources-card]");
const noteNumber = document.querySelector("[data-note-number]");
const noteTitle = document.querySelector("[data-note-title]");
const noteCopy = document.querySelector("[data-note-copy]");
const noteCue = document.querySelector("[data-note-cue]");
let current = 0;
let notesOpen = false;
let sourcesOpen = false;
function clampIndex(index) {
return Math.max(0, Math.min(slides.length - 1, index));
}
function syncHash() {
history.replaceState(null, "", `#${current + 1}`);
}
function updateNotes() {
const note = notes[current] || {};
noteNumber.textContent = `Slide ${current + 1}`;
noteTitle.textContent = note.title || slides[current]?.dataset.title || "";
noteCopy.textContent = note.copy || "";
noteCue.textContent = note.cue || "";
}
function showSlide(index, options = {}) {
current = clampIndex(index);
slides.forEach((slide, slideIndex) => {
const active = slideIndex === current;
slide.classList.toggle("active", active);
slide.setAttribute("aria-hidden", String(!active));
});
currentLabel.textContent = String(current + 1).padStart(2, "0");
totalLabel.textContent = String(slides.length).padStart(2, "0");
progress.style.width = `${((current + 1) / slides.length) * 100}%`;
updateNotes();
if (notesOpen) speakerCard.focus?.();
if (!options.skipHash) syncHash();
}
function changeSlide(delta) {
showSlide(current + delta);
}
function setNotes(open) {
notesOpen = open;
speakerCard.classList.toggle("open", open);
speakerCard.setAttribute("aria-hidden", String(!open));
if (open && sourcesOpen) setSources(false);
}
function setSources(open) {
sourcesOpen = open;
sourcesCard.classList.toggle("open", open);
sourcesCard.setAttribute("aria-hidden", String(!open));
if (open && notesOpen) setNotes(false);
}
function launchDemo(relativeUrl) {
const url = new URL(relativeUrl, window.location.href);
window.open(url.href, "lumen-demo");
}
document.querySelector("[data-prev]").addEventListener("click", () => changeSlide(-1));
document.querySelector("[data-next]").addEventListener("click", () => changeSlide(1));
document.querySelector("[data-notes-toggle]").addEventListener("click", () => setNotes(!notesOpen));
document.querySelector("[data-notes-close]").addEventListener("click", () => setNotes(false));
document.querySelector("[data-sources-toggle]").addEventListener("click", () => setSources(!sourcesOpen));
document.querySelector("[data-sources-close]").addEventListener("click", () => setSources(false));
document.querySelectorAll("[data-demo]").forEach((button) => {
button.addEventListener("click", () => launchDemo(button.dataset.demo));
});
window.addEventListener("keydown", (event) => {
if (event.defaultPrevented || ["INPUT", "TEXTAREA"].includes(document.activeElement?.tagName)) return;
if (["ArrowRight", "PageDown"].includes(event.key) || event.code === "Space") {
event.preventDefault();
changeSlide(1);
return;
}
if (["ArrowLeft", "PageUp"].includes(event.key)) {
event.preventDefault();
changeSlide(-1);
return;
}
if (event.key === "Home") {
event.preventDefault();
showSlide(0);
return;
}
if (event.key === "End") {
event.preventDefault();
showSlide(slides.length - 1);
return;
}
if (event.key.toLowerCase() === "n") {
event.preventDefault();
setNotes(!notesOpen);
return;
}
if (event.key.toLowerCase() === "s") {
event.preventDefault();
setSources(!sourcesOpen);
return;
}
if (event.key === "Escape") {
setNotes(false);
setSources(false);
}
});
window.addEventListener("hashchange", () => {
const target = Number.parseInt(location.hash.slice(1), 10);
if (Number.isFinite(target)) showSlide(target - 1, { skipHash: true });
});
const initial = Number.parseInt(location.hash.slice(1), 10);
showSlide(Number.isFinite(initial) ? initial - 1 : 0, { skipHash: true });
```
### demo-interface/model-council.html
```html
Lumen — Model Council
```
### demo-interface/index.html
```html
Lumen — AI Ad Placement
```
### demo-interface/test_tokenrouter_analysis.py
```python
import json
import tempfile
import unittest
from pathlib import Path
from tokenrouter_analysis import (
DEFAULT_MODELS,
analysis_fingerprint,
parse_json_content,
read_cached_analysis,
scene_definitions,
)
REPO_ROOT = Path(__file__).resolve().parents[1]
class TokenRouterAnalysisTests(unittest.TestCase):
def test_three_scene_assets_exist(self):
scenes = scene_definitions(REPO_ROOT)
self.assertEqual(len(scenes), 3)
for scene in scenes:
self.assertTrue(scene["path"].is_file(), scene["path"])
def test_default_council_has_three_providers(self):
self.assertEqual(
[provider for _, _, provider in DEFAULT_MODELS],
["claude", "gemini", "openai"],
)
def test_parses_plain_and_fenced_json(self):
expected = {"summary": "ok", "scenes": []}
self.assertEqual(parse_json_content(json.dumps(expected)), expected)
self.assertEqual(
parse_json_content(f"```json\n{json.dumps(expected)}\n```"),
expected,
)
def test_cache_is_only_reused_for_current_fingerprint(self):
with tempfile.TemporaryDirectory() as directory:
cache = Path(directory) / "analysis.json"
cache.write_text(
json.dumps(
{
"fingerprint": analysis_fingerprint(REPO_ROOT),
"models": [{"model": "test"}],
}
),
encoding="utf-8",
)
result = read_cached_analysis(cache, REPO_ROOT)
self.assertTrue(result["cached"])
cache.write_text('{"fingerprint":"stale"}', encoding="utf-8")
self.assertIsNone(read_cached_analysis(cache, REPO_ROOT))
def test_scene_intelligence_links_to_model_council(self):
app = (REPO_ROOT / "demo-interface" / "app.js").read_text(encoding="utf-8")
council = (REPO_ROOT / "demo-interface" / "model-council.js").read_text(
encoding="utf-8"
)
self.assertIn('href="/demo-interface/model-council.html"', app)
self.assertIn("Claude Opus 4.8", council)
self.assertIn("Gemini 3.1 Pro", council)
self.assertIn("GPT-5.5", council)
self.assertNotIn("fetch(", council)
if __name__ == "__main__":
unittest.main()
```
### demo-interface/test_creator_studio.py
```python
import subprocess
import tempfile
import unittest
from pathlib import Path
from serve_demo import PRECACHED_PRODUCT_ASSETS, REPO_ROOT, parse_pika_result
class PikaResultParsingTests(unittest.TestCase):
def test_reads_strict_json_result(self):
with tempfile.TemporaryDirectory() as directory:
output = Path(directory) / "result.json"
output.write_text(
'{"asset_url":"https://cdn.pika.art/product.png","source":"pika-mcp"}',
encoding="utf-8",
)
self.assertEqual(
parse_pika_result(output, "", "")["asset_url"],
"https://cdn.pika.art/product.png",
)
def test_recovers_image_url_from_non_json_output(self):
with tempfile.TemporaryDirectory() as directory:
output = Path(directory) / "result.json"
output.write_text(
"Finished asset: https://cdn.pika.art/generated/water-bottle.png",
encoding="utf-8",
)
self.assertEqual(
parse_pika_result(output, "", "")["asset_url"],
"https://cdn.pika.art/generated/water-bottle.png",
)
def test_missing_asset_has_demo_safe_error(self):
with tempfile.TemporaryDirectory() as directory:
output = Path(directory) / "missing.json"
with self.assertRaisesRegex(RuntimeError, "demo window"):
parse_pika_result(output, "No asset produced", "")
class GreatValueCampaignTests(unittest.TestCase):
campaign_root = REPO_ROOT / "demo-interface" / "assets" / "campaign" / "great-value"
def test_precached_product_matches_demo_upload(self):
digest = "8fbf4ac70d4681576f39b807d9ee9ef7bca51c13f6832372a00afb2964d29dd6"
self.assertEqual(
PRECACHED_PRODUCT_ASSETS[digest],
"/demo-interface/assets/campaign/great-value/product-render.png",
)
self.assertTrue((REPO_ROOT / PRECACHED_PRODUCT_ASSETS[digest].lstrip("/")).is_file())
def test_campaign_media_is_present_and_playable(self):
for filename in ("bridge-clean.mp4", "bridge-billboard.mp4"):
video = self.campaign_root / filename
self.assertTrue(video.is_file())
result = subprocess.run(
[
"ffprobe",
"-v",
"error",
"-show_entries",
"format=duration",
"-of",
"default=noprint_wrappers=1:nokey=1",
str(video),
],
capture_output=True,
text=True,
check=True,
)
self.assertAlmostEqual(float(result.stdout.strip()), 10.04, delta=0.1)
def test_campaign_flow_is_wired_to_all_three_stages(self):
app = (REPO_ROOT / "demo-interface" / "app.js").read_text(encoding="utf-8")
for hook in (
"data-generate-product",
"data-run-video-analysis",
"data-generate-campaign",
"bridge-billboard.mp4",
):
self.assertIn(hook, app)
def test_creator_studio_regression_states(self):
app = (REPO_ROOT / "demo-interface" / "app.js").read_text(encoding="utf-8")
self.assertIn("Pepsi example", app)
self.assertIn("OpenCV · ORB feature tracking + homography", app)
self.assertIn("updateFacadeTracking", app)
for removed_copy in (
"Slow aerial push · solved",
"184 stable feature paths",
"34.8% usable frame area",
"Clear for full shot",
):
self.assertNotIn(removed_copy, app)
if __name__ == "__main__":
unittest.main()
```
### demo-interface/model-council.css
```css
@font-face {
font-family: "Urbanist";
src: url("/demo-interface/fonts/Urbanist.ttf") format("truetype");
font-weight: 100 900;
font-display: swap;
}
:root {
font-family: "Urbanist", sans-serif;
color: #17130d;
background: #f2efe6;
}
* { box-sizing: border-box; }
body { margin: 0; }
img { display: block; width: 100%; }
.topbar {
position: sticky;
top: 0;
z-index: 10;
height: 64px;
padding: 0 28px;
display: grid;
grid-template-columns: 1fr auto 1fr;
align-items: center;
border-bottom: 2px solid #17130d;
background: rgba(242,239,230,.92);
backdrop-filter: blur(16px);
}
.topbar a { width: max-content; color: inherit; font-size: 13px; font-weight: 750; text-decoration: none; }
.brand, .status { display: flex; align-items: center; gap: 9px; font-size: 11px; font-weight: 850; text-transform: uppercase; letter-spacing: .14em; }
.brand i { width: 14px; height: 14px; border-radius: 50%; background: conic-gradient(#e23b2b 0 33%, #f4b400 0 66%, #2456e6 0); }
.status { justify-self: end; letter-spacing: 0; text-transform: none; }
.status i { width: 7px; height: 7px; border-radius: 50%; background: #1f8f4e; box-shadow: 0 0 0 4px rgba(31,143,78,.12); }
.intro {
max-width: 1460px;
margin: 0 auto;
padding: 52px 28px 34px;
display: grid;
grid-template-columns: 1fr minmax(440px, .8fr);
gap: 70px;
align-items: end;
}
.eyebrow { color: #e23b2b; font-size: 11px; font-weight: 850; letter-spacing: .2em; text-transform: uppercase; }
h1 { margin: 14px 0; font-size: clamp(48px, 7vw, 92px); line-height: .87; letter-spacing: -.06em; }
.intro p { max-width: 650px; margin: 0; color: rgba(23,19,13,.64); line-height: 1.55; }
.intro aside { display: grid; grid-template-columns: repeat(3, 1fr); border: 1px solid rgba(23,19,13,.28); background: #fff; }
.intro aside div { padding: 18px; border-right: 1px solid rgba(23,19,13,.18); }
.intro aside div:last-child { border: 0; }
.intro aside span, .intro aside b { display: block; }
.intro aside span { margin-bottom: 7px; color: rgba(23,19,13,.48); font-size: 9px; font-weight: 850; letter-spacing: .11em; text-transform: uppercase; }
.intro aside b { font-size: 12px; }
.models { max-width: 1460px; margin: 0 auto; padding: 0 28px 44px; display: grid; gap: 24px; }
.model { --model: #2456e6; overflow: hidden; border: 2px solid #17130d; background: #fff; box-shadow: 12px 12px 0 rgba(23,19,13,.08); }
.provider-gemini { --model: #e23b2b; }
.provider-openai { --model: #1f8f4e; }
.model > header { padding: 20px; display: grid; grid-template-columns: auto 230px 1fr; gap: 16px; align-items: center; border-bottom: 1px solid rgba(23,19,13,.2); }
.model-mark { width: 48px; height: 48px; display: grid; place-items: center; border-radius: 50%; background: var(--model); color: #fff; font-size: 11px; font-weight: 900; }
.model header small { color: rgba(23,19,13,.5); font-size: 10px; }
.model h2 { margin: 2px 0 0; font-size: 24px; letter-spacing: -.03em; }
.model header p { margin: 0; color: rgba(23,19,13,.68); line-height: 1.45; }
.scene-grid { display: grid; grid-template-columns: repeat(3, 1fr); }
.scene-card { min-width: 0; border-right: 1px solid rgba(23,19,13,.18); }
.scene-card:last-child { border: 0; }
.scene-card.winner { background: color-mix(in srgb, var(--model) 6%, white); }
.scene-image { position: relative; height: 235px; overflow: hidden; background: #080808; }
.scene-image img { height: 100%; object-fit: cover; opacity: .88; }
.target { position: absolute; width: 46px; height: 46px; transform: translate(-50%,-50%); border: 1px solid #fff; border-radius: 50%; box-shadow: 0 0 0 7px rgba(255,255,255,.18); }
.target::before, .target::after { content: ""; position: absolute; left: 50%; top: 50%; background: #fff; transform: translate(-50%,-50%); }
.target::before { width: 64px; height: 1px; }
.target::after { width: 1px; height: 64px; }
.target i { position: absolute; left: 50%; top: 50%; width: 8px; height: 8px; transform: translate(-50%,-50%); border-radius: 50%; background: var(--model); }
.confidence, .pick { position: absolute; top: 13px; padding: 6px 8px; color: #fff; font-size: 9px; font-weight: 850; text-transform: uppercase; letter-spacing: .08em; }
.confidence { right: 13px; background: rgba(0,0,0,.8); }
.pick { left: 13px; background: var(--model); }
.scene-copy { padding: 18px; }
.scene-copy > span { color: var(--model); font-size: 9px; font-weight: 850; letter-spacing: .12em; text-transform: uppercase; }
.scene-copy h3 { margin: 7px 0; font-size: 20px; }
.scene-copy > p { min-height: 40px; margin: 0 0 16px; color: rgba(23,19,13,.67); font-size: 13px; line-height: 1.4; }
dl { margin: 0; display: grid; gap: 10px; }
dl div { display: grid; grid-template-columns: 38px 1fr; gap: 8px; }
dt { color: rgba(23,19,13,.45); font-size: 9px; font-weight: 850; text-transform: uppercase; }
dd { margin: 0; font-size: 12px; line-height: 1.4; }
.model > footer { padding: 15px 20px; display: flex; align-items: center; justify-content: space-between; gap: 24px; border-top: 1px solid rgba(23,19,13,.2); background: #f7f4ed; }
.model footer span { color: rgba(23,19,13,.45); font-size: 9px; font-weight: 850; text-transform: uppercase; }
.model footer b { margin-left: 8px; }
.model footer p { display: inline; margin-left: 9px; color: rgba(23,19,13,.6); font-size: 11px; }
.proof { display: grid; justify-items: end; gap: 4px; }
.proof code { color: rgba(23,19,13,.5); font-size: 9px; }
@media (max-width: 900px) {
.brand { display: none; }
.topbar { grid-template-columns: 1fr auto; }
.intro { grid-template-columns: 1fr; gap: 30px; }
.scene-grid { grid-template-columns: 1fr; }
.scene-card { border-right: 0; border-bottom: 1px solid rgba(23,19,13,.18); }
.model > header { grid-template-columns: auto 1fr; }
.model header p { grid-column: 1 / -1; }
.model > footer { align-items: flex-start; flex-direction: column; }
.proof { justify-items: start; }
}
```
### demo-interface/model-council.js
```javascript
const SCENES = {
"harvey-office": {
title: "Harvey's office",
subtitle: "Suits interview backdrop",
image: "/demo-interface/posters/suits-neutral.jpg",
},
"the-interview": {
title: "The Interview",
subtitle: "Frame 338 · hero conversation",
image: "/demo-interface/assets/scene/interview-frame-338.png",
},
"manhattan-building": {
title: "Manhattan building",
subtitle: "Bridge approach · glass façade",
image: "/demo-interface/assets/campaign/great-value/bridge-clean-poster.jpg",
},
};
const MODELS = [
{
name: "Claude Opus 4.8",
model: "anthropic/claude-opus-4.8",
provider: "claude",
requestId: "20260621160025839687793tMIPyfrE",
tokens: 8191,
summary: "The interview scene offers a purpose-built glass coffee table with existing drinkware that naturally hosts a packaged product, while the office lacks a clear foreground surface and the Manhattan aerial has no usable product support.",
winnerSceneId: "the-interview",
winnerReason: "A central glass coffee table already holds glasses and a tray, providing a stable, well-lit, story-safe surface that reads as natural product placement.",
scenes: [
{
sceneId: "harvey-office",
surface: "Soft-focus foreground furniture",
placement: "Upright on the striped sofa arm or low surface at lower left, clear of the subject.",
confidence: 32,
why: "It stays below face level and avoids the central subject while blending into the soft focus.",
risk: "No confirmed hard tabletop is visible; the sofa arm is rounded.",
x: 8, y: 82,
},
{
sceneId: "the-interview",
surface: "Glass coffee table",
placement: "Just left of the water pitcher and tray, centered between the seated figures.",
confidence: 86,
why: "Firm level contact, matching drinkware, good light, and a locked story-safe position.",
risk: "Existing glasses, pitcher, and planter create possible clutter and occlusion.",
x: 55, y: 66,
},
{
sceneId: "manhattan-building",
surface: "Lower glass façade",
placement: "Use the façade as signage rather than pretending a physical package is supported.",
confidence: 18,
why: "The façade is the only plausible branded plane in an aerial cityscape.",
risk: "It is advertising integration, not believable physical product placement.",
x: 18, y: 55,
},
],
},
{
name: "Gemini 3.1 Pro",
model: "google/gemini-3.1-pro-preview",
provider: "gemini",
requestId: "20260621160314829574400LF9LzuOm",
tokens: 5807,
summary: "The interview scene provides an ideal, contextually natural surface for a beverage, while the office lacks a foreground table and the aerial shot offers no physical support for a packaged product.",
winnerSceneId: "the-interview",
winnerReason: "A prominent, stable, contextually appropriate glass coffee table sits in the center of the action.",
scenes: [
{
sceneId: "harvey-office",
surface: "Grand piano lid",
placement: "On the closed piano lid behind the subject's left shoulder.",
confidence: 45,
why: "The piano is horizontal, stable, and fits the upscale office environment.",
risk: "The heavy background blur severely reduces brand legibility.",
x: 62, y: 52,
},
{
sceneId: "the-interview",
surface: "Glass coffee table",
placement: "Left side of the table, slightly behind the black mug and clear of the plant.",
confidence: 95,
why: "It is logical, well-lit, in focus, and visible without obstructing either character.",
risk: "Complex glass reflections and gesture occlusion must be matched carefully.",
x: 42, y: 72,
},
{
sceneId: "manhattan-building",
surface: "Glass building façade",
placement: "A large-scale graphic reflection or digital billboard on the left façade.",
confidence: 15,
why: "No product-scale support exists, so a 2D architectural integration is safest.",
risk: "A physical package would require absurd scale or appear to float.",
x: 15, y: 60,
},
],
},
{
name: "GPT-5.5",
model: "openai/gpt-5.5",
provider: "openai",
requestId: "20260621155919989306205pkq1UFjq",
tokens: 7266,
summary: "The interview scene is strongest because it has a clear coffee-table support, while the office has only soft background furniture and the Manhattan exterior lacks a believable product-scale surface.",
winnerSceneId: "the-interview",
winnerReason: "Clear stable tabletop placement with good visibility and no face interference.",
scenes: [
{
sceneId: "harvey-office",
surface: "Front-right striped sofa cushion",
placement: "A small can or bottle upright on the visible cushion near the lower-right edge.",
confidence: 42,
why: "It is a visible physical surface outside the primary face and story area.",
risk: "Soft upholstery is weaker and less stable than a hard tabletop.",
x: 91, y: 83,
},
{
sceneId: "the-interview",
surface: "Center glass coffee table",
placement: "Just left of the tray and slightly behind the black mug, with contact reflection.",
confidence: 91,
why: "Clear horizontal support and existing drinkware make the product feel native.",
risk: "Reflections and nearby table objects must be matched to avoid a pasted-on look.",
x: 45, y: 73,
},
{
sceneId: "manhattan-building",
surface: "Mid-lower glass façade",
placement: "A window-panel graphic or billboard-style treatment rather than a physical bottle.",
confidence: 28,
why: "The façade is a large planar region with consistent geometry and no faces.",
risk: "A real
[truncated — 2612 more characters]
```
### demo-interface/serve_demo.py
```python
#!/usr/bin/env python3
"""Serve the Lumen demo and bridge Creator Studio requests to Pika MCP."""
from __future__ import annotations
import base64
import hashlib
import json
import mimetypes
import os
from pathlib import Path
import random
import re
import subprocess
import tempfile
import time
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
from urllib.error import URLError
from urllib.request import Request, urlopen
from uuid import uuid4
from tokenrouter_analysis import (
get_or_run_analysis,
read_cached_analysis,
scene_manifest,
)
REPO_ROOT = Path(__file__).resolve().parents[1]
GENERATED_ROOT = REPO_ROOT / "demo-interface" / "generated"
UPLOAD_ROOT = GENERATED_ROOT / "uploads"
ASSET_ROOT = GENERATED_ROOT / "assets"
CACHE_PATH = GENERATED_ROOT / "asset-cache.json"
MODEL_ANALYSIS_CACHE_PATH = GENERATED_ROOT / "model-analysis-cache.json"
MAX_UPLOAD_BYTES = 12 * 1024 * 1024
MAX_DOWNLOAD_BYTES = 25 * 1024 * 1024
# Deliberate render-feel delay (seconds) so a cached/instant result still reads
# as a live Pika generation in the demo.
CACHE_DELAY_RANGE = (3.0, 4.0)
PRECACHED_PRODUCT_ASSETS = {
# 6957106505870228476.jpeg — Great Value water demo reference.
"8fbf4ac70d4681576f39b807d9ee9ef7bca51c13f6832372a00afb2964d29dd6":
"/demo-interface/assets/campaign/great-value/product-render.png",
}
DATA_URL_RE = re.compile(
r"^data:(image/(?:png|jpeg|webp));base64,([A-Za-z0-9+/=\s]+)$"
)
EXTENSIONS = {"image/png": ".png", "image/jpeg": ".jpg", "image/webp": ".webp"}
HTTPS_IMAGE_RE = re.compile(
r"https://[^\s\"'<>]+?\.(?:png|jpe?g|webp)(?:\?[^\s\"'<>]*)?",
re.IGNORECASE,
)
def json_bytes(payload: dict) -> bytes:
return json.dumps(payload, separators=(",", ":")).encode("utf-8")
def safe_filename(name: str, mime_type: str) -> str:
stem = re.sub(r"[^A-Za-z0-9._-]+", "-", Path(name).stem).strip("-._") or "product"
return f"{stem[:60]}-{uuid4().hex[:10]}{EXTENSIONS[mime_type]}"
def parse_image(data_url: str, filename: str) -> tuple[Path, str]:
match = DATA_URL_RE.fullmatch(data_url or "")
if not match:
raise ValueError("Upload a PNG, JPG, or WEBP product photo.")
mime_type, encoded = match.groups()
try:
image_bytes = base64.b64decode(encoded, validate=True)
except ValueError as exc:
raise ValueError("The uploaded photo is not valid image data.") from exc
if not image_bytes or len(image_bytes) > MAX_UPLOAD_BYTES:
raise ValueError("The product photo must be between 1 byte and 12 MB.")
UPLOAD_ROOT.mkdir(parents=True, exist_ok=True)
path = UPLOAD_ROOT / safe_filename(filename, mime_type)
path.write_bytes(image_bytes)
return path, hashlib.sha256(image_bytes).hexdigest()
def read_cache() -> dict:
try:
return json.loads(CACHE_PATH.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return {}
def write_cache(cache: dict) -> None:
GENERATED_ROOT.mkdir(parents=True, exist_ok=True)
CACHE_PATH.write_text(json.dumps(cache, indent=2), encoding="utf-8")
def parse_pika_result(output_path: Path, stdout: str, stderr: str) -> dict:
candidates = []
try:
candidates.append(output_path.read_text(encoding="utf-8"))
except OSError:
pass
candidates.extend([stdout or "", stderr or ""])
for text in candidates:
stripped = text.strip()
if not stripped:
continue
try:
payload = json.loads(stripped)
except json.JSONDecodeError:
payload = None
if isinstance(payload, dict):
asset_url = payload.get("asset_url") or payload.get("url")
if isinstance(asset_url, str) and asset_url.startswith("https://"):
return {"asset_url": asset_url, "source": payload.get("source", "pika-mcp")}
urls = HTTPS_IMAGE_RE.findall(stripped)
if urls:
return {"asset_url": urls[-1], "source": "pika-mcp"}
raise RuntimeError("Pika did not return a finished asset within the demo window.")
def run_pika(image_path: Path) -> dict:
schema = {
"type": "object",
"properties": {
"asset_url": {"type": "string"},
"source": {"type": "string"},
},
"required": ["asset_url", "source"],
"additionalProperties": False,
}
prompt = f"""
Create a production-ready, brand-accurate product asset from the attached
photograph. Use the Pika MCP tools, not a mock and not a local image library.
The uploaded photo is a real-world phone shot: the product may be tilted,
crinkled, dented, badly lit, or shot at an angle. Do NOT just remove the
background of that flawed photo. Instead GENERATE a clean, flawless, upright,
factory-fresh studio version of the SAME branded product, preserving the exact
brand, label text, logo, colors, and packaging shape from the reference.
Required flow:
1. Upload the exact local image at {image_path} with Pika upload_asset to get a
public reference URL.
2. Call Pika generate_image with provider "nano-banana-pro", passing that URL in
reference_images, aspect_ratio "3:4", resolution "2K". Prompt it to produce a
single, perfectly upright, undistorted, pristine studio product shot of the
same item — smooth and flawless with NO crinkles, dents, warping, or tilt —
reproducing the reference label/logo/text exactly, centered on a pure flat
white seamless background, the whole product fully in frame.
3. Call Pika remove_background on the generated image URL to get a transparent
PNG.
4. Return only the final transparent PNG URL.
Set source to "pika-mcp".
""".strip()
with tempfile.TemporaryDirectory(prefix="lumen-pika-") as temp_dir:
temp = Path(temp_dir)
schema_path = temp / "schema.json"
output_path = temp / "result.json"
schema_path.write_text(json.dumps(schema), encoding="utf-8")
command = [
"codex",
"exec",
"--e
[truncated — 6579 more characters]
```
### demo-interface/tokenrouter_analysis.py
```python
"""Cached multi-model scene analysis through TokenRouter's OpenAI-compatible API."""
from __future__ import annotations
import base64
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timezone
import hashlib
import json
import os
from pathlib import Path
import re
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
TOKENROUTER_URL = os.environ.get(
"TOKENROUTER_BASE_URL",
"https://api.tokenrouter.com/v1/chat/completions",
)
DEFAULT_MODELS = (
("Claude Opus 4.8", "anthropic/claude-opus-4.8", "claude"),
("Gemini 3.1 Pro", "google/gemini-3.1-pro-preview", "gemini"),
("GPT-5.5", "openai/gpt-5.5", "openai"),
)
PROMPT_VERSION = "lumen-scene-placement-v1"
JSON_FENCE_RE = re.compile(r"```(?:json)?\s*(\{.*\})\s*```", re.DOTALL | re.IGNORECASE)
def configured_models() -> tuple[tuple[str, str, str], ...]:
raw = os.environ.get("TOKENROUTER_MODELS", "").strip()
if not raw:
return DEFAULT_MODELS
models = []
for index, model_id in enumerate(part.strip() for part in raw.split(",")):
if not model_id:
continue
provider = model_id.split("/", 1)[0] if "/" in model_id else "model"
models.append((model_id.split("/")[-1], model_id, f"{provider}-{index}"))
return tuple(models) or DEFAULT_MODELS
def scene_definitions(repo_root: Path) -> tuple[dict, ...]:
return (
{
"id": "harvey-office",
"title": "Harvey's office",
"subtitle": "Suits interview backdrop",
"path": repo_root / "demo-interface" / "posters" / "suits-neutral.jpg",
"url": "/demo-interface/posters/suits-neutral.jpg",
"mime": "image/jpeg",
},
{
"id": "the-interview",
"title": "The Interview",
"subtitle": "Frame 338 · hero conversation",
"path": repo_root / "demo-interface" / "assets" / "scene" / "interview-frame-338.png",
"url": "/demo-interface/assets/scene/interview-frame-338.png",
"mime": "image/png",
},
{
"id": "manhattan-building",
"title": "Manhattan building",
"subtitle": "Bridge approach · glass façade",
"path": (
repo_root
/ "demo-interface"
/ "assets"
/ "campaign"
/ "great-value"
/ "bridge-clean-poster.jpg"
),
"url": "/demo-interface/assets/campaign/great-value/bridge-clean-poster.jpg",
"mime": "image/jpeg",
},
)
def scene_manifest(repo_root: Path) -> list[dict]:
return [
{
"id": scene["id"],
"title": scene["title"],
"subtitle": scene["subtitle"],
"imageUrl": scene["url"],
}
for scene in scene_definitions(repo_root)
]
def analysis_fingerprint(repo_root: Path) -> str:
digest = hashlib.sha256(PROMPT_VERSION.encode("utf-8"))
for _, model_id, _ in configured_models():
digest.update(model_id.encode("utf-8"))
for scene in scene_definitions(repo_root):
digest.update(scene["id"].encode("utf-8"))
digest.update(scene["path"].read_bytes())
return digest.hexdigest()
def data_url(scene: dict) -> str:
encoded = base64.b64encode(scene["path"].read_bytes()).decode("ascii")
return f"data:{scene['mime']};base64,{encoded}"
def analysis_prompt(scenes: tuple[dict, ...]) -> str:
scene_list = "\n".join(
f'- "{scene["id"]}": {scene["title"]} — {scene["subtitle"]}' for scene in scenes
)
return f"""
You are the placement intelligence engine for Lumen, a native in-scene product
placement system. Analyze the three attached scene images in the exact order
listed below and recommend the best believable place to insert a packaged
consumer product such as a bottle or can.
Scenes:
{scene_list}
Judge geometry, support/contact, scale, lighting, occlusion, visual salience,
face/story safety, and whether a placement could remain stable through video.
Do not identify real people. Do not propose placing products on people, in
mid-air, or over faces. If a scene lacks a strong physical support surface,
recommend the safest alternate placement area and say so plainly.
Return ONLY valid JSON in this exact shape:
{{
"summary": "one sentence comparing the three scenes",
"winner_scene_id": "one of the scene ids",
"winner_reason": "short reason",
"scenes": [
{{
"scene_id": "scene id",
"recommended_surface": "specific visible surface or region",
"placement_description": "precise location and orientation",
"confidence": 0.0,
"why_it_works": "two concise sentences",
"watch_out": "main risk",
"coordinates": {{"x": 0.0, "y": 0.0}}
}}
]
}}
confidence, x, and y must be numbers from 0 to 1. Coordinates are the visual
center of the proposed placement measured from the image's top-left.
""".strip()
def request_payload(model_id: str, repo_root: Path) -> dict:
scenes = scene_definitions(repo_root)
content = [{"type": "text", "text": analysis_prompt(scenes)}]
for scene in scenes:
content.extend(
(
{"type": "text", "text": f'IMAGE FOR SCENE "{scene["id"]}"'},
{
"type": "image_url",
"image_url": {"url": data_url(scene), "detail": "high"},
},
)
)
payload = {
"model": model_id,
"messages": [{"role": "user", "content": content}],
"max_tokens": 5000 if "gemini-3.1-pro" in model_id else 2400,
"response_format": {"type": "json_object"},
}
if "claude-opus-4.8" not in model_id:
payload["temperature"] = 0.15
return payload
def parse_json_content(content: str) -> dict:
text = (content or "").strip()
fence = JSON_FENCE_RE.fullmatch(text)
if fence:
[truncated — 7250 more characters]
```
[2 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]