Project Info

Winner

Nitrode: Nitrode Turbo Mode Award

Live-it

We wanted to give people the power to experience any image—real or imagined—as an explorable 3D world. Whether it's a childhood photo, a city street, or a prehistoric jungle, users should be able to reimagine the setting and step into it. This idea led us to build Live-it: a pipeline that lets users modify an image via prompt, generate a cinematic video using Veo 3, and turn that into a walkable 3D scene. Live-it allows users to: Enter or upload an image, and optionally rewrite it using a creative prompt. Generate a stylized video from that image using Veo 3 (Gemini API). Reconstruct the scene into 3D using fast and high-quality Gaussian Splatting. Walk through the scene in a real-time 3D engine (Nitrode or web-based renderer). Frontend: React + TypeScript for the UI, including prompt editing, video preview, and 3D scene interaction. Backend: Node.js server that handles prompt submission, Veo 3 API calls, and routes for rendering. Veo 3 Integration: Gemini API for generating high-quality, 8-second stylized cinematic videos. 3D Reconstruction: Started with NeRFStudio but shifted to 3D Gaussian Splatting due to speed and quality. Used VGGT, a recent vision model, to infer accurate camera trajectories from Veo videos. Languages and Infra: Python, CUDA, C++, and Node.js running on a cloud instance (T4/Colab Pro/own server). Slow convergence of NeRF on T4 GPUs caused rendering lags and noisy outputs. COLMAP camera estimations were highly inaccurate on synthetic Veo videos, producing distorted splats. Integrating multiple models (Gemini → VGGT → GSP) in one unified pipeline required careful output formatting. Latency between prompt → video → 3D posed difficulty for live previewing during a 36-hour hackathon. Integrated end-to-end prompt-to-3D generation in a single user-friendly web app. Achieved real-time 3D previews using Gaussian Splatting + VGGT for fast camera estimation. Built a modular backend that can scale to new input modalities (videos, text descriptions, photos). Enabled scene-level creativity: users can style their environment before walking through it. Prompt engineering for Veo 3 affects not just visual tone, but downstream mesh quality. NeRF is powerful but impractical for hackathon-paced iteration—Gaussian Splatting wins on speed/quality tradeoff. VGGT-style direct trajectory inference significantly improves 3D fidelity on synthetic content. Splat rendering is a game-changer for demo-ready NeRF-based workflows. Add VR or headset support to explore scenes with full immersion. Support temporal continuity (multi-video, scene stitching). Implement prompt-driven re-styling after 3D generation: switch from forest to cyberpunk in real-time. Experiment with Volumetric Audio + Soundscape AI to add dynamic sound layers to the 3D world. Extend to multiplayer walkthroughs—collaboratively explore a memory or place.

LiveIt

Demo

An app for generating cinematic videos using the Veo 3 API and proceeding to 3D rendering using multiple AI 3D rendering techniques.

Features

Text Prompt Input: Enter video descriptions via textarea or upload text files
Veo 3 API Integration: Generate 8-second cinematic videos using veo-3.0-generate-preview
Real-time Loading: Beautiful spinner animations during video generation
Video Preview: In-browser MP4 video player with controls
Regeneration: Edit prompts and regenerate videos
3D Processing: Proceed to 3D rendering with finalized videos
Modern UI: Clean, responsive design with gradient backgrounds

Project Structure

live-it/
├── client/                 # React frontend
│   ├── public/
│   ├── src/
│   │   ├── App.tsx        # Main application component
│   │   ├── App.css        # Application styles
│   │   ├── index.tsx      # React entry point
│   │   └── index.css      # Global styles
│   ├── package.json
│   └── tsconfig.json
├── server/                 # Node.js backend
│   ├── index.js           # Express server
│   ├── package.json
│   └── .env               # Environment variables
├── package.json           # Root package.json
└── README.md

Setup Instructions

Prerequisites

Node.js (v16 or higher)
npm or yarn
Veo 3 API key

Installation

Clone the repository

git clone <repository-url>
cd epicvideo3d

Install root dependencies
```
npm install
```
Install server dependencies
```
cd server
npm install
cd ..
```
Install client dependencies
```
cd client
npm install
cd ..
```

Environment Configuration

cd server
cp .env.example .env

Edit server/.env with your API credentials:

PORT=5000
VEO3_API_KEY=your_actual_veo3_api_key
VEO3_API_URL=https://api.veo3.com/v1/generate-preview

Running the Application

Development Mode

Run both frontend and backend simultaneously:

npm run dev

This will start:

Frontend: http://localhost:3000
Backend: http://localhost:5000

Individual Services

Backend only:

npm run server

Frontend only:

npm run client

API Endpoints

Backend Routes

POST /api/generate-video - Generate video from text prompt
GET /api/video-status/:jobId - Check video generation status
POST /api/proceed-to-3d - Initiate 3D rendering process
GET /api/health - Health check endpoint

Usage Flow

Enter Prompt: Type a video description or upload a text file
Generate: Click "Generate Video" to call Veo 3 API
Wait: Loading spinner shows while video is being generated
Preview: Generated MP4 video appears in the player
Edit/Regenerate: Modify prompt and regenerate if needed
Proceed: Click "Proceed to 3D" to send video for 3D processing

Technology Stack

Frontend

React 18 with TypeScript
Modern CSS with gradients and animations
Responsive design for mobile/desktop
File upload handling

Backend

Node.js with Express
Axios for API calls
CORS enabled
Environment variable configuration
Error handling and logging

Environment Variables

Variable	Description	Default
`PORT`	Server port	`5000`
`VEO3_API_KEY`	Veo 3 API authentication key	Required
`VEO3_API_URL`	Veo 3 API endpoint	`https://api.veo3.com/v1/generate-preview`

##Production Deployment

Build the frontend:
```
cd client
npm run build
```
Start the server:
```
cd server
npm start
```
Configure environment variables for production

Contributing

Fork the repository
Create a feature branch
Make your changes
Test thoroughly
Submit a pull request

License

IMPORTANT NOTES

Make sure you run gh repo clone donmccurdy/three-gltf-viewer in the root node. We used this open source project for our basic preview

MIT License

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

CIn code
C++In code
CSSIn code
ExpressIn code
Google GeminiIn code
HTMLIn code
JavaScriptIn code
PythonIn code
PyTorchIn code
ReactIn code
TypeScriptIn code
Node.jsClaimed

11 of 12 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

2.7 MB

Source files

213

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.

Repository

ekagra1602/live-it

392 files · 206.7 MB · @ 01ea78e

Structure

Interface
38 files · 10%
Screens, components and styles rendered to the user.
API & routing
3 files · 1%
Request entry points: routes, handlers and controllers.
Application logic
178 files · 45%
Domain rules, services and shared utilities.
+9 more
Data & schema
2 files · 1%
Schema definitions, migrations and data access.

Supporting

Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.

Languages

JavaScript50%
Python34%
CSS4%
TypeScript4%
C++3%
C2%
Other (4)4%

Share of indexed source by file size. Binary and vendored files are excluded.

Dependencies

client/package.json

npm · 20

@react-three/drei
@react-three/fiber
@testing-library/jest-dom
@testing-library/react
@testing-library/user-event
@types/jest
@types/node
@types/react
@types/react-dom
@types/three
dat.gui
framer-motion
lucide-react
react
react-dom
react-scripts
simple-dropzone
three
+2 more

vggt/pyproject.toml

pypi · 17

einops
huggingface_hub
numpy
opencv-python
Pillow
safetensors
+11 more

gsplat/examples/requirements.txt

pypi · 15

git
imageio[ffmpeg]
matplotlib
numpy
opencv-python
Pillow
pyyaml
scikit-learn
splines
tensorboard
tensorly
torchmetrics[image]
tqdm
tyro
viser

three-gltf-viewer/package.json

npm · 8

dat.gui
gltf-validator
query-string
simple-dropzone
three
vhtml
+2 more

server/package.json

npm · 7

@google/genai
axios
cors
dotenv
express
multer
+1 more

vggt/requirements.txt

pypi · 7

einops
huggingface_hub
numpy
Pillow
safetensors
torch
torchvision

gsplat/docs/requirements.txt

pypi · 6

furo
sphinx
sphinx-copybutton
sphinx-design
sphinxcontrib-bibtex
sphinxcontrib-video

package.json

npm · 2

@google/generative-ai
+1 more

Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.