# Project export: Rewrite This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: Reimagine reality. Explore any history. - Devpost: https://devpost.com/software/rewrite-2snpat - GitHub: https://github.com/NoahCodes23/TreeHacks2026 - Video: https://www.youtube.com/embed/dJSc6apgca8?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 0 GitHub contributor(s) — ## Devpost submission (written by the team) ### Inspiration We were inspired by the idea of bringing any image or idea to life as an explorable world. Whether it's stepping into a historical photograph, wandering through a painting, exploring a fantasy landscape from your imagination, or experiencing a scene from a book—we wanted to create a tool that could transform any static moment into an immersive, interactive experience. History education is one powerful use case we envision, whether in museums or classrooms, but the possibilities are endless. ### What it does If the Cuban Missile Crisis had escalated instead of de-escalated, what would the world look like today? Rewrite turns history into an explorable, living world. A teacher enters a VR environment generated in real time by diffusion models, while an LLM agent reasons about historical trends and counterfactual decisions to evolve the world dynamically. The experience is streamed live to students via Zoom, who can propose alternate timelines that directly influence what the teacher sees, with a narration agent explaining the changes as they happen. ### How we built it Rewrite is built on Hunyuan Worldplay 1.5 as the core model, which generates environments in real time from initial images or text prompts. On the frontend, a mobile phone mounted on the VR headset uses its built-in IMU sensors to track the teacher’s position and head orientation. These signals are transformed into model-interpretable movement commands, such as WASD and directional inputs. The VR experience is implemented in Unity using C#. Python handles model inference, while a TypeScript backend connects visual context to audio generation with Suno AI. The system continuously generates the next video frame, with user movements influencing the prompt stream in real time. ### Challenges we ran into Getting streaming to work across all components while maintaining performance was a major challenge. Our initial inference ran at around 1 fps even on an NVIDIA DGX Spark. To reach real-time performance, we deployed the model on Modal using H200 GPUs x4, with tensor parallelism and applied extensive low-level optimizations, including using splitting frames into chunks, FlashAttention, SageAttention, etc. In the end, we achieved the inference speed of 12 fps, which is enough for real-time streaming. We also encountered networking issues when connecting to NVIDIA DGX in the beginning, which we ultimately resolved by using Tailscale to connect machines across subnets. ### Accomplishments we're proud of We're proud of achieving real-time performance. Going from 1 fps to a 12 fps, explorable experience was a big technical challenge. Successfully implementing the streaming architecture between all our components (video generation, audio synthesis, VR rendering, and user controls) was another major accomplishment that required careful coordination and optimization. ### What we learned This project gave us deep insights into streaming video and audio at scale, and we learned a lot about running extremely compute-intensive ML models in production-like environments. We gained hands-on experience with the practical challenges of real-time generative AI. ### What's next We want to give users even more control over their worlds. Our next steps include allowing users to directly prompt the environment in real-time with custom commands and integrating VR motion tracking directly with the world controls, so head movements and hand gestures can influence exploration. The goal is to create a fully interactive platform where anyone can explore and shape the worlds they imagine. ## README (from the GitHub repository) # Elysia with Bun runtime ## Getting Started To get started with this template, simply paste this command into your terminal: ```bash bun create elysia ./elysia-example ``` ## Development To start the development server run: ```bash bun run dev ``` Open http://localhost:3000/ with your browser to see the result. ## Detected evidence (automated analysis) Indexed codebase: 7 recognized source files, 28 KB. - Anthropic (technology) — detected in the code - TypeScript (language) — detected in the code - C# (language) — claimed on Devpost, not found in the code - Python (language) — claimed on Devpost, not found in the code - PyTorch (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (13 of 13) ``` .gitignore .idea/.gitignore .idea/copilot.data.migration.ask2agent.xml .idea/inspectionProfiles/Project_Default.xml .idea/modules.xml .idea/treehacks-backend.iml .idea/vcs.xml bun.lock package.json README.md src/index.ts suno.md tsconfig.json ``` ### Dependencies - package.json: @anthropic-ai/sdk@^0.74.0, bun-types@latest, elysia@latest ### Recent commits (newest first) - Init - suno endpoints mostly done - Initial commit (via bun create) ## Key source files (fetched from GitHub, selected and truncated for size) ### suno.md ```markdown Welcome to the Suno API for TreeHacks 2026! This API allows you to programmatically generate music, covers, and stems using Suno's AI music models. ## Getting Started ### 1. Get Your API Key Please have one of your team members visit the **Suno booth at TreeHacks** to receive your team’s unique API token! This token is required for all API requests and is specifically configured for TreeHacks participants. If you encounter any issues, please message Krish Chopra on Slack. ### 1.2. 🎁 Free Suno Pro + Credits **Exclusive TreeHacks offer -** use this code for **free Suno Pro** (includes **2,500 credits**): `TREEHACKS1PRO` To redeem, sign in and go to [suno.com/account](http://suno.com/account). Choose **Monthly**, then **Pro**, and enter the above code at checkout to receive 1 month free. Perfect for experimenting with Suno's features during and after the hackathon! *Note*: You cannot have an active subscription before redeeming - you’ll only see the coupon area if you’re checking out from a non-subscribed state using the website! ### 1.5. Try the Starter App Check out our **starter application** that demonstrates the song generation and polling workflow: https://github.com/suno-ai/treehacks-starter-app This example app shows you exactly how to generate music, poll for results, and stream audio before generation is complete! ### 2. Base URL All API endpoints are accessible at: ``` https://studio-api.prod.suno.com/api/v2/external/hackathons/ ``` ### 3. Authentication All requests must include your API token in the Authorization header: ``` Authorization: Bearer YOUR_TREEHACKS_TOKEN ``` Replace `YOUR_TREEHACKS_TOKEN` with your API token (emailed to one of your team members / provided at the Suno booth). ## Rate Limits - **Music generation**: 60 songs per minute per user - **Stem separation**: 5 requests per minute per user - **Clip fetching**: 100 requests per minute per user ## Audio Format & Streaming All generated audio is provided in **MP3 format** with high-quality encoding suitable for streaming and download. ### 🎵 Real-Time Streaming **One of Suno's coolest features!** You don't have to wait for generation to complete: 1. **Submit generation** → Get back a clip ID 2. **Poll for status** (~30 seconds) → When `status` becomes `"streaming"`, `audio_url` is available 3. **Start playing immediately** → The song streams in real-time as it generates 4. **Keep listening** → Audio continues seamlessly until the full song is complete (~1-2 minutes total) Perfect for quick feedback and testing during your hackathon demo! 🚀 ## Suno API Features for TreeHacks - ✅ Music generation (simple and custom mode) - ✅ Cover generation (remake songs in different styles) - ✅ Stem separation (extract individual vocals/instruments from songs) - ✅ Real-time generation status updates --- ## API Endpoints ### 1. Generate Music Generate new music using either simple prompts or custom lyrics. **Endpoint:** ``` POST /generate ``` **Request B [truncated — 12932 more characters] ``` ### package.json ``` { "name": "treehacks-backend", "version": "1.0.50", "scripts": { "test": "echo \"Error: no test specified\" && exit 1", "dev": "bun run --watch src/index.ts" }, "dependencies": { "@anthropic-ai/sdk": "^0.74.0", "elysia": "latest" }, "devDependencies": { "bun-types": "latest" }, "module": "src/index.js" } ``` ### src/index.ts ```typescript import Elysia, { t } from "elysia"; import Anthropic from "@anthropic-ai/sdk"; const SUNO_API_BASE = "https://studio-api.prod.suno.com/api/v2/external/hackathons"; const SUNO_API_TOKEN = process.env.SUNO_API_KEY; const ANTHROPIC_API_KEY = process.env.ANTHROPIC_API_KEY; // Validate required environment variables if (!ANTHROPIC_API_KEY) { console.error("⚠️ ANTHROPIC_API_KEY is not set in environment variables"); } if (!SUNO_API_TOKEN) { console.error("⚠️ SUNO_API_KEY is not set in environment variables"); } // Initialize Anthropic Claude const anthropic = new Anthropic({ apiKey: ANTHROPIC_API_KEY, }); const CLAUDE_MODEL = "claude-sonnet-4-20250514"; console.log(`Using Claude model: ${CLAUDE_MODEL}`); // Type definitions for Suno API // Request interface for /generate endpoint interface GenerateRequest { topic?: string; tags?: string; negative_tags?: string; prompt?: string; make_instrumental?: boolean; cover_clip_id?: string; } // Response interface for /generate endpoint interface GenerateResponse { id: string; request_id: string; created_at: string; status: "submitted"; title: string; metadata: { tags?: string; prompt?: string; gpt_description_prompt?: string; type?: string; }; } // Response interface for /clips endpoint interface ClipResponse { id: string; request_id: string; created_at: string; status: "submitted" | "streaming" | "complete" | "failed"; title: string; audio_url?: string; image_url?: string; image_large_url?: string; metadata: { duration?: number; tags?: string; prompt?: string; gpt_description_prompt?: string; type?: string; }; } // Helper function to generate music async function generateMusic(request: GenerateRequest): Promise { const response = await fetch(`${SUNO_API_BASE}/generate`, { method: "POST", headers: { "Authorization": `Bearer ${SUNO_API_TOKEN}`, "Content-Type": "application/json", }, body: JSON.stringify(request), }); if (!response.ok) { console.log(`Suno API error details: ${await response.text()}`); throw new Error(`Suno API error: ${response.status} ${response.statusText}`); } return response.json(); } // Helper function to get clip status async function getClipStatus(clipId: string): Promise { const response = await fetch(`${SUNO_API_BASE}/clips?ids=${clipId}`, { method: "GET", headers: { "Authorization": `Bearer ${SUNO_API_TOKEN}`, }, }); if (!response.ok) { throw new Error(`Suno API error: ${response.status} ${response.statusText}`); } return response.json(); } // Helper function to poll until streaming or complete async function pollForAudio(clipId: string, maxAttempts = 60, intervalMs = 2000): Promise { for (let i = 0; i < maxAttempts; i++) { const clips = await getClipStatus(clipId); const clip = clips[0]; if (clip.status === "streaming" || clip.status === "complete") { return clip; } if (clip.status === "failed") { throw new Error("Music generation failed"); } await new Promise(resolve => setTimeout(resolve, intervalMs)); } throw new Error("Timeout waiting for music generation"); } // Helper function to truncate tags to fit within maxLength characters // Removes entire tags (not partial) to stay under the limit function truncateTags(tags: string, maxLength: number = 100): string { if (tags.length <= maxLength) { return tags; } const tagArray = tags.split(" "); let result = ""; for (const tag of tagArray) { const newResult = result ? `${result} ${tag}` : tag; if (newResult.length <= maxLength) { result = newResult; } else { break; } } return result; } // Helper function to analyze image and generate music tags using Claude async function analyzeImageForMusicTags(imageData: string, mimeType: string): Promise { const prompt = `Analyze this image and generate space-separated tags for music that would fit the vibe, environment, or setting of this image. The tags should describe musical genres, instruments, moods, and atmospheres that match the image. Output ONLY the tags, nothing else. No explanations, no punctuation except spaces between tags. Keep tags concise and relevant for an AI music generator. Example output: ambient peaceful piano nature serene calm atmospheric Generate tags for this image:`; const response = await anthropic.messages.create({ model: CLAUDE_MODEL, max_tokens: 200, messages: [ { role: "user", content: [ { type: "image", source: { type: "base64", media_type: mimeType as "image/jpeg" | "image/png" | "image/gif" | "image/webp", data: imageData, }, }, { type: "text", text: prompt, }, ], }, ], }); // Extract text from response const textContent = response.content.find(block => block.type === "text"); const tags = textContent && textContent.type === "text" ? textContent.text.trim().toLowerCase() : ""; // Truncate to 100 characters, removing whole tags return truncateTags(tags, 100); } const app = new Elysia() .get("/", () => "Hello Elysia") // Generate music endpoint - simple mode .post("/generate", async ({ body }) => { return await generateMusic(body); }, { body: t.Object({ topi [truncated — 5207 more characters] ``` ### .idea/vcs.xml ```xml ``` ### .idea/copilot.data.migration.ask2agent.xml ```xml ``` ### .idea/modules.xml ```xml ``` ### .idea/inspectionProfiles/Project_Default.xml ```xml ```