# Project export: xxx This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2025 - Tagline: xxx - Devpost: https://devpost.com/software/xxx-lrki7u - GitHub: https://github.com/zcsabbagh/zu-lm - Team: 3 GitHub contributor(s) — zcsabbag (31 commits), uche-afk (28 commits), Emily Zhang (10 commits) ## Devpost submission (written by the team) ### Overview WELCOME TO ZU-LM A brave new world... 4-D, Interactive Voice Podcasting Edge Computing Retrieval Augmented Generation ### Inspiration What It Does How We Built It Challenges We Faced Accomplishments We're Proud Of What We Learned ## README (from the GitHub repository) # ZU-LM A research-driven podcast generator that creates engaging conversations about any topic. ## Environment Setup This project requires several API keys to function properly. Follow these steps to set up your environment: 1. Copy the example environment files: ```bash # Root directory cp .env.example .env # zu-chat directory cd zu-chat cp .env.example .env.local # researcher directory cd ../researcher cp .env.example .env ``` 2. Obtain the required API keys: - [ElevenLabs](https://elevenlabs.io/) - For text-to-speech - [Groq](https://groq.com/) - For LLM inference - [OpenAI](https://openai.com/) - For AI capabilities - [Luma AI](https://lumalabs.ai/) - For image generation - [Perplexity](https://www.perplexity.ai/) - For search capabilities 3. Fill in your API keys in the respective .env files ## Required API Keys The following API keys are required for full functionality: - `ELEVENLABS_API_KEY` - For text-to-speech generation - `GROQ_API_KEY` - For LLM inference - `OPENAI_API_KEY` - For AI capabilities - `LUMAAI_API_KEY` - For image generation - `PERPLEXITY_API_KEY` - For search functionality ## Optional Configuration Some features can be configured through environment variables: - `LOCAL_LLM` - Specify which local LLM to use (default: "deepseek-r1:8b") - `MAX_WEB_RESEARCH_LOOPS` - Control research depth (default: 1) - `SEARCH_API` - Search provider to use (default: "perplexity") ## Run the project To run the frontend server: ``` cd zu-chat npm install npm run dev ``` To run the Rust backend: ``` cd researcher cargo run ``` Enjoy! ## Detected evidence (automated analysis) Indexed codebase: 63 recognized source files, 411 KB. - CSS (language) — detected in the code - JavaScript (language) — detected in the code - LangChain (technology) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Rust (language) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - Vercel AI SDK (technology) — detected in the code - Mistral AI (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (80 of 80) ``` .env.example .gitattributes .gitignore .gitmodules agent_animation.py Cargo.toml package.json README.md researcher/.env.example researcher/.gitignore researcher/Cargo.toml researcher/langgraph.json researcher/src/assistant/configuration.rs researcher/src/assistant/debate.rs researcher/src/assistant/graph.rs researcher/src/assistant/groq.rs researcher/src/assistant/mod.rs researcher/src/assistant/prompts.rs researcher/src/assistant/state.rs researcher/src/assistant/utils.rs researcher/src/lib.rs researcher/src/main.rs researcher/src/server.rs src/demo.js src/index.js src/lib.rs src/luma.js src/main.rs src/schemas.js src/stream.js src/text.js src/voice.js tsconfig.json zu-chat/.env.example zu-chat/.gitignore zu-chat/components.json zu-chat/eslint.config.mjs zu-chat/log.txt zu-chat/next.config.ts zu-chat/package.json zu-chat/postcss.config.mjs zu-chat/README.md zu-chat/src/app/api/enrich/route.ts zu-chat/src/app/api/generate/images/route.ts zu-chat/src/app/api/generate/route.ts zu-chat/src/app/api/research/config/route.ts zu-chat/src/app/api/research/route.ts zu-chat/src/app/api/research/status/route.ts zu-chat/src/app/globals.css zu-chat/src/app/layout.tsx zu-chat/src/app/page.tsx zu-chat/src/app/research/App.css zu-chat/src/app/research/page.tsx zu-chat/src/app/test/page.tsx zu-chat/src/components/Chat.tsx zu-chat/src/components/podcast/EnrichmentPopup.tsx zu-chat/src/components/podcast/PodcastControls.tsx zu-chat/src/components/podcast/PodcastNavigation.tsx zu-chat/src/components/podcast/PodcastPlayer.tsx zu-chat/src/components/podcast/PodcastSegment.tsx zu-chat/src/components/podcast/PodcastTest.tsx zu-chat/src/components/research/ResearchHeader.tsx zu-chat/src/components/research/ResearchPerspectives.tsx zu-chat/src/components/ResearchFlow.tsx zu-chat/src/components/ResearchInputNode.tsx zu-chat/src/components/ui/button.tsx zu-chat/src/components/ui/card.tsx zu-chat/src/components/ui/hover-card.tsx zu-chat/src/components/ui/input.tsx zu-chat/src/components/ui/label.tsx zu-chat/src/components/ui/switch.tsx zu-chat/src/components/ui/tabs.tsx zu-chat/src/lib/constants.ts zu-chat/src/lib/luma.ts zu-chat/src/lib/schemas.ts zu-chat/src/lib/text.ts zu-chat/src/lib/utils.ts zu-chat/src/lib/voice.ts zu-chat/tailwind.config.ts zu-chat/tsconfig.json ``` ### Dependencies - package.json: @ai-sdk/groq@latest, @ai-sdk/openai@^1.1.11, ai@^4.1.40, dotenv@latest, elevenlabs@^1.51.0, lumaai@^1.4.0, node-fetch@^3.3.2, tailwindcss-animate@^1.0.7 - researcher/Cargo.toml: anyhow@1.0, async-stream@0.3, async-trait@0.1, axum@0.7, dotenv@0.15, enum-as-inner@0.6, futures@0.3, http@1.0, langchain@0.2.2, ollama-rs@0.1, reqwest@0.11, serde@1.0, serde_json@1.0, thiserror@1.0, tokio@1.0, tokio-stream@0.1, tower-http@0.5, tracing@0.1, url@2.5 - zu-chat/package.json: @11labs/react@^0.0.7, @ai-sdk/groq@^1.1.9, @ai-sdk/openai@^1.1.11, @eslint/eslintrc@^3, @radix-ui/react-hover-card@^1.1.6, @radix-ui/react-label@^2.1.2, @radix-ui/react-slot@^1.1.2, @radix-ui/react-switch@^1.1.3, @radix-ui/react-tabs@^1.1.3, @types/d3@^7.4.3, @types/dagre@^0.7.52, @types/node@^20, @types/react@^19, @types/react-dom@^19, @xyflow/react@^12.4.3, ai@^4.1.40, class-variance-authority@^0.7.1, clsx@^2.1.1, crypto@^1.0.1, d3@^7.9.0, dagre@^0.8.5, elevenlabs@^1.51.0, eslint@^9, eslint-config-next@15.1.7, lucide-react@^0.475.0, lumaai@^1.4.0, next@15.1.7, postcss@^8, react@^19.0.0, react-d3-tree@^3.6.5, react-dom@^19.0.0, react-markdown@^9.0.3, tailwind-merge@^3.0.1, tailwindcss@^3.4.1, tailwindcss-animate@^1.0.7, typescript@^5, zod@^3.24.2 ### Recent commits (newest first) - Update README.md - Update README.md - chore: update gitignore and remove ignored files - minor - edit readme - update submodule name again - rename submodule - added exo submodule - final - fix - added feynman review - ALMOST DONE - merge - changes - finished integrating enrichment with main - Merge branch 'main' of https://github.com/zcsabbagh/zu-lm - enrich mode now working - new node - added voice agent to /test route - Merge branch 'main' of https://github.com/zcsabbagh/zu-lm ## Key source files (fetched from GitHub, selected and truncated for size) ### Cargo.toml ``` [workspace] members = ["researcher"] resolver = "2" ``` ### package.json ``` { "name": "zu-lm", "type": "module", "scripts": { "start": "ts-node src/index.ts", "build": "tsc", "dev": "ts-node src/index.ts" }, "dependencies": { "@ai-sdk/groq": "latest", "@ai-sdk/openai": "^1.1.11", "ai": "^4.1.40", "dotenv": "latest", "elevenlabs": "^1.51.0", "lumaai": "^1.4.0", "node-fetch": "^3.3.2", "tailwindcss-animate": "^1.0.7" } } ``` ### researcher/Cargo.toml ``` [package] name = "researcher" version = "0.1.0" edition = "2021" [[bin]] name = "researcher" path = "src/main.rs" [dependencies] serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" langchain = "0.2.2" tokio = { version = "1.0", features = ["full", "net"] } async-trait = "0.1" anyhow = "1.0" dotenv = "0.15" reqwest = { version = "0.11", features = ["json"] } thiserror = "1.0" tracing = "0.1" url = "2.5" enum-as-inner = "0.6" ollama-rs = "0.1" # For Ollama integration axum = { version = "0.7", features = ["json", "macros"] } tower-http = { version = "0.5", features = ["cors", "fs"] } futures = "0.3" tokio-stream = "0.1" async-stream = "0.3" http = "1.0" ``` ### zu-chat/package.json ``` { "name": "zu-chat", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev --turbopack", "build": "next build", "start": "next start", "lint": "next lint" }, "dependencies": { "@11labs/react": "^0.0.7", "@ai-sdk/groq": "^1.1.9", "@ai-sdk/openai": "^1.1.11", "@radix-ui/react-hover-card": "^1.1.6", "@radix-ui/react-label": "^2.1.2", "@radix-ui/react-slot": "^1.1.2", "@radix-ui/react-switch": "^1.1.3", "@radix-ui/react-tabs": "^1.1.3", "@types/d3": "^7.4.3", "@types/dagre": "^0.7.52", "@xyflow/react": "^12.4.3", "ai": "^4.1.40", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "crypto": "^1.0.1", "d3": "^7.9.0", "dagre": "^0.8.5", "elevenlabs": "^1.51.0", "lucide-react": "^0.475.0", "lumaai": "^1.4.0", "next": "15.1.7", "react": "^19.0.0", "react-d3-tree": "^3.6.5", "react-dom": "^19.0.0", "react-markdown": "^9.0.3", "tailwind-merge": "^3.0.1", "tailwindcss-animate": "^1.0.7", "zod": "^3.24.2" }, "devDependencies": { "@eslint/eslintrc": "^3", "@types/node": "^20", "@types/react": "^19", "@types/react-dom": "^19", "eslint": "^9", "eslint-config-next": "15.1.7", "postcss": "^8", "tailwindcss": "^3.4.1", "typescript": "^5" } } ``` ### src/index.js ```javascript import { createText, MODELS as TEXT_MODELS } from './text.js'; import { createTextStream, MODELS as STREAM_MODELS } from './stream.js'; import { textToSpeech, playPodcastTranscript } from './voice.js'; export { // Text generation createText, createTextStream, TEXT_MODELS, STREAM_MODELS, // Voice synthesis textToSpeech, playPodcastTranscript }; ``` ### src/main.rs ```rust use researcher::{init, Configuration}; #[tokio::main] async fn main() { println!("Starting application..."); // Initialize environment variables from .env init(); println!("Initialized environment variables"); // Now try to create the configuration match Configuration::from_runnable_config(None) { Ok(config) => { println!("Successfully loaded configuration"); // Your research logic here } Err(e) => { eprintln!("Configuration error: {}", e); } } } ``` ### researcher/src/main.rs ```rust use researcher::{ init, assistant::configuration::Configuration, server::run_server, }; #[tokio::main] async fn main() { println!("Starting application..."); // Initialize environment variables from .env init(); println!("Initialized environment variables"); // Now try to create the configuration match Configuration::from_runnable_config(None) { Ok(config) => { println!("Successfully loaded configuration"); // Run the server run_server(config).await; } Err(e) => { eprintln!("Configuration error: {}", e); } } } ``` ### researcher/src/server.rs ```rust use axum::{ routing::{post, get, put}, Router, Json, extract::State, http::StatusCode, response::{IntoResponse, Response, sse::{Event, Sse}}, }; use std::{ sync::Arc, net::SocketAddr, convert::Infallible, time::{Duration, SystemTime, UNIX_EPOCH}, env, sync::OnceLock, }; use crate::assistant::{ configuration::Configuration, state::{SummaryStateInput, StatusUpdate}, graph::ResearchGraph, configuration::ResearchMode, }; use tower_http::cors::CorsLayer; use futures::stream::Stream; use tokio::sync::broadcast; use serde_json::json; use serde::Deserialize; use tokio::sync::Mutex; use http::{Method, header}; use http::header::HeaderValue; // Increase channel capacity const CHANNEL_CAPACITY: usize = 100; static STATUS_CHANNEL: OnceLock> = OnceLock::new(); fn get_status_channel() -> broadcast::Sender { STATUS_CHANNEL.get_or_init(|| { let (tx, _) = broadcast::channel(CHANNEL_CAPACITY); tx }).clone() } #[derive(Clone)] pub struct AppState { graph: Arc>, status_tx: broadcast::Sender, } #[derive(Deserialize)] struct ConfigUpdate { local_llm: Option, max_web_research_loops: Option, research_mode: Option, } #[derive(serde::Deserialize)] pub struct ResearchRequest { topic: String, } #[derive(serde::Serialize)] struct ResearchResponse { summary: String, status: String, } // Custom error type for our API struct ApiError(anyhow::Error); impl IntoResponse for ApiError { fn into_response(self) -> Response { ( StatusCode::INTERNAL_SERVER_ERROR, Json(ResearchResponse { summary: format!("Error: {}", self.0), status: "Error occurred".to_string(), }) ).into_response() } } // Convert anyhow::Error into our ApiError impl From for ApiError { fn from(err: anyhow::Error) -> Self { ApiError(err) } } // Add this new struct for the config response #[derive(serde::Serialize)] struct ConfigResponse { local_llm: String, max_web_research_loops: i32, research_mode: ResearchMode, groq_model: String, } pub async fn run_server(config: Configuration) { let status_tx = get_status_channel(); let mut graph = ResearchGraph::new(config); graph.set_status_sender(status_tx.clone()); let state = Arc::new(AppState { graph: Arc::new(Mutex::new(graph)), status_tx, }); let frontend_origin = env::var("FRONTEND_URL") .unwrap_or_else(|_| "http://localhost:3000".to_string()); let cors = CorsLayer::new() .allow_methods([Method::GET, Method::POST, Method::PUT]) .allow_headers([ header::CONTENT_TYPE, header::ACCEPT, header::CACHE_CONTROL, header::CONNECTION, ]) .allow_credentials(true) .allow_origin(frontend_origin.parse::().unwrap()); let app = Router::new() .route("/research", post(handle_research)) .route("/config", put(update_config)) .route("/config", get(get_config)) .route("/status", get(status_stream)) .layer(cors) .with_state(state); let port = env::var("PORT").unwrap_or_else(|_| "4000".to_string()).parse::().unwrap_or(4000); let addr = SocketAddr::from(([0, 0, 0, 0], port)); println!("Starting server on http://localhost:{}", port); println!("Allowing CORS for origin: {}", frontend_origin); axum::serve( tokio::net::TcpListener::bind(&addr).await.unwrap(), app.into_make_service(), ) .await .unwrap(); } async fn handle_research( State(state): State>, Json(request): Json, ) -> Response { let input = SummaryStateInput { research_topic: request.topic, }; // Send initial status update let mut status = StatusUpdate::default(); status.phase = "init".to_string(); status.message = format!("Starting research on topic: {}", input.research_topic); status.timestamp = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs(); if let Err(e) = state.status_tx.send(status) { eprintln!("Failed to send initial status update: {}", e); } let mut graph = state.graph.lock().await; // Update the graph's status sender graph.set_status_sender(state.status_tx.clone()); match graph.process_research(input).await { Ok(output) => { // Send final status update let mut status = StatusUpdate::default(); status.phase = "complete".to_string(); status.message = output.running_summary.clone(); status.timestamp = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs(); let _ = state.status_tx.send(status); ( StatusCode::OK, Json(ResearchResponse { summary: output.running_summary, status: "Research completed".to_string(), }) ).into_response() }, Err(e) => { eprintln!("Research error: {:?}", e); let error_message = e.to_string(); let mut status = StatusUpdate::default(); status.phase = "error".to_string(); status.message = format!("Error: {}", error_message); status.timestamp = SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs(); let _ = state.status_tx.send(status); ( StatusCode::INTERNAL_SERVER_ERROR, Json(ResearchResponse { summary: format!("Error: {}", error_message), status: "Error occurred".to_string(), }) ).into_response() }, } } async fn status_stream( [truncated — 3501 more characters] ``` ### zu-chat/src/app/layout.tsx ```typescript import type { Metadata } from "next"; import { Geist, Geist_Mono } from "next/font/google"; import "./globals.css"; const geistSans = Geist({ variable: "--font-geist-sans", subsets: ["latin"], }); const geistMono = Geist_Mono({ variable: "--font-geist-mono", subsets: ["latin"], }); export const metadata: Metadata = { title: "Create Next App", description: "Generated by create next app", }; export default function RootLayout({ children, }: Readonly<{ children: React.ReactNode; }>) { return ( {children} ); } ``` ### zu-chat/src/app/page.tsx ```typescript "use client"; import { useState, useEffect, useRef } from "react"; import { useChat } from "ai/react"; import Link from "next/link"; import { Button } from "@/components/ui/button"; import { Card } from "@/components/ui/card"; import { PodcastControls } from "@/components/podcast/PodcastControls"; import { PodcastPlayer } from "@/components/podcast/PodcastPlayer"; import { ResearchHeader } from "@/components/research/ResearchHeader"; import { ResearchPerspectives } from "@/components/research/ResearchPerspectives"; import { DEFAULT_RESEARCH_SUMMARY } from "@/lib/constants"; import { useConversation } from "@11labs/react"; const LANGUAGES = [ { value: "English", label: "English" }, { value: "Spanish", label: "Spanish" }, { value: "Chinese", label: "Chinese" }, { value: "Russian", label: "Russian" }, { value: "Arabic", label: "Arabic" }, { value: "Japanese", label: "Japanese" }, { value: "Korean", label: "Korean" }, { value: "French", label: "French" }, { value: "German", label: "German" }, { value: "Italian", label: "Italian" }, { value: "Portuguese", label: "Portuguese" }, ] as const; const DURATIONS = [ { value: "1", label: "1 minute" }, { value: "2", label: "2 minutes" }, { value: "3", label: "3 minutes" }, { value: "5", label: "5 minutes" }, ] as const; interface SpeakerImage { speaker: string; imageUrl: string | null; error?: string; } export default function Home() { const [isGenerating, setIsGenerating] = useState(false); const [isGeneratingImages, setIsGeneratingImages] = useState(false); const [isReady, setIsReady] = useState(false); const [language, setLanguage] = useState("English"); const [duration, setDuration] = useState("3"); const [audioSegments, setAudioSegments] = useState([]); const [transcript, setTranscript] = useState([]); const [currentSegment, setCurrentSegment] = useState(0); const [displayedMessages, setDisplayedMessages] = useState([]); const [speakerImages, setSpeakerImages] = useState([]); const [trackOne, setTrackOne] = useState(null); const [trackTwo, setTrackTwo] = useState(null); const [selectedTrack, setSelectedTrack] = useState<"one" | "two">("one"); const audioRef = useRef(null); const { messages, setMessages } = useChat(); const [isTestMode, setIsTestMode] = useState(false); useEffect(() => { // Load research summaries from localStorage const trackOne = localStorage.getItem("researchSummaryTrackOne"); const trackTwo = localStorage.getItem("researchSummaryTrackTwo"); if (trackOne) setTrackOne(trackOne); if (trackTwo) setTrackTwo(trackTwo); }, []); useEffect(() => { // Handle audio playback and transcript synchronization if (isReady && audioSegments.length > 0 && currentSegment < audioSegments.length) { const audio = audioRef.current; if (audio) { // Stop any current playback audio.pause(); // Create and play the new segment const blob = new Blob([audioSegments[currentSegment]], { type: "audio/mpeg" }); const url = URL.createObjectURL(blob); audio.src = url; // Start playing automatically audio.play().catch((error) => { console.error("Error playing audio:", error); }); // Clean up previous URL if it exists if (audio.dataset.previousUrl) { URL.revokeObjectURL(audio.dataset.previousUrl); } audio.dataset.previousUrl = url; // Update displayed message if not already shown if (!displayedMessages[currentSegment]) { setDisplayedMessages((prev) => { const newMessages = [...prev]; newMessages[currentSegment] = { role: "assistant", content: `${transcript[currentSegment].speaker}: ${transcript[currentSegment].text}`, }; return newMessages; }); } audio.onended = () => { if (currentSegment < audioSegments.length - 1) { setCurrentSegment((prev) => prev + 1); } }; // Cleanup function return () => { audio.pause(); if (audio.dataset.previousUrl) { URL.revokeObjectURL(audio.dataset.previousUrl); } }; } } }, [audioSegments, currentSegment, transcript, isReady, displayedMessages]); const handleGenerate = async () => { setIsGenerating(true); setIsReady(false); setDisplayedMessages([]); setCurrentSegment(0); setSpeakerImages([]); try { const researchSummary = selectedTrack === "one" ? trackOne : trackTwo; // Generate podcast audio const response = await fetch("/api/generate", { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({ language, minutes: duration, researchSummary: researchSummary || DEFAULT_RESEARCH_SUMMARY, }), }); if (!response.ok) { throw new Error("Failed to generate podcast"); } const data = await response.json(); if (data.error) { throw new Error(data.error); } // Convert base64 audio segments to ArrayBuffer const segments = data.audioSegments.map((base64Audio: string) => { try { const binaryString = atob(base64Audio); const bytes = new Uint8Array(binaryString.length); for (let i = 0; i < binaryString.length; i++) { bytes[i] = binaryString.charCodeAt(i); } return bytes.buffer; } catch (error) { console.error("Error decoding audio segment:", error); throw new Error("Failed to decode audio data"); } }); setAudioSegments(segments); setTranscript(data.transcript); // Generate images for all segments setIsGeneratingImages(tr [truncated — 6300 more characters] ``` [53 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]