# Project export: Juliette This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: Every lecture deserves a second voice. Juliette is a TA built right into your lecture. - Devpost: https://devpost.com/software/j-vp1gfb - GitHub: https://github.com/enbaocao/juliette - Demo: https://juliette-six.vercel.app/ - Video: https://www.youtube.com/embed/8IsAe0S3TJ0?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 4 GitHub contributor(s) — impeccablejosh (43 commits), en (37 commits), lemoncarrot (15 commits), Claude Sonnet 4.5 (14 commits) ## Devpost submission (written by the team) ### Inspiration We went up to our professors and our TAs and asked for pain points in the process. The first few conversations we had, we were just talking about grading and doing unnecessary work and optimizing things from the teacher's standpoint. But then we decided to talk to our CS 109 professor, Juliette Woodrow, who actually does research on AI and learning. Some of her papers feature AI feedback on problem sets and more one-on-one personalized learning. So we wanted to make a platform that would make online learning more accessible for both the student and the teacher. One of our teammates, Luke, had this brilliant scenario: in lecture, you're confused — what do you do? You ask the guy next to you. But when you're listening to a Zoom lecture or watching a YouTube video to learn something, you don't have that person next to you. So we made Juliette, in the spirit of Valentine's Day — you're Romeo, and you have your Juliette next to you to answer your questions. ### What it does Juliette can ingest YouTube videos or be a live Zoom companion to: Explain topics with simple, digestible answers and comprehension-check questions Generate animations in beautiful Manim to visualize concepts Give you personalized practice problems to get you interested in the work — like if you were learning probability and you were interested in sports, it would give you a sports statistics question The teacher has a dashboard where she or he can see what questions the students are asking, and therefore get feedback on what areas need the most improvement. Through this, we hope we can make online learning more enjoyable for both the student and the teacher. ### How we built it We built Juliette on Next.js 15 with TypeScript and Tailwind CSS, backed by Supabase for our database, auth, and storage. The AI pipeline uses OpenAI Whisper for transcription, GPT for answering questions across all three modes, and embeddings for retrieving the right transcript chunks. Animations are rendered through a Docker-based Manim worker using predefined templates filled in by the LLM. For the live classroom experience, we integrated the Zoom Apps SDK** to embed Juliette as an in-meeting Q&A panel. ### Challenges we ran into Chunking transcripts into meaningful 45–90 second segments with accurate timestamps so the AI retrieves the right context for each question Constraining Manim animation generation to predefined templates rather than arbitrary code — balancing creative flexibility with reliability Building a real-time teacher dashboard that surfaces actionable question trends without overwhelming instructors Making the Zoom panel feel native and responsive inside a meeting environment ### Accomplishments we're proud of The personalized practice problem generation — seeing it tailor a probability question around sports or music based on a student's interests feels genuinely engaging Getting the full pipeline working end-to-end: video upload → transcription → chunked retrieval → three distinct AI response modes The teacher dashboard providing real, useful insight into where students are struggling Shipping a working Zoom integration as an in-meeting panel ### What we learned Talking to Professor Woodrow taught us that the best AI learning tools don't replace teachers — they create feedback loops. Juliette helps students get unstuck in the moment, and simultaneously gives instructors visibility into where their class is struggling. We learned to design for both sides of the classroom, not just one. ### What's next Adding vector embeddings for smarter, semantic transcript retrieval beyond keyword matching Expanding the Manim template library so Juliette can visualize a wider range of concepts Supporting live multi-student sessions where the teacher can see questions aggregated in real time during a Zoom lecture Integrating with Canvas and other LMS platforms so Juliette fits naturally into existing course workflows And so goes the story of our project, Juliette. ## README (from the GitHub repository) # Juliette - AI Educational Video Assistant An AI agent that ingests educational videos, answers student questions with Manim-style animations, personalized practice problems, or simple explanations, and integrates with Zoom for live Q&A. ## Features - **Video Upload & Transcription**: Upload educational videos or provide URLs for automatic transcription with timestamps - **AI-Powered Q&A**: Three response modes: - Simple explanations with check questions - Personalized practice problems based on student interests - Animated visualizations using Manim templates - **Zoom Integration**: Live Q&A panel in Zoom meetings for classroom use - **Teacher Feedback**: Alternative explanations and resource suggestions ## Tech Stack - **Frontend**: Next.js (App Router) + TypeScript + Tailwind CSS - **Backend**: Supabase (Auth, Postgres, Storage, Vector Search) - **AI**: OpenAI (Transcription, Q&A, Embeddings) - **Animations**: Manim (Docker-based render worker) - **Real-time**: Zoom Apps SDK ## Getting Started ### Prerequisites - Node.js 18+ and npm - Supabase account - OpenAI API key - (Optional) Zoom Developer account for Zoom integration ### Installation 1. Clone the repository: ```bash git clone cd juliette ``` 2. Install dependencies: ```bash npm install ``` 3. Set up environment variables: ```bash cp .env.local.example .env.local ``` Edit `.env.local` and add your credentials: - Supabase URL and keys - OpenAI API key - Zoom credentials (if using Zoom integration) 4. Set up Supabase database: Run the SQL migrations in `supabase/migrations/` to create the necessary tables: - `videos` - stores uploaded video metadata - `transcript_chunks` - timestamped transcript segments - `questions` - student questions and AI responses - `jobs` - background task queue 5. Run the development server: ```bash npm run dev ``` Open [http://localhost:3000](http://localhost:3000) in your browser. ## Development ### Available Scripts - `npm run dev` - Start development server with Turbopack - `npm run build` - Build for production - `npm start` - Start production server - `npm run lint` - Run ESLint ### Project Structure ``` juliette/ ├── app/ # Next.js App Router pages ├── components/ # React components ├── lib/ # Utilities and configurations │ ├── supabase.ts # Supabase client │ └── openai.ts # OpenAI client ├── utils/ # Helper functions ├── public/ # Static assets └── supabase/ # Database migrations and schemas ``` ## Architecture See [Project.md](./Project.md) for detailed MVP architecture and implementation plan. ## License MIT ## Detected evidence (automated analysis) Indexed codebase: 155 recognized source files, 782 KB. - Anthropic (technology) — detected in the code - CSS (language) — detected in the code - Express (technology) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - SQL (language) — detected in the code - Supabase (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - Docker (technology) — claimed on Devpost, not found in the code - PostgreSQL (technology) — claimed on Devpost, not found in the code - Vercel (technology) — claimed on Devpost, not found in the code - AI coding agent: Claude Code — evidence: config files committed to the repository; commit authorship or trailers ## Codebase structure (from repository index) ### Files (120 of 297) ``` .claude/settings.local.json .env.local.example .eslintrc.json .gitattributes .gitignore [backup_folder]/live-sessions-id-route/[id]/route.ts [backup_folder]/live-sessions-id-route/[sessionId]/route.ts [backup_folder]/live-sessions-id-route/active/route.ts [backup_folder]/live-sessions-id-route/check/route.ts [backup_folder]/live-sessions-id-route/end/route.ts [backup_folder]/live-sessions-id-route/link-video/route.ts [backup_folder]/live-sessions-id-route/questions/route.ts [backup_folder]/live-sessions-id-route/start/route.ts ALTERNATIVE_APPROACH.md app/api/ask/route.ts app/api/generate-animation/route.ts app/api/live-sessions/[sessionId]/route.ts app/api/live-sessions/active/route.ts app/api/live-sessions/check/route.ts app/api/live-sessions/end/route.ts app/api/live-sessions/link-video/route.ts app/api/live-sessions/questions/route.ts app/api/live-sessions/start/route.ts app/api/practice-demo/route.ts app/api/rtms/start/route.ts app/api/rtms/status/route.ts app/api/rtms/stop/route.ts app/api/transcribe-recording/route.ts app/api/upload-youtube/route.ts app/api/upload/route.ts app/api/videos/[id]/route.ts app/api/videos/list/route.ts app/api/videos/route.ts app/auth/callback/route.ts app/error.tsx app/global-error.tsx app/globals.css app/layout.tsx app/live/dashboard/[sessionId]/page.tsx app/page.tsx app/practice-sandbox/layout.tsx app/practice-sandbox/page.tsx app/practice/[id]/page.tsx app/student/page.tsx app/teacher/analytics/page.tsx app/teacher/live/page.tsx app/teacher/page.tsx app/teacher/questions/page.tsx app/teacher/videos/page.tsx app/teacher/zoom/page.tsx app/test-animation/layout.tsx app/test-animation/page.tsx app/upload/page.tsx app/videos/[id]/ask/layout.tsx app/videos/[id]/ask/page.tsx app/videos/[id]/page.tsx app/zoom/auth/page.tsx app/zoom/panel/layout.tsx app/zoom/panel/page.tsx check-jobs.ts CLAUDE.md components/auth/AuthForm.tsx components/auth/SignOutButton.tsx components/Header.tsx components/qa/AnswerDisplay.tsx components/qa/QuestionForm.tsx components/teacher/TeacherAudioRecorder.tsx components/ui/Toast.tsx components/ui/ToastProvider.tsx components/upload/VideoUpload.tsx components/UserMenu.tsx components/videos/StatusAutoRefresh.tsx components/zoom/HostControls.tsx components/zoom/LiveSessionPanel.tsx components/zoom/ManimVideoTab.tsx components/zoom/ScreenRecorder.tsx components/zoom/SimpleStudentView.tsx components/zoom/StudentView.tsx components/zoom/TranscriptionStatus.tsx CURRENT_STATUS.md DIRECT_TRANSCRIPTION_FLOW.md docker/.dockerignore docker/manim-api-server.ts docker/manim-api.Dockerfile ENV_SETUP_SIMPLIFIED.md hooks/useZoomApp.ts IMPLEMENTATION_STATUS.md lib/animation-library.ts lib/anthropic.ts lib/file-logger.ts lib/openai.ts lib/supabase-server.ts lib/supabase.ts lib/supabase/client.ts lib/supabase/middleware.ts lib/supabase/server.ts lib/types.ts lib/zoom.ts LICENSE MANIM_API.md MANIM_OPUS_UPGRADE.md MANIM_TEMPLATES.md manim-sandbox/.cache/fontconfig/0bd3dc0958fa2205aaaa8ebb13e2872b-le64.cache-9 manim-sandbox/.cache/fontconfig/188ac73a183f12857f63bb60a4a6d603-le64.cache-9 manim-sandbox/.cache/fontconfig/32b6488e5b8292a2e95c79d947e009e8-le64.cache-9 manim-sandbox/.cache/fontconfig/3830d5c3ddfd5cd38a049b759396e72e-le64.cache-9 manim-sandbox/.cache/fontconfig/4c599c202bc5c08e2d34565a40eac3b2-le64.cache-9 manim-sandbox/.cache/fontconfig/573ec803664ed168555e0e8b6d0f0c7f-le64.cache-9 manim-sandbox/.cache/fontconfig/57e423e26b20ab21d0f2f29c145174c3-le64.cache-9 manim-sandbox/.cache/fontconfig/6333f38776742d18e214673cd2c24e34-le64.cache-9 manim-sandbox/.cache/fontconfig/6cc790b63b123a6a96ee260fcdad32a9-le64.cache-9 manim-sandbox/.cache/fontconfig/707971e003b4ae6c8121c3a920e507f5-le64.cache-9 manim-sandbox/.cache/fontconfig/7ef2298fde41cc6eeb7af42e48b7d293-le64.cache-9 manim-sandbox/.cache/fontconfig/95530828ff6c81d309f8258d8d02a23e-le64.cache-9 manim-sandbox/.cache/fontconfig/a4e60e8d1e10d2fdff3fe3037a1845fb-le64.cache-9 manim-sandbox/.cache/fontconfig/c855463f699352c367813e37f3f70ea7-le64.cache-9 manim-sandbox/.cache/fontconfig/CACHEDIR.TAG manim-sandbox/.cache/fontconfig/d3e5c4ee2ceb1fc347f91d4cefc53bc0-le64.cache-9 manim-sandbox/.cache/fontconfig/d589a48862398ed80a3d6066f4f56f4c-le64.cache-9 manim-sandbox/.cache/fontconfig/d82eb4fd963d448e2fcb7d7b793b5df3-le64.cache-9 [177 more files omitted for size] ``` ### Dependencies - manim-sandbox/requirements.txt: numpy@>=1.21.0, scipy@>=1.7.0 - package.json: @anthropic-ai/sdk@^0.74.0, @supabase/ssr@^0.6.0, @supabase/supabase-js@^2.48.0, @types/cors@^2.8.19, @types/express@^5.0.6, @types/node@^22, @types/react@^19, @types/react-dom@^19, @zoom/appssdk@^0.16.36, autoprefixer@^10.4.20, clsx@^2.1.1, cors@^2.8.6, dotenv@^17.3.1, eslint@^9, eslint-config-next@^15.1.6, express@^5.2.1, framer-motion@^12.34.0, geist@^1.7.0, katex@^0.16.28, lucide-react@^0.564.0, next@^15.1.6, openai@^4.77.3, postcss@^8, react@^19.0.0, react-dom@^19.0.0, react-katex@^3.1.0, react-markdown@^10.1.0, rehype-katex@^7.0.1, remark-math@^6.0.0, tailwind-merge@^3.4.0, tailwindcss@^3.4.1, tsc-alias@^1.8.16, tsx@^4.21.0, typescript@^5, youtube-transcript@^1.2.1 ### Recent commits (newest first) - adjusted now that we have public domain - periodic teacher upload+ manim save - auto videos - vercel fix - fix app session pick - fix zoom app pleaseeee - zoom app fix - zoom app techer session model - fix plz - zapp fix? - vercel fix - Merge branch 'main' of https://github.com/enbaocao/juliette - update screen record - Update ask/practice UI and auth flow adjustments - Merge branch 'main' of https://github.com/enbaocao/juliette - screenrecord fix? - Fix auth redirects to use site URL instead of localhost - audio recording - audio whisper fix - minor bug fixing ## Key source files (fetched from GitHub, selected and truncated for size) ### mission.md ```markdown I am trying to build an AI agent that (1) ingests educational video, (2) answers student questions with manim-style animations, personalized practice, or simple explanations, and (3) integrates with Zoom for live Q&A + teacher feedback (including pulling from other internet videos). This is the tech stack that I want to implement ``` ### SETUP_CLAUDE_API.md ```markdown # Setup Claude API for Animation Generation ## Quick Setup The animation generation now uses **Claude (Anthropic API)** instead of OpenAI for better code generation! --- ## Get Your Claude API Key ### Option 1: Use Existing Key (If you have one) If you already have an Anthropic API key, skip to "Add to Environment". ### Option 2: Create New Key 1. Go to https://console.anthropic.com 2. Sign in or create account 3. Go to **API Keys** section 4. Click **Create Key** 5. Copy your key (starts with `sk-ant-...`) --- ## Add to Environment Add this line to your `.env.local` file: ```bash ANTHROPIC_API_KEY=sk-ant-your-key-here ``` Your `.env.local` should now have: ```bash # Supabase NEXT_PUBLIC_SUPABASE_URL=... NEXT_PUBLIC_SUPABASE_ANON_KEY=... SUPABASE_SERVICE_ROLE_KEY=... # OpenAI (for transcription) OPENAI_API_KEY=sk-proj-... # Anthropic (for animation generation) ANTHROPIC_API_KEY=sk-ant-... # Zoom (optional) ZOOM_CLIENT_ID=... ZOOM_CLIENT_SECRET=... ``` --- ## Restart Server After adding the key: ```bash # The dev server will auto-reload, but if not: # Stop server (Ctrl+C) and restart: npm run dev ``` --- ## Test It Visit http://localhost:3001/test-animation and try generating an animation! --- ## Why Claude? - ✅ **Better at following templates** - Understands structured patterns - ✅ **More reliable code generation** - Fewer syntax errors - ✅ **Follows constraints better** - Keeps code minimal - ✅ **Great with instructions** - Respects the template guidelines --- ## Cost Comparison **Claude Sonnet 3.5:** - Input: $3 per 1M tokens - Output: $15 per 1M tokens - ~$0.01-0.02 per animation **OpenAI GPT-4:** - Input: $10 per 1M tokens - Output: $30 per 1M tokens - ~$0.03-0.05 per animation **Claude is cheaper and better for this use case!** 💰 --- ## Troubleshooting **Error: "API key not found"** - Make sure you added `ANTHROPIC_API_KEY=...` to `.env.local` - Check there are no quotes around the key - Restart dev server **Error: "Invalid API key"** - Verify key starts with `sk-ant-` - Check you copied the full key - Regenerate key in Anthropic Console if needed --- Ready to test? Add your key and visit http://localhost:3001/test-animation! 🎬 ``` ### package.json ``` { "name": "juliette", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev --turbopack", "build": "next build", "start": "next start", "lint": "next lint", "worker:transcription": "tsx workers/transcription-worker.ts", "worker:youtube": "tsx workers/youtube-transcript-worker.ts", "worker:youtube-download": "tsx workers/youtube-download-worker.ts", "logs:youtube-download": "tail -n 200 -f logs/youtube-download-worker.log", "logs:transcription": "tail -n 200 -f logs/transcription-worker.log" }, "dependencies": { "@anthropic-ai/sdk": "^0.74.0", "@supabase/ssr": "^0.6.0", "@supabase/supabase-js": "^2.48.0", "@zoom/appssdk": "^0.16.36", "clsx": "^2.1.1", "cors": "^2.8.6", "dotenv": "^17.3.1", "express": "^5.2.1", "framer-motion": "^12.34.0", "geist": "^1.7.0", "katex": "^0.16.28", "lucide-react": "^0.564.0", "next": "^15.1.6", "openai": "^4.77.3", "react": "^19.0.0", "react-dom": "^19.0.0", "react-katex": "^3.1.0", "react-markdown": "^10.1.0", "rehype-katex": "^7.0.1", "remark-math": "^6.0.0", "tailwind-merge": "^3.4.0", "youtube-transcript": "^1.2.1" }, "devDependencies": { "@types/cors": "^2.8.19", "@types/express": "^5.0.6", "@types/node": "^22", "@types/react": "^19", "@types/react-dom": "^19", "autoprefixer": "^10.4.20", "eslint": "^9", "eslint-config-next": "^15.1.6", "postcss": "^8", "tailwindcss": "^3.4.1", "tsc-alias": "^1.8.16", "tsx": "^4.21.0", "typescript": "^5" } } ``` ### manim-sandbox/requirements.txt ``` # Manim is already included in the base image # Add any additional dependencies here numpy>=1.21.0 scipy>=1.7.0 ``` ### manim-sandbox/Dockerfile ``` FROM manimcommunity/manim:stable # Set working directory WORKDIR /manim # Copy requirements if we need additional packages COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt # Copy templates and examples COPY templates/ ./templates/ COPY examples/ ./examples/ # Default command CMD ["bash"] ``` ### manim-sandbox/docker-compose.yml ```yaml version: '3.8' services: manim: build: . volumes: # Mount current directory to /manim in container - ./:/manim # Output directory for rendered videos - ./output:/manim/media working_dir: /manim # Run commands like: docker-compose run manim manim -pql examples/01_basic_shapes.py entrypoint: [] command: bash ``` ### app/layout.tsx ```typescript import type { Metadata } from "next"; import { GeistSans } from "geist/font/sans"; import "./globals.css"; import Header from "@/components/Header"; import ToastProvider from "@/components/ui/ToastProvider"; export const metadata: Metadata = { metadataBase: new URL(process.env.NEXT_PUBLIC_SITE_URL || "https://juliette-six.vercel.app"), icons: { icon: "/logo.png", }, title: { default: "Juliette | AI Educational Video Assistant", template: "%s | Juliette", }, description: "Juliette helps students learn from educational videos with AI-powered explanations, personalized practice problems, and animated visualizations. Upload videos, ask questions, get instant answers.", keywords: ["education", "AI", "video learning", "practice problems", "animations", "student", "teacher"], authors: [{ name: "Juliette" }], openGraph: { type: "website", locale: "en_US", siteName: "Juliette", url: "/", images: [ { url: "/banner.png", width: 1200, height: 630, alt: "Juliette - AI Educational Video Assistant", }, ], }, twitter: { card: "summary_large_image", images: ["/banner.png"], }, robots: { index: true, follow: true, }, }; export default function RootLayout({ children, }: Readonly<{ children: React.ReactNode; }>) { return (
{children}
); } ``` ### app/page.tsx ```typescript import Image from 'next/image'; import Link from 'next/link'; import { Suspense } from 'react'; import AuthForm from '@/components/auth/AuthForm'; import { createClient } from '@/lib/supabase/server'; import { Upload, LayoutDashboard, PlayCircle } from 'lucide-react'; export const metadata = { title: "Welcome", description: "Juliette is your AI educational assistant. Upload videos, ask questions, and get personalized explanations, practice problems, and animated visualizations.", }; export default async function Home({ searchParams }: { searchParams: Promise<{ next?: string; error?: string }> }) { const { next, error } = await searchParams; let user: { email?: string } | null = null; try { const supabase = await createClient(); const { data } = await supabase.auth.getUser(); user = data.user; } catch { // Auth failed - show logged-out state } return (
{/* Left Column: Content */}

Welcome, Romeo

{user ? (

What would you like to do today?

{/* Student Option - Featured */}
🎓

Join Live Session

Record Zoom lectures & ask AI-powered questions

{/* Other Options */}

Upload Video

Add lecture videos

Dashboard

Teacher analytics

Browse Videos

Watch recorded lectures

) : (
Loading…
}> {error === 'auth' && (

Authentication failed. Please try again.

)}
)}
{/* Right Column: Image */}
Students studying late at night
); } ``` ### app/test-animation/layout.tsx ```typescript import type { Metadata } from 'next'; export const metadata: Metadata = { title: 'Animation Test', description: 'Test Manim animation generation for educational visualizations.', }; export default function TestAnimationLayout({ children, }: { children: React.ReactNode; }) { return children; } ``` ### app/practice-sandbox/layout.tsx ```typescript import type { Metadata } from 'next'; export const metadata: Metadata = { title: 'Practice Sandbox', description: 'Test the practice problem generator with custom prompts and student interests.', }; export default function PracticeSandboxLayout({ children, }: { children: React.ReactNode; }) { return children; } ``` [146 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]