# Project export: Step Step Learn

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2026
- Tagline: An AI-powered dance-along game that transforms personalized music into an interactive, movement-based experience for kids.
- Devpost: https://devpost.com/software/step-step-learn
- GitHub: https://github.com/laureny17/JAEL
- Team: 4 GitHub contributor(s) — Lauren Yoo (21 commits), Angelina Ning (13 commits), JamesL425 (4 commits), eszczep (1 commits)

## Devpost submission (written by the team)

### Inspiration

Every child learns differently. We wondered: What if a child could turn their own science homework or favorite story into a custom music video they could dance to?

### What it does

Lyric Generation: Takes a topic (e.g., "The Water Cycle") and generates age-appropriate, catchy lyrics. Music Creation: Uses Suno to turn those lyrics into a high-quality song in any genre (Pop, Disco, Hip-Hop). AI Choreography: Our custom pipeline analyzes the song’s rhythm and semantics to generate a 3D character that dances in sync with the words. Interactive Play: Kids follow the character’s moves on a 3x3 grid, paired with hand motions, and are scored based on the accuracy of their movements.

### How we built it

Backend stack: Express, MongoDB, Node.js, ClaudeSDK, Suno API, OpenAI Whisper Frontend stack: React, Three.js, OpenCV, MediaPipe ClaudeSDK to generate lyrics, break of lyrics into semantically meaningful fragments, and translate lyric fragments into 3D pose data (arm and elbow angles, feet positions on a grid) while following physical constraints. Suno API to generate full tracks based on lyrics. OpenAI Whisper to transcribe audio from Suno into words mapped to timestamps, so that we could align the music with movements in our UI. OpenCV to register upper body motions, Mediapipe to process margin of error relative to specified movements Three.js to build the 3D model in our UI. Floor grid: We used acrylic tiles and cushioning foam, so tiles would compress and rebound when stepped on. Additionally, we used Hall effect sensors paired with magnets inserted into the foam to detect compression of a tile. We wired the sensors to an Arduino Uno.

### Challenges we ran into

3D Modeling: Mastering Three.js required a deep dive into skeletal animation. Managing dozens of interconnected joints and ensuring smooth interpolation between AI-generated keyframes was a significant hurdle. AI Context & Logic Failures: We faced "long context" struggles where Claude initially failed to maintain the mapping between lyric fragments and Whisper’s word-level timestamps. It took extensive prompt engineering and iterative experimentation to get the model to respect rhythmic timing. Hardware Noise & Sensor Fusion: Our original vision included hand sensors for upper-body tracking, but we encountered significant accelerometer noise, making relative positioning too unstable for accurate gameplay. Dance Tile Design: Designing the physical input—getting floor tiles to compress and rebound reliably to detect "steps"—presented a classic mechanical engineering challenge that required multiple iterations. We originally tried 3D printing springs but this was too costly and time-consuming. We also considered using copper strips to detect conductivity. Ultimately, we decided on Pipeline Orchestration: Integrating a chain of multiple models (Suno, Whisper, Claude, Three.js) meant that a failure in any one link could break the entire experience, necessitating robust error-handling and iterative experimentation.

### Accomplishments we're proud of

DIY tile boards: We had to iterate through many designs to make sure the tiles compressed and decompressed properly. 3D model on the UI: We had to learn how to use Three.js for 3D modeling, constrained to physical reality Agent pipeline: We integrated many models and had to iteratively experiment and design validation/schema to ensure robustness. Pivot for hand movement detection: After various sensors didn't work, we pivoted to using a computer vision-based pipeline involving OpenCV and MediaPipe.

## README (from the GitHub repository)

# JAEL

- `frontend/`: Next.js app
- `backend/`: Agent workflow skeleton (Claude lyrics -> Suno music)


## Detected evidence (automated analysis)

Indexed codebase: 61 recognized source files, 196 KB.
- Anthropic (technology) — detected in the code
- CSS (language) — detected in the code
- Express (technology) — detected in the code
- Flask (technology) — detected in the code
- MongoDB (technology) — detected in the code
- Next.js (technology) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- C++ (language) — claimed on Devpost, not found in the code
- Node.js (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (73 of 73)

```
.gitignore
backend/package.json
backend/pose-detection/pose_server.py
backend/pose-detection/README.md
backend/pose-detection/requirements.txt
backend/README.md
backend/src/agents/danceAgent.ts
backend/src/agents/tools.ts
backend/src/cli/test-workflow.ts
backend/src/clients/claudeClient.ts
backend/src/clients/sunoClient.ts
backend/src/clients/whisperClient.ts
backend/src/config/env.ts
backend/src/db/db.ts
backend/src/db/mongo.ts
backend/src/prompts/danceMovePrompt.ts
backend/src/prompts/danceSystemPrompt.ts
backend/src/prompts/lyricFragmentPrompt.ts
backend/src/prompts/poseGenerationPrompt.ts
backend/src/routes/danceRoutes.ts
backend/src/routes/songRoutes.ts
backend/src/server.ts
backend/src/types/dance.ts
backend/tsconfig.json
backend/vitest.config.ts
dance_tile_code.ino
frontend/.gitignore
frontend/app/api/dance/create/route.ts
frontend/app/api/songs/[songId]/leaderboard/route.ts
frontend/app/api/songs/[songId]/route.ts
frontend/app/api/songs/[songId]/scores/route.ts
frontend/app/api/songs/route.ts
frontend/app/globals.css
frontend/app/layout.tsx
frontend/app/level/[songId]/page.tsx
frontend/app/page.tsx
frontend/app/play/page.tsx
frontend/components/Character.tsx
frontend/components/FootIcon.tsx
frontend/components/GameHUD.tsx
frontend/components/HandIcon.tsx
frontend/components/LaneDisplay.tsx
frontend/components/PoseDetector.tsx
frontend/components/Scene.tsx
frontend/components/ScoreDisplay.tsx
frontend/eslint.config.mjs
frontend/lib/animation/pose-animator.ts
frontend/lib/backend-url.ts
frontend/lib/constants.ts
frontend/lib/detected-pose-store.ts
frontend/lib/game-store.ts
frontend/lib/ik/bone-lookup.ts
frontend/lib/ik/two-bone-ik.ts
frontend/lib/POSE_FORMAT.md
frontend/lib/pose-player.ts
frontend/lib/poses/foot-positions.ts
frontend/lib/poses/hand-poses.ts
frontend/lib/poses/weight-shift.ts
frontend/lib/score-store.ts
frontend/lib/scoring.ts
frontend/lib/store.ts
frontend/lib/test-arrow-sequence.ts
frontend/lib/test-sequence.ts
frontend/lib/types.ts
frontend/lib/use-pose-checker.ts
frontend/next.config.ts
frontend/package.json
frontend/postcss.config.mjs
frontend/public/character-timmy-2.glb
frontend/public/character-timmy.glb
frontend/README.md
frontend/tsconfig.json
README.md
```

### Dependencies

- backend/package.json: @anthropic-ai/claude-agent-sdk@^0.1.0, @anthropic-ai/sdk@^0.74.0, @types/express@^4.17.21, @types/node@^20.10.0, @vitest/coverage-v8@^1.0.0, axios@^1.13.5, dotenv@^16.3.1, express@^4.18.2, mongodb@^6.16.0, openai@^6.22.0, tsx@^4.7.0, typescript@^5.3.3, vitest@^1.0.0
- backend/pose-detection/requirements.txt: flask, mediapipe@==0.10.14, numpy, opencv-python
- frontend/package.json: @mediapipe/tasks-vision@^0.10.32, @react-three/drei@^10.7.7, @react-three/fiber@^9.5.0, @tailwindcss/postcss@^4, @types/node@^20, @types/react@^19, @types/react-dom@^19, @types/three@^0.182.0, eslint@^9, eslint-config-next@16.1.6, next@16.1.6, react@19.2.3, react-dom@19.2.3, tailwindcss@^4, three@^0.182.0, typescript@^5, zustand@^5.0.11

### Recent commits (newest first)

- Merge branch 'main' of https://github.com/laureny17/JAEL
- fix audio bug
- add small vfx + fix name & favicon
- Add files via upload
- fix scoring, add hand motion guides, fix alignment of guides
- fix scoring to reflect accuracy of move matching
- add backup plan for when suno is down
- Merge branch 'main' of https://github.com/laureny17/JAEL
- styling fixes for homepage + integrate w/ angelina
- Update return from dance agent run
- Update
- update readme
- Merge branch 'main' of https://github.com/laureny17/JAEL
- clean up root
- fix: resolve camera deadlock from React Strict Mode double-mount
- feat: add in-browser pose detection with MediaPipe PoseLandmarker
- add mongo connection and other pages
- added more ui to gameplay screen (pause, camera, countdown, lyrics)
- Merge branch 'main' of https://github.com/laureny17/JAEL
- stylize display for game sequences

## Key source files (fetched from GitHub, selected and truncated for size)

### frontend/lib/POSE_FORMAT.md

```markdown
# Pose Sequence JSON Format

A pose sequence is an array of timestamped keyframes. The frontend interpolates smoothly between keyframes to animate the 3D character.

## Structure

```json
[
  {
    "time": 0,
    "pose": {
      "leftShoulderAngle": 10,
      "rightShoulderAngle": 10,
      "leftElbowAngle": 160,
      "rightElbowAngle": 160,
      "leftHandShape": "open",
      "rightHandShape": "open",
      "leftFoot": "M",
      "rightFoot": "M"
    }
  },
  {
    "time": 1.5,
    "pose": { ... }
  }
]
```

## Fields

### `time` (number)

Seconds from the start of the sequence. Keyframes must be in chronological order. The sequence loops back to the beginning after the last keyframe.

### Arm Angles (degrees)

| Field | Range | Description |
|---|---|---|
| `leftShoulderAngle` | 0 - 180 | Angle between the spine (torso line) and the left upper arm. 0 = arm hanging at side, 90 = arm horizontal, 180 = arm straight overhead. |
| `rightShoulderAngle` | 0 - 180 | Same for the right arm. |
| `leftElbowAngle` | 0 - 180 | Non-reflex angle between upper arm and forearm. 0 = fully bent (forearm folded against upper arm), 180 = fully extended (straight arm). |
| `rightElbowAngle` | 0 - 180 | Same for the right arm. |

Arms are rendered in the 2D frontal plane (no depth). The shoulder angle rotates the arm outward/upward from the torso, and the elbow angle opens or closes the forearm relative to the upper arm.

### Hand Shapes

| Value | Description |
|---|---|
| `"open"` | All fingers extended, slight natural spread (5 fingers showing) |
| `"fist"` | All fingers and thumb curled tightly |
| `"one"` | Index finger extended, rest curled, thumb tucked |
| `"peace"` | Index + middle extended in a V shape, rest curled, thumb tucked |
| `"three"` | Index + middle + ring extended, rest curled, thumb tucked |
| `"four"` | All fingers except thumb extended, thumb tucked |
| `"heart"` | Thumbs up (thumb extended, all fingers curled) |
| `"flat"` | All fingers together, flat palm, thumb tucked to side |
| `"pointing"` | Index finger extended, rest curled, thumb tucked |

### Foot Positions

Feet are placed on a cross-shaped grid (no diagonals). The grid is oriented from the character's perspective (facing the camera):

```
         T          Back (tiptoe stance)
      L  M  R       Middle (flat foot)
         B          Front (heel stance)
```

| Value | Position |
|---|---|
| `"T"` | Top-center (back, tiptoe) |
| `"L"` | Middle-left (flat foot) |
| `"M"` | Middle-center (neutral stance, flat foot) |
| `"R"` | Middle-right (flat foot) |
| `"B"` | Bottom-center (front, heel) |

**Constraints:**
- Both feet on the same position is only allowed when both are `"M"`.
- In the same row, feet cannot cross (e.g., right foot can't be in left column if left foot is in the same row).

```

### frontend/package.json

```
{
  "name": "frontend",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "eslint"
  },
  "dependencies": {
    "@mediapipe/tasks-vision": "^0.10.32",
    "@react-three/drei": "^10.7.7",
    "@react-three/fiber": "^9.5.0",
    "next": "16.1.6",
    "react": "19.2.3",
    "react-dom": "19.2.3",
    "three": "^0.182.0",
    "zustand": "^5.0.11"
  },
  "devDependencies": {
    "@tailwindcss/postcss": "^4",
    "@types/node": "^20",
    "@types/react": "^19",
    "@types/react-dom": "^19",
    "@types/three": "^0.182.0",
    "eslint": "^9",
    "eslint-config-next": "16.1.6",
    "tailwindcss": "^4",
    "typescript": "^5"
  }
}

```

### backend/package.json

```
{
  "name": "jael-backend2",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "dev": "tsx watch src/server.ts",
    "build": "tsc",
    "start": "node dist/server.js",
    "test": "vitest",
    "test:watch": "vitest --watch",
    "test:coverage": "vitest --coverage",
    "workflow": "tsx src/cli/test-workflow.ts"
  },
  "dependencies": {
    "@anthropic-ai/claude-agent-sdk": "^0.1.0",
    "@anthropic-ai/sdk": "^0.74.0",
    "axios": "^1.13.5",
    "dotenv": "^16.3.1",
    "express": "^4.18.2",
    "mongodb": "^6.16.0",
    "openai": "^6.22.0"
  },
  "devDependencies": {
    "@types/express": "^4.17.21",
    "@types/node": "^20.10.0",
    "@vitest/coverage-v8": "^1.0.0",
    "tsx": "^4.7.0",
    "typescript": "^5.3.3",
    "vitest": "^1.0.0"
  }
}

```

### backend/pose-detection/requirements.txt

```
opencv-python
mediapipe==0.10.14
numpy
flask

```

### backend/src/server.ts

```typescript
import express from "express";
import { env } from "./config/env.js";
import danceRoutes from "./routes/danceRoutes.js";
import { connectMongo } from "./db/mongo.js";
import songRoutes from "./routes/songRoutes.js";

const app = express();

app.use(express.json());

app.use("/api/dance", danceRoutes);
app.use("/api/songs", songRoutes);

app.get("/health", (req, res) => {
  res.json({ status: "ok" });
});

async function start() {
  await connectMongo();
  app.listen(env.port, () => {
    console.log(`Backend server running on port ${env.port}`);
  });
}

start().catch((err) => {
  console.error("Failed to start server:", err);
  process.exit(1);
});

```

### frontend/app/layout.tsx

```typescript
import type { Metadata } from "next";
import { Geist_Mono, Nunito } from "next/font/google";
import "./globals.css";

const geistMono = Geist_Mono({
  variable: "--font-geist-mono",
  subsets: ["latin"],
});

const nunito = Nunito({
  variable: "--font-nunito",
  subsets: ["latin"],
});

export const metadata: Metadata = {
  title: "step step learn",
  description: "generate dance levels from your ideas and learn through movement",
};

export default function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode;
}>) {
  return (
    <html lang="en">
      <head>
        <link rel="stylesheet" href="https://use.typekit.net/pxe1cjr.css" />
      </head>
      <body
        className={`${nunito.variable} ${geistMono.variable} antialiased lowercase`}
      >
        {children}
      </body>
    </html>
  );
}

```

### frontend/app/page.tsx

```typescript
'use client';

import { useEffect, useState } from 'react';
import Link from 'next/link';

const MAX_PROMPT = 160;
const LENGTH_OPTIONS = [
  { id: 'short', label: 'Short', seconds: 60 },
  { id: 'medium', label: 'Medium', seconds: 90 },
  { id: 'long', label: 'Long', seconds: 120 },
];
const RISE_AND_FALL_SONG_ID = '6991eb84fba63cc2e3ad092f';

type SongListItem = {
  id: string;
  title?: string;
  prompt?: string;
  lengthSeconds?: number;
  createdAt?: string;
};

function formatDuration(seconds?: number): string {
  if (!seconds || Number.isNaN(seconds)) return '0:00';
  const total = Math.max(0, Math.floor(seconds));
  const mins = Math.floor(total / 60);
  const secs = total % 60;
  return `${mins}:${String(secs).padStart(2, '0')}`;
}

export default function Home() {
  const [prompt, setPrompt] = useState('');
  const [audience, setAudience] = useState('');
  const [lengthId, setLengthId] = useState('short');
  const [isSubmitting, setIsSubmitting] = useState(false);
  const [mode, setMode] = useState<'create' | 'play' | null>(null);
  const [songs, setSongs] = useState<SongListItem[]>([]);
  const [songsError, setSongsError] = useState<string | null>(null);
  const [loadingSongs, setLoadingSongs] = useState(false);
  const [createError, setCreateError] = useState<string | null>(null);
  const [showSunoUnavailable, setShowSunoUnavailable] = useState(false);

  const length = LENGTH_OPTIONS.find((opt) => opt.id === lengthId) ?? LENGTH_OPTIONS[0];

  useEffect(() => {
    let cancelled = false;

    async function loadSongs() {
      setLoadingSongs(true);
      setSongsError(null);

      try {
        const response = await fetch('/api/songs', { cache: 'no-store' });
        if (!response.ok) {
          throw new Error(`Could not load songs (${response.status}).`);
        }

        const payload = await response.json();
        const nextSongs = Array.isArray(payload?.songs) ? payload.songs : [];

        if (!cancelled) {
          setSongs(nextSongs);
        }
      } catch (error) {
        if (!cancelled) {
          setSongsError(error instanceof Error ? error.message : 'Failed to load songs.');
        }
      } finally {
        if (!cancelled) {
          setLoadingSongs(false);
        }
      }
    }

    void loadSongs();

    return () => {
      cancelled = true;
    };
  }, []);

  async function handleGenerate() {
    if (!prompt.trim()) return;
    setIsSubmitting(true);
    setCreateError(null);
    setShowSunoUnavailable(false);

    try {
      const response = await fetch('/api/songs', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          prompt: prompt.trim(),
          audienceDescriptor: audience.trim(),
          lengthSeconds: length.seconds,
        }),
      });

      const raw = await response.text();
      let data: { id?: string; error?: string } = {};
      try {
        data = JSON.parse(raw) as { id?: string; error?: string };
      } catch {
        data = { error: raw };
      }

      if (!response.ok) {
        if (response.status === 503) {
          setShowSunoUnavailable(true);
          throw new Error("Suno's services are currently unavailable.");
        }
        throw new Error(data.error || `Generation failed (${response.status}).`);
      }

      if (data.id) {
        window.location.href = `/play?songId=${data.id}`;
      }
    } catch (error) {
      setCreateError(error instanceof Error ? error.message : 'Generation failed.');
    } finally {
      setIsSubmitting(false);
    }
  }

  return (
    <div className="min-h-screen w-screen overflow-x-hidden overflow-y-auto lowercase" style={{ background: '#5a3a3b' }}>
      <div className="px-6 pt-16 pb-10 flex flex-col items-center text-center">
        <div className="text-5xl md:text-6xl font-bold mb-4 lowercase" style={{ color: '#f8f4f2' }}>
          step step learn
        </div>
        <div className="max-w-2xl text-base md:text-lg mb-6 lowercase" style={{ color: '#f8f4f2', opacity: 0.8 }}>
          generate dance levels from your ideas and learn through movement — then play and compare scores with everyone!
        </div>
        <div className="flex flex-wrap justify-center gap-3">
          <button
            type="button"
            className="px-5 py-2 rounded-full border-2 text-sm lowercase"
            style={{
              borderColor: '#462c2d',
              backgroundColor: mode === 'create' ? '#462c2d' : '#f8f4f2',
              color: mode === 'create' ? '#f8f4f2' : '#462c2d',
            }}
            onClick={() => setMode('create')}
          >
            create new level
          </button>
          <button
            type="button"
            className="px-5 py-2 rounded-full border-2 text-sm lowercase"
            style={{
              borderColor: '#462c2d',
              backgroundColor: mode === 'play' ? '#462c2d' : '#f8f4f2',
              color: mode === 'play' ? '#f8f4f2' : '#462c2d',
            }}
            onClick={() => setMode('play')}
          >
            play existing levels
          </button>
        </div>
      </div>

      {mode === 'create' ? (
        <div className="px-6 pt-6 pb-14 flex justify-center">
          <div className="w-full max-w-3xl rounded-3xl border-2 p-6" style={{ backgroundColor: '#f8f4f2', borderColor: '#f8f4f2' }}>
            <div className="text-sm mb-2 lowercase" style={{ color: '#462c2d' }}>
              new to step step learn?{' '}
              <Link href={`/play?songId=${RISE_AND_FALL_SONG_ID}`} className="underline">
                try a short example here
              </Link>
            </div>
            <div className="text-lg font-semibold mb-4 lowercase" style={{ color: '#f8f4f2' }}>
              generate a level
            </div>

            <label className="block text-sm mb-2 lowercase" style={{ color: '#462c2d' }}>
              prompt
            </label>
            <textarea
              value={prompt}

[truncated — 6555 more characters]
```

### frontend/app/play/page.tsx

```typescript
'use client';

import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { useRouter, useSearchParams } from 'next/navigation';
import { Scene } from '@/components/Scene';
import { PoseDetector } from '@/components/PoseDetector';
import { usePosePlayer } from '@/lib/pose-player';
import { usePoseChecker } from '@/lib/use-pose-checker';
import { GameHUD } from '@/components/GameHUD';
import { useGameStore } from '@/lib/game-store';
import type { ArrowSequence, GridPosition, HandCueSequence, PoseSequence } from '@/lib/types';

type PipelineResult = {
  lyrics?: {
    title?: string;
    lyrics?: string;
  };
  track?: {
    audioUrl?: string;
    audio_url?: string;
  };
  poses?: PoseSequence;
  poseSequence?: PoseSequence;
};

type SongDetailResponse = {
  song?: PipelineResult & {
    id: string;
    prompt?: string;
    title?: string;
    lengthSeconds?: number;
  };
};

function PosePlayer({ sequence, paused }: { sequence: PoseSequence; paused: boolean }) {
  usePosePlayer(sequence, paused);
  return null;
}

function PoseChecker({ sequence, paused }: { sequence: PoseSequence; paused: boolean }) {
  usePoseChecker(sequence, paused);
  return null;
}

function buildFootArrows(sequence: PoseSequence, foot: 'leftFoot' | 'rightFoot'): ArrowSequence {
  if (sequence.length === 0) return [];

  const arrows: ArrowSequence = [];
  let previous: GridPosition = 'M';

  for (const keyframe of sequence) {
    const direction = keyframe.pose[foot];
    if (direction !== previous && direction !== 'M') {
      arrows.push({ time: keyframe.time, direction });
    }
    previous = direction;
  }

  return arrows;
}

function normalizePoseSequence(sequence: PoseSequence): PoseSequence {
  return [...sequence]
    .filter((keyframe) => Number.isFinite(keyframe.time) && keyframe.time >= 0)
    .sort((a, b) => a.time - b.time);
}

function buildHandCues(sequence: PoseSequence): HandCueSequence {
  if (sequence.length === 0) return [];

  // Keep every timed instruction so this lane mirrors the exact hand sequence
  // driving the model, without dropping repeated or rapid changes.
  return sequence.map((keyframe) => ({
    time: keyframe.time,
    label: `L: ${keyframe.pose.leftHandShape} | R: ${keyframe.pose.rightHandShape}`,
  }));
}

function toLyricLines(rawLyrics?: string): string[] {
  if (!rawLyrics) return [];

  return rawLyrics
    .split('\n')
    .map((line) => line.trim())
    .filter((line) => line.length > 0 && !/^\[.*\]$/.test(line));
}

export default function Play() {
  const [paused, setPaused] = useState(false);
  const [secondsLeft, setSecondsLeft] = useState(0);
  const [showEnd, setShowEnd] = useState(false);
  const [playerName, setPlayerName] = useState('');
  const [savingScore, setSavingScore] = useState(false);
  const [loadingPipeline, setLoadingPipeline] = useState(true);
  const [pipelineError, setPipelineError] = useState<string | null>(null);
  const [sequence, setSequence] = useState<PoseSequence>([]);
  const [rightArrows, setRightArrows] = useState<ArrowSequence>([]);
  const [leftArrows, setLeftArrows] = useState<ArrowSequence>([]);
  const [handCues, setHandCues] = useState<HandCueSequence>([]);
  const [arrowDuration, setArrowDuration] = useState(1);
  const [lyrics, setLyrics] = useState<string[]>([]);
  const [songAudioUrl, setSongAudioUrl] = useState<string | null>(null);
  const [bgPulse, setBgPulse] = useState(0);
  const lyricsContainerRef = useRef<HTMLDivElement | null>(null);
  const lyricLineRefs = useRef<Array<HTMLDivElement | null>>([]);
  const audioRef = useRef<HTMLAudioElement | null>(null);
  const searchParams = useSearchParams();
  const router = useRouter();
  const score = useGameStore((s) => s.score);
  const currentTime = useGameStore((s) => s.currentTime);
  const songId = searchParams.get('songId');

  const currentLine = useMemo(() => {
    if (lyrics.length === 0) return 0;
    if (arrowDuration <= 0) return 0;
    const progress = Math.min(0.999, Math.max(0, currentTime / arrowDuration));
    return Math.min(lyrics.length - 1, Math.floor(progress * lyrics.length));
  }, [lyrics, currentTime, arrowDuration]);

  const setLyricLineRef = useCallback(
    (idx: number) => (el: HTMLDivElement | null) => {
      lyricLineRefs.current[idx] = el;
    },
    []
  );

  useEffect(() => {
    const container = lyricsContainerRef.current;
    const activeLine = lyricLineRefs.current[currentLine];
    if (!container || !activeLine) return;

    const top = activeLine.offsetTop - container.offsetTop;
    container.scrollTo({ top, behavior: 'smooth' });
  }, [currentLine]);

  useEffect(() => {
    const audio = audioRef.current;
    if (!audio) return;

    const shouldPlay = Boolean(songAudioUrl) && !paused && !loadingPipeline && !showEnd && !pipelineError;
    if (!shouldPlay) {
      audio.pause();
      return;
    }

    const playPromise = audio.play();
    if (playPromise && typeof playPromise.catch === 'function') {
      playPromise.catch(() => {
        // Autoplay may be blocked; we retry on the next user interaction below.
      });
    }
  }, [songAudioUrl, paused, loadingPipeline, showEnd, pipelineError]);

  useEffect(() => {
    if (!songAudioUrl || paused || loadingPipeline || showEnd || pipelineError) return;

    const tryResume = () => {
      const audio = audioRef.current;
      if (!audio) return;
      void audio.play().catch(() => {
        // Ignore; browser may still reject until a stronger gesture.
      });
    };

    window.addEventListener('pointerdown', tryResume, { passive: true });
    window.addEventListener('keydown', tryResume);
    window.addEventListener('touchstart', tryResume, { passive: true });

    return () => {
      window.removeEventListener('pointerdown', tryResume);
      window.removeEventListener('keydown', tryResume);
      window.removeEventListener('touchstart', tryResume);
    };
  }, [songAudioUrl, paused, loadingPipeline, showEnd, pipelineError]);

  useEffect(() => {
    
[truncated — 11363 more characters]
```

### frontend/app/api/songs/route.ts

```typescript
import { NextRequest, NextResponse } from 'next/server';
import { getBackendBaseUrl } from '@/lib/backend-url';

const BASE = getBackendBaseUrl();

export async function GET() {
  try {
    const res = await fetch(`${BASE}/api/songs`, { cache: 'no-store' });
    const text = await res.text();

    return new NextResponse(text, {
      status: res.status,
      headers: { 'Content-Type': res.headers.get('content-type') ?? 'application/json' },
    });
  } catch (error) {
    const message = error instanceof Error ? error.message : 'Proxy request failed';
    return NextResponse.json({ error: message }, { status: 502 });
  }
}

export async function POST(req: NextRequest) {
  try {
    const body = await req.text();
    const res = await fetch(`${BASE}/api/songs`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body,
      cache: 'no-store',
    });

    const text = await res.text();
    return new NextResponse(text, {
      status: res.status,
      headers: { 'Content-Type': res.headers.get('content-type') ?? 'application/json' },
    });
  } catch (error) {
    const message = error instanceof Error ? error.message : 'Proxy request failed';
    return NextResponse.json({ error: message }, { status: 502 });
  }
}

```

### frontend/app/level/[songId]/page.tsx

```typescript
'use client';

import { useEffect, useMemo, useState } from 'react';
import Link from 'next/link';
import { useParams } from 'next/navigation';

type SongDetail = {
  id: string;
  title?: string;
  prompt?: string;
  audienceDescriptor?: string;
  lengthSeconds?: number;
  lyrics?: {
    title?: string;
    lyrics?: string;
  };
};

type ScoreEntry = {
  id: string;
  name: string;
  score: number;
  createdAt?: string;
};

function formatDuration(seconds?: number): string {
  if (!seconds || Number.isNaN(seconds)) return '0:00';
  const total = Math.max(0, Math.floor(seconds));
  const mins = Math.floor(total / 60);
  const secs = total % 60;
  return `${mins}:${String(secs).padStart(2, '0')}`;
}

export default function LevelDetailPage() {
  const params = useParams<{ songId: string }>();
  const songId = params?.songId;

  const [song, setSong] = useState<SongDetail | null>(null);
  const [scores, setScores] = useState<ScoreEntry[]>([]);
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState<string | null>(null);

  useEffect(() => {
    let cancelled = false;

    async function load() {
      if (!songId) {
        setLoading(false);
        setError('Missing song id.');
        return;
      }

      setLoading(true);
      setError(null);

      try {
        const [songRes, leaderboardRes] = await Promise.all([
          fetch(`/api/songs/${songId}`, { cache: 'no-store' }),
          fetch(`/api/songs/${songId}/leaderboard?limit=20`, { cache: 'no-store' }),
        ]);

        if (!songRes.ok) {
          throw new Error(`Could not load level (${songRes.status}).`);
        }
        if (!leaderboardRes.ok) {
          throw new Error(`Could not load leaderboard (${leaderboardRes.status}).`);
        }

        const songJson = await songRes.json();
        const leaderboardJson = await leaderboardRes.json();

        if (!cancelled) {
          setSong(songJson?.song ?? null);
          setScores(Array.isArray(leaderboardJson?.scores) ? leaderboardJson.scores : []);
        }
      } catch (e) {
        if (!cancelled) {
          setError(e instanceof Error ? e.message : 'Failed to load level details.');
        }
      } finally {
        if (!cancelled) {
          setLoading(false);
        }
      }
    }

    void load();
    return () => {
      cancelled = true;
    };
  }, [songId]);

  const title = useMemo(() => (song?.title || 'untitled').toLowerCase(), [song?.title]);

  return (
    <div className="min-h-screen w-screen overflow-x-hidden overflow-y-auto px-10 md:px-16 lg:px-28 py-12 lowercase" style={{ background: '#5a3a3b' }}>
      <div className="max-w-5xl mx-auto">
        <div className="flex flex-wrap items-center justify-between gap-3 mb-8">
          <Link href="/" className="px-4 py-2 rounded-full border-2 text-xs" style={{ borderColor: '#f8f4f2', color: '#f8f4f2' }}>
            back to home
          </Link>
          {songId && (
            <Link href={`/play?songId=${songId}`} className="px-5 py-2 rounded-full text-sm font-semibold" style={{ backgroundColor: '#f8f4f2', color: '#462c2d' }}>
              play this level
            </Link>
          )}
        </div>

        {loading && (
          <div className="text-sm" style={{ color: '#f8f4f2', opacity: 0.9 }}>loading level details…</div>
        )}

        {!loading && error && (
          <div className="text-sm" style={{ color: '#f8f4f2' }}>{error}</div>
        )}

        {!loading && !error && song && (
          <div className="grid grid-cols-1 lg:grid-cols-3 gap-8">
            <div className="lg:col-span-2 rounded-2xl border-2 p-6" style={{ borderColor: '#462c2d', backgroundColor: '#f8f4f2' }}>
              <div className="text-xl font-semibold" style={{ color: '#462c2d' }}>{title}</div>
              <div className="text-xs mt-2" style={{ color: '#462c2d', opacity: 0.7 }}>
                length: {formatDuration(song.lengthSeconds)}
              </div>

              <div className="mt-6 text-sm font-semibold" style={{ color: '#462c2d' }}>
                prompt
              </div>
              <div className="mt-2 text-sm leading-relaxed" style={{ color: '#462c2d' }}>
                {song.prompt || 'No prompt saved for this level.'}
              </div>

              <div className="mt-6 text-sm font-semibold" style={{ color: '#462c2d' }}>
                audience notes
              </div>
              <div className="mt-2 text-sm leading-relaxed" style={{ color: '#462c2d' }}>
                {song.audienceDescriptor?.trim() || 'No audience notes were provided.'}
              </div>

              <div className="mt-6 text-sm font-semibold" style={{ color: '#462c2d' }}>
                lyrics
              </div>
              <div
                className="mt-2 text-sm leading-relaxed whitespace-pre-wrap rounded-lg border p-3 max-h-80 overflow-y-auto"
                style={{ color: '#462c2d', borderColor: '#462c2d22', backgroundColor: '#fff' }}
              >
                {song.lyrics?.lyrics?.trim() || 'No lyrics were saved for this level.'}
              </div>
            </div>

            <div className="rounded-2xl border-2 p-6" style={{ borderColor: '#462c2d', backgroundColor: '#f8f4f2' }}>
              <div className="text-sm font-semibold mb-4" style={{ color: '#462c2d' }}>
                leaderboard
              </div>

              {scores.length === 0 ? (
                <div className="text-xs" style={{ color: '#462c2d', opacity: 0.7 }}>
                  no scores yet. be the first to submit.
                </div>
              ) : (
                <div className="flex flex-col gap-2">
                  {scores.map((entry, idx) => (
                    <div key={entry.id} className="rounded-lg px-3 py-2 border" style={{ borderColor: '#462c2d22', backgroundColor: '#fff' }}>
                      <div className="text-xs font-semibold" style={{ color: '#462c2d' }}>
                        {idx + 1}. {entry.name}
       
[truncated — 355 more characters]
```

[50 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]