# Project export: Stream Director

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: UC Berkeley AI Hackathon 2026
- Tagline: An AI co-director that watches your livestream in real time and flashes on-screen cues, reactions, scene changes, even catch-up summaries after ad breaks (only when something's actually worth it).
- Devpost: https://devpost.com/software/stream-director
- GitHub: https://github.com/anushharish1/stream-director
- Video: https://www.youtube.com/embed/QqgOVRGyGv4?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 0 GitHub contributor(s) — 

## Devpost submission (written by the team)

### Inspiration

I'm building a creator brand ("aycee") where my life plays out like a TV show. The problem: when you're solo streaming, there's no director calling the beats, hyping moments, or framing chaos in real time. So I built one.

### What it does

Stream Director listens to a streamer's mic live and decides, in real time, whether to flash an on-screen director cue such as a quick reaction, a big "episode" title card for scene changes, or a "while you were away" catch-up summary after an ad break. It defaults to silence and only fires when something genuinely notable happens, so cues feel earned instead of spammy. How I built it Deepgram streams live speech-to-text from the mic A rolling 60-second transcript buffer feeds into Claude every 15 seconds Claude decides between three response types: a quick REACTION cue, a bigger SCENE/episode title card, or (after a simulated ad break) a CATCHUP summary A lightweight Express server exposes the latest cue via a polling endpoint A branded HTML/CSS overlay (designed for OBS browser sources) animates the cues in Challenges I ran into The hardest part was tuning the AI to know when to stay silent. A naive version fires a cue on every beat, which is noisy and useless. I built in a hard cooldown plus an explicit "default to silence" instruction so the system only reacts to things that actually matter, which took several rounds of live testing and prompt tuning to get right. Accomplishments that I'm proud of Getting the AI to actually know when to stay quiet. Most reactive AI demos fire constantly and feel gimmicky. Getting Stream Director to sit silent through minutes of filler talk and only fire when something genuinely happened took real iteration, and seeing it correctly catch a topic change or a joke landing live, on the first real test, felt like a genuine "it works" moment. I'm also proud that I built three distinct, working cue types (REACTION, SCENE, CATCHUP) solo in one weekend, each requiring its own reasoning logic and visual treatment, and got the entire pipeline — mic to transcript to AI decision to live overlay — running end-to-end, multiple times, with no manual triggers. What I learned The technical integration (Deepgram, Claude, Express) was the easy part. The real lesson was that prompting an LLM to make a judgment call, deciding when NOT to act, is a fundamentally different and harder problem than prompting it to generate content. Cooldowns, explicit "default to silence" instructions, and a lot of live testing were what actually made the system feel intelligent rather than noisy. I also learned a lot about real-time audio pipelines — buffering live transcripts, managing polling-based UI updates, and debugging issues that only show up when multiple async pieces (mic input, API calls, browser rendering) are running simultaneously.

### What's next

Hooking into real Twitch chat and ad-break APIs instead of the simulated versions used in this demo, and running it live on my own channel.

## README (from the GitHub repository)

No README available.

## Detected evidence (automated analysis)

Indexed codebase: 2 recognized source files, 11 KB.
- Anthropic (technology) — detected in the code
- Express (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Node.js (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (5 of 5)

```
.DS_Store
.gitignore
index.js
package.json
public/overlay.html
```

### Dependencies

- package.json: @anthropic-ai/sdk@^0.105.0, @deepgram/sdk@^5.4.0, dotenv@^17.4.2, express@^5.2.1, mic@^2.1.2

### Recent commits (newest first)

- Stream Director - AI Hackathon 2026

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "name": "stream-director",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "type": "commonjs",
  "dependencies": {
    "@anthropic-ai/sdk": "^0.105.0",
    "@deepgram/sdk": "^5.4.0",
    "dotenv": "^17.4.2",
    "express": "^5.2.1",
    "mic": "^2.1.2"
  }
}

```

### index.js

```javascript
require('dotenv').config();
const { DeepgramClient } = require('@deepgram/sdk');
const Anthropic = require('@anthropic-ai/sdk');
const mic = require('mic');
const express = require('express');

let transcriptBuffer = [];

function addToBuffer(text) {
  const now = Date.now();

  if (adBreakActive) {

    adBreakBuffer.push(text);

    return; 

  }

  transcriptBuffer.push({ text, timestamp: now });
  const cutoff = now - 60000;
  transcriptBuffer = transcriptBuffer.filter(entry => entry.timestamp >= cutoff);
}

function getBufferText() {
  return transcriptBuffer.map(entry => entry.text).join(' ');
}

let latestCue = { cue: null, timestamp: 0 };

const app = express();
app.use(express.static('public'));
app.get('/latest-cue', (req, res) => {
  res.json(latestCue);
});
app.listen(3000, () => {
  console.log('🖥️  Overlay running at http://localhost:3000/overlay.html');
});

const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

let lastCueTime = 0;
let lastCueText = null;
let adBreakActive = false;
let adBreakBuffer = [];

const CUE_SYSTEM_PROMPT = `You are a live TV director watching a stream's transcript in real time.
Your job is to decide whether to flash an on-screen cue, and if so, what kind.

You will receive the last ~30-60 seconds of transcript, plus the most
recent cue you fired (if any) and how long ago it fired.

DEFAULT TO SILENCE. Only fire a cue when something genuinely notable
just happened.

There are two types of cues:
- REACTION: a quick directorial reaction to a beat — a joke landing, energy
  shifting, a funny moment. Under 6 words. Optionally include one emoji.
- SCENE: a bigger moment — a clear topic change, a new "chapter" starting,
  a dramatic turn. Styled like a TV episode title card. Format as
  "EPISODE: <short title>" — under 5 words after "EPISODE:".

Rules:
- Never fire two cues within 45 seconds of each other.
- Never fire a cue that just restates what was said.
- SCENE cues should be rarer than REACTION cues — only for real shifts,
  not every topic mention.
- If nothing notable happened, respond with exactly: NONE

Respond in exactly one of these formats:
REACTION: <text>
SCENE: EPISODE: <text>
or
NONE`;

async function maybeGenerateCue() {
  const now = Date.now();
  const secondsSinceLastCue = (now - lastCueTime) / 1000;

  if (secondsSinceLastCue < 45) {
    return; // still in cooldown, skip the API call entirely
  }

  const bufferText = getBufferText();
  if (!bufferText || bufferText.trim().length === 0) {
    return;
  }

  const userMessage = `[Last ~60s of transcript]: "${bufferText}"

Last cue fired: ${lastCueText ? `"${lastCueText}" (${Math.round(secondsSinceLastCue)} seconds ago)` : 'none yet'}`;

  try {
    const response = await anthropic.messages.create({
      model: 'claude-sonnet-4-6',
      max_tokens: 50,
      system: CUE_SYSTEM_PROMPT,
      messages: [{ role: 'user', content: userMessage }],
    });

    const reply = response.content[0].text.trim();

    if (reply.startsWith('REACTION:')) {
      const cueText = reply.replace('REACTION:', '').trim();
      console.log('\n🎬 REACTION FIRED:', cueText, '\n');
      lastCueTime = now;
      lastCueText = cueText;
      latestCue = { cue: cueText, type: 'reaction', timestamp: now };
    } else if (reply.startsWith('SCENE:')) {
      const cueText = reply.replace('SCENE:', '').trim();
      console.log('\n🎭 SCENE FIRED:', cueText, '\n');
      lastCueTime = now;
      lastCueText = cueText;
      latestCue = { cue: cueText, type: 'scene', timestamp: now };
    }

  } catch (err) {
    console.error('Claude error:', err);
  }
}

async function generateCatchupCue() {
  const summary = adBreakBuffer.join(' ');
  if (!summary || summary.trim().length === 0) {
    console.log('(nothing happened during ad break)');
    return;
  }

  const prompt = `While the viewer was watching an ad, here's everything that happened on stream: "${summary}"

Summarize this in one punchy "while you were away" catch-up line for viewers coming back. Under 8 words. Format as:
CATCHUP: <text>`;

  try {
    const response = await anthropic.messages.create({
      model: 'claude-sonnet-4-6',
      max_tokens: 50,
      messages: [{ role: 'user', content: prompt }],
    });

    const reply = response.content[0].text.trim();
    if (reply.startsWith('CATCHUP:')) {
      const cueText = reply.replace('CATCHUP:', '').trim();
      console.log('\n📺 CATCHUP FIRED:', cueText, '\n');
      latestCue = { cue: cueText, type: 'catchup', timestamp: Date.now() };
    }
  } catch (err) {
    console.error('Catchup error:', err);
  }

  adBreakBuffer = [];
}

async function main() {
  const client = new DeepgramClient({ apiKey: process.env.DEEPGRAM_API_KEY });

  const connection = await client.listen.v1.connect({
    model: 'nova-3',
    language: 'en',
    punctuate: 'true',
    smart_format: 'true',
    interim_results: 'true',
    encoding: 'linear16',
    sample_rate: '16000',
    channels: '1',
  });

  connection.on('open', () => {
    console.log('✅ Connected to Deepgram. Start talking...\n');

    const micInstance = mic({
      rate: '16000',
      channels: '1',
      debug: false,
      exitOnSilence: 6,
    });

    const micInputStream = micInstance.getAudioStream();

    micInputStream.on('data', (data) => {
      connection.socket.send(data);
    });

    micInputStream.on('error', (err) => {
      console.error('Mic error:', err);
    });

    micInstance.start();
  });

connection.on('message', (data) => {
  if (data.type === 'Results') {
    const transcript = data.channel?.alternatives?.[0]?.transcript;
    const isFinal = data.is_final;
    if (transcript && transcript.length > 0 && isFinal) {
      console.log('📝', transcript);
      addToBuffer(transcript);
    }
  }
});

  connection.on('error', (err) => {
    console.error('Deepgram error:', err);
  });

  connection.on('close', () => {
    console.log('Connection closed.');
  });

  setInterval(() => {
    console.log('\n🪟 B
[truncated — 546 more characters]
```

### public/overlay.html

```html
<!DOCTYPE html>
<html>
<head>
  <style>
    @import url('https://fonts.googleapis.com/css2?family=Bebas+Neue&family=Inter:wght@600&display=swap');

    body {
      margin: 0;
      background: transparent;
      overflow: hidden;
      font-family: 'Inter', sans-serif;
    }

    #cue-wrapper {
      position: absolute;
      bottom: 70px;
      left: 50%;
      transform: translateX(-50%) translateY(20px);
      opacity: 0;
      transition: opacity 0.4s ease, transform 0.4s ease;
      text-align: center;
    }

    #cue-wrapper.show {
      opacity: 1;
      transform: translateX(-50%) translateY(0px);
    }

    #cue-label {
      font-family: 'Bebas Neue', sans-serif;
      font-size: 22px;
      letter-spacing: 6px;
      color: #ffcc00;
      text-align: center;
      margin-bottom: 8px;
      text-shadow: 0 0 8px rgba(255, 204, 0, 0.6), 0 2px 4px rgba(0,0,0,0.8);
    }

    #cue {
      background: rgba(10, 10, 10, 0.85);
      border-left: 4px solid #ffcc00;
      padding: 14px 28px;
      font-size: 32px;
      font-weight: 600;
      color: white;
      white-space: nowrap;
      border-radius: 4px;
      box-shadow: 0 4px 20px rgba(0,0,0,0.4);
    }

    #scene-card {
      position: absolute;
      top: 40%;
      left: 50%;
      transform: translate(-50%, -50%) scale(0.9);
      opacity: 0;
      transition: opacity 0.5s ease, transform 0.5s ease;
      text-align: center;
    }


    #scene-card.show {
      opacity: 1;
      transform: translate(-50%, -50%) scale(1);
    }

    #scene-label {
      font-family: 'Bebas Neue', sans-serif;
      font-size: 16px;
      letter-spacing: 6px;
      color: #ffcc00;
      margin-bottom: 6px;
    }

    #scene-title {
      font-family: 'Bebas Neue', sans-serif;
      font-size: 56px;
      letter-spacing: 3px;
      color: white;
      text-shadow: 0 0 20px rgba(0,0,0,0.9), 0 4px 8px rgba(0,0,0,0.8);
    }

    #catchup-card {
  position: absolute;
  top: 20px;
  left: 50%;
  transform: translateX(-50%) translateY(-100px);
  opacity: 0;
  transition: opacity 0.5s ease, transform 0.5s ease;
  text-align: center;
}

#catchup-card.show {
  opacity: 1;
  transform: translateX(-50%) translateY(0px);
}

#catchup-label {
  font-family: 'Bebas Neue', sans-serif;
  font-size: 16px;
  letter-spacing: 5px;
  color: #0a0a0a;
  background: #ffcc00;
  display: inline-block;
  padding: 5px 18px;
  border-radius: 3px 3px 0 0;
}

#catchup-text {
  background: rgba(10, 10, 10, 0.9);
  color: white;
  font-family: 'Inter', sans-serif;
  font-size: 26px;
  font-weight: 600;
  padding: 14px 32px;
  border-radius: 0 0 6px 6px;
  box-shadow: 0 4px 20px rgba(0,0,0,0.5);
  max-width: 90vw;
  word-wrap: break-word;
}
  </style>
</head>
<body>
  <div id="cue-wrapper">
    <div id="cue-label">AYCEE • LIVE</div>
    <div id="cue"></div>
  </div>

  <div id="scene-card">
    <div id="scene-label">EPISODE</div>
    <div id="scene-title"></div>
  </div>

  <div id="catchup-card">
    <div id="catchup-label">WHILE YOU WERE AWAY</div>
    <div id="catchup-text"></div>
  </div>

  <script>
    const wrapper = document.getElementById('cue-wrapper');
    const cueEl = document.getElementById('cue');

    async function poll() {
  try {
    const res = await fetch('/latest-cue');
    const data = await res.json();
    if (data.cue && data.timestamp !== window.lastShown) {
      window.lastShown = data.timestamp;

      if (data.type === 'scene') {
        const titleText = data.cue.replace(/^EPISODE:\s*/i, '');
        document.getElementById('scene-title').textContent = titleText;
        document.getElementById('scene-card').classList.add('show');
        setTimeout(() => {
          document.getElementById('scene-card').classList.remove('show');
        }, 4500);
      } else if (data.type === 'catchup') {
        document.getElementById('catchup-text').textContent = data.cue;
        document.getElementById('catchup-card').classList.add('show');
        setTimeout(() => {
          document.getElementById('catchup-card').classList.remove('show');
        }, 6000);
      } else {
        cueEl.textContent = data.cue;
        wrapper.classList.add('show');
        setTimeout(() => wrapper.classList.remove('show'), 4000);
      }
    }
  } catch (err) {
    console.error('Poll error:', err);
  }
}

    setInterval(poll, 2000);
  </script>
</body>
</html>

```