# Project export: ZoneOut

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: Zoned out during a lecture? Your very own lecture buddy is here to save you! With real-time Q&A based on audio, text & visual context from the lectures, you'll be acing your classes again in no time!
- Devpost: https://devpost.com/software/zoneout-atz5pe
- GitHub: https://github.com/NxtGenLegend/TreeHacks
- Video: https://www.youtube.com/embed/Be4lfvVzMRQ?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner (Zoom: Best Use of Zoom APIs ($250 Git Card + Herschel Duffle Bags [1st] & $100 Git Card + Hoodies [2nd] & $50 Git Card + Bottles [3rd]))
- Team: 1 GitHub contributor(s) — Yahia Salman (11 commits)

## Devpost submission (written by the team)

### Inspiration

We've all been in classes or meetings where we ZoneOut, even if for a few seconds, and came back to see that the cure to cancer has been invented! Inspired by attending online lectures this Friday (we're dedicated students), we found out that this isn't as uncommon as you'd think. As a matter of fact, the average retention rate of a student after just 45 mins of online learning is just 61%, which is HIGHER than the average person. Moreover, the average drop in engagement is 87% (scaling exponentially!) on average, in meetings with larger sets of participants.

### What it does

ZoneOut utilizes textual, visual & audio contexts from meetings, enabling our AI assistant to teach, revise & explain any concept, In-Depth & in Real-Time, to users to maintain higher levels of retention, productivity & engagement.

### How we built it

We built ZoneOut with a complex, yet execution directed architecture, developed completely on Windsurf. We used the Zoom API to connect the client to a Real-Time Media Server (RTMS) via a Handshake protocol, and consequently sampled data at microintervals as well as based on when a sentence/section of an idea being discussed finished. We collected textual data from the chat, audio data via live transcripts and visual screensharing/camera data with help from Zoom's API. We then used OpenAI for embedding the text & images with Chain-Of-Thought (CoT) Reasoning to keep context well-fitted and connected, independent of context window sizes and to keep images & text associated with one another. We also used parallel computation to allow us to index this data using Chroma concurrently, associating images with concepts in both audio & textual formats in different timestamps. Finally, we used a similarity search RAG system with ChromaDB for the audio/transcript & textual data, and a vision-based RAG system on ColPali (VLM), which we accelerated using a caching system that we developed, allowing us to use it without reloading it into memory again & again. The outputs of both RAG systems are then passed through OpenAI's API to format it nicely. We also optimized sampling parameters to avoid hallucinations caused by excess external information or misunderstanding contextual information. We then send this data back to the client, who's now back in the loop of everything that's happening!

### Challenges we ran into

Originally, the RTMS faced issues with streaming audio & video. After a lot of debugging & troubleshooting, we found & cured the error by handling edgecases through intensive vision programming back & forth, sending our sample code to the Zoom team so they can debug other teams. Then, our VLM workflow turned out to be too slow as the VLM was being loaded into memory repeatedly. So again, after coding a lot of reacharounds, we finally implemented our own caching system to supercharge our VLM, which now works with various forms of handwriting effectively. We also faced hallucinations wherein the model knew information it should not, and misinterpreted information it had. We cured this using indexing & CoT, to reach the product we have today!

### Accomplishments we're proud of

This hackathon has been a proud technical moment for all 3 of us. Our achievements stem from our challenges. We very quickly figured out the edge case of professors writing on whiteboards, both virtual & real, instead of explaining things like equations. So we developed a multi-language model workflow to work around. Another proud accomplishment was improving the Zoom RTMS repo, as we were the first people that figured it out, turning our curiosity into open source contributions in Zoom's repos. Next was integrating a complex parallel workflow to interpret & contextualize images, text & audio data altogether, particularly because of how LLMs & VLMs can be very funky sometimes. After that, was when we implemented our own caching system to boost our VLM system, after having faced a barrage of vision problems. Finally, was our creative use of prompt engineering, context windows & frontend-backend structuring for Windsurf to swap between entirely different frontend frameworks (HTML/CSS & React) & even simple backend worflows without breaking the frontend or the backend, letting us build very quickly, despite initial samples & software not being completely compatible, causing issues in the webSockets & handshake protocol, amongst other incompatibility issues.

## README (from the GitHub repository)


## Installation & setup

This app requires [FFmpeg](https://github.com/FFmpeg/FFmpeg) and [Node.js version 14]() or higher.

The app can be run locally by cloning and installing packages with npm or on [Docker](https://www.docker.com/).

**npm** <br/>
To setup with npm, install dependencies and run the app:

```bash
cd rtms-mock-server-sample

# Install dependencies
npm install

# Start the server
npm start
```

**Docker** <br/>
To setup with Docker, run the following:

```bash
cd rtms-mock-server-sample

# Option 1: Using docker-compose (recommended)
docker-compose up -d

# Option 2: Manual docker commands

# Build Docker image
docker build -t rtms-mock-server .

# Run the container
docker run -d \
  -p 9092:9092 \
  -p 8081:8081 \
  -v $(pwd)/data:/app/data \
  --name rtms-mock-server \
  rtms-mock-server

# View logs
docker logs -f rtms-mock-server
```

To stop the container:

```bash
docker stop rtms-mock-server
```

To restart the container:

```bash
docker start rtms-mock-server
```

## Using the sample client

Start the server (npm or Docker) and open the mock server at [http://localhost:9092](http://localhost:9092). The sample client at `./client` can now be used to consume media from the mock server.

In a new terminal, run the sample client:

```bash
node client/server.js
```

This opens up a server at `localhost:8000`. For webhook validation, the client will need to be exposed to the internet with a tunnel, like [ngrok](https://ngrok.com/).

```bash
ngrok http 8000
```

The ngrok URL will be used to validate the webhook endpoint. Copy your URL and paste it into the webhook URL field on the mock server (http://localhost:9092). Click validate. In the RTMS server and client you'll see confirmation of the validation.

You can now start a meeting and start streaming media to the client.

## To run the backend

Delete the current index from the indexes folder

run 

```bash
python3 reset_everything.py
```

comment out this line in the app.py: 
```bash
RAG = model_manager.get_model(device="mps")
```

and uncomment these

```bash
RAG = RAGMultiModalModel.from_pretrained(pretrained_model_name_or_path="/Users/yahiasalman/Desktop/RetainAll/RetainBackend/app/models/colqwen2-v1.0", index_root="./index", device="mps")
RAG.index(input_path="./saved_frame.jpg", index_name="TreeIndex", store_collection_with_index=True, overwrite=True)
```

and run 
```bash
python3 app.py
```

then you can run 

```bash
./start.sh
```

and the server should start running on port 8010

Make sure that you have redis and celery installed!




## Detected evidence (automated analysis)

Indexed codebase: 29 recognized source files, 162 KB.
- CSS (language) — detected in the code
- Express (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- OpenAI (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (68 of 68)

```
.dockerignore
.DS_Store
.gitignore
app.py
chroma_db/chroma.sqlite3
data/rtms_credentials.json
docker-compose.yml
Dockerfile
index/TreeIndex/embeddings/embeddings_0.pt
intertest.py
license.md
main.js
model_manager.py
package.json
public/css/styles.css
public/index.html
public/js/api.js
public/js/audio-processor.js
public/js/config.js
public/js/mediaHandler.js
public/js/uiController.js
public/js/webSocket.js
readme.md
recordings/KLhvT3WEBT6Srse3TgWRGs/audio.raw
recordings/KLhvT3WEBT6Srse3TgWRGs/metadata.jsonl
recordings/KLhvT3WEBT6Srse3TgWRGs/video.raw
recordings/PLhvT3WEBT6Srse3TgWRGt/audio.raw
recordings/PLhvT3WEBT6Srse3TgWRGt/metadata.jsonl
recordings/PLhvT3WEBT6Srse3TgWRGt/video.raw
recordings/QLhvT3WEBT6Srse3TgWRGu/audio.raw
recordings/QLhvT3WEBT6Srse3TgWRGu/metadata.jsonl
recordings/QLhvT3WEBT6Srse3TgWRGu/video.raw
recordings/RLhvT3WEBT6Srse3TgWRGv/audio.raw
recordings/RLhvT3WEBT6Srse3TgWRGv/metadata.jsonl
recordings/RLhvT3WEBT6Srse3TgWRGv/video.raw
recordings/SLhvT3WEBT6Srse3TgWRGw/audio.raw
recordings/SLhvT3WEBT6Srse3TgWRGw/metadata.jsonl
recordings/SLhvT3WEBT6Srse3TgWRGw/video.raw
recordings/TNhvT3WEBT6Srse3TgWRGr/audio.raw
recordings/TNhvT3WEBT6Srse3TgWRGr/metadata.jsonl
recordings/TNhvT3WEBT6Srse3TgWRGr/video.raw
recordings/ULhvT3WEBT6Srse3TgWRGx/audio.raw
recordings/ULhvT3WEBT6Srse3TgWRGx/metadata.jsonl
recordings/ULhvT3WEBT6Srse3TgWRGx/video.raw
recordings/VLhvT3WEBT6Srse3TgWRGy/audio.raw
recordings/VLhvT3WEBT6Srse3TgWRGy/metadata.jsonl
recordings/VLhvT3WEBT6Srse3TgWRGy/video.raw
recordings/WLhvT3WEBT6Srse3TgWRGz/audio.raw
recordings/WLhvT3WEBT6Srse3TgWRGz/metadata.jsonl
recordings/WLhvT3WEBT6Srse3TgWRGz/video.raw
recordings/XLhvT3WEBT6Srse3TgWRG0/audio.raw
recordings/XLhvT3WEBT6Srse3TgWRG0/metadata.jsonl
recordings/XLhvT3WEBT6Srse3TgWRG0/video.raw
reset_everything.py
server/config/serverConfig.js
server/constants/messageTypes.js
server/handlers/mediaHandler.js
server/handlers/signalingHandler.js
server/handlers/webhookHandler.js
server/handlers/wsHandler.js
server/setup/serverSetup.js
server/utils/credentialsManager.js
server/utils/mediaUtils.js
server/utils/wsUtils.js
start.sh
test_client/.env
test_client/client_readme.md
test_client/server.js
```

### Dependencies

- package.json: cors@^2.8.5, dotenv@^16.4.7, express@^4.17.1, fluent-ffmpeg@^2.1.3, multer@^1.4.5-lts.1, node-fetch@^2.6.1, ws@^8.2.3

### Recent commits (newest first)

- Update readme.md
- Update readme.md
- everything
- final
- bc
- final changes
- FINISHED I THINK
- I think final
- made correct code and changed how video is sent
- added proper indexing
- adding apis
- Initial commit
- Version 1.0 release
- fixed the keep alive issue
- minor updates
- minor updates
- added docker
- updated docker
- removed unused ports and added print statement to webhook validation for the client
- fixing the readme.md

## Key source files (fetched from GitHub, selected and truncated for size)

### license.md

```markdown
Copyright 2025 Zoom Communications, Inc

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

```

### test_client/client_readme.md

```markdown
# Zoom RTMS Client

A client to test the RTMS Mock Server. This client connects to the RTMS Mock Server and sends messages to it.

## Overview

This client:
- Listens for Zoom webhook events
- Handles RTMS connection initialization
- Manages WebSocket connections for both signaling and media data
- Processes real-time media streams from RTMS Mock Server

## Prerequisites

- Node.js (v14 or higher)
- npm
- A Zoom account with RTMS enabled
- RTMS credentials (stored in `data/rtms_credentials.json`)

## Setup

1. Install dependencies:
```

## Using ngrok for Webhook Testing

To receive webhooks from the RTMS Mock Server, your client needs to be publicly accessible. ngrok is a useful tool for this:

1. Install ngrok from [https://ngrok.com/download](https://ngrok.com/download)

2. Start ngrok on port 8000:
```bash
ngrok http 8000
```

3. Copy the generated HTTPS URL (e.g., `https://abc123.ngrok.io`)

4. Use this URL as your webhook endpoint in the RTMS Mock Server UI

Note: The ngrok URL changes each time you restart ngrok unless you have a paid account.

## How It Works

1. The client starts an Express server on port 8000 to receive Zoom webhooks
2. When a meeting (session) is started in RTMS Mock Server:
   - Mock server sends a webhook to the client
   - Client establishes a signaling WebSocket connection
   - After successful handshake, connects to media WebSocket
   - Begins receiving real-time media data



## Limitations

This is a minimal implementation focused on core functionality. For production use, consider adding:
- Data processing
```

### docker-compose.yml

```yaml
version: '3.8'

services:
  rtms-mock:
    build: .
    ports:
      - "9092:9092"  # Handshake server
      - "8081:8081"  # Media server
    volumes:
      - ./data:/usr/src/app/data
    environment:
      - NODE_ENV=production
    restart: unless-stopped 
```

### Dockerfile

```
# Use Node.js LTS version
FROM node:18-slim

# Install FFmpeg for media processing
RUN apt-get update && \
    apt-get install -y ffmpeg && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Create app directory
WORKDIR /usr/src/app

# Copy package files
COPY package*.json ./

# Install dependencies
RUN npm install

# Copy app source
COPY . .

# Create data directory
RUN mkdir -p data

# Expose ports
EXPOSE 9092 8081

# Start the application
CMD ["node", "main.js"] 
```

### package.json

```
{
  "name": "rtms-mock-server",
  "version": "1.0.0",
  "description": "RTMS Mock Server",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/zoom/rtms-mock-server-sample.git"
  },
  "bugs": {
    "url": "https://github.com/zoom/rtms-mock-server-sample/issues"
  },
  "homepage": "https://github.com/zoom/rtms-mock-server-sample#readme",
  "main": "main.js",
  "scripts": {
    "start": "node main.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "cors": "^2.8.5",
    "dotenv": "^16.4.7",
    "express": "^4.17.1",
    "fluent-ffmpeg": "^2.1.3",
    "multer": "^1.4.5-lts.1",
    "node-fetch": "^2.6.1",
    "ws": "^8.2.3"
  }
}

```

### main.js

```javascript
const ServerSetup = require('./server/setup/serverSetup');
const WSHandler = require('./server/handlers/wsHandler');
const MediaHandler = require('./server/handlers/mediaHandler');
const CONFIG = require('./server/config/serverConfig');
const express = require('express');
const webhookRouter = require("./server/handlers/webhookHandler");

// Initialize global state
global.isHandshakeServerActive = false;
global.mediaServer = null;
global.signalingWebsocket = null;
global.wss = null;
global.logsWss = null;

// Setup servers
const handshakeServer = ServerSetup.setupHandshakeServer();
const mediaHttpServer = ServerSetup.setupMediaServer();

// Setup WebSocket servers
global.wss = WSHandler.setupWebSocketServer(handshakeServer);
global.isHandshakeServerActive = true;

// Setup media server
function initializeMediaServer() {
    if (!global.mediaServer || global.mediaServer.isClosed) {
        global.mediaServer = MediaHandler.setupMediaServer(mediaHttpServer);
        console.log("Media server initialized");
    }
    return global.mediaServer;
}

// Initial media server setup
initializeMediaServer();

// Handle WebSocket upgrade requests
handshakeServer.on("upgrade", (request, socket, head) => {
    // Ensure media server is initialized before handling upgrade
    initializeMediaServer();
    WSHandler.handleUpgrade(request, socket, head);
});

// Add webhook router
const app = require('express')();
app.use("/", webhookRouter);

console.log("Starting WSS servers...");
```

### app.py

```python
from fastapi import FastAPI, UploadFile, File, Form, HTTPException, Request
from fastapi.middleware.cors import CORSMiddleware
import os
from datetime import datetime
import base64
import chromadb
import openai
import random
from byaldi import RAGMultiModalModel
from pydantic import BaseModel
import cv2
import time
import numpy as np
import celery
from celery import Celery
import redis
import requests
import json
import datetime
from model_manager import ModelManager

PERPLEXITY_API_KEY = "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"

model_manager = ModelManager()

#create a celery app
celery_app = Celery(
    "RetainBackendCelery", 
    broker="redis://localhost:6379/0", 
    backend="redis://localhost:6379/0"  
)

celery_app.conf.update(
    broker_connection_retry_on_startup=True,
    task_serializer="json",
    accept_content=["json"],
    result_serializer="json",
    timezone="UTC",
    enable_utc=True,
)

open_client = openai.OpenAI(api_key="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")

class VideoFrame(BaseModel):
    video_data: str
    timestamp: int

TIME_BETWEEN_FRAMES = 5
last_frame_timestamp = time.time()

# RAG = RAGMultiModalModel.from_pretrained(pretrained_model_name_or_path="/Users/yahiasalman/Desktop/RetainAll/RetainBackend/app/models/colqwen2-v1.0", index_root="./index", device="mps")
RAG = model_manager.get_model(device="mps")
# RAG.index(input_path="./saved_frame.jpg", index_name="TreeIndex", store_collection_with_index=True, overwrite=True)

seen = set()
# Connect to ChromaDB (runs locally)
client = chromadb.PersistentClient(path="./chroma_db")

# Create a collection
collection = client.get_or_create_collection(name="sentences")

app = FastAPI()

# Add CORS middleware to allow requests from your web app
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],  # In production, replace with your actual origin
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

def get_embedding(text):
    client = openai.OpenAI(api_key="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")
    response = client.embeddings.create(
        input=text,
        model="text-embedding-ada-002"
    )
    return response.data[0].embedding

def search_similar(query):
    query_vector = get_embedding(query)
    results = collection.query(query_embeddings=[query_vector], n_results=10)
    return [hit["text"] for hit in results["metadatas"][0]]

def delete_all_embeddings():
    client.delete_collection("sentences") 
    collection = client.get_or_create_collection(name="sentences")
    
def add_embedding(text):
    collection.add(ids=[str(len(seen))], embeddings=[get_embedding(text)], metadatas=[{"text": text}])

def initialize_index():
    RAG.index(input_path="./saved_frame.jpg", index_name="TreeIndex")

def search_vlm_index(query):
    return RAG.search(query=query, k=3)

async def search_transcript_index(query: str):
    return search_similar(query)

@celery_app.task
def add_to_index(file_path: str):
    try:
        # Initialize RAG model inside the worker process
        RAG = model_manager.get_model(device="mps")
        
        # Read the image and convert to base64
        with open(file_path, "rb") as image_file:
            image_data = image_file.read()
            base64_data = base64.b64encode(image_data).decode('utf-8')
        
        # Add to index with base64 in metadata
        RAG.add_to_index(
            input_item=file_path, 
            store_collection_with_index=True,
            # metadata={"base64": base64_data}
        )
        
        os.remove(file_path)
        return {"message": "Successfully added to index"}
    except Exception as e:
        print(f"Error adding to index: {str(e)}")
        os.remove(file_path)
        return {"error": str(e)}
    

# def get_chroma_results(chroma_input: str) -> str:
#     """Fetch search results from the Chroma API."""
#     chroma_input = chroma_input.replace(" ", "+")
#     url = f"http://localhost:8000/search_chroma?query={chroma_input}"
    
#     headers = {
#         "Content-Type": "application/json",
#         "Accept": "application/json"
#     }
    
#     try:
#         response = requests.post(url, headers=headers)
#         response.raise_for_status()  # Raise an error for bad status codes
        
#         result = response.json()
#         print("Search results:", result.get("data", []))
        
#         return ", ".join(result.get("data", []))
    
#     except requests.RequestException as e:
#         print("Failed to search Chroma:", str(e))
    
#     print(chroma_input)
#     return ""

def get_perplexity_response(query):
    url = "https://api.perplexity.ai/chat/completions"
    
    chroma_db_results = search_similar(query)
    headers = {
        "Authorization": f"Bearer {PERPLEXITY_API_KEY}",
        "Content-Type": "application/json"
    }
    
    content = []
    
    extended_content = [{"type": "text", "text": result} for result in chroma_db_results]
    
    content.extend(extended_content)
    
    vlm_results = search_vlm_index(query)
    extended_content_vlm = [{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{result.base64}"}} for result in vlm_results]
    content.extend(extended_content_vlm)
    
    content.extend([{"type": "text", "text": query}])
    
    response = open_client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": '''You are a helpful assistant for a zoom lecture. You are given a set of screenshots 
                from the lecture and sentences that the teacher has said. You are also 
                given a question from the user. You need to answer the question based on th
[truncated — 5982 more characters]
```

### test_client/server.js

```javascript
const express = require('express');
const crypto = require('crypto');
const WebSocket = require('ws');
const fs = require('fs');
const path = require('path');
const { spawn } = require('child_process');
const ffmpeg = require('fluent-ffmpeg');
require('dotenv').config({ path: path.join(__dirname, '.env') });

const app = express();
const PORT = 8000;

// Load environment variable
const BASE_URL = process.env.BASE_URL;

// Middleware to parse incoming JSON payloads
app.use(express.json());

// Configuration - Replace with actual values
const ZOOM_SECRET_TOKEN = 'DyBoLm8OZoJT2Pi3-kY2px'; // Webhook secret for validation
const CLIENT_SECRET = 'YZnKVUufg7N18Oej6gHHqNWc7CG5jQ6N'; // Secret key for generating HMAC signatures

// Add these constants at the top with other constants
const RECORDINGS_DIR = path.join(process.cwd(), 'recordings');
const H264_START_CODE = Buffer.from([0x00, 0x00, 0x00, 0x01]);
const DEFAULT_H264_SPS = Buffer.from([
    0x00, 0x00, 0x00, 0x01, 0x67, 0x42, 0x00, 0x2A,  // Updated SPS with 640x480
    0x95, 0xA8, 0x1E, 0x00, 0x89, 0xF9, 0x50, 0x00,
    0x00, 0x03, 0x00, 0x01, 0x00, 0x00, 0x03, 0x00,
    0x32, 0x8F, 0x16, 0x2E, 0x48
]);
const DEFAULT_H264_PPS = Buffer.from([
    0x00, 0x00, 0x00, 0x01, 0x68, 0xce, 0x06, 0xe2
]);

// Create recordings directory if it doesn't exist
if (!fs.existsSync(RECORDINGS_DIR)) {
    fs.mkdirSync(RECORDINGS_DIR);
}

/**
 * Function to generate HMAC signature
 * 
 * @param {string} clientId - The client ID of the RTMS application
 * @param {string} meetingUuid - The UUID of the Zoom meeting
 * @param {string} streamId - The RTMS stream ID
 * @param {string} secret - The secret key used for signing
 * @returns {string} HMAC SHA256 signature
 */
function generateSignature(clientId, meetingUuid, streamId, secret) {
    const message = `${clientId},${meetingUuid},${streamId}`;
    return crypto.createHmac("sha256", secret).update(message).digest("hex");
}

/**
 * Webhook endpoint to receive events from Zoom
 */
app.post('/', (req, res) => {
    console.log('Received request:', JSON.stringify(req.body, null, 2));

    const { event, payload } = req.body;

    // Handle Zoom Webhook Endpoint Validation
    if (event === 'endpoint.url_validation' && payload?.plainToken) {
        console.log('Processing Zoom endpoint validation...');
        const hashForValidate = crypto.createHmac('sha256', ZOOM_SECRET_TOKEN)
            .update(payload.plainToken)
            .digest('hex');

        console.log(`Validation response:`, {
            plainToken: payload.plainToken,
            encryptedToken: hashForValidate
        });

        return res.json({
            plainToken: payload.plainToken,
            encryptedToken: hashForValidate
        });
    }

    // Handle RTMS Event when a meeting starts streaming
    if (payload?.event === 'meeting.rtms.started' && payload?.payload?.object) {
        console.log('Processing RTMS Event: meeting.rtms.started');

        try {
            const {
                clientId,
                payload: {
                    event: rtmsEvent, // Extract event name
                    payload: {
                        operator_id,
                        object: { meeting_uuid, rtms_stream_id, server_urls }
                    }
                }
            } = req.body;

            console.log('Extracted RTMS Data:', {
                rtmsEvent,
                clientId,
                meeting_uuid,
                rtms_stream_id,
                server_urls
            });

            // Establish WebSocket connection with RTMS signaling server
            connectToRTMSWebSocket(clientId, meeting_uuid, rtms_stream_id, server_urls);
        } catch (error) {
            console.error('Error processing RTMS event:', error);
        }
    } 
    // Log other Zoom events for debugging
    else if (event) {
        console.log(`Processing Zoom event: ${event}`);
    } 
    // Handle unknown event types
    else {
        console.log("Received an event but couldn't determine the type.");
    }

    res.sendStatus(200);
});

/**
 * Connects to the RTMS signaling WebSocket server
 * 
 * @param {string} clientId - The client ID
 * @param {string} meetingUuid - The meeting UUID
 * @param {string} streamId - The RTMS stream ID
 * @param {string} serverUrl - WebSocket URL for signaling server
 */
function connectToRTMSWebSocket(clientId, meetingUuid, streamId, serverUrl) {
    console.log(`Connecting to RTMS WebSocket server: ${serverUrl}`);

    const ws = new WebSocket(serverUrl, { rejectUnauthorized: false });

    // Set a timeout to prevent hanging if the connection is unresponsive
    const connectionTimeout = setTimeout(() => {
        console.error('Connection to WebSocket server timed out.');
        process.exit(1);
    }, 10000); // 10 seconds timeout

    ws.on("open", () => {
        clearTimeout(connectionTimeout);
        console.log("Connected to WebSocket server");

        // Periodically log connection status
        const connectionCheckInterval = setInterval(() => {
            console.log("Still connected...");
        }, 20000);

        // Generate authentication signature
        const signature = generateSignature(clientId, meetingUuid, streamId, CLIENT_SECRET);

        // Prepare handshake message for signaling server
        const handshakeMessage = {
            msg_type: "SIGNALING_HAND_SHAKE_REQ",
            protocol_version: 1,
            meeting_uuid: meetingUuid,
            rtms_stream_id: streamId,
            signature: signature
        };

        console.log("Sending handshake message:", JSON.stringify(handshakeMessage, null, 2));
        ws.send(JSON.stringify(handshakeMessage));

        // Handle WebSocket closure
        ws.on("close", (code, reason) => {
            clearInterval(connectionCheckInterval);
            console.log("RTMS WebSocket closed:", code, reason.toString());
            if (code === 1005) {
                console.log("Closing RTMS WebSocket 
[truncated — 14833 more characters]
```

### reset_everything.py

```python
from app import delete_all_embeddings

if __name__ == "__main__":
    delete_all_embeddings()
```

### start.sh

```shell
#!/bin/bash

# Check if Redis is running
if ! redis-cli ping > /dev/null 2>&1; then
    # Start Redis only if it's not running
    echo "Starting Redis..."
    redis-server --save "" --appendonly no &
    sleep 5
else
    echo "Redis is already running..."
fi

# Start Celery in the background
echo "Starting Celery worker..."
celery -A app:celery_app worker --loglevel=info &

# Start FastAPI
echo "Starting FastAPI server..."
uvicorn app:app --reload --port 8010 --host 0.0.0.0
```

[20 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]