Project Info

ClipStudy.ai

Inspiration

Many students struggle to efficiently review specific concepts in long lecture recordings. Even with YouTube’s auto-generated chapter labels, users still have to manually scroll through these sections, and these labels often miss hyperspecific or nuanced topics, making it hard to quickly access exactly what’s needed.

What it does

ClipStudy allows users to paste in a YouTube video link and use natural language to ask for the specific concept they want to review. Our backend then analyzes the video transcript, finds the relevant timestamps, and automatically jumps to that point in our embedded video.

How we built it

We built ClipStudy.ai by developing a local script to fetch and clean YouTube video transcripts using yt-dlp and custom SRT parsing logic. Once the transcript is processed, we leverage Claude 4 Sonnet to analyze the transcript and identify segments relevant to a user’s query, using a prompt-based approach for granular concept detection. The backend coordinates transcript extraction and segment analysis, while the frontend presents an interface for users to submit links and review the identified video segments. All intermediate data (transcripts and segment metadata) are managed locally for efficiency and privacy.

Challenges we ran into

Curating and cleaning diverse YouTube transcripts with varying quality and formats Fine-tuning prompt engineering with Claude 4 Sonnet to accurately detect nuanced educational concepts Balancing fast response times with thorough transcript analysis for a smooth user experience

Accomplishments we're proud of

Successfully automated the end-to-end process from transcript extraction to concept-based video navigation Developed a robust transcript cleaning pipeline that handles a wide range of YouTube lecture formats Achieved accurate, AI-powered detection of nuanced concepts and seamless video segment jumping Built a privacy-focused system that processes and stores all user data locally

What we learned

The power of large language models for granular concept detection in long-form educational videos The value of prompt engineering for improving AI accuracy and relevance

What's next

Expand support to additional video platforms beyond YouTube, such as Vimeo, Zoom recordings, and educational portals. Integrate with popular notetaking and productivity apps such as Notion. Add support for YouTube playlists and video-searching inside the system (backend code exists).

ClipStudy

Paste a YouTube lecture URL, search for a concept, and jump straight to the relevant moments in the video.

ClipStudy downloads English auto-captions with yt-dlp, cleans overlapping subtitle text, then uses Anthropic Claude to return timestamped segments (title, summary, relevance score). The Next.js UI embeds the YouTube player and seeks to those segments on click.

Collaborative project (multiple contributors in git history). Status: working prototype / demo — not a production SaaS.

How it works

YouTube URL + search query
        │
        ▼
Next.js UI  ──/api/* proxy──►  Flask (port 3001)
                                    │
                    ┌───────────────┼───────────────┐
                    ▼               ▼               ▼
              yt-dlp auto-subs   SRT clean/merge   Claude segment JSON
                    │                               │
                    └──────────► UI: player + transcript + clickable segments

User submits a YouTube URL and a concept query (e.g. "matrix multiplication").
Backend fetches English auto-generated captions via yt-dlp.
Captions are cleaned (overlap removal / sentence merge).
Claude returns ranked segments with start, end, title, summary, relevance_score.
Frontend shows the video, synced transcript, and segment cards that seekTo the YouTube player.

Features

Concept search over a single YouTube video via captions + LLM
Timestamped, ranked segment results
Click a segment to seek the embedded player
Live transcript panel that highlights / scrolls with playback
Optional playlist analysis script under backend/test/ (experimental; not wired into the UI)
Local Docker Compose setup for backend + frontend

Stack

Layer	Tech
Frontend	Next.js 15, React 19, Radix UI Themes, Tailwind CSS 4, `react-youtube`
Backend	Python 3.11, Flask, Flask-CORS
Captions	`yt-dlp` (English auto-subs → SRT)
Analysis	Anthropic Claude (`claude-3-5-sonnet-20241022`)
Deploy helpers	Docker Compose, `deploy.sh`

Prerequisites

Python 3.11+
Node.js 18+ and pnpm
yt-dlp on PATH for local (non-Docker) runs
Anthropic API key

Setup

git clone git@github.com:nependyala/ClipStudy.git
cd ClipStudy
cp .env.example .env
# Edit .env and set ANTHROPIC_API_KEY=...

Backend

cd backend
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt
python app.py               # http://127.0.0.1:3001

Frontend

cd frontend
pnpm install
pnpm dev                    # http://localhost:3000

The Next.js config proxies /api/* to the Flask server (BACKEND_URL, default http://localhost:3001).

One-shot local script

./deploy.sh

Docker

export ANTHROPIC_API_KEY=your_anthropic_api_key_here
docker compose up --build -d
# App: http://localhost:3000  API: http://localhost:3001

On non-Apple Silicon machines, remove or change platform: linux/arm64 in docker-compose.yaml.

Configuration

Variable	Where	Purpose
`ANTHROPIC_API_KEY`	`.env` / compose	Required for segment analysis
`BACKEND_URL`	frontend / Docker	Flask base URL for Next.js rewrites
`FLASK_HOST` / `FLASK_PORT`	backend	Bind address (default `0.0.0.0:3001`)
`FLASK_DEBUG`	backend	Flask debug mode

API

Endpoint	Description
`GET /api/hello`	Health check
`GET /api/get/<video_id>?prompt=<query>`	Fetch captions, analyze, return transcript + segments
`GET /api/info/<video_id>`	Whether a cached transcript exists on disk

Example:

curl "http://127.0.0.1:3001/api/get/rfG8ce4nNh0?prompt=matrix%20multiplication"

Response includes transcript, segments[] (start, end, title, summary, relevance_score), and total_segments.

CLI (without Flask):

cd backend/transcript_extraction
python decide_clip.py "https://www.youtube.com/watch?v=VIDEO_ID" "your concept query"

Project structure

ClipStudy/
├── backend/
│   ├── app.py                      # Flask API
│   ├── requirements.txt
│   ├── transcript_extraction/
│   │   ├── transcript_fetch.py     # yt-dlp + SRT cleaning
│   │   ├── decide_clip.py          # Claude segment analysis
│   │   └── temporary_files/        # Runtime cache (gitignored)
│   └── test/analyze_playlist.py    # Experimental playlist script
├── frontend/                       # Next.js app (src/app/page.jsx)
├── docs/ARCHITECTURE.md
├── docker-compose.yaml
├── deploy.sh
└── .env.example

Current limitations

English auto-captions only; videos without them will fail
Single-video UI flow (playlist script is separate / experimental)
File-based transcript/segment cache — no database or auth
Analysis quality depends on caption quality and the LLM
Collaborative history includes an older project name ("SmartLLMs") in some docs/commits

Security note

Do not commit API keys. If a key was ever committed in git history, rotate it in the Anthropic console immediately.

License

No license file is present; treat as private/personal unless you add one.

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

CSSIn code
JavaScriptIn code
Next.jsIn code
PythonIn code
ReactIn code
Tailwind CSSIn code
TypeScriptIn code
FlaskClaimed

7 of 8 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

293 KB

Source files

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.