# Project export: focusr ai

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: Humans lose focus; we bring it back. Sit back, relax, and watch your lecture while focusr.ai catches everything you missed.
- Devpost: https://devpost.com/software/focusr-ai
- GitHub: https://github.com/isabellasocci/focusr-ai
- Demo: https://focusr-ai.vercel.app/
- Video: https://www.youtube.com/embed/4JakhQYLe2g?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 3 GitHub contributor(s) — Isabella Socci (14 commits), Ishan Chawla (5 commits), Tyler Emanuel (2 commits)

## Devpost submission (written by the team)

### Inspiration

Staying engaged during a lecture isn’t always easy. A brief lapse in focus—whether due to distractions, fatigue, or simply a wandering mind—can mean missing a key concept, making it harder to follow along with the rest of the lesson. In fast-paced courses, those small moments of disengagement add up, leaving students struggling to catch up. For those with learning disabilities, this is even more so the case. We built focusr.ai to help bridge that gap. By tracking engagement in real-time through eye movement, our tool identifies when a student zones out and ensures that missed topics get extra emphasis in the AI-generated summary. Instead of rewatching an entire lecture or sifting through lengthy PowerPoints, our users can quickly review exactly what they need to stay on track. This project isn’t about forcing focus—it’s about helping students learn more effectively, even when attention slips.

### How we built it

We built focusr.ai as a web application that tracks engagement, transcribes lectures, and generates AI-powered summaries to help students stay on top of their learning. The frontend uses React and Tailwind CSS, while the backend, built with Flask, handles audio processing, AI interactions, and engagement tracking. To detect focus lapses, we use Gemini’s multimodal processing, analyzing eye movement patterns. OpenAI Whisper transcribes lectures, and Perplexity AI creates context-aware summaries, prioritizing missed sections. An interactive engagement chart visualizes attention levels, helping students quickly review key moments. Hosted on Vercel with cloud/local storage, focusr.ai ensures that distractions don’t become setbacks, making learning more adaptive and accessible.

### Challenges we ran into

A major challenge we faced was engagement tracking for long lectures. Using OpenCV and MediaPipe for eye-tracking was too slow for extended recordings. We switched to Gemini and used images rather than videos, which efficiently analyzed video input directly, eliminating the need for uploading unnecessarily large files. This drastically improved speed and accuracy, making real-time engagement tracking viable for full-length lectures.

### Accomplishments we're proud of

We as a team all contributed to this project, and we are thrilled at the intuitive way in which the UI/UX has come together. We’re proud of how we streamlined the entire learning workflow—instead of just transcribing audio, our system prioritizes the moments when a student loses focus, ensuring they can efficiently catch up without rewatching everything. We originally thought about going with another project idea because we thought we wouldn't be able to learn and finish everything on time for focusr.ai, but we successfully brought it all together into a working product that makes reviewing lectures more effective.

### What we learned

All team members gained a much deeper understanding of the languages, libraries, and implementation techniques in their respective area of building focusr. The members: Ishan - frontend, Tyler - backend (engagement analysis), and Isabella - backend (transcript generation); all leaped into React.js, Flask, Git/Github, as well as component integration. Every member of our team worked with a language, technology, or library they hadn't previously worked with before, so we're proud to have learned while creating something we're all proud of.

### What's next

for focusr.ai? We see focusr.ai expanding its impact on both students and educators. From a teacher’s perspective, implementing this system across multiple students in a class could provide valuable insights into engagement trends. If a significant percentage of students consistently lose focus at the same point in a lecture, instructors could identify and refine less engaging sections to improve overall learning outcomes. For students, we want to move beyond video uploads and explore live engagement tracking during lectures. This could allow real-time feedback, helping students refocus before they fall too far behind. Expanding in this direction also opens the door to making focusr.ai a Zoom app, extending its use beyond in-person lectures to online meetings, virtual classrooms, and remote work settings, making it a more versatile engagement tool.

## README (from the GitHub repository)

## **focusr.ai** 
humans lose focus; we bring it back

## **Overview**
focusr.ai is a **web app** designed to track student engagement during lectures. It allows students to **record or upload lecture videos**, analyzes their **engagement levels using eye-tracking software**, and generates **AI-enhanced lecture summaries** that emphasize sections where the student was distracted.

## **Features**
- **Upload or Record Lecture Audio/Video**: Supports **MP4, WAV, M4A, and WebM** file formats.
- **Eye Tracking for Engagement Analysis**: Uses **Google Gemini** to detect eye movement and identify disengagement.
- **AI-Powered Summarization**: Integrates **OpenAI Whisper** and **Perplexity Sonar** to generate detailed lecture summaries.
- **Visualized Engagement Data**: Provides an **interactive chart** displaying engagement trends over time.
- **Web-Based Interface**: Built with **React & Flask** for seamless user experience.

## **Tech Stack**
### **Frontend (React + Tailwind CSS)**
- React.js (UI Components)
- React Router (Navigation)
- Chart.js (Engagement Visualization)
- Tailwind CSS (Styling)

### **Backend (Flask API + Python)**
- Flask (REST API for file processing & AI interactions)
- OpenAI Whisper (Audio transcription)
- Perplexity - Sonar (lecture summarization)
- Gemini Vision (engagement detection)

### **Cloud & Storage**
- Vercel (Frontend Deployment)
- Flask (Backend Hosting)

## **Installation & Setup**
### **Clone our repo**
```sh
git clone https://github.com/yourusername/focusr-ai.git
cd focusr-ai
```

### **Backend Setup (Flask API)**
```sh
cd backend
python -m venv venv
source venv/bin/activate  # (Windows: venv\Scripts\activate)
pip install -r requirements.txt
```

**Set Up API Keys** in `.env` file (create one if missing):
```
OPENAI_API_KEY=your-openai-api-key
PERPLEXITY_API_KEY=your-perplexity-api-key
```

**Run the Flask server:**
```sh
python app.py
```

### **Frontend Setup (React App)**
```sh
cd ../frontend
npm install
npm start  # should run on http://localhost:3000
```

## **Usage**
1. **Go to the upload page** and **upload a lecture video or audio file**.
2. The system **transcribes and analyzes engagement levels** based on eye movement.
3. AI generates a **customized summary** emphasizing areas of disengagement.
4. Review the **engagement chart** to visualize focus levels throughout the lecture.

## **Deployment**
### **Frontend Deployment (Vercel)**
```sh
cd frontend
vercel
```

### **Backend Deployment (Render / Flask on a Cloud Server)**
1. Ensure all dependencies are installed.
2. Deploy via Render / DigitalOcean / AWS Lambda, etc.

## **Future Improvements**
- **Live Eye Tracking** via Webcam
- **Support for Real-Time Audio Processing**
- **Integration with Learning Management Systems (LMS)**
- **Zoom app and API Integration for online classes**

Happy learning! ʕ•ᴥ•ʔ


## Detected evidence (automated analysis)

Indexed codebase: 19 recognized source files, 31 KB.
- CSS (language) — detected in the code
- Flask (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Google Gemini (technology) — claimed on Devpost, not found in the code
- Vercel (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (26 of 26)

```
.gitignore
backend/app.py
backend/requirements.txt
backend/video_processor.py
backend/video_routes.py
frontend/.gitignore
frontend/package.json
frontend/public/index.html
frontend/public/manifest.json
frontend/public/robots.txt
frontend/README.md
frontend/src/App.css
frontend/src/App.js
frontend/src/App.test.js
frontend/src/components/Info.js
frontend/src/components/Timestamps.js
frontend/src/components/UploadAudio.js
frontend/src/components/UploadVideo.js
frontend/src/index.css
frontend/src/index.js
frontend/src/pages/Home.js
frontend/src/pages/Upload.js
frontend/src/reportWebVitals.js
frontend/src/setupTests.js
package.json
README.md
```

### Dependencies

- backend/requirements.txt: flask, google-cloud-aiplatform, google-oauth2-tool, multiprocess, openai, requests, sounddevice, tqdm, vertexai, wavio
- frontend/package.json: @testing-library/dom@^10.4.0, @testing-library/jest-dom@^6.6.3, @testing-library/react@^16.2.0, @testing-library/user-event@^13.5.0, chart.js@^4.4.7, react@^19.0.0, react-chartjs-2@^5.3.0, react-dom@^19.0.0, react-dropzone@^14.3.5, react-router-dom@^7.1.5, react-scripts@^5.0.1, web-vitals@^2.1.4
- package.json: react-router-dom@^7.1.5

### Recent commits (newest first)

- add formatting
- website title
- added favicon + text edits
- Update README.md
- Update README.md
- Merge pull request #4 from isabellasocci/timestamp
- add timestamp functionality
- added drag & drop + css
- pls do not put secret keys on the internet
- update frontend
- update front end
- update gitignore
- add env and config
- Merge pull request #3 from isabellasocci/add-tracking
- add tracking code
- add gitignore
- add to gitignore
- add eye tracking
- add eye tracking
- Update README.md

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "dependencies": {
    "react-router-dom": "^7.1.5"
  }
}

```

### backend/requirements.txt

```
# backend/requirements.txt
flask
openai
sounddevice
wavio
requests
vertexai
google-cloud-aiplatform
tqdm
google-oauth2-tool
multiprocess
```

### frontend/package.json

```
{
  "name": "frontend",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@testing-library/dom": "^10.4.0",
    "@testing-library/jest-dom": "^6.6.3",
    "@testing-library/react": "^16.2.0",
    "@testing-library/user-event": "^13.5.0",
    "chart.js": "^4.4.7",
    "react": "^19.0.0",
    "react-chartjs-2": "^5.3.0",
    "react-dom": "^19.0.0",
    "react-dropzone": "^14.3.5",
    "react-router-dom": "^7.1.5",
    "react-scripts": "^5.0.1",
    "web-vitals": "^2.1.4"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  },
  "proxy": "http://localhost:5000"
}

```

### backend/app.py

```python
from flask import Flask, request, jsonify
from flask_cors import CORS
from video_routes import video_bp

app = Flask(__name__)
CORS(app)

@app.route('/')
def home():
    return "Flask server is running"

@app.errorhandler(500)
def internal_error(e):
    return {"error": str(e)}, 500

@app.errorhandler(404)
def not_found(e):
    return {"error": "Resource not found"}, 404

app.register_blueprint(video_bp, url_prefix="/video")

@app.route('/summary/generate', methods=['POST', 'OPTIONS'])
def generate_summary():
    if request.method == 'OPTIONS':
        return jsonify({}), 200
        
    data = request.json
    # Your summary generation logic here
    return jsonify({"summary": "Summary text here"})

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5001, debug=True)

```

### frontend/src/index.js

```javascript
import React from "react";
import ReactDOM from "react-dom/client";
import App from "./App";
import "./index.css";

const root = ReactDOM.createRoot(document.getElementById("root"));
root.render(
    <React.StrictMode>
        <App />
    </React.StrictMode>
);

```

### frontend/src/App.js

```javascript
import React from "react";
import { BrowserRouter as Router, Routes, Route } from "react-router-dom";
import Home from "./pages/Home";
import Upload from "./pages/Upload";

function App() {
    return (
        <Router>
            <Routes>
                <Route path="/" element={<Home />} />
                <Route path="/upload" element={<Upload />} />
            </Routes>
        </Router>
    );
}

export default App;

```

### backend/video_routes.py

```python
import os
from flask import Blueprint, request, jsonify
from flask_cors import CORS
from video_processor import process_video, find_no_strands


video_bp = Blueprint('video', __name__)
CORS(video_bp)

UPLOADS_FOLDER = os.path.join(os.path.dirname(os.path.abspath(__file__)), "uploads")

@video_bp.route('/upload', methods=['POST'])
def upload_video():
    try:
        
        if 'video' not in request.files:
            return jsonify({"error": "No video file provided"}), 400
            
        file = request.files['video']
        if not file:
            return jsonify({"error": "Empty file"}), 400
        
        
        if not os.path.exists(UPLOADS_FOLDER):
            os.makedirs(UPLOADS_FOLDER)
            
        file_path = os.path.join(UPLOADS_FOLDER, file.filename)
        file.save(file_path)
        
        data = process_video(file_path)
        
        result = find_no_strands(data)

        disengagement_periods = [
            {
                "start_time": start,
                "end_time": end,
                "duration": end - start,
                "message": f"Disengaged from {start} to {end} seconds"
            }
            for start, end in result
        ]
        
        return jsonify({
            "success": True,
            "disengagement_periods": disengagement_periods
        })
        
    except Exception as e:
        return jsonify({"error": str(e)}), 500

```

### backend/video_processor.py

```python
import os
import subprocess
from google.oauth2 import service_account
from google.auth.transport.requests import Request
import vertexai
from vertexai.generative_models import GenerativeModel, Part, Image
from multiprocessing import Pool, cpu_count, Manager
import glob
from tqdm import tqdm

UPLOADS_FOLDER = os.path.join(os.path.dirname(os.path.dirname((__file__))), "uploads")

# Authentication setup
credentials = service_account.Credentials.from_service_account_file(
'focusr-ai-key.json',
scopes=['https://www.googleapis.com/auth/cloud-platform']
)

if credentials.expired:
    credentials.refresh(Request())

# Vertex AI initialization
PROJECT_ID = 'focusr-ai'
REGION = 'us-central1'

vertexai.init(project=PROJECT_ID, location=REGION)

def analyze_frame(args):
    frame_path, progress_dict = args
    model = GenerativeModel("gemini-pro-vision")
    image = Image.load_from_file(frame_path)
    image_part = Part.from_image(image)
    prompt = "Is the user's gaze focused on the camera in this image? Provide a one word yes or no answer only."
    response = model.generate_content([image_part, prompt])
    frame_number = int(frame_path.split('_')[1].split('.')[0])
    
    # Update progress
    result = (frame_path, response.text.upper(), frame_number)
    progress_dict['current_frame'] = f"Frame: {frame_path}, Engagement: {response.text.upper()}, Timestamp: {frame_number}s"
    
    return result

def process_video(video_path):
    frames_dir = "frames"
    if not os.path.exists(frames_dir):
        os.makedirs(frames_dir)
    
    # Extract frames
    print("Extracting frames...")
    subprocess.run([
        'ffmpeg', '-i', video_path, 
        '-vf', 'fps=1', 
        f'{frames_dir}/frame_%04d.png'
    ], stderr=subprocess.DEVNULL)
    
    # Get all frame paths
    frame_paths = sorted(glob.glob(os.path.join(frames_dir, "frame_*.png")))
    
    # Create a manager for sharing progress information
    with Manager() as manager:
        progress_dict = manager.dict()
        progress_dict['current_frame'] = ""
        
        # Create arguments for the pool
        args = [(frame_path, progress_dict) for frame_path in frame_paths]
        
        # Create a pool of workers
        num_processes = cpu_count() - 1  # Leave one core free
        with Pool(processes=num_processes) as pool:
            # Use tqdm to create a progress bar
            results = []
            with tqdm(total=len(frame_paths), desc="Processing frames") as pbar:
                for result in pool.imap_unordered(analyze_frame, args):
                    # Store the engagement and timestamp
                    engagement = result[1]  # Get the engagement status
                    timestamp = result[2]   # Get the timestamp
                    results.append((engagement.strip(), timestamp))
                    pbar.update(1)
    
    # Sort results by timestamp
    results.sort(key=lambda x: x[1])
    
    return results


def find_no_strands(data, min_length=5):
    no_strands = []
    current_strand = []
    
    for engagement, timestamp in data:
        if engagement.strip().upper() == 'NO.':
            current_strand.append(timestamp)
        else:
            if len(current_strand) >= min_length:
                no_strands.append((current_strand[0], current_strand[-1]))
            current_strand = []
    
    if len(current_strand) >= min_length:
        no_strands.append((current_strand[0], current_strand[-1]))
    
    return no_strands

if __name__ == "__main__":
    video_path = "short_test.mp4"
    data = process_video(video_path)
    result = find_no_strands(data)
    for res in result:
        print("You were disengaged from " + str(res[0]) + " to " + str(res[1]) + " seconds")

```

### frontend/src/setupTests.js

```javascript
// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';

```

### frontend/src/App.test.js

```javascript
import { render, screen } from '@testing-library/react';
import App from './App';

test('renders learn react link', () => {
  render(<App />);
  const linkElement = screen.getByText(/learn react/i);
  expect(linkElement).toBeInTheDocument();
});

```

[10 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]