# Project export: ZoomPulse This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2025 - Tagline: vibe check how your Zoom meet went - Devpost: https://devpost.com/software/convocriticpoopify - GitHub: https://github.com/alanliu2009/treehack25 - Video: https://www.youtube.com/embed/ch1SE_Ccn6Q?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 3 GitHub contributor(s) — alanliu2009 (8 commits), Maggie Liu (7 commits), juliading (2 commits) ## Devpost submission (written by the team) ### Inspiration As meeting hosts and educators, it may be difficult to discern audience engagement and get live feedback from presentations. We want to offer a quick and efficient method for hosts to gauge how well the participants might be digesting their content, such that they can easily reflect on others' experiences to better recognize students' learning styles and adapt teaching approaches to improve quality of learning for more students. As students ourselves, we feel that sometimes it could be uncomfortable providing direct feedback to our instructors. Traditional methods to the feedback cycle often require students to offer feedback at certain time intervals, such as at the middle and end of a course. It can be a struggle to verbalize our thoughts and we exponentially lose recollection of details as time passes. Despite the insights that these methods offer, they can lack accurate representation of all students at relevant time intervals. With ZoomPulse, Zoom hosts can automatically receive aggregated and immediate feedback about their meeting based on participant expressions, post end of the recording. ### What it does Once ZoomPulse is running in the background, hosts can run and record their Zoom meetings. After the recording is complete and the host has finished the meeting, ZoomPulse quickly generates the average sentiment of the participants and display it onto the website. ZoomPulse's approach to determining the sentiment involves taking snapshots of the Zoom recording, analyzing each frame for emotions, and taking aggregates of the sentiment to determine overall meeting satisfaction. ### How we built it ZoomPulse is a web application built on the Flask framework, with Python running in the backend and ReactJS materializing what our users see and interact with. We implemented a webhook to receive events sent by Zoom to our application's endpoint on our server side. The webhook has been developed such that any Zoom meeting video recorded on Cloud will be retrieved after the corresponding meeting ends. We also employed the Python Deepface library to assist in facial attribute analysis in regards to participant emotions throughout the meeting recording. ### Challenges we ran into Zoom Setting up webhook with Zoom Trying to work with Real-Time Media Streams, ultimately deciding to pivot our approach Setting up webhook with Zoom Trying to work with Real-Time Media Streams, ultimately deciding to pivot our approach Web Dev Learning how to use ReactJS and building a full-stack web application from scratch Learning how to use ReactJS and building a full-stack web application from scratch Overall, our team have gained tremendous experiences with web development and with the features of the Zoom Developer portal. ### What's next So much! We see lots of potential features and verticals that ZoomPulse can embark in during the near future. Line charts for visual fluctuation of sentiment: we would like to implement a visualization of the change in attendee sentiment throughout the meeting. With the support of AI tools to analyze the topic of discussions at certain points of the meeting, we can help hosts identify positive and negative sentiments for specific parts of the meeting. This can also be adapted for visualization of per-user engagement. Evaluate levels of participation => user participation leaderboard page for post-meeting: with data about each user's use of the polling feature and raise hand feature, ZoomPulse can offer another way to share engagement results with Zoom meeting hosts. As well, this feature allows ZoomPulse to also increase engagement for the users by building a leaderboard page that users can view after the meeting. This can be an incentive for the attendees to be more engaged and interact with the host more throughout the meeting through polls and reactions. Exploring the use of ZoomPulse in the workplace: In addition to fostering more inclusive, accessible learning experiences for students everywhere, we anticipate ZoomPulse's ability to enhance empathy between people to be valued in general workplaces. Just as student engagement is important in effective teaching, team meetings are most productive when the attendees are engaged and participating as well. ZoomPulse can contribute to a collaborative space that brings more transparency to the team, helping team leads and managers determine areas for improvements, keep track of the team's mental well-being and note tasks for prioritizing. ## README (from the GitHub repository) # ZoomPulse ### TreeHacks 2025 by Maggie Liu, Julia Ding, and Alan Liu ## Setup - Install ngrok and run `ngrok http 5000` in your terminal to retrieve a target URL, and use it as the web target on Zoom App Marketplace. - Run `pip install -r requirements.txt` in both the frontend and backend directories. ## Running the Program 1. Start by enabling the frontend development server by running `npm start`. 2. Enable the backend webhook with `python3 app.py`. 3. Now, record a Zoom call to the cloud, and your call will automatically retrieved by the webhook. 4. Check the webpage to get a review of the video sentiment! ## Detected evidence (automated analysis) Indexed codebase: 20 recognized source files, 23 KB. - CSS (language) — detected in the code - Flask (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code ## Codebase structure (from repository index) ### Files (31 of 31) ``` .DS_Store .gitignore backend/.flaskenv backend/.gitignore backend/app.py backend/downloader.py backend/frame_extractor.py backend/recordings/.DS_Store backend/sentiment.py backend/server.py frontend/.gitignore frontend/package.json frontend/postcss.config.js frontend/public/index.html frontend/public/manifest.json frontend/public/robots.txt frontend/README.md frontend/src/App.css frontend/src/App.js frontend/src/App.test.js frontend/src/components/card.jsx frontend/src/components/progress.jsx frontend/src/index.css frontend/src/index.js frontend/src/reportWebVitals.js frontend/src/Sentiment.js frontend/src/setupTests.js frontend/tailwind.config.js package.json README.md requirements.txt ``` ### Dependencies - frontend/package.json: @testing-library/dom@^10.4.0, @testing-library/jest-dom@^6.6.3, @testing-library/react@^16.2.0, @testing-library/user-event@^13.5.0, autoprefixer@^10.4.20, lucide-react@^0.475.0, postcss@^8.5.2, react@^19.0.0, react-dom@^19.0.0, react-scripts@^5.0.1, web-vitals@^2.1.4 - package.json: @zoom/videosdk@^2.1.5, socket.io-client@^4.8.1 - requirements.txt: blinker@==1.9.0, click@==8.1.8, deepface, Flask@==3.1.0, importlib_metadata@==8.6.1, itsdangerous@==2.2.0, Jinja2@==3.1.5, MarkupSafe@==3.0.2, numpy, opencv-python, requests, Werkzeug@==3.1.3, zipp@==3.21.0 ### Recent commits (newest first) - Update README.md - it update but - need test rn - connecty - Update README.md - tryna connect but cant test - editted title + attempted to centre it - Merge branch 'main' of github.com:alanliu2009/treehack25 - automatically process video when downloaded - refreshing auth token - draft 2 - draft 1 - remove old downloader - taking download url to process mp4 - frame processor - readme - flask react setup - Initial commit ## Key source files (fetched from GitHub, selected and truncated for size) ### package.json ``` { "dependencies": { "@zoom/videosdk": "^2.1.5", "socket.io-client": "^4.8.1" } } ``` ### requirements.txt ``` blinker==1.9.0 click==8.1.8 Flask==3.1.0 importlib_metadata==8.6.1 itsdangerous==2.2.0 Jinja2==3.1.5 MarkupSafe==3.0.2 Werkzeug==3.1.3 zipp==3.21.0 requests opencv-python deepface numpy ``` ### frontend/package.json ``` { "name": "frontend", "version": "0.1.0", "private": true, "proxy": "http://localhost:8000", "dependencies": { "@testing-library/dom": "^10.4.0", "@testing-library/jest-dom": "^6.6.3", "@testing-library/react": "^16.2.0", "@testing-library/user-event": "^13.5.0", "lucide-react": "^0.475.0", "react": "^19.0.0", "react-dom": "^19.0.0", "react-scripts": "^5.0.1", "web-vitals": "^2.1.4" }, "scripts": { "start": "react-scripts start", "start-server": "cd ../backend && flask run --no-debugger", "build": "react-scripts build", "test": "react-scripts test", "eject": "react-scripts eject" }, "eslintConfig": { "extends": [ "react-app", "react-app/jest" ] }, "browserslist": { "production": [ ">0.2%", "not dead", "not op_mini all" ], "development": [ "last 1 chrome version", "last 1 firefox version", "last 1 safari version" ] }, "devDependencies": { "autoprefixer": "^10.4.20", "postcss": "^8.5.2" } } ``` ### backend/server.py ```python from app import app if __name__ == "__main__": app.run() ``` ### backend/app.py ```python from flask import Flask, request, jsonify from downloader import download_zoom_recording from frame_extractor import extract_frames # Assuming you have this function for frame extraction from sentiment import analyze_faces # Assuming you have this function for sentiment analysis import threading import os app = Flask(__name__) # Load your Zoom verification token ZOOM_VERIFICATION_TOKEN = os.getenv("ZOOM_VERIFICATION_TOKEN", "YAA7Hqj3R0WLj5f7oIVnSQ") RECORDINGS_FOLDER = "recordings" latest_video_path = None # latest_sentiment_score = None @app.route('/process_video', methods=['GET']) def process_video(): # global latest_sentiment_score, latest_video_path global latest_video_path if not latest_video_path or not os.path.exists(latest_video_path): print("No video found for processing.") return """Process the video after downloading (extract frames, analyze sentiment).""" print("Extracting frames from video...") frames = extract_frames(latest_video_path) # Function to extract frames print("Analyzing sentiment...") avg_sentiment = analyze_faces(frames) # Function to analyze sentiment print(f"Meeting Sentiment Score: {avg_sentiment}") if avg_sentiment > 0: print("Meeting had a **positive** sentiment 😊") elif avg_sentiment < 0: print("Meeting had a **negative** sentiment 😞") else: print("Meeting was **neutral** 😐") ret_sent = 50 * (avg_sentiment + 1) print(ret_sent) return str(ret_sent) @app.route('/zoom-webhook', methods=['POST']) def zoom_webhook(): global latest_video_path data = request.json print(f"Received Webhook: {data}") # Verify the request is from Zoom if "token" in data and data["token"] != ZOOM_VERIFICATION_TOKEN: return jsonify({"error": "Unauthorized"}), 403 # Respond to Zoom's challenge request (initial verification) if data.get("event") == "endpoint.url_validation": return jsonify({"plainToken": data["payload"]["plainToken"]}) # Process recording completion events if data.get("event") == "recording.completed": recording_files = data['payload']['object']['recording_files'] download_token = data['download_token'] # Extract the token print(f"✅ Download Token: {download_token}") for file in recording_files: if file['file_extension'].upper() == 'MP4': recording_url = file['download_url'] # Start the download in a separate thread threading.Thread(target=download_zoom_recording, args=(recording_url, download_token)).start() # Once the download completes, process the video # Wait for the file to download (you can implement a better check for file existence) file_path = os.path.join(RECORDINGS_FOLDER, "meeting.mp4") if os.path.exists(file_path): latest_video_path = file_path threading.Thread(target=process_video).start() break return jsonify({"status": "received"}), 200 # @app.route('/process_video', methods=['GET']) # def process_video_endpoint(): # """Trigger video processing and return the result.""" # threading.Thread(target=process_video).start() # return jsonify({"message": "Processing started"}) # @app.route('/video_status', methods=['GET']) # def video_status(): # """Check if the video processing is done and return the score.""" # if latest_sentiment_score is None: # return jsonify({"status": "processing"}), 202 # return jsonify({"status": "done", "sentiment_score": latest_sentiment_score}) if __name__ == "__main__": app.run(host="0.0.0.0", port=8000, debug=True) ``` ### frontend/src/index.js ```javascript import React from 'react'; import ReactDOM from 'react-dom/client'; import './index.css'; import App from './App'; import reportWebVitals from './reportWebVitals'; const root = ReactDOM.createRoot(document.getElementById('root')); root.render( ); // If you want to start measuring performance in your app, pass a function // to log results (for example: reportWebVitals(console.log)) // or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals reportWebVitals(); ``` ### frontend/src/App.js ```javascript // import logo from './logo.svg'; import './App.css'; import React, { useState, useEffect } from "react"; // import SentimentAnalysis from './Sentiment'; // import { useState } from "react"; import { Progress } from "./components/progress.jsx"; import { Card, CardContent, CardHeader, CardTitle } from "./components/card.jsx"; // import { Progress } from "@/components/progress"; // import { Card, CardContent, CardHeader, CardTitle } from "@/components/card"; import { Frown, Smile, Meh } from "lucide-react"; function App() { const [sentiment, setSentiment] = useState(15); // Example sentiment value // IFFFF LOADING, should try and get a diff UI going up here const [loading, setLoading] = useState(true); const [error, setError] = useState(null); useEffect(() => { const fetchData = async () => { try { // const response = await fetch("http://127.0.0.1:8000/process_video"); const response = await fetch("/process_video", { method: "GET" }) if (!response.ok) { throw new Error("Network response was not ok"); } const result = await response.text(); const intResult = parseInt(result); setSentiment(intResult); // Update the state with Flask API response } catch (err) { setError(err.message); } finally { setLoading(false); } }; fetchData(); }, []); // Empty dependency array ensures it runs once when component mounts const getSentimentIcon = (value) => { if (value > 66) return ; if (value > 33) return ; return ; }; const getSentimentColor = (value) => { if (value > 66) return "bg-green-500"; if (value > 33) return "bg-yellow-500"; return "bg-red-500"; }; return (
here's how your meeting went :) {getSentimentIcon(sentiment)}
{/* */}

Sentiment Score: {sentiment}%
); } export default App; ``` ### frontend/postcss.config.js ```javascript module.exports = { plugins: { tailwindcss: {}, autoprefixer: {}, }, }; ``` ### frontend/tailwind.config.js ```javascript /** @type {import('tailwindcss').Config} */ module.exports = { content: ["./pages/**/*.{js,ts,jsx,tsx}", "./components/**/*.{js,ts,jsx,tsx}"], theme: { extend: {}, }, plugins: [], }; ``` ### backend/sentiment.py ```python from deepface import DeepFace sentiment_mapping = { "happy": 1, "neutral": 0, "surprise": 0.5, "sad": -1, "angry": -1, "fear": -0.5, "disgust": -1 } def analyze_faces(frame_list): scores = [] for frame in frame_list: try: result = DeepFace.analyze(frame, actions=['emotion']) emotion = result[0]['dominant_emotion'] scores.append(sentiment_mapping.get(emotion, 0)) # Default to neutral except: scores.append(0) # Assume neutral if no face is detected avg_sentiment = sum(scores) / len(scores) if scores else 0 return avg_sentiment ``` [11 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]