# Project export: Speech Master This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 10.0 - Tagline: Leveraging ML to improve public speaking - Devpost: https://devpost.com/software/speech-master - GitHub: https://github.com/shahdivyank/speechmaster - Demo: https://www.figma.com/file/4K7YLO6OBMkJxFm3DElFMo/speechmaster?type=design&node-id=0-1&mode=design&t=kWm58njNRzqRPG9d-0 - Result: winner (Loved by Ddoski: 2nd Overall) - Team: 5 GitHub contributor(s) — Divyank Shah (51 commits), menthy-wu (35 commits), Shing Hung (27 commits), theo (25 commits), Menthy Wu (10 commits) ## Devpost submission (written by the team) ### Inspiration The post-COVID era has increased the number of in-person events and need for public speaking. However, more individuals are anxious to publicly articulate their ideas, whether this be through a presentation for a class, a technical workshop, or preparing for their next interview. It is often difficult for audience members to catch the true intent of the presenter, hence key factors including tone of voice, verbal excitement and engagement, and physical body language can make or break the presentation. A few weeks ago during our first project meeting, we were responsible for leading the meeting and were overwhelmed with anxiety. Despite knowing the content of the presentation and having done projects for a while, we understood the impact that a single below-par presentation could have. To the audience, you may look unprepared and unprofessional, despite knowing the material and simply being nervous. Regardless of their intentions, this can create a bad taste in the audience's mouths. As a result, we wanted to create a judgment-free platform to help presenters understand how an audience might perceive their presentation. By creating Speech Master, we provide an opportunity for presenters to practice without facing a real audience while receiving real-time feedback. Purpose Speech Master aims to provide a practice platform for practice presentations with real-time feedback that captures details in regard to your body language and verbal expressions. In addition, presenters can invite real audience members to practice where the audience member will be able to provide real-time feedback that the presenter can use to improve. While presenting, presentations will be recorded and saved for later reference for them to go back and see various feedback from the ML models as well as live audiences. They are presented with a user-friendly dashboard to cleanly organize their presentations and review for upcoming events. After each practice presentation, the data is aggregated during the recording and process to generate a final report. The final report includes the most common emotions expressed verbally as well as times when the presenter's physical body language could be improved. The timestamps are also saved to show the presenter when the alerts rose and what might have caused such alerts in the first place with the video playback. Tech Stack We built the web application using Next.js v14, a React-based framework that seamlessly integrates backend and frontend development. We deployed the application on Vercel, the parent company behind Next.js. We designed the website using Figma and later styled it with TailwindCSS to streamline the styling allowing developers to put styling directly into the markup without the need for extra files. To maintain code formatting and linting via Prettier and EsLint. These tools were run on every commit by pre-commit hooks configured by Husky. Hume AI provides the Speech Prosody model with a streaming API enabled through native WebSockets allowing us to provide emotional analysis in near real-time to a presenter. The analysis would aid the presenter in depicting the various emotions with regard to tune, rhythm, and timbre. Google and Tensorflow provide the MoveNet model is a large improvement over the prior PoseNet model which allows for real-time pose detection. MoveNet is an ultra-fast and accurate model capable of depicting 17 body points and getting 30+ FPS on modern devices. To handle authentication, we used Next Auth to sign in with Google hooked up to a Prisma Adapter to interface with CockroachDB, allowing us to maintain user sessions across the web app. Cloudinary, an image and video management system, was used to store and retrieve videos. Socket.io was used to interface with Websockets to enable the messaging feature to allow audience members to provide feedback to the presenter while simultaneously streaming video and audio. We utilized various services within Git and Github to host our source code, run continuous integration via Github Actions, make pull requests, and keep track of issues and projects. Challenges It was our first time working with Hume AI and a streaming API. We had experience with traditional REST APIs which are used for the Hume AI batch API calls, but the streaming API was more advantageous to provide real-time analysis. Instead of an HTTP client such as Axios, it required creating our own WebSockets client and calling the API endpoint from there. It was also a hurdle to capture and save the correct audio format to be able to call the API while also syncing audio with the webcam input. We also worked with Tensorflow for the first time, an end-to-end machine learning platform. As a result, we faced many hurdles when trying to set up Tensorflow and get it running in a React environment. Most of the documentation uses Python SDKs or vanilla HTML/CSS/JS which were not possible for us. Attempting to convert the vanilla JS to React proved to be more difficult due to the complexities of execution orders and React's useEffect and useState hooks. Eventually, a working solution was found, however, it can still be improved to better its performance and bring fewer bugs. We originally wanted to use the Youtube API for video management where users would be able to post and retrieve videos from their personal accounts. Next Auth and YouTube did not originally agree in terms of available scopes and permissions, but once resolved, more issues arose. We were unable to find documentation regarding a Node.js SDK and eventually even reached our quota. As a result, we decided to drop YouTube as it did not provide a feasible solution and found Cloudinary. Accomplishments We are proud of being able to incorporate Machine Learning into our applications for a meaningful purpose. We did not want to reinvent the wheel by creating our own models but rather use the existing and incredibly powerful models to create new solutions. Although we did not hit all the milestones that were hoping to achieve, we are still proud of the application that we were able to make in such a short amount of time and be able to deploy the project as well. Most notably, we are proud of our Hume AI and Tensorflow integrations that took our application to the next level. Those 2 features took the most time, but they were also the most rewarding as in the end, we got to see real-time updates of our emotional and physical states. We are proud of being able to run the application and get feedback in real-time, which gives small cues to the presenter on what to improve without risking distracting the presenter completely. ### What we learned Each of the developers learned something valuable as each of us worked with a new technology that we did not know previously. Notably, Prisma and its integration with CockroachDB and its ability to make sessions and general usage simple and user-friendly. Interfacing with CockroachDB barely had problems and was a powerful tool to work with. We also expanded our knowledge with WebSockets, both native and Socket.io. Our prior experience was more rudimentary, but building upon that knowledge showed us new powers that WebSockets have both when used internally with the application and with external APIs and how they can introduce real-time analysis. Future of Speech Master The first step for Speech Master will be to shrink the codebase. Currently, there is tons of potential for components to be created and reused. Structuring the code to be more strict and robust will ensure that when adding new features the codebase will be readable, deployable, and functional. The next priority will be responsiveness, due to the lack of time many components appear strangely on different devices throwing off the UI and potentially making the application unusable. Once the current codebase is restructured, then we would be able to focus on optimization primarily on the machine learning models and audio/visual. Currently, there are multiple instances of audio and visual that are being used to show webcam footage, stream footage to other viewers, and sent to HumeAI for analysis. By reducing the number of streams, we should expect to see significant performance improvements with which we can upgrade our audio/visual streaming to use something more appropriate and robust. In terms of new features, Speech Master would benefit greatly from additional forms of audio analysis such as speed and volume. Different presentations and environments require different talking speeds and volumes of speech required. Given some initial parameters, Speech Master should hopefully be able to reflect on those measures. In addition, having transcriptions that can be analyzed for vocabulary and speech, ensuring that appropriate language is used for a given target audience would drastically improve the way a presenter could prepare for a presentation. ## README (from the GitHub repository) # Speech Master ![Next.js](https://img.shields.io/badge/next.js-000000?style=for-the-badge&logo=nextdotjs&logoColor=white) ![JavaScript](https://img.shields.io/badge/javascript-%23323330.svg?style=for-the-badge&logo=javascript&logoColor=%23F7DF1E) ![TailwindCSS](https://img.shields.io/badge/tailwindcss-%2338B2AC.svg?style=for-the-badge&logo=tailwind-css&logoColor=white) ![Figma](https://img.shields.io/badge/figma-%23F24E1E.svg?style=for-the-badge&logo=figma&logoColor=white)
![CockroachLabs](https://img.shields.io/badge/Cockroach%20Labs-6933FF?style=for-the-badge&logo=Cockroach%20Labs&logoColor=white) ![Prisma](https://img.shields.io/badge/Prisma-3982CE?style=for-the-badge&logo=Prisma&logoColor=white) ![TensorFlow](https://img.shields.io/badge/TensorFlow-%23FF6F00.svg?style=for-the-badge&logo=TensorFlow&logoColor=white) ![Socket.io](https://img.shields.io/badge/Socket.io-black?style=for-the-badge&logo=socket.io&badgeColor=010101)
![ESLint](https://img.shields.io/badge/ESLint-4B3263?style=for-the-badge&logo=eslint&logoColor=white) ![Prettier](https://img.shields.io/badge/prettier-1A2C34?style=for-the-badge&logo=prettier&logoColor=F7BA3E) ![GitHub Actions](https://img.shields.io/badge/GitHub_Actions-2088FF?style=for-the-badge&logo=github-actions&logoColor=white) ## Node.js Hackathon Website Template runs on Node.js Version 18.17.0 and higher. Please ensure you have Node.js installed via the [official website](https://nodejs.org/en). ## Next.js This project is built using [Next.js](https://nextjs.org), a React framework. Next.js is automatically installed when you install all dependencies for this project. ## Environment Variables The following environment variables are required and must be stored in an `.env` file: ```env NEXT_PUBLIC_DATABASE_URL= NEXT_PUBLIC_GOOGLE_CLIENT_ID= NEXT_PUBLIC_GOOGLE_CLIENT_SECRET= NEXTAUTH_URL= NEXTAUTH_SECRET= NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME= NEXT_PUBLIC_CLOUDINARY_API_KEY= NEXT_PUBLIC_CLOUDINARY_API_SECRET= ``` ## Commands ### Dependencies ```bash # Install dependencies npm i # Add dependency npm i # Remove dependency npm un ``` ### Running the Website Locally ```bash # Open a browser at localhost:3000 npm run dev ``` ### Formatting Code via Prettier ```bash # Rewrite code recursively with proper formatting npm run format # Show formatting differences recursively npm run check ``` ### Linting Code via Eslint ```bash npm run eslint ``` ### Build the Website ```bash npm run build ``` ## Detected evidence (automated analysis) Indexed codebase: 51 recognized source files, 81 KB. - CSS (language) — detected in the code - JavaScript (language) — detected in the code - Next.js (technology) — detected in the code - PostgreSQL (technology) — detected in the code - React (technology) — detected in the code - SQL (language) — detected in the code - Tailwind CSS (technology) — detected in the code - TensorFlow (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (60 of 60) ``` .eslintignore .eslintrc.json .github/workflows/dev.yaml .gitignore .husky/pre-commit .prettierignore jsconfig.json next.config.js package.json postcss.config.js prisma/init_database.sql prisma/migrations/20231028045150_initial_migration/migration.sql prisma/migrations/20231028203053_/migration.sql prisma/migrations/20231028212014_update_video/migration.sql prisma/migrations/20231029083417_add_posture_and_hume/migration.sql prisma/migrations/20231029100203_changeemotion/migration.sql prisma/migrations/20231029101144_timestampcase/migration.sql prisma/migrations/20231029122326_changehume/migration.sql prisma/migrations/20231029152619_add_message/migration.sql prisma/migrations/migration_lock.toml prisma/schema.prisma prismaClient.js README.md socket.js src/app/api/auth/[...nextauth]/route.js src/app/api/message/route.js src/app/api/video/route.js src/app/api/videos/route.js src/app/dashboard/page.js src/app/globals.css src/app/join/[...id]/page.js src/app/layout.js src/app/live/page.js src/app/page.js src/app/review/[id]/page.js src/components/AudioAnalysis.jsx src/components/AudioPlayer.jsx src/components/BlobToBase64.jsx src/components/BreakDown.jsx src/components/Category.jsx src/components/Checkbox.jsx src/components/Details.jsx src/components/NavBar.jsx src/components/Postures.jsx src/components/ProtectedPage.jsx src/components/Recording.jsx src/components/Signin.jsx src/components/TitleBar.jsx src/components/ToolBar.jsx src/components/useInterval.jsx src/components/VideoPlayer.jsx src/components/VideoRecording.jsx src/data/Categories.js src/data/Emotions.js src/data/mockTags.js src/data/Posture.js src/data/Recordings.js src/data/User.js src/pages/api/socket.js tailwind.config.js ``` ### Dependencies - package.json: @auth/prisma-adapter@^1.0.5, @cloudinary/react@^1.11.2, @cloudinary/url-gen@^1.12.0, @nivo/pie@^0.83.0, @prisma/client@^5.5.2, @tensorflow-models/pose-detection@^2.1.3, @tensorflow/tfjs-backend-webgl@^4.12.0, @tensorflow/tfjs-converter@^4.12.0, @tensorflow/tfjs-core@^4.12.0, autoprefixer@^10, axios@^1.6.0, bootstrap@^5.3.2, cloudinary@^1.41.0, eslint@^8.52.0, eslint-config-google@^0.14.0, eslint-config-prettier@^9.0.0, eslint-plugin-react@^7.33.2, fs@^0.0.1-security, husky@^8.0.0, next@14.0.0, next-auth@^4.24.4, pg@^8.11.3, postcss@^8, prisma@^5.5.2, react@^18, react-audio-voice-recorder@^2.2.0, react-bootstrap@^2.9.1, react-circular-progressbar@^2.1.0, react-dom@^18, react-hot-toast@^2.4.1, react-icons@^4.11.0, react-webcam@^7.2.0, socket.io@^4.7.2, socket.io-client@^4.7.2, tailwindcss@^3 ### Recent commits (newest first) - fix login - fix login - fix play - Merge pull request #63 from shahdivyank/pondohoo/justifyAround - api fix - chagne next domian - d - justify around - Merge pull request #62 from shahdivyank/menthy/dashboard - Merge pull request #61 from shahdivyank/menthy/message - Merge branch 'dev' of https://github.com/shahdivyank/speechmaster into menthy/message - menthy/addMessage - Merge pull request #60 from shahdivyank/shing/buttons - format buttons - Merge pull request #59 from shahdivyank/shahdivyank/messaging - remove useless apis - fix recording - add messaging - Merge pull request #58 from shahdivyank/menthy/dashboard - remove transcript ## Key source files (fetched from GitHub, selected and truncated for size) ### package.json ``` { "name": "microphone", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint", "eslint": "npx eslint .", "check": "npx prettier --check .", "format": "npx prettier --write .", "prepare": "husky install" }, "dependencies": { "@auth/prisma-adapter": "^1.0.5", "@cloudinary/react": "^1.11.2", "@cloudinary/url-gen": "^1.12.0", "@nivo/pie": "^0.83.0", "@prisma/client": "^5.5.2", "@tensorflow-models/pose-detection": "^2.1.3", "@tensorflow/tfjs-backend-webgl": "^4.12.0", "@tensorflow/tfjs-converter": "^4.12.0", "@tensorflow/tfjs-core": "^4.12.0", "axios": "^1.6.0", "bootstrap": "^5.3.2", "cloudinary": "^1.41.0", "fs": "^0.0.1-security", "next": "14.0.0", "next-auth": "^4.24.4", "pg": "^8.11.3", "react": "^18", "react-audio-voice-recorder": "^2.2.0", "react-bootstrap": "^2.9.1", "react-circular-progressbar": "^2.1.0", "react-dom": "^18", "react-hot-toast": "^2.4.1", "react-icons": "^4.11.0", "react-webcam": "^7.2.0", "socket.io": "^4.7.2", "socket.io-client": "^4.7.2" }, "devDependencies": { "autoprefixer": "^10", "eslint": "^8.52.0", "eslint-config-google": "^0.14.0", "eslint-config-prettier": "^9.0.0", "eslint-plugin-react": "^7.33.2", "husky": "^8.0.0", "postcss": "^8", "prisma": "^5.5.2", "tailwindcss": "^3" } } ``` ### src/app/layout.js ```javascript /* eslint-disable new-cap */ "use client"; import "./globals.css"; import TitleBar from "@/components/TitleBar"; import { SessionProvider } from "next-auth/react"; import "bootstrap/dist/css/bootstrap.min.css"; import { Poppins } from "next/font/google"; import ProtectedPage from "@/components/ProtectedPage"; import { Toaster } from "react-hot-toast"; import { usePathname } from "next/navigation"; const poppins = Poppins({ subsets: ["latin"], display: "swap", weight: ["100", "200", "300", "400", "500", "600", "700", "800", "900"], variable: "--poppins-font", }); export default function RootLayout({ children, session }) { const pathname = usePathname(); return ( {pathname !== "/" && pathname !== "/dashboard" && } {children} ); } ``` ### src/app/page.js ```javascript "use client"; import Image from "next/image"; import divider from "../../public/divider.svg"; import logo from "../../public/logo2.svg"; import { BsGoogle } from "react-icons/bs"; import { signIn } from "next-auth/react"; const login = () => { return (

LOGIN

WELCOME

signIn("google")} className="hover:bg-sm-orange transition-transform hover:text-sm-white hover:opacity-50 border-[1px] border-stone-700 text-stone-700 text-xs py-1.5 px-8 mt-16 cursor-pointer flex items-center" > LOGIN WITH GOOGLE
); }; export default login; ``` ### src/app/dashboard/page.js ```javascript "use client"; import ToolBar from "@/components/ToolBar"; import Recording from "@/components/Recording"; import NavBar from "@/components/NavBar"; import { useState, useEffect } from "react"; import { colors } from "@/data/Categories"; import axios from "axios"; import { AiOutlineLoading3Quarters } from "react-icons/ai"; const home = () => { const [loading, setLoading] = useState(true); const [recordings, setRecordings] = useState([]); useEffect(() => { axios .get("/api/videos") .then((response) => { setLoading(false); setRecordings( response.data.map((r) => ({ ...r, categories: r.categories.split(","), selected: false, })), ); }) .catch((error) => { console.log(error); }); }, []); return (
{loading ? (
Loading...
) : ( recordings.filter((r) => !r.hidden).length === 0 && (
No videos to display
) )}
{recordings .filter((r) => !r.hidden) .map((recording, index) => ( ))}
); }; export default home; ``` ### src/app/live/page.js ```javascript "use client"; import { BsPlayCircle, BsStopCircle, BsShare } from "react-icons/bs"; import Webcam from "react-webcam"; import { useRef, useState, useCallback, useEffect } from "react"; import axios from "axios"; import * as poseDetection from "@tensorflow-models/pose-detection"; import toast from "react-hot-toast"; import { useSession } from "next-auth/react"; import { socket } from "../../../socket"; import AudioPlayer from "@/components/AudioPlayer"; import * as tf from "@tensorflow/tfjs-core"; import "@tensorflow/tfjs-backend-webgl"; import Checkbox from "@/components/Checkbox"; import { useRouter } from "next/navigation"; import { useInterval } from "@/components/useInterval"; import { AiOutlineLoading3Quarters } from "react-icons/ai"; const Live = () => { const router = useRouter(); const [loading, setLoading] = useState(false); const [startTime, setStartTime] = useState(null); const { data: session } = useSession(); const [recording, setRecording] = useState(false); const webcamRef = useRef(null); const mediaRecorderRef = useRef(null); const [notifs, setNotifs] = useState([]); const [recordedVideo, setRecordedVideo] = useState([]); const [title, setTitle] = useState(""); const [base64Audio, setBase64Audio] = useState(); const [tags, setTags] = useState({ Workshops: false, "Class Presentations": false, Other: false, }); const [body, setBody] = useState("upper"); const [DBEmotions, setDBEmotions] = useState([]); const [messages, setMessages] = useState([]); const handleShare = () => { const link = `${process.env.NEXT_PUBLIC_DOMAIN}/join/${session.user.id}`; navigator.clipboard.writeText(link); toast("✅ Link copied to clipboard"); }; const capture = useCallback(() => { socket.emit("frames", webcamRef.current.getScreenshot()); }, [webcamRef]); useEffect(() => { console.log(DBEmotions); const loadup = async () => { await tf.setBackend("webgl"); await tf.ready(); }; loadup(); // load(); socket.connect(); socket.emit("join", session.user.id); socket.on("message", (message) => { setMessages([...messages, { message, timestamp: new Date() }]); }); const id = setInterval(() => { capture(); }, 20); return () => { clearInterval(id); socket.disconnect(); }; }, []); const handleStartRecording = useCallback(() => { setStartTime(new Date()); setRecording(true); mediaRecorderRef.current = new MediaRecorder(webcamRef.current.stream, { mimeType: "video/webm", }); mediaRecorderRef.current.addEventListener( "dataavailable", handleDataAvailable, ); mediaRecorderRef.current.start(); }, [webcamRef, setRecording, mediaRecorderRef]); const handleDataAvailable = useCallback( ({ data }) => { if (data.size > 0) { setRecordedVideo((prev) => prev.concat(data)); } }, [setRecordedVideo], ); const handleStopRecording = useCallback(() => { mediaRecorderRef.current.stop(); setRecording(false); }, [mediaRecorderRef, webcamRef, setRecording]); const handleUpload = useCallback(async () => { setLoading(true); if (recordedVideo.length) { const blob = new Blob(recordedVideo, { type: "video/webm", }); const reader = new FileReader(); reader.readAsDataURL(blob); reader.onloadend = function () { const base64data = reader.result; axios .post(`/api/video`, { file: base64data, title: title, categories: Object.keys(tags).filter((tag) => tags[tag]), created: startTime, postures: notifs, humes: DBEmotions, messages: messages, }) .then((res) => { toast("✅ Video Uploaded Successfully"); setRecordedVideo([]); router.push("/dashboard"); setLoading(false); }) .catch((err) => { toast("❌ Internal Server Error"); }); // console.log("final notifs", notifs); }; } }, [recordedVideo]); // const load = async () => { // const detector = await poseDetection.createDetector( // poseDetection.SupportedModels.MoveNet, // { modelType: poseDetection.movenet.modelType.SINGLEPOSE_LIGHTNING } // ); // }; useInterval(async () => { const detector = await poseDetection.createDetector( poseDetection.SupportedModels.MoveNet, { modelType: poseDetection.movenet.modelType.SINGLEPOSE_LIGHTNING }, ); detect(detector); }, 3000); const detect = async (detector) => { console.log(notifs); if (detector && webcamRef) { if (webcamRef.current.video) { const poses = await detector.estimatePoses(webcamRef.current.video); if (poses[0]) { checkHeadTilt(poses[0].keypoints); checkShoulderTilt(poses[0].keypoints); console.log(body); body !== "upper" && checkHipTilt(poses[0].keypoints); body !== "upper" && checkLegTitle(poses[0].keypoints); } } } requestAnimationFrame(detect); }; const checkHeadTilt = async (poses) => { const leftEye = poses[1]; const leftEar = poses[3]; const rightEye = poses[2]; const rightEar = poses[4]; if (leftEar.y < leftEye.y || rightEar.y < rightEye.y) { setNotifs((prev) => [ ...prev, { timestamp: new Date(), type: "face", message: "Look towards the audience!", }, ]); console.log("LOOKING DOWN"); } }; const checkShoulderTilt = async (poses) => { const leftShoulder = poses[5]; const rightShoulder = poses[6]; const margin = 25; if ( leftShoulder.y + margin < rightShoulder.y || leftShoulder.y - margin > rightShoulder.y || rightShoulder.y + margin < leftShoulder.y || rightShoulder.y - margin > leftShoulder.y [truncated — 8259 more characters] ``` ### src/app/api/message/route.js ```javascript import { NextResponse } from "next/server"; import prisma from "../../../../prismaClient"; export async function POST(req) { const res = NextResponse; const { message, videoId } = await req.json(); console.log(message); const response = await prisma.messages.create({ data: { videoId: videoId, userImg: "img", message: message, timestamp: new Date(), }, }); return res.json(response); } ``` ### src/app/api/videos/route.js ```javascript import cloudinary from "cloudinary"; import { NextResponse } from "next/server"; import prisma from "../../../../prismaClient"; import { getServerSession } from "next-auth/next"; import { authOptions } from "@/app/api/auth/[...nextauth]/route"; cloudinary.config({ cloud_name: process.env.NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME, api_key: process.env.NEXT_PUBLIC_CLOUDINARY_API_KEY, api_secret: process.env.NEXT_PUBLIC_CLOUDINARY_API_SECRET, }); export async function GET(req) { const res = NextResponse; const videoId = req.nextUrl.searchParams.get("videoId"); if (videoId) { const response = await prisma.videos.findUnique({ where: { identifier: videoId, }, }); if (response) return res.json(response); else return res.json(500); } else { const session = await getServerSession(authOptions); const response = await prisma.videos.findMany({ where: { userId: session.user.id, }, orderBy: [ { created: "desc", }, ], }); if (response) return res.json(response); else return res.json(500); } } ``` ### src/app/join/[...id]/page.js ```javascript "use client"; import { useEffect, useState } from "react"; import { socket } from "../../../../socket"; import { BiSolidSend } from "react-icons/bi"; import toast from "react-hot-toast"; import { useSession } from "next-auth/react"; const Page = ({ params }) => { const [frame, setFrame] = useState(null); const [message, setMessage] = useState(""); const [messages, setMessages] = useState([]); const { data: session } = useSession(); useEffect(() => { socket.connect(); socket.emit("join", params.id[0]); socket.on("frames", (image) => { setFrame(image); }); socket.on("audio", (audio) => { const voice = new Audio(audio); voice.play(); }); return () => socket.disconnect(); }, []); const handleSend = (e) => { e.preventDefault(); if (message === "") { toast("❌ Please enter a message!"); return; } socket.emit("message", { message, id: params.id[0], img: session.user.image, }); setMessages([...messages, { message, timestamp: new Date() }]); setMessage(""); }; return (
{frame && } {!frame && (
Apologies! This connection is not taking longer than expected! If this continues to happen, please close this tab and open the link again.
)}
{messages.map(({ message, timestamp }, index) => (
{message} {timestamp.toLocaleTimeString()}
))}
setMessage(e.target.value)} />
); }; export default Page; ``` ### src/app/api/video/route.js ```javascript import cloudinary from "cloudinary"; import { NextResponse } from "next/server"; import prisma from "../../../../prismaClient"; import { getServerSession } from "next-auth/next"; import { authOptions } from "@/app/api/auth/[...nextauth]/route"; import { POSTURES } from "@/data/Posture"; import { EMOTIONS_NEG, EMOTIONS_POS } from "@/data/Emotions"; cloudinary.config({ cloud_name: process.env.NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME, api_key: process.env.NEXT_PUBLIC_CLOUDINARY_API_KEY, api_secret: process.env.NEXT_PUBLIC_CLOUDINARY_API_SECRET, }); export async function POST(req) { const session = await getServerSession(authOptions); const res = NextResponse; const { file, title, categories, created, postures, humes, messages } = await req.json(); const video = await cloudinary.v2.uploader.upload(file, { resource_type: "video", }); const score = Math.abs( 100 - 2 * postures.length - 2 * humes.filter((hume) => EMOTIONS_NEG.includes(hume.emotionName)).length + 2 * humes.filter((hume) => EMOTIONS_POS.includes(hume.emotionName)).length, ); await prisma.videos.create({ data: { identifier: video.public_id, userId: session.user.id, created: created, score: score, title: title, categories: categories.join(","), }, }); if (postures.length > 0) await prisma.posture.createMany({ data: postures.map((posture) => { return { ...posture, videoId: video.public_id, type: POSTURES[posture.type], }; }), }); if (humes.length > 0) await prisma.hume.createMany({ data: humes.map((hume) => { return { ...hume, videoId: video.public_id, }; }), }); if (messages.length > 0) await prisma.messages.createMany({ data: messages.map((message) => { return { ...message, userImg: "img", videoId: video.public_id, }; }), }); return res.json(200); } export async function GET(req) { const res = NextResponse; const videoId = req.nextUrl.searchParams.get("videoId"); const response = await prisma.videos.findUnique({ where: { identifier: videoId, }, }); const postures = await prisma.posture.findMany({ where: { videoId: videoId, }, orderBy: [ { timestamp: "desc", }, ], }); const humes = await prisma.hume.findMany({ where: { videoId: videoId, }, orderBy: [ { timestamp: "desc", }, ], }); const messages = await prisma.messages.findMany({ where: { videoId: videoId, }, orderBy: [ { timestamp: "desc", }, ], }); return res.json({ video: response, postures: postures.map((posture) => { return { ...posture, check: "postures", timestamp: Math.abs(new Date(posture.timestamp) - new Date(response.created)) / 1000, }; }), humes: humes.map((hume) => { return { ...hume, check: "humes", timestamp: Math.abs(new Date(hume.timestamp) - new Date(response.created)) / 1000, }; }), messages: messages.map((message) => { return { ...message, check: "messages", timestamp: Math.abs(new Date(message.timestamp) - new Date(response.created)) / 1000, }; }), }); } export async function PUT(req) { const res = NextResponse; const { title, videoId, action } = await req.json(); if (action === "delete") { const snapShot = await prisma.videos.deleteMany({ where: { identifier: { in: videoId, }, }, }); if (snapShot) return res.json(snapShot); else return res.json(500); } else { const snapShot = await prisma.videos.update({ where: { identifier: videoId, }, data: { title: title, }, }); if (snapShot) return res.json(snapShot); else return res.json(500); } } ``` ### src/app/review/[id]/page.js ```javascript "use client"; import { BsArrowRightShort } from "react-icons/bs"; import { CircularProgressbar } from "react-circular-progressbar"; import "react-circular-progressbar/dist/styles.css"; import "react-circular-progressbar/dist/styles.css"; import VideoPlayer from "@/components/VideoPlayer"; import { useEffect, useState } from "react"; import axios from "axios"; import toast from "react-hot-toast"; import Postures from "@/components/Postures"; import Details from "@/components/Details"; import BreakDown from "@/components/BreakDown"; import { EMOTIONS_NEG, EMOTIONS_POS } from "@/data/Emotions"; const review = ({ params }) => { const [title, setTitle] = useState(""); const [vedio, setVedio] = useState({}); const [messages, setMessages] = useState([]); const [breakdownView, setBreakdownView] = useState(false); const [postures, setPostures] = useState([]); const [humes, setHumes] = useState([]); const [selectedTag, setSelectedTag] = useState({}); useEffect(() => { axios.get(`/api/video?videoId=${params.id}`).then((res) => { setVedio(res.data.video); setTitle(res.data.video.title); setPostures(res.data.postures); setHumes(res.data.humes); setMessages(res.data.messages); console.log(res.data); }); }, []); return (