# Project export: EduScribe This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 10.0 - Tagline: Using AI to turn minutes into moments - Devpost: https://devpost.com/software/eduscribe - GitHub: https://github.com/VinnyXP/EduScribe - Demo: https://vectara-eduscribe.onrender.com/ - Result: winner (Best Use of Vectara) - Team: 2 GitHub contributor(s) — VinnyXP (12 commits), banyar-shin (1 commits) ## Devpost submission (written by the team) ### Inspiration In our journey as students, we have come to recognize the profound predicament that ensues when attempting to glean meaningful insights from lengthy video content. The struggle to synthesize information effectively from such resources is palpable. In light of this predicament, we embarked on a mission to craft a solution that not only ameliorates this issue but also bestows a boon to our fellow students by saving their precious time. We sought to empower them with a tool that not only facilitates the curation of their educational content but also ensures the utmost convenience by allowing them to revisit videos pertinent to specific topics. ### What it does Our innovative product, EduScribe, is endowed with multifaceted capabilities aimed at enhancing the educational experience of its users. It excels in the art of distilling high-quality summaries from YouTube videos, thereby rendering the extraction of essential insights an effortless endeavor. Furthermore, EduScribe takes the onus of maintaining a comprehensive record of the user's viewing history, facilitating easy reference to previously watched videos. This feature is invaluable for those seeking to revisit content related to a specific subject matter. In addition, EduScribe offers a succinct overview of search results based on the user's content history, thereby streamlining the process of finding pertinent information. ### How we built it The development of EduScribe was an endeavor that harnessed the capabilities of three cutting-edge technologies, namely Vectara AI, Together.ai, and Convex. The fusion of these technological marvels resulted in the creation of our robust product. Vectara AI contributed its prowess in frontend development, ensuring a seamless user interface. Together.ai played a pivotal role in facilitating data migration, a task often fraught with challenges. Convex, while a potent tool, presented its own set of challenges, particularly in comprehending the syntax for database querying. Additionally, the process of fine-tuning an AI model for summary generation and implementing zero-shot prompting for the LLaMa 2 models posed intricate challenges that we had to surmount. ### Challenges we ran into Our journey in developing EduScribe was not without its share of obstacles. Foremost among these was the formidable challenge of querying from the Convex database. The intricacies of database querying, coupled with the necessity of understanding and implementing the correct syntax, proved to be a daunting task. On the frontend, integrating Vectara AI presented its own set of difficulties, albeit surmountable. Data migration, while a crucial step in the development process, was not without its complications. Fine-tuning an AI model for summary generation, a task requiring precision and expertise, demanded a significant investment of time and effort. Moreover, implementing zero-shot prompting for the LLaMa 2 models, though a promising avenue, presented its own unique challenges that required creative problem-solving. ### Accomplishments we're proud of Our journey in developing EduScribe culminated in a series of accomplishments that we hold in high esteem. Most notably, we successfully crafted a full-stack project, reflecting our proficiency in both frontend and backend development. The crowning achievement of our endeavor was the creation of a sophisticated machine learning model for summary generation, a feat that showcases our commitment to innovation and excellence in the field of artificial intelligence. ### What we learned The development of EduScribe was a crucible of learning, where we immersed ourselves in a plethora of new and diverse technologies. Throughout this journey, we gained invaluable insights and knowledge, transcending our previous boundaries and expanding our technological horizons. Our acquisition of expertise in a variety of domains and the cultivation of our problem-solving skills have left an indelible mark on our development team. ### What's next The horizon of possibilities for EduScribe is broad and promising. Our future endeavors include the transformation of EduScribe into a browser extension, thereby increasing its accessibility and usability. This expansion will further enhance the educational experience of our users, making EduScribe an indispensable tool for the academic journey. ## README (from the GitHub repository) # EduScribe Using AI to turn watching minutes into moments CalHacks 10.0 Project (Best Use of Vectara) ## Inspiration In our journey as students, we have come to recognize the profound predicament that ensues when attempting to glean meaningful insights from lengthy video content. The struggle to synthesize information effectively from such resources is palpable. In light of this predicament, we embarked on a mission to craft a solution that not only ameliorates this issue but also bestows a boon to our fellow students by saving their precious time. We sought to empower them with a tool that not only facilitates the curation of their educational content but also ensures the utmost convenience by allowing them to revisit videos pertinent to specific topics. ## What it does Our innovative product, EduScribe, is endowed with multifaceted capabilities aimed at enhancing the educational experience of its users. It excels in the art of distilling high-quality summaries from YouTube videos, thereby rendering the extraction of essential insights an effortless endeavor. Furthermore, EduScribe takes the onus of maintaining a comprehensive record of the user's viewing history, facilitating easy reference to previously watched videos. This feature is invaluable for those seeking to revisit content related to a specific subject matter. In addition, EduScribe offers a succinct overview of search results based on the user's content history, thereby streamlining the process of finding pertinent information. ## How we built it The development of EduScribe was an endeavor that harnessed the capabilities of three cutting-edge technologies, namely Vectara AI, Together.ai, and Convex. The fusion of these technological marvels resulted in the creation of our robust product. Vectara AI contributed its prowess in frontend development, ensuring a seamless user interface. Together.ai played a pivotal role in facilitating data migration, a task often fraught with challenges. Convex, while a potent tool, presented its own set of challenges, particularly in comprehending the syntax for database querying. Additionally, the process of fine-tuning an AI model for summary generation and implementing zero-shot prompting for the LLaMa 2 models posed intricate challenges that we had to surmount. ## Challenges we ran into Our journey in developing EduScribe was not without its share of obstacles. Foremost among these was the formidable challenge of querying from the Convex database. The intricacies of database querying, coupled with the necessity of understanding and implementing the correct syntax, proved to be a daunting task. On the frontend, integrating Vectara AI presented its own set of difficulties, albeit surmountable. Data migration, while a crucial step in the development process, was not without its complications. Fine-tuning an AI model for summary generation, a task requiring precision and expertise, demanded a significant investment of time and effort. Moreover, implementing zero-shot prompting for the LLaMa 2 models, though a promising avenue, presented its own unique challenges that required creative problem-solving. ## Accomplishments that we're proud of Our journey in developing EduScribe culminated in a series of accomplishments that we hold in high esteem. Most notably, we successfully crafted a full-stack project, reflecting our proficiency in both frontend and backend development. The crowning achievement of our endeavor was the creation of a sophisticated machine learning model for summary generation, a feat that showcases our commitment to innovation and excellence in the field of artificial intelligence. ## What we learned The development of EduScribe was a crucible of learning, where we immersed ourselves in a plethora of new and diverse technologies. Throughout this journey, we gained invaluable insights and knowledge, transcending our previous boundaries and expanding our technological horizons. Our acquisition of expertise in a variety of domains and the cultivation of our problem-solving skills have left an indelible mark on our development team. ## What's next for EduScribe The horizon of possibilities for EduScribe is broad and promising. Our future endeavors include the transformation of EduScribe into a browser extension, thereby increasing its accessibility and usability. This expansion will further enhance the educational experience of our users, making EduScribe an indispensable tool for the academic journey. ## Detected evidence (automated analysis) Indexed codebase: 68 recognized source files, 146 KB. - CSS (language) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - Node.js (technology) — claimed on Devpost, not found in the code - Python (language) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (77 of 77) ``` eduscribe/_gitignore eduscribe/.eslintrc.cjs eduscribe/.gitignore eduscribe/components.json eduscribe/convex/_generated/api.d.ts eduscribe/convex/_generated/api.js eduscribe/convex/_generated/dataModel.d.ts eduscribe/convex/_generated/server.d.ts eduscribe/convex/_generated/server.js eduscribe/convex/_schema.ts eduscribe/convex/auth.config.js eduscribe/convex/myActions.ts eduscribe/convex/myFunctions.ts eduscribe/convex/README.md eduscribe/convex/tsconfig.json eduscribe/example.js eduscribe/index.html eduscribe/package.json eduscribe/postcss.config.js eduscribe/README.md eduscribe/src/App.css eduscribe/src/App.tsx eduscribe/src/components/typography/code.tsx eduscribe/src/components/typography/link.tsx eduscribe/src/components/ui/accordion.tsx eduscribe/src/components/ui/alert-dialog.tsx eduscribe/src/components/ui/alert.tsx eduscribe/src/components/ui/aspect-ratio.tsx eduscribe/src/components/ui/avatar.tsx eduscribe/src/components/ui/badge.tsx eduscribe/src/components/ui/button.tsx eduscribe/src/components/ui/calendar.tsx eduscribe/src/components/ui/card.tsx eduscribe/src/components/ui/checkbox.tsx eduscribe/src/components/ui/collapsible.tsx eduscribe/src/components/ui/command.tsx eduscribe/src/components/ui/context-menu.tsx eduscribe/src/components/ui/dialog.tsx eduscribe/src/components/ui/dropdown-menu.tsx eduscribe/src/components/ui/form.tsx eduscribe/src/components/ui/hover-card.tsx eduscribe/src/components/ui/input.tsx eduscribe/src/components/ui/label.tsx eduscribe/src/components/ui/menubar.tsx eduscribe/src/components/ui/navigation-menu.tsx eduscribe/src/components/ui/popover.tsx eduscribe/src/components/ui/progress.tsx eduscribe/src/components/ui/radio-group.tsx eduscribe/src/components/ui/scroll-area.tsx eduscribe/src/components/ui/select.tsx eduscribe/src/components/ui/separator.tsx eduscribe/src/components/ui/sheet.tsx eduscribe/src/components/ui/skeleton.tsx eduscribe/src/components/ui/slider.tsx eduscribe/src/components/ui/switch.tsx eduscribe/src/components/ui/table.tsx eduscribe/src/components/ui/tabs.tsx eduscribe/src/components/ui/textarea.tsx eduscribe/src/components/ui/toast.tsx eduscribe/src/components/ui/toaster.tsx eduscribe/src/components/ui/toggle.tsx eduscribe/src/components/ui/tooltip.tsx eduscribe/src/components/ui/use-toast.ts eduscribe/src/ErrorBoundary.tsx eduscribe/src/index.css eduscribe/src/lib/utils.tsx eduscribe/src/LoginPage.tsx eduscribe/src/main.tsx eduscribe/src/manifest.json eduscribe/src/SigningUp.tsx eduscribe/tailwind.config.js eduscribe/tsconfig.json eduscribe/tsconfig.node.json eduscribe/vite.config.ts eduscribe/webpack.config.js eduscribe/yt-script.js README.md ``` ### Dependencies - eduscribe/package.json: @clerk/clerk-react@^4.27.1, @hookform/resolvers@^3.3.2, @radix-ui/react-accordion@^1.1.2, @radix-ui/react-alert-dialog@^1.0.5, @radix-ui/react-aspect-ratio@^1.0.3, @radix-ui/react-avatar@^1.0.4, @radix-ui/react-checkbox@^1.0.4, @radix-ui/react-collapsible@^1.0.3, @radix-ui/react-context-menu@^2.1.5, @radix-ui/react-dialog@^1.0.5, @radix-ui/react-dropdown-menu@^2.0.6, @radix-ui/react-hover-card@^1.0.7, @radix-ui/react-icons@^1.3.0, @radix-ui/react-label@^2.0.2, @radix-ui/react-menubar@^1.0.4, @radix-ui/react-navigation-menu@^1.1.4, @radix-ui/react-popover@^1.0.7, @radix-ui/react-progress@^1.0.3, @radix-ui/react-radio-group@^1.1.3, @radix-ui/react-scroll-area@^1.0.5, @radix-ui/react-select@^2.0.0, @radix-ui/react-separator@^1.0.3, @radix-ui/react-slider@^1.1.2, @radix-ui/react-slot@^1.0.2, @radix-ui/react-switch@^1.0.3, @radix-ui/react-tabs@^1.0.4, @radix-ui/react-toast@^1.1.5, @radix-ui/react-toggle@^1.0.3, @radix-ui/react-tooltip@^1.0.7, @types/node@^20.8.9, @types/react@^18.2.21, @types/react-dom@^18.2.7, @typescript-eslint/eslint-plugin@^6.7.0, @typescript-eslint/parser@^6.7.0, @vitejs/plugin-react@^4.0.4, autoprefixer@^10.4.16, axios@^1.6.0, class-variance-authority@^0.7.0, clsx@^2.0.0, cmdk@^0.2.0, convex@^1.4.1, date-fns@^2.30.0, dotenv@^16.3.1, eslint@^8.49.0, eslint-plugin-react-hooks@^4.6.0, eslint-plugin-react-refresh@^0.4.3, form-data@^4.0.0, npm-run-all@^4.1.5, postcss@^8.4.30, react@^18.2.0, react-day-picker@^8.9.1, react-dom@^18.2.0, react-hook-form@^7.47.0, tailwind-merge@^1.14.0, tailwindcss@^3.3.3, tailwindcss-animate@^1.0.7, typescript@^5.2.2, vite@^4.4.9, youtube-captions-scraper@^2.0.1, youtube-transcript@^1.0.6, zod@^3.22.4 ### Recent commits (newest first) - Update README.md - Merge pull request #2 from VinnyXP/vinh-ts - Merge branch 'main' into vinh-ts - Tweak to max_tokens constant and handling setNewUrl - Set up .env and API keys (#1) - remove the non used hook - Completed Convex fullstack app - Connected to Vectura AI DB - Testing with tanuj - update frontend, writing query to display analysis now - established the pipeline to get video transcript - created script to complete transcripts, need to adapt to convex - Set up .env and API keys - Initial convex setup - Initial commit ## Key source files (fetched from GitHub, selected and truncated for size) ### eduscribe/package.json ``` { "name": "eduscribe", "private": true, "version": "0.0.0", "type": "module", "scripts": { "dev": "npm-run-all dev:init --parallel dev:frontend dev:backend", "dev:frontend": "vite --open", "dev:backend": "convex dev", "dev:init": "convex dev --until-success && convex dashboard", "build": "tsc && vite build", "lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0", "preview": "vite preview" }, "dependencies": { "@clerk/clerk-react": "^4.27.1", "@hookform/resolvers": "^3.3.2", "@radix-ui/react-accordion": "^1.1.2", "@radix-ui/react-alert-dialog": "^1.0.5", "@radix-ui/react-aspect-ratio": "^1.0.3", "@radix-ui/react-avatar": "^1.0.4", "@radix-ui/react-checkbox": "^1.0.4", "@radix-ui/react-collapsible": "^1.0.3", "@radix-ui/react-context-menu": "^2.1.5", "@radix-ui/react-dialog": "^1.0.5", "@radix-ui/react-dropdown-menu": "^2.0.6", "@radix-ui/react-hover-card": "^1.0.7", "@radix-ui/react-icons": "^1.3.0", "@radix-ui/react-label": "^2.0.2", "@radix-ui/react-menubar": "^1.0.4", "@radix-ui/react-navigation-menu": "^1.1.4", "@radix-ui/react-popover": "^1.0.7", "@radix-ui/react-progress": "^1.0.3", "@radix-ui/react-radio-group": "^1.1.3", "@radix-ui/react-scroll-area": "^1.0.5", "@radix-ui/react-select": "^2.0.0", "@radix-ui/react-separator": "^1.0.3", "@radix-ui/react-slider": "^1.1.2", "@radix-ui/react-slot": "^1.0.2", "@radix-ui/react-switch": "^1.0.3", "@radix-ui/react-tabs": "^1.0.4", "@radix-ui/react-toast": "^1.1.5", "@radix-ui/react-toggle": "^1.0.3", "@radix-ui/react-tooltip": "^1.0.7", "axios": "^1.6.0", "class-variance-authority": "^0.7.0", "clsx": "^2.0.0", "cmdk": "^0.2.0", "convex": "^1.4.1", "date-fns": "^2.30.0", "dotenv": "^16.3.1", "form-data": "^4.0.0", "react": "^18.2.0", "react-day-picker": "^8.9.1", "react-dom": "^18.2.0", "react-hook-form": "^7.47.0", "tailwind-merge": "^1.14.0", "tailwindcss-animate": "^1.0.7", "youtube-captions-scraper": "^2.0.1", "youtube-transcript": "^1.0.6", "zod": "^3.22.4" }, "devDependencies": { "@types/node": "^20.8.9", "@types/react": "^18.2.21", "@types/react-dom": "^18.2.7", "@typescript-eslint/eslint-plugin": "^6.7.0", "@typescript-eslint/parser": "^6.7.0", "@vitejs/plugin-react": "^4.0.4", "autoprefixer": "^10.4.16", "eslint": "^8.49.0", "eslint-plugin-react-hooks": "^4.6.0", "eslint-plugin-react-refresh": "^0.4.3", "npm-run-all": "^4.1.5", "postcss": "^8.4.30", "tailwindcss": "^3.3.3", "typescript": "^5.2.2", "vite": "^4.4.9" } } ``` ### eduscribe/src/main.tsx ```typescript import { ClerkProvider, useAuth } from "@clerk/clerk-react"; import { ConvexReactClient} from "convex/react"; import { ConvexProviderWithClerk } from "convex/react-clerk"; import React from "react"; import ReactDOM from "react-dom/client"; import App from "./App"; import { ErrorBoundary } from "./ErrorBoundary"; import "./index.css"; const convex = new ConvexReactClient(import.meta.env.VITE_CONVEX_URL as string); ReactDOM.createRoot(document.getElementById("root")!).render( ); ``` ### eduscribe/src/App.tsx ```typescript import { useState } from "react" import { Button } from "@/components/ui/button"; import { Input } from "./components/ui/input"; import { useAction } from "convex/react"; import { api } from "../convex/_generated/api"; import { Textarea } from "./components/ui/textarea"; //import { useUser } from "@clerk/clerk-react"; export default function App() { const [newUrl, setNewUrl] = useState("") const [summary, setSummary] = useState("") const extractTranscript = useAction(api.myFunctions.extractTranscript) function handleUrlChange(text) { setNewUrl(text) if (!text) { setSummary("") } } return (

EduScribe

Using AI to turn watching minutes into seconds

handleUrlChange(event.target.value)} placeholder="Enter your YouTube video url here" />