# Project export: Quizcribe

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 11.0
- Tagline: Retaining information from lectures is hard. Built for learners of all levels, Quizscribe provides AI-powered summaries and interactive quizzes for efficient and convenient self-regulated learning.
- Devpost: https://devpost.com/software/quizcribe
- GitHub: https://github.com/notrichardpeng/Quizcribe
- Video: https://www.youtube.com/embed/CBIhPNf5tdw?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 4 GitHub contributor(s) — notrichardpeng (71 commits), Michael Siu (33 commits), AmieeYL (32 commits), Zhen Liu (20 commits)

## Devpost submission (written by the team)

### Inspiration

Quizcribe is inspired by the problem that many students face in retaining the knowledge that they learned from class lectures or open-course videos. Through AI-powered summaries and interactive quizzes, we hope to make learning more effective and expand the effectiveness of self-regulated learning.

### What it does

Quizcribe is an educational software that supplements students with AI-powered study resources generated from inputted videos or audio. By entering a link to a video, such as one from YouTube, Quizcribe automatically transcribes the video, returns a detailed summary of its content, and generates an interactive quiz game for knowledge testing and review.

### How we built it

Frontend: Next.js/React Backend: Django/Python APIs: Deepgram (Speech to Text), Google Gemini 1.5, YouTube DL

### Challenges we ran into

Extracting video URL from web URL (Resolve) Gemini prompt design and output format had to be very precise (Resolved) Frontend and backend connection issues with generated interactive quizzes (Resolved)

### Accomplishments we're proud of

The summaries and interactive quizzes generated by Quizcribe is very accurate to the content of the inputted video/audio Our team was able to extend Quizcribe's video transcription and processing service to 16 different languages

### What we learned

Fast-paced app development Use of AI and LLM in video/audio transcription and processing Using Next.js for frontend development Developing REST API in Django Version control with Git

### What's next

Support for video file upload and other URL domains e.g. Coursera, Khan Academy, Zoom Enable translation to languages other than English Expanding support for more than 16 languages Increasing the efficiency and accuracy of video transcription and summary/quiz generation

## README (from the GitHub repository)

# Quizcribe
![屏幕截图 2024-10-20 112001](https://github.com/user-attachments/assets/bcb28a88-70d2-49b2-a65e-5ca9e130e8b4)

Calhacks 11.0 Michael Siu, Richard Peng, Yiling Yuan, Zhen Liu

## Overview
Quizcribe is an educational software that supplements students with AI-powered study resources generated from inputted videos or audio.
By entering a link to a video, such as one from YouTube, Quizcribe automatically transcribes the video, returns a detailed summary of its content, and generates an interactive quiz game for knowledge testing and review.  

Through providing a more versatile, time-efficient, and fun study medium, Quizcribe aims to expand the effectiveness of free online education, further expanding the accessibility of educational resources to people of all backgrounds. 

Main Features:
* Automated speech-to-text transcription that converts videos/audio of any length into text.
* AI-powered video summary and interactive quiz generation.
* Video detection and transcription support for 16 languages. 

## Running & Installation
To install dependencies:
```
pip install -r requirements.txt
```
```
cd calhacksmryz
npm install
```

Run App and API Server:
```
npm run dev
```
```
cd backend
python manage.py runserver
```
The app should be at `localhost:3000`

## Tech Stack
* Frontend: Next.js/React, Tailwind CSS, Material UI
* Backend: Django/Python
* API: Deepgram (Speech to Text), Google Gemini 1.5, YouTube DL

## Key Functionalities

### Video/Audio to Transcript Generation using Deepgram
* Users are prompted to input a link, which is passed to the YouTube DL and extracted into a video file URL.
* Deepgram AI's Speech-To-Text API is then used to convert the audio into transcripts.
* Deepgram AI's built-in language detection system allows for the transcription of videos/audio in 16 languages.

### Content Summarization using Google Gemini
* Gemini API is used to automatically provide consistently structured summaries for the plain text transcripts.
### Interactive Quiz Generation using Google Gemini
* Using a chain of thought approach, problem sets are generated from the summarized content. The logic behind this feature involves step-by-step prompt engineering that ensures the generated questions are relevant, thought-provoking, and aligned with the key concepts in the summarized content.
* Corrent answer are determined by AI-powered logic reasoning.

### Demo
![屏幕截图 2024-10-20 112138](https://github.com/user-attachments/assets/b9d00b12-2ddc-4c5c-958e-a594b7e33154)
_Summary to video url provided_

![image (2)](https://github.com/user-attachments/assets/e7b33303-5641-4325-8a1c-6dd0e53fd341)
_Quiz_

![image (3)](https://github.com/user-attachments/assets/293ff408-c2a7-4627-b823-908b60d44961)
_Result_


## Detected evidence (automated analysis)

Indexed codebase: 25 recognized source files, 164 KB.
- CSS (language) — detected in the code
- Django (technology) — detected in the code
- JavaScript (language) — detected in the code
- Next.js (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- Google Gemini (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (32 of 32)

```
.gitignore
backend/ai_core/__init__.py
backend/ai_core/gemini_app.py
backend/ai_core/transcribe.py
backend/backend/__init__.py
backend/backend/asgi.py
backend/backend/settings.py
backend/backend/urls.py
backend/backend/wsgi.py
backend/manage.py
backend/quizcribe_api/__init__.py
backend/quizcribe_api/admin.py
backend/quizcribe_api/apps.py
backend/quizcribe_api/urls.py
backend/quizcribe_api/views.py
calhacksmryz/.eslintrc.json
calhacksmryz/.gitignore
calhacksmryz/app/api/route.js
calhacksmryz/app/layout.js
calhacksmryz/app/page.js
calhacksmryz/app/quiz/card.jsx
calhacksmryz/app/quiz/page.js
calhacksmryz/app/quiz/result.jsx
calhacksmryz/app/styles/globals.css
calhacksmryz/jsconfig.json
calhacksmryz/next.config.mjs
calhacksmryz/package.json
calhacksmryz/postcss.config.mjs
calhacksmryz/README.md
calhacksmryz/tailwind.config.js
README.md
requirements.txt
```

### Dependencies

- calhacksmryz/package.json: @emotion/react@^11.13.3, @emotion/styled@^11.13.0, @mui/icons-material@^6.1.4, @mui/material@^6.1.4, eslint@^8, eslint-config-next@14.2.15, next@^14.2.15, postcss@^8, react@^18.3.1, react-dom@^18.3.1, react-simple-typewriter@^5.0.1, tailwindcss@^3.4.1
- requirements.txt: deepgram_sdk@==3.7.4, Django@==5.1.2, django-cors-headers@==4.5.0, djangorestframework@==3.15.2, google-generativeai@==0.8.3, python-dotenv@==1.0.1, yt_dlp@==2024.10.7

### Recent commits (newest first)

- Update README.md
- Update README.md
- README
- last commit?
- Merge branch 'main' of github.com:notrichardpeng/calhacksmryz into main
- merge main
- Merge pull request #38 from notrichardpeng/frontv2
- more regex
- delete upload.js
- Merge branch 'main' into frontv2
- Merge branch 'main' of https://github.com/notrichardpeng/calhacksmryz
- upload file name display
- Merge pull request #37 from WingchunSiu/main
- edit comments
- initial upload button
- className error
- Update README.md
- Update README.md
- Merge pull request #36 from notrichardpeng/readme-patch
- Update README.md

## Key source files (fetched from GitHub, selected and truncated for size)

### requirements.txt

```
deepgram_sdk==3.7.4
Django==5.1.2
djangorestframework==3.15.2
python-dotenv==1.0.1
yt_dlp==2024.10.7
google-generativeai==0.8.3
django-cors-headers==4.5.0

```

### calhacksmryz/package.json

```
{
  "name": "calhacksmryz",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "@emotion/react": "^11.13.3",
    "@emotion/styled": "^11.13.0",
    "@mui/icons-material": "^6.1.4",
    "@mui/material": "^6.1.4",
    "next": "^14.2.15",
    "react": "^18.3.1",
    "react-dom": "^18.3.1",
    "react-simple-typewriter": "^5.0.1"
  },
  "devDependencies": {
    "eslint": "^8",
    "eslint-config-next": "14.2.15",
    "postcss": "^8",
    "tailwindcss": "^3.4.1"
  }
}

```

### calhacksmryz/app/layout.js

```javascript
import localFont from "next/font/local";
import "./styles/globals.css";

const geistSans = localFont({
  src: "./fonts/GeistVF.woff",
  variable: "--font-geist-sans",
  weight: "100 900",
});
const geistMono = localFont({
  src: "./fonts/GeistMonoVF.woff",
  variable: "--font-geist-mono",
  weight: "100 900",
});

export const metadata = {
  title: "Quizcribe",
  description: "",
};

export default function RootLayout({ children }) {
  return (
    <html lang="en">
      <body
        className={`${geistSans.variable} ${geistMono.variable} antialiased`}
      >
        {children}
      </body>
    </html>
  );
}

```

### calhacksmryz/app/page.js

```javascript
'use client';
import React, { useState } from 'react';
import { useRouter } from 'next/navigation';
import {
  Container,
  TextField,
  Button,
  Box,
  Typography,
  CircularProgress,
  LinearProgress
} from "@mui/material";
import { Typewriter } from "react-simple-typewriter";

export default function Home() {
  const router = useRouter();
  const HOST_URL = "http://localhost:8000/quizcribe_api"

  const [url, setUrl] = useState('');
  const [file, setFile] = useState(null);
  const [fetched, setFetched] = useState('');
  const [isHidden, setIsHidden] = useState(false);
  const [isTesting, setIsTesting] = useState(false);

  const handleInputChange = (e) => {
    setUrl(e.target.value);
  };

  const handleFileChange = (e) => {
    const file = e.target.files[0]
    if (file) {
      setFile(file)
      setUrl(file.name)
    }
  };

  const handleSubmit = async () => {
    try {
      setIsHidden(true);

      const response = await fetch(`${HOST_URL}/transcribe/`, {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({ url }),
      });

      const data = await response.json();

      if (response.ok) {
        setFetched(data["response"]);
        setIsHidden(false);
      } else {
        console.error("Error:", data.error);
      }
    } catch (error) {
      console.error("Error submitting URL:", error);
    }
  };

  const enterQuiz = async () => {
    setIsTesting(true);

    const response = await fetch(`${HOST_URL}/generate-qa/`, {
      method: 'POST',
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ "text": fetched }),
    });

    const data = await response.json();
    sessionStorage.setItem("questions", JSON.stringify(data["response"]));
    router.push("/quiz");

  };

  return (
    <>
      <Container
        maxWidth="lg"
        sx={{
          minHeight: "100vh",
          display: "flex",
          flexDirection: "column",
          justifyContent: "center",
          alignItems: "center",
          overflow: "hidden",
          py: 10,
          px: { xs: 2, md: 4 },
        }}
      >
        <Box component="header" sx={{ width: { xs: "100%", md: "55%" } }}>
          <h1 style={{ fontSize: "3rem", fontWeight: "bold", fontFamily: 'Helvetica Neue' }}>
            <Typewriter
              words={["Quizcribe"]}
              cursor
              cursorStyle="_"
              loop={100}
              typeSpeed={100}
              deleteSpeed={80}
              delaySpeed={5000}
            ></Typewriter>
          </h1>

          <Typography variant="body1" fontWeight="semibold" sx={{ mt: 2, fontFamily: 'Helvetica Neue' }}>
            Rewriting open education.
          </Typography>
        </Box>

        {isHidden ? (
          <Box
            sx={{
              textAlign: "center",
              fontWeight: "semibold",
              mt: 10  
            }}
          >
            <CircularProgress size="3rem" />
            <p>Dissecting video... This may take a while...</p>
          </Box>
        ) : (
          <Box
            className='bg-white'
            component="form"
            onSubmit={handleSubmit}
            sx={{
              mt: 10,
              display: "flex",
              alignItems: "center",
              flexDirection: "column",
              gap: 2,
              p: 4,
              backgroundColor: "background.paper",
              borderRadius: "12px",
              width: { xs: "100%", md: "60%" },
              boxShadow: 1,
            }}
          >
            <Box
              sx={{
                display: "flex",
                gap: 2,
                width: "100%",
                alignItems: "center",
              }}
            >
              <TextField
                label="Video URL"
                variant="outlined"
                fullWidth
                value={url}
                onChange={handleInputChange}
              />
              <input
                accept="video/*"
                style={{ display: 'none' }}
                id="file-input"
                type="file"
                onChange={handleFileChange}
              />
              <label htmlFor="file-input">
                <Button 
                className="bg-zinc-700 text-white"
                component="span" 
                variant="contained"
                sx={{ 
                  height: '56px',
                  whiteSpace: 'nowrap' }}>
                Upload
                </Button>
              </label>
            </Box>

            <Button
              className="bg-zinc-700 text-white"
              variant="contained"
              onClick={handleSubmit}
              sx={{ width: "100%" }} 
            >
              Quizcribe
            </Button>
          </Box>
        )}

        {fetched && (
          <>
            <Box
              className='bg-white text-black'
              component="form"
              onSubmit={handleSubmit}
              sx={{
                mt: 10,
                display: "grid",
                gap: 2,
                p: 4,
                backgroundColor: "background.paper",
                borderRadius: "12px",
                width: { xs: "100%", md: "90%" },
                boxShadow: 1,
              }}
            >
              <Typography className="text-zinc-800" variant="h4" fontWeight="bold" sx={{ justifyContent: 'center', textAlign: 'center', }}>
                Summary
              </Typography>
              <Typography variant="h6" component="div">
                {fetched}
              </Typography>

            </Box>

            <Box sx={{ mt: 10, display: 'flex', justifyContent: 'center', alignItems: 'center' }}>
              {isTesting ? (
                <div style={{ textAlign: "center" }}>
                  <LinearProgress />
                  <p>Generating your quiz... Please wait...</p>
                </div>
[truncated — 1876 more characters]
```

### calhacksmryz/app/api/route.js

```javascript
import { NextResponse } from 'next/server';

export async function POST(req) {
  const { url } = await req.json();

  if (!url) {
    return NextResponse.json({ error: 'URL is required' }, { status: 400 });
  }

  return NextResponse.json({ url }, { status: 200 });
}
```

### calhacksmryz/app/quiz/page.js

```javascript
'use client';

import Head from 'next/head'
import { useRouter } from 'next/navigation';
import { useState, useEffect } from 'react';
import CardTemp from './card';
import QuizResult from './result';
import { Card, CardContent, CardActions, CardMedia, Box, Button, Typography, IconButton } from '@mui/material';

export default function QuizPage() {
  const questions = JSON.parse(sessionStorage.getItem('questions'));
  
  const [userAnswers, setUserAnswers] = useState([]);
  const [currentQuestionIndex, setCurrentQuestionIndex] = useState(0);
  const [selectedChoice, setSelectedChoice] = useState(null);
  const [score, setScore] = useState(0);
  const [submitted, setSubmitted] = useState(false);
  const [isCorrect, setIsCorrect] = useState(null);
  const [isFinished, setIsFinished] = useState(false);
  const totalQuestions = questions?.length;

  const handleChoiceSelect = (choice) => {
    if(!submitted){
      setSelectedChoice(choice);
      setSubmitted(false);
      setIsCorrect(null);  
    }
  };
  
  const handleSubmit = () => {
      if (selectedChoice) {
          setUserAnswers([...userAnswers, selectedChoice.text]);
          if (selectedChoice.correct) {
              setScore(score + 1);
              setIsCorrect(true);
          } else {
              setIsCorrect(false);
          }
          setSubmitted(true);
      }
  };

  const handleNext = () => {
    if (submitted) {
      if (currentQuestionIndex < questions.length - 1) {
        setCurrentQuestionIndex(currentQuestionIndex + 1);
        setSelectedChoice(null);
        setSubmitted(false);
        setIsCorrect(null);
      } else {
        setIsFinished(true)
      }
    }
  };

  return (
    <>
      <Head>
        <title>QuizPage</title>
        <meta name="viewport" content="initial-scale=1.0, width=device-width" />
      </Head>

      <div className='h-[100vh] w-[100vw] flex justify-center ease-linear duration-300 items-center'>
        {questions !== undefined && (
          !isFinished ? (
              <CardTemp
                  question={questions[currentQuestionIndex]}
                  questionIndex={currentQuestionIndex + 1}
                  totalQuestions={totalQuestions}
                  onChoiceSelect={handleChoiceSelect}
                  onSubmit={handleSubmit}
                  onNext={handleNext}
                  selectedChoice={selectedChoice}
                  submitted={submitted}
                  isCorrect={isCorrect}
              />
          ) : (
              <QuizResult score={score} totalQuestions={totalQuestions} questions={questions} userAnswers={userAnswers} />
          )
        )}
      </div>
    </>
  )
}
```

### calhacksmryz/tailwind.config.js

```javascript
/** @type {import('tailwindcss').Config} */
module.exports = {
  content: [
    "./pages/**/*.{js,ts,jsx,tsx,mdx}",
    "./components/**/*.{js,ts,jsx,tsx,mdx}",
    "./app/**/*.{js,ts,jsx,tsx,mdx}",
  ],
  theme: {
    extend: {
      colors: {
        background: "var(--background)",
        foreground: "var(--foreground)",
      },
    },
  },
  plugins: [],
};

```

### backend/manage.py

```python
#!/usr/bin/env python
"""Django's command-line utility for administrative tasks."""
import os
import sys


def main():
    """Run administrative tasks."""
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "backend.settings")
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)


if __name__ == "__main__":
    main()

```

### backend/quizcribe_api/admin.py

```python
from django.contrib import admin

# Register your models here.

```

### backend/quizcribe_api/apps.py

```python
from django.apps import AppConfig


class quizcribe_apiConfig(AppConfig):
    default_auto_field = "django.db.models.BigAutoField"
    name = "quizcribe_api"

```

[12 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]