# Project export: Study Suite

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2024
- Tagline: Research shows students learn 20% more content through the active recall method. Our project converts lectures into quizzes that help student retain and understand the information they learn best.
- Devpost: https://devpost.com/software/study-suite
- GitHub: https://github.com/Daniel-Moraes1/UnifyAI
- Result: winner (Fetch.ai: Beyond Boundaries AI Agent Prize (4x Engraved iPads + 1v1 with CEO))
- Team: 3 GitHub contributor(s) — Daniel Moraes (13 commits), Ronald Arifin (10 commits), natesh-reddy (1 commits)

## Devpost submission (written by the team)

### Inspiration

Inspired by a shared vision for transformative education, our team is embarking on a project to enhance learning from recorded lectures. We believe that integrating interactive quizzes will foster active participation, deepen understanding, and improve retention for students. Through this initiative, we aim to revolutionize the learning experience by empowering students to engage with the material in a more dynamic and effective way.

### What it does

Through a series of meticulously designed agents, we facilitate every step of the process. Initially, our system effortlessly transforms video content into detailed lecture transcripts, ensuring accessibility and clarity. Next, these transcripts are seamlessly converted into interactive quizzes, engaging students in active learning. Our quiz auto-grader feature streamlines assessment, providing instant feedback and allowing students to revisit incorrect responses for further understanding. Moreover, our system offers valuable suggestions, pinpointing timestamps for review based on quiz mistakes, and recommending related lectures to deepen comprehension. By integrating these functionalities, we aim to optimize learning outcomes and empower students to navigate their educational pursuits with confidence and efficiency.

### How we built it

Agents is the whole idea of the project, so we needed a way to create responsive agents,and make them communicate with each other. Our project was constructed with a sophisticated blend of cutting-edge technologies. Leveraging Fetch.ai's framework, uagents, we orchestrated a network of AI agents to execute various tasks seamlessly. For the conversion of video to text transcripts, we harnessed the power of advanced APIs dedicated to this purpose, ensuring accuracy and efficiency in the process. Additionally, we integrated Language Model APIs, such as LLM's, to dynamically generate quizzes from the extracted transcripts, enabling personalized and contextually relevant assessment content. Through the strategic amalgamation of these tools and frameworks, we engineered a robust and adaptable system that streamlines the creation of interactive learning materials while harnessing the capabilities of artificial intelligence to enhance the educational experience for students.

### Challenges we ran into

Undertanding the different ways agents, protocols, and delta v communicate is our main struggle. We needed to understand the concepts from the network including Context, Protocol, Mailbox, and other things.

### Accomplishments we're proud of

We're proud to have created an end-to-end pipeline using AI agents, leveraging cutting-edge technologies to enhance the learning experience. Our seamless integration of advanced tools transforms video content into detailed transcripts and generates personalized quizzes, resulting in a dynamic and engaging platform. This accomplishment showcases our commitment to innovation and our dedication to empowering students with effective educational solutions.

### What we learned

Throughout this project, we've learned to harness Fetch.ai's uagents for building interactive AI agents and effectively utilize the latest text generation APIs. This journey has honed our skills in deploying uagents for seamless communication and collaboration while also mastering the capabilities of text generation APIs to dynamically create learning materials. This experience has broadened our technical expertise and highlighted the transformative potential of AI in education.

### What's next

Different format of questions like - Truth or False and even open ended questions add more agents to increase the functionality of the agents. Integrating wispr with the app to make a just voice integrated agent.

## README (from the GitHub repository)

# UnifyAI
LLM agents with the power to control multiplatform functionality

## Inspiration

Inspired by a shared vision for transformative education, our team is embarking on a project to enhance learning from recorded lectures. We believe that integrating interactive quizzes will foster active participation, deepen understanding, and improve retention for students. Through this initiative, we aim to revolutionize the learning experience by empowering students to engage with the material in a more dynamic and effective way.

## What it does

Through a series of meticulously designed agents, we facilitate every step of the process. Initially, our system effortlessly transforms video content into detailed lecture transcripts, ensuring accessibility and clarity. Next, these transcripts are seamlessly converted into interactive quizzes, engaging students in active learning. Our quiz auto-grader feature streamlines assessment, providing instant feedback and allowing students to revisit incorrect responses for further understanding. Moreover, our system offers valuable suggestions, pinpointing timestamps for review based on quiz mistakes, and recommending related lectures to deepen comprehension. By integrating these functionalities, we aim to optimize learning outcomes and empower students to navigate their educational pursuits with confidence and efficiency.

## How we built it

Agents is the whole idea of the project, so we needed a way to create responsive agents,and make them communicate with each other. 

Our project was constructed with a sophisticated blend of cutting-edge technologies. Leveraging [Fetch.ai](http://fetch.ai/)'s framework, uagents, we orchestrated a network of AI agents to execute various tasks seamlessly. For the conversion of video to text transcripts, we harnessed the power of advanced APIs dedicated to this purpose, ensuring accuracy and efficiency in the process. Additionally, we integrated Language Model APIs, such as LLM's, to dynamically generate quizzes from the extracted transcripts, enabling personalized and contextually relevant assessment content. Through the strategic amalgamation of these tools and frameworks, we engineered a robust and adaptable system that streamlines the creation of interactive learning materials while harnessing the capabilities of artificial intelligence to enhance the educational experience for students.

## Challenges we ran into

Undertanding the different ways agents, protocols, and delta v communicate is our main struggle. We needed to understand the concepts from the network including Context, Protocol, Mailbox, and other things.

## Accomplishments that we're proud of

We're proud to have created an end-to-end pipeline using AI agents, leveraging cutting-edge technologies to enhance the learning experience. Our seamless integration of advanced tools transforms video content into detailed transcripts and generates personalized quizzes, resulting in a dynamic and engaging platform. This accomplishment showcases our commitment to innovation and our dedication to empowering students with effective educational solutions.

## What we learned
We learned that Fetch AI is amazing in ways that we could 

## What's next for Study Suite

- Different format of questions like - Truth or False and even open ended questions
- add more agents to increase the functionality of the agents.
- Integrating wispr with the app to make a just voice integrated agent.


## Detected evidence (automated analysis)

Indexed codebase: 15 recognized source files, 23 KB.
- Python (language) — detected in the code
- OpenAI (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (19 of 19)

```
.gitignore
Agents/GraderAgent.py
Agents/helper.py
Agents/homeAgent.py
Agents/protocol/__init__.py
Agents/protocol/grader_proto.py
Agents/protocol/home_proto.py
Agents/protocol/protocol_models.py
Agents/protocol/summarizer_proto.py
Agents/protocol/yt_proto.py
Agents/question_gen.py
Agents/test.py
Agents/URLFetcherAgent.py
Agents/youtubeAgent.py
Agents/youtubeUser.py
poetry.lock
pyproject.toml
README.md
requirements.txt
```

### Dependencies

- requirements.txt: tortoise, tortoise-orm, uagents, uagents-ai-engine, youtube-transcript-api

### Recent commits (newest first)

- Merge pull request #1 from natesh-reddy/patch-1
- Update README.md
- Merge branch 'main' of https://github.com/Daniel-Moraes1/QuizFetcher
- imports working
- grade answers
- fasdfla
- imports
- Resolved merge conflicts
- Agent models and communication
- f
- delete questions model
- refactor
- added lazy loading questions
- Merge branch 'main' of https://github.com/Daniel-Moraes1/QuizFetcher
- added questions generator and transcript fix
- Reconstruction of Agents setup.
- endpoint changed
- Merge branch 'main' of https://github.com/Daniel-Moraes1/QuizFetcher
- grader agent
- gitignore

## Key source files (fetched from GitHub, selected and truncated for size)

### requirements.txt

```
youtube-transcript-api
uagents
uagents-ai-engine
# parse_qs
# urlparse
tortoise
tortoise-orm
```

### pyproject.toml

```
[tool.poetry]
name = "quizfetcher"
version = "0.1.0"
description = ""
authors = ["Michael Wiradharma <michael.wiradharma@berkeley.edu>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

```

### Agents/URLFetcherAgent.py

```python
from uagents import Agent, Context, Model, Protocol
from uagents.setup import fund_agent_if_low
from protocol import yt_proto

AGENT_MAILBOX_KEY = "3c1184b5-bf92-404a-9dfd-e77d01d23092"

yt = Agent(name="youtubeTranscriptGrabber",
            seed="youtube_transcript_grabber",
            mailbox=f"{AGENT_MAILBOX_KEY}@https://agentverse.ai",
            port=8001,
            endpoint=["http://localhost:8001/quiz_gen"])

print(f"agent name and addres: {yt.name} at {yt.address}")

fund_agent_if_low(yt.wallet.address())

yt.include(yt_proto, publish_manifest=True);

if __name__ == "__main__":
    yt.run()
```

### Agents/GraderAgent.py

```python
from uagents import Agent
from uagents.setup import fund_agent_if_low
from protocol import grader_proto

#AGENT_MAILBOX_KEY = "83342e4b-18c6-4d93-bcb6-bc958de3fd13"
AGENT_MAILBOX_KEY = "4fdd2708-d5aa-4ded-a2c6-24f16d6fad55"
grader = Agent(name="StudySuite",
            seed="StudySuite Agent",
            mailbox=f"{AGENT_MAILBOX_KEY}@https://agentverse.ai",
            port=8001,
            endpoint=["http://localhost:8001/studysuite"])

print(f"agent name and addres: {grader.name} at {grader.address}")

fund_agent_if_low(grader.wallet.address())

grader.include(grader_proto, publish_manifest=True);

if __name__ == "__main__":
    grader.run()
```

### Agents/youtubeUser.py

```python
from uagents import Agent, Context, Model, Protocol, Bureau
from uagents.setup import fund_agent_if_low
from youtube_transcript_api import YouTubeTranscriptApi


user = Agent(name="userAgent", 
            seed="userAgent")

class Begin(Model):
    url: str


@user.on_interval(5)
async def interval_task(ctx: Context):
    ctx.logger.info("User executing periodic task...")

@user.on_message(model=Begin)
async def handle_message(ctx: Context, sender: str, msg: Begin):
    transcript = getTranscript(msg.url)
    quiz = generateQuiz(transcript)

    ctx.logger.info(f"Got message from {sender}: {url}")

    
    if response.status_code == 200:
        ctx.logger.info(f"Got message from {sender}: {url}")

    else:
        return None
    

def getTranscript(url):
    id = url[url.index("v=")+2:]
    transcript = YouTubeTranscriptApi.get_transcript(id)
    return transcript

def generateQuiz(transcript)

    


if __name__ == "__main__":
    yt.run()
```

### Agents/youtubeAgent.py

```python
from uagents import Agent, Context, Model
from youtube_transcript_api import YouTubeTranscriptApi
import requests


yt = Agent(name="youtubeTranscriptGrabber", 
            seed="youtube_transcript_grabber",
            port=8001,
            endpoint=["http://localhost:8001/submit"])


class Message(Model):
    string_field: str


@yt.on_interval(5)
async def interval_task(ctx: Context):
    ctx.logger.info("Executing periodic task...")

@yt.on_message(Message)
async def handle_message(ctx: Context, sender: str, url: Message):
    transcript = getTranscript(url)
    ctx.logger.info(f"Got message from {sender}: {url}")

    
    if response.status_code == 200:
        ctx.logger.info(f"Got message from {sender}: {url}")

    else:
        return None
    

def getTranscript(url):
    id = url[url.index("v=")+2:]
    transcript = YouTubeTranscriptApi.get_transcript(id)
    return transcript

def createQuiz(transcript):
    

if __name__ == "__main__":
    yt.run()
```

### Agents/homeAgent.py

```python
from uagents import Agent, Context
from uagents.setup import fund_agent_if_low
from protocol import home_proto

#AGENT_MAILBOX_KEY = "83342e4b-18c6-4d93-bcb6-bc958de3fd13"
AGENT_MAILBOX_KEY = "ababf13b-88fd-4fc9-ad8d-a5b2289b3784"

home = Agent(name="StudySuite",
            seed="StudySuite",
            mailbox=f"{AGENT_MAILBOX_KEY}@https://agentverse.ai",
            port=8001,
            endpoint=["http://localhost:8001/welcome"]
            )

print(f"agent name and addres: {home.name} at {home.address}")
# @home.on_event("startup")
# async def welcome(ctx: Context):
#     ctx.logger.info(f"Welcome to {ctx.name}! My purpose is to help you study.")
#     ctx.logger.info(f"Currently, you can generate quizzes and summaries from videos to check and improve your understanding!")
#     ctx.logger.info(f"Would you like to start by generating a quiz or a summary?")

fund_agent_if_low(home.wallet.address())

# home.include(home_proto, publish_manifest=True)

if __name__ == "__main__":
    home.run()

```

### Agents/test.py

```python
from youtube_transcript_api import YouTubeTranscriptApi
import requests
from openai import OpenAI
import os
from dotenv import load_dotenv

load_dotenv()
client = OpenAI(api_key=os.getenv('OPENAI_API_KEY'))



def getTranscript(url):
    id = url[url.index("v=")+2:]
    transcript = YouTubeTranscriptApi.get_transcript(id)
    return transcript


from openai import OpenAI
client = OpenAI()

completion = client.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
  ]
)

print(completion.choices[0].message)


def createQuiz(transcript):
    json_format = {
        "question": "blblablabla",
        "options": ["blablabla", "blablabla", "blablabla", "blablabla"],
        "answer_index": 1,
    }
    # we just have to prompt the json to fix
    response = client.chat.completions.create(
    model = "gpt-3.5-turbo",
    messages=[
        {"role": "system", "content": "Create me 1 multiple question quiz with 3 options. Return me the json format: " + str(json_format) + "based on " + str(transcript)},
        {"role": "user", "content": "Hello!"}
    ],
    max_tokens=25000,
    )

    return response.choices[0].message


def main():
    url = "https://www.youtube.com/watch?v=3KaffTIZ5II"
    transcript = getTranscript(url)
    print(os.getenv("OPENAI_API_KEY"))
    quiz = createQuiz(transcript)
    print(quiz)

    return 0

main()
    
```

### Agents/question_gen.py

```python
from youtube_transcript_api import YouTubeTranscriptApi
import requests
from openai import OpenAI
import os
from dotenv import load_dotenv
import json

from urllib.parse import urlparse

load_dotenv()
client = OpenAI(api_key=os.getenv('OPENAI_API_KEY'))


def getTranscript(url):
    url_data = urlparse(url)
    video_id = ''

    if 'youtu.be' in url_data.netloc:
        # Extract video ID from path for shortened URLs
        video_id = url_data.path[1:]  # Remove the leading '/'
    else:
        # Extract video ID from query parameters for standard URLs
        query_params = dict(qc.split('=') for qc in url_data.query.split('&') if qc)
        if 'v' in query_params:
            video_id = query_params['v']

    if not video_id:
        raise ValueError("Could not extract video ID from URL.")

    transcript = YouTubeTranscriptApi.get_transcript(video_id)
    return transcript


# given a transcript, we create a quiz question
def createQuiz(transcript: str):
    json_format = """{
        "question": "blblablabla",
        "options": ["blablabla", "blablabla", "blablabla", "blablabla"],
        "answer_index": 1,
    }"""
    prompt = f"Create me 1 multiple question quiz with 3 options. Return me the json format: {str(json_format)} based on {str(transcript)}. Remember just the json string."
    # we just have to prompt the json to fix
    response = client.chat.completions.create(
    model = "gpt-4-0125-preview",
    messages=[
        {"role": "system", "content": "You are a question generator AI. Don't generate a question if it's not necessary. "},
        {"role": "user", "content": prompt}
    ],
    max_tokens=1500,
    )

    return response.choices[0].message.content

# a question generator that lazy loads. Call next on a gnerator to get the question
def quiz_generator(transcripts: list[str]):
    i = 0
    while i < len(transcripts):
        transcript = ' '.join([d['text'] for d in transcripts[i:i+9]])
        transcript = transcript.replace("\n", " ")
        # print(transcript)
        quiz = createQuiz(transcript)[7:-3]
        # print(quiz)
        testObj = json.loads(quiz)
        
        yield testObj  # This yields control back to the caller, resuming from here when next() is called again.
        i += 9

def get_next_question(question_gen):
    return next(question_gen)

# 
def get_summarized_text(transcripts):
    i = 0
    while i < len(transcripts):
        transcript = ' '.join([d['text'] for d in transcripts[i:i+9]])
        transcript = transcript.replace("\n", " ")
        # print(transcript)
        quiz = createQuiz(transcript)[7:-3]
        # print(quiz)
        testObj = json.loads(quiz)
        
        yield testObj  # This yields control back to the caller, resuming from here when next() is called again.
        i += 9
def main():
    # VIDEO about productivity
    url1 = "https://youtu.be/0CmtDk-joT4?si=q5AMUjRgaD37Xgnm"

    # VIDEO about ai video
    url2 = "https://youtu.be/NXpdyAWLDas?si=AMRXgzlvinyFApKI"
    transcripts = getTranscript(url1)
    # we loop every 10 transcript to create a quiz
    quizzes = quiz_generator(transcripts)  # This creates a generator object but doesn't start the function yet.
    print(get_next_question(quizzes))

    return 0

main()
    
```

### Agents/helper.py

```python
from youtube_transcript_api import YouTubeTranscriptApi
import requests
from openai import OpenAI
import os
from dotenv import load_dotenv
import json

from urllib.parse import urlparse

load_dotenv()
client = OpenAI(api_key=os.getenv('OPENAI_API_KEY'))


def getTranscript(url):
    url_data = urlparse(url)
    video_id = ''

    if 'youtu.be' in url_data.netloc:
        # Extract video ID from path for shortened URLs
        video_id = url_data.path[1:]  # Remove the leading '/'
    else:
        # Extract video ID from query parameters for standard URLs
        query_params = dict(qc.split('=') for qc in url_data.query.split('&') if qc)
        if 'v' in query_params:
            video_id = query_params['v']

    if not video_id:
        raise ValueError("Could not extract video ID from URL.")

    transcript = YouTubeTranscriptApi.get_transcript(video_id)
    return transcript


# given a transcript, we create a quiz question
def createQuiz(transcript: str):
    json_format = """{
        "question": "blblablabla",
        "options": ["blablabla", "blablabla", "blablabla", "blablabla"],
        "answer_index": 1,
    }"""
    prompt = f"Create me 1 multiple question quiz with 3 options. Return me the json format: {str(json_format)} based on {str(transcript)}. Remember just the json string."
    # we just have to prompt the json to fix
    response = client.chat.completions.create(
    model = "gpt-4-0125-preview",
    messages=[
        {"role": "system", "content": "You are a question generator AI. Don't generate a question if it's not necessary. "},
        {"role": "user", "content": prompt}
    ],
    max_tokens=1500,
    )

    return response.choices[0].message.content

# a question generator that lazy loads. Call next on a gnerator to get the question
def quiz_generator(transcripts: list[str]):
    i = 0
    while i < len(transcripts):
        transcript = ' '.join([d['text'] for d in transcripts[i:i+9]])
        transcript = transcript.replace("\n", " ")
        # print(transcript)
        quiz = createQuiz(transcript)[7:-3]
        # print(quiz)
        testObj = json.loads(quiz)
        
        yield testObj  # This yields control back to the caller, resuming from here when next() is called again.
        i += 9

def get_next_question(question_gen):
    return next(question_gen)

def get_summary_helper(transcript: str):
    prompt = f"As a text string summarizer expert. Summarize this {transcript}"
    # we just have to prompt the json to fix
    response = client.chat.completions.create(
    model = "gpt-4-0125-preview",
    messages=[
        {"role": "system", "content": "You are a class notes summarizer. Give me the notes only. No other fluff. Keep it concise"},
        {"role": "user", "content": prompt}
    ],
    max_tokens=1500,
    )

    return response.choices[0].message.content

def get_summary(transcripts, chunk_size=10):
    summaries = ""  # Initialize a list to hold all summaries
    total_transcripts = len(transcripts)
    start = 0  # Initialize start index for chunk processing

    while start < total_transcripts:
        end = min(start + chunk_size, total_transcripts)  # Determine end index for the current chunk
        transcript_chunk = ' '.join([transcripts[i]['text'] for i in range(start, end)])
        transcript_chunk = transcript_chunk.replace("\n", " ")
        # print("\n\n")
        # print(transcript_chunk)
        summary_result = get_summary_helper(transcript_chunk)  
        summaries += summary_result
        start += chunk_size  # Move to the next chunk
    return summaries  # Return the aggregated list of summaries

def main():
    # VIDEO about productivity
    url1 = "https://youtu.be/0CmtDk-joT4?si=q5AMUjRgaD37Xgnm"

    # VIDEO about ai video
    url2 = "https://youtu.be/NXpdyAWLDas?si=AMRXgzlvinyFApKI"
    transcripts = getTranscript(url1)
    # we loop every 10 transcript to create a quiz
    # quizzes = quiz_generator(transcripts)  # This creates a generator object but doesn't start the function yet.
    # print(get_next_question(quizzes))

    transcripts2 = getTranscript(url2)
    summary = get_summary(transcripts2, len(transcripts) // 5)
    print(summary)

    return 0

main()
    
```

[6 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]