# Project export: gradeAI
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: TreeHacks 2024
- Tagline: Efficient teaching, effective learning
- Devpost: https://devpost.com/software/gradeai-s9ptdc
- GitHub: https://github.com/justbustin/gradeAI
- Result: winner (Fetch.ai: Smartest AI Agent Prize (Top 2 Teams - $3k Cash))
- Team: 3 GitHub contributor(s) — ArnavJPortfolio (19 commits), Kenny Wan (16 commits), justbustin (6 commits)
## Devpost submission (written by the team)
### Inspiration
As engineering students who attend public universities, class sizes of 300 are no rarity. Sadly, that also means that oftentimes, our most important classes can take weeks or months to supply us with feedback for our assignments, hamstringing the learning process and creating a larger disconnect between the professor and the student. A quarter system is 10 weeks long, and having 3 weeks of ungraded assignments means learning 30% of the class without any feedback. We're also keenly aware that now more than ever, teachers are overworked. The average teacher logs upwards of 180 hours per year on grading alone, making it the most worked-on task outside of actual teaching inside the classroom. The solutions they have right now are few. Hiring extra teaching assistants to aid with grading is an extensive process that demands thorough interviewing to ensure the applicants actually know the material. Simultaneously, money is something that most universities can't spare on extra graders, meaning those that are qualified face heavily inflated workloads. At the moment, it seems there's no feasible out. With less time spent grading, there's more time spent with students. Enter gradeAI, our AI-based tool that aims to streamline grading by automatically processing assignments and evaluating them based on a given rubric. Our goal isn't to replace the teacher or the TA, but rather to appraise assignments quickly, let educators review and alter judgements made, and expedite grading large volumes of similar assignments.
### What it does
It utilizes AI Agents from Fetch.ai to break down multiple steps of grading assignments. First on the frontend, the teacher can create an assignment in a course, then name it and provide a rubric. As students submit their homework, the assignment will start getting graded after the due date. We have multiple AI Agents with the first being locally run and ported publicaly using Fetch.AI's mailbox. This first agent is responsible for pulling submitted homework files and the associated rubric from our Google Cloud Platform storage. This agent also processes the text and is the handler for sending homework to be graded in our pipeline, and eventually write the processed grades to our database. Next in the process is our pipeline of 4 agents on the Agentverse! The first agent is responsible for sending all of the processed text in a clean format to our next agent which parses the homework into readable chunks. The second agent-let's name it Parser-takes each problem number and associated work with it, and puts them all into an array! Parser does this for both the solution and homework submitted! This agent then sends this data to our third AI Agent and let's name this one Solver! This is where all of the grading happens. This AI Agent cross examines each problem's solution and attempt one at a time, which increases accuracy of grading. We ask this AI Agent to return data in a JSON format with the grade, confidence level, summary, and details for each problem. From there, Solver sends all of the graded problems that it did one by one to ensure accuracy to our last AI Agent on the Agentverse! This last agent aggregates all of the data and sends it back to our first agent being locally ran! Our locally ran agent then writes everything to our database, where we show it on our frontend!
### How we built it
We used the Reflex framework and Fetch.AI to accomplish all of our goals. This proved to be both difficult and convenient, as we never used these technologies before, but everything was written in Python! Thanks to Reflex, we built our entire website in Python, and Fetch.AI is all in python, which was great. In terms of Fetch.AI, we used many of their products such as the Agentverse, mailbox, template agents, and a little bit of DeltaV! We utilized the Agentverse, as it provided a convenient way to deploy our agents and have them run 24/7, which allowed us to keep a constant pipeline! The integrated development environment was also helpful in debugging and creating agents from scratch. Mailbox provided by Fetch.AI allowed our locally ran AI Agent to communicate with all of the deployed agents. Using a combination of all of these tools, we were able to piece together multiple agents with using a combination of openAI's API and OCR. Reflex was also a great tool and we utilized it's documentation and ability to wrap React code to create nice and complex components with functionality.
### Challenges we ran into
We took a big challenge in using technology we've never touched before! Reflex and Fetch.AI were the two major components of our project, and there was a big learning curve. One of the issues we ran into was a lack of documentation. Being a relatively newer product that also had a recent name change, Reflex didn't have as many resources as many of the tools we were more accustomed to. As a result, it took us a while to get the hang of what we were doing, and complex issues such as managing routes and backend integration with GCP were made even more difficult as we searched for solutions. Simpler issues like embedding PDF views of files was also made demanding, and our time frame for design was expanded due to underestimation of what we thought wouldn't be issues.
### What we learned
In terms of our web framework, we chose to use Reflex to build out our web app that incorporated Fetch.ai, FAST APIs, and OpenAI APIs. For Reflex, we learned how to develop the architecture of our application by fully understanding the Reflex documentation, and of course, reaching out to the Reflex developers for assistance. Additionally, we learned to link the various pages together and to incorporate components into our pages in order to fully immerse the user(s) and to provide a platform for the students and teachers to view the graded papers and their respective results.
### What's next
Given a time frame of 36 hours, our team had many more features planned than we were able to execute. Primarily, we'd love to add a plagiarism checker, LLM integration for student follow-ups, and textbook breakdowns for relevant questions in the homework. If gradeAI were a product in wide circulation, plagiarism would undoubtedly be the number one threat to its efficacy; as such, this implement demands our most immediate attention. Textbook breakdowns would also be a great quality of life feature we'd love to be able to tackle, but in truth, the feature we'd most look towards would be the LLM integration. Our undertaking of this project was meant to challenge us from the very beginning, and LLM is something we're all excited about and ready to learn and apply. Outside of features, we're looking to simply run and test a wider variety of materials so gradeAI can be the product it was meant to be. We've been nothing but thoroughly impressed with what we've been able to make, and we hope educators will be too. There's a long road to come for gradeAI, but whatever comes our way, we're confident we have the tools to ace it.
## README (from the GitHub repository)
# Welcome to Reflex!
This is the base Reflex template - installed when you run `reflex init`.
If you want to use a different template, pass the `--template` flag to `reflex init`.
For example, if you want a more basic starting point, you can run:
```bash
reflex init --template blank
```
## About this Template
This template has the following directory structure:
```bash
├── README.md
├── assets
├── rxconfig.py
└── {your_app}
├── __init__.py
├── components
│ ├── __init__.py
│ └── sidebar.py
├── pages
│ ├── __init__.py
│ ├── dashboard.py
│ ├── index.py
│ └── settings.py
├── styles.py
├── templates
│ ├── __init__.py
│ └── template.py
└── {your_app}.py
```
See the [Project Structure docs](https://reflex.dev/docs/getting-started/project-structure/) for more information on general Reflex project structure.
### Adding Pages
In this template, the pages in your app are defined in `{your_app}/pages/`.
Each page is a function that returns a Reflex component.
For example, to edit this page you can modify `{your_app}/pages/index.py`.
See the [pages docs](https://reflex.dev/docs/components/pages/) for more information on pages.
In this template, instead of using `rx.add_page` or the `@rx.page` decorator,
we use the `@template` decorator from `{your_app}/templates/template.py`.
To add a new page:
1. Add a new file in `{your_app}/pages/`. We recommend using one file per page, but you can also group pages in a single file.
2. Add a new function with the `@template` decorator, which takes the same arguments as `@rx.page`.
3. Import the page in your `{your_app}/pages/__init__.py` file and it will automatically be added to the app.
### Adding Components
In order to keep your code organized, we recommend putting components that are
used across multiple pages in the `{your_app}/components/` directory.
In this template, we have a sidebar component in `{your_app}/components/sidebar.py`.
### Adding State
As your app grows, we recommend using [substates](https://reflex.dev/docs/state/substates/)
to organize your state.
You can either define substates in their own files, or if the state is
specific to a page, you can define it in the page file itself.
## Detected evidence (automated analysis)
Indexed codebase: 45 recognized source files, 147 KB.
- CSS (language) — detected in the code
- JavaScript (language) — detected in the code
- Next.js (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- FastAPI (technology) — claimed on Devpost, not found in the code
- OpenAI (technology) — claimed on Devpost, not found in the code
## Codebase structure (from repository index)
### Files (60 of 60)
```
.gitignore
.web/.gitignore
.web/bun.lockb
.web/components/reflex/chakra_color_mode_provider.js
.web/components/reflex/radix_themes_color_mode_provider.js
.web/env.json
.web/jsconfig.json
.web/next.config.js
.web/package.json
.web/pages/_app.js
.web/pages/_document.js
.web/pages/404.js
.web/pages/assignment.js
.web/pages/dashboard.js
.web/pages/grading.js
.web/pages/index.js
.web/pages/settings.js
.web/postcss.config.js
.web/reflex.install_frontend_packages.cached
.web/reflex.json
.web/styles/styles.css
.web/styles/tailwind.css
.web/tailwind.config.js
.web/utils/client_side_routing.js
.web/utils/components.js
.web/utils/context.js
.web/utils/helpers/dataeditor.js
.web/utils/helpers/range.js
.web/utils/state.js
.web/utils/stateful_components.js
.web/utils/theme.js
backend_tooling/__init__.py
backend_tooling/amazing-city-414621-61f39de69c52.json
backend_tooling/local_interval_trigger.py
backend_tooling/mnemonic_seed
gradeAI/__init__.py
gradeAI/components/__init__.py
gradeAI/components/sidebar.py
gradeAI/gradeAI.py
gradeAI/pages/__init__.py
gradeAI/pages/amazing-city-414621-61f39de69c52.json
gradeAI/pages/assignment.py
gradeAI/pages/dashboard.py
gradeAI/pages/grading.py
gradeAI/pages/index.py
gradeAI/pages/settings.py
gradeAI/pages/zoom_calendly_embed.py
gradeAI/styles.py
gradeAI/templates/__init__.py
gradeAI/templates/template.py
hosted_agents/parsing_agent.py
hosted_agents/problem_agent.py
hosted_agents/submission_agent.py
README.md
requirements.txt
rxconfig.py
temp_score_hw1_grades_hw1.json
test_additional_tooling/__init__.py
test_additional_tooling/amazing-city-414621-61f39de69c52.json
test_additional_tooling/Test_Text_Extraction_PDF_Word_OCR.ipynb
```
### Dependencies
- .web/package.json: @chakra-ui/icons@2.0.19, @chakra-ui/react@2.6.1, @chakra-ui/system@2.5.7, @emotion/react@^11.11.1, @radix-ui/react-progress@^1.0.3, @radix-ui/themes@^2.0.0, autoprefixer@10.4.14, axios@1.4.0, framer-motion@10.16.4, json5@2.2.3, next@14.0.1, next-sitemap@4.1.8, next-themes@0.2.0, postcss@8.4.31, react@18.2.0, react-dom@18.2.0, react-dropzone@14.2.3, react-markdown@8.0.7, react-pdf@^7.7.0, react-syntax-highlighter@15.5.0, rehype-katex@6.0.3, rehype-raw@6.1.1, remark-gfm@3.0.1, remark-math@5.1.1, silly@^0.2.0, socket.io-client@4.6.1, tailwindcss@3.3.2, universal-cookie@4.0.4
- requirements.txt: reflex@==0.4.0
### Recent commits (newest first)
- Merge pull request #6 from justbustin/kenny/teacher-view
- Added Upload to GCP functionality from Web App
- added styling
- Finished Agent Chain
- styling changes
- connected to GCP, integrated PDFs
- Added Upload to GCP functionality from Web App
- Merge remote-tracking branch 'origin/main'
- Finished Agent Chain
- Merge pull request #5 from justbustin/agent
- adding code of agents hosted on agentverse
- Merge branch 'main' of https://github.com/justbustin/gradeAI
- Merge branch 'main' of https://github.com/justbustin/gradeAI
- Added Upload to GCP functionality from Web App
- Merge pull request #4 from justbustin/pdf
- pdf view
- Merge branch 'main' of https://github.com/justbustin/gradeAI
- Finished Agent Chain
- Finished Agent Chain
- Merge pull request #3 from justbustin/kenny/teacher-view
## Key source files (fetched from GitHub, selected and truncated for size)
### requirements.txt
```
reflex==0.4.0
```
### .web/package.json
```
{
"name": "reflex",
"scripts": {
"dev": "next dev",
"export": "next build",
"export-sitemap": "next build && next-sitemap",
"prod": "next start"
},
"dependencies": {
"@chakra-ui/icons": "2.0.19",
"@chakra-ui/react": "2.6.1",
"@chakra-ui/system": "2.5.7",
"@emotion/react": "^11.11.1",
"@radix-ui/react-progress": "^1.0.3",
"@radix-ui/themes": "^2.0.0",
"axios": "1.4.0",
"framer-motion": "10.16.4",
"json5": "2.2.3",
"next": "14.0.1",
"next-sitemap": "4.1.8",
"next-themes": "0.2.0",
"react": "18.2.0",
"react-dom": "18.2.0",
"react-dropzone": "14.2.3",
"react-markdown": "8.0.7",
"react-pdf": "^7.7.0",
"react-syntax-highlighter": "15.5.0",
"rehype-katex": "6.0.3",
"rehype-raw": "6.1.1",
"remark-gfm": "3.0.1",
"remark-math": "5.1.1",
"silly": "^0.2.0",
"socket.io-client": "4.6.1",
"universal-cookie": "4.0.4"
},
"devDependencies": {
"autoprefixer": "10.4.14",
"postcss": "8.4.31",
"tailwindcss": "3.3.2"
}
}
```
### gradeAI/pages/index.py
```python
"""The home page of the app."""
from gradeAI import styles
from gradeAI.templates import template
from typing import List
import reflex as rx
class ForeachState(rx.State):
pageRoute: List[str] = [
"/dashboard",
"/settings",
],
className: List[str] = [
"MATH61: Discrete Math"
"LIFE101: Touching Grass"
"LIFE144: Finding a Partner For You"
"LIFE188: Shower Techniques"
"LIFE31: Intro to Leaving Your Room"
"MATH203: Graduate Topology"
]
def classCard(className: str):
return rx.card(
rx.chakra.link(
rx.flex(
rx.avatar(src="/reflex_banner.png"),
rx.box(
rx.heading("Quick Start"),
rx.text(
className
),
),
spacing="2",
),
# href={route}
),
as_child=True,
)
@template(route="/", title="Home", image="/github.svg")
def index() -> rx.Component:
"""The home page.
Returns:
The UI for the home page.
"""
return rx.box(
rx.text('Courses', marginLeft="0.1em", marginTop="1.7em", font_weight=500, font_size="1.4em", margin_bottom="0.5em"),
rx.grid(
rx.card(
rx.chakra.link(
rx.flex(
rx.box(
rx.flex(
rx.image(src="calculator.svg", width='1.6em', height='1.6em', marginTop='0.7em', marginRight='0.3em', color="#cb997e"),
rx.box(
rx.text("MATH 61", font_size="2em", font_weight="bold", font_family="Work Sans"),
),
)
),
rx.box(
rx.flex(
rx.image(src="friends.svg", width='2em', height='2em', marginTop='0.5em'),
rx.text(
"271", marginTop='0.7em'
),
)
),
# spacing="8",
justify="between",
),
rx.text(
"Introduction to Discrete Structures", marginLeft="0.2em", marginTop="-0.4em", font_family="Work Sans"
),
rx.text("5 assignments, 1 Exam", marginLeft="0.2em", marginTop="-0.2em", font_family="Work Sans"),
rx.chakra.divider(marginTop="0.5em", width="45em", marginLeft="-10em"),
rx.image(src="wavyred.png", width="35em", height="12em", margin_top="-2em"),
href="/assignment",
text_decoration="none"
),
as_child=True,
height="40vh",
),
rx.card(
rx.chakra.link(
rx.flex(
rx.box(
rx.flex(
rx.image(src="shower.svg", width='1.6em', height='1.6em', marginTop='0.7em', marginRight='0.3em', color="#cb997e"),
rx.box(
rx.text("LIFE 142", font_size="2em", font_weight="bold", font_family="Work Sans"),
),
)
),
rx.box(
rx.flex(
rx.image(src="friends.svg", width='2em', height='2em', marginTop='0.5em'),
rx.text(
"400", marginTop='0.7em'
),
)
),
# spacing="8",
justify="between",
),
rx.text(
"Shower Techniques", marginLeft="0.2em", marginTop="-0.4em", font_family="Work Sans"
),
rx.text("0 assignments, 0 Exams", marginLeft="0.2em", marginTop="-0.2em", font_family="Work Sans"),
rx.chakra.divider(marginTop="0.5em", width="45em", marginLeft="-10em"),
rx.image(src="wavyorange.png", width="35em", height="12em", margin_top="-2em"),
href="/dashboard",
text_decoration="none"
),
as_child=True,
height="40vh",
),
rx.card(
rx.chakra.link(
rx.flex(
rx.box(
rx.flex(
rx.image(src="couple.svg", width='1.6em', height='1.6em', marginTop='0.7em', marginRight='0.3em', color="#cb997e"),
rx.box(
rx.text("LOVE 96", font_size="2em", font_weight="bold", font_family="Work Sans"),
),
)
),
rx.box(
rx.flex(
rx.image(src="friends.svg", width='2em', height='2em', marginTop='0.5em'),
rx.text(
"69", marginTop='0.7em'
),
)
),
# spacing="8",
justify="between",
),
rx.text(
"Introduction to Finding Love", marginLeft="0.2em", marginTop="-0.4em", font_family="Work Sans"
),
rx.text("2 assignments, 0 Exams", marginLeft="0.2em", marginTop="-0.2em", font_family="Work Sans"),
rx.chakr
[truncated — 5534 more characters]
```
### .web/pages/index.js
```javascript
/** @jsxImportSource @emotion/react */
import { Fragment, useContext } from "react"
import { EventLoopContext } from "/utils/context"
import { Event, getBackendURL, isTrue } from "/utils/state"
import { Box as RadixThemesBox, Card as RadixThemesCard, Dialog as RadixThemesDialog, Flex as RadixThemesFlex, Grid as RadixThemesGrid, Text as RadixThemesText } from "@radix-ui/themes"
import env from "/env.json"
import { Box, Divider, HStack, Link } from "@chakra-ui/react"
import NextLink from "next/link"
import NextHead from "next/head"
export function Fragment_1762bb90abdb81b879b2a22edbbe01a1 () {
const [addEvents, connectError] = useContext(EventLoopContext);
return (
{isTrue(connectError !== null) ? (
{`Connection Error`}
{`Cannot connect to server: `}
{(connectError !== null) ? connectError.message : ''}
{`. Check if server is reachable at `}
{getBackendURL(env.EVENT).href}
) : (
)}
)
}
export default function Component() {
return (
{`Home`}
{`Grades`}
{`Account`}
{`Search classes...`}
{`Courses`}
{`MATH 61`}
{`271`}
{`Introduction to Discrete Structures`}
{`5 assignments, 1 Exam`}
{`LIFE 142`}
{`400`}
{`Shower Techniques`}
