# Project export: calvin This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2025 - Tagline: your canvas (lms) ai assistant - Devpost: https://devpost.com/software/calvin-3jpoqh - GitHub: https://github.com/flotoria/treehacks25 - Video: https://www.youtube.com/embed/uxybdbsNkM8?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 1 GitHub contributor(s) — Ryan Pham (7 commits) ## Devpost submission (written by the team) ### Inspiration the inspiration for this project actually came from my friend, calvin. he thought of this idea back back a long time ago (65 million years ago). my motivation for this project came from wanting to learn how to use the canvas apis and some of the sponsors' libraries / frameworks / services, so i thought that it would be an interesting project to do while drinking pocari sweat and eating the free food. ### What it does to preface, canvas is a learning management system (lms) that is used by many schools including stanford, and you can imagine this application as an add-on to the lms. calvin is your personal chatbot when it comes to canvas courses and assignments. select a school subject then click on an assignment. you can, then, ask calvin about your assignment. also, you can even ask the chatbot about all your classes. you don't have to specify which assignment. you can even whiteboard out your query if you feel like it. ### How we built it frontend next.js - the core of our frontend shadcn/ui - the component library backend fastapi - the core of our backend chromadb - the vector database groq - used their hosted llm models canvas api - the api we used to fetch the assignments / coursework langgraph - to create cool ai agents dev tools windsurf ide - sped up development time by a century system design ### Challenges we ran into i went solo experimented with many vector databases before deciding to use chromadb slept early and wanted to relax during this hackathon i wanted to have fun got distracted by llamas ### Accomplishments we're proud of my first solo hackathon made it easier for students to ask questions about their canvas assignments implemented the whiteboard feature so students can draw out their questions more and more and more rag ### What we learned learned how to use chromadb's cloud service learned how to use groq's llama vision models learned to have lots of fun ### What's next implement tool calling integrate ai agents that can do deep research or problem solving ## README (from the GitHub repository) # calvin your canvas AI assistant (created @ treehacks '25) ## Detected evidence (automated analysis) Indexed codebase: 26 recognized source files, 58 KB. - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - Next.js (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - LangChain (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (36 of 36) ``` be/.gitignore be/app.py be/canvas_api.py be/chroma.py be/groq_api.py be/langgraph_helper.py be/llm.py be/poetry.lock be/pyproject.toml fe/.gitignore fe/app/chat/page.tsx fe/app/globals.css fe/app/layout.tsx fe/app/page.tsx fe/bun.lock fe/components.json fe/components/DrawingCanvas.tsx fe/components/sidebar/LeftSidebarContainer.tsx fe/components/ui/avatar.tsx fe/components/ui/button.tsx fe/components/ui/dialog.tsx fe/components/ui/input.tsx fe/components/ui/scroll-area.tsx fe/components/ui/select.tsx fe/components/ui/skeleton.tsx fe/components/ui/spinner.tsx fe/eslint.config.mjs fe/lib/utils.ts fe/next.config.ts fe/package.json fe/postcss.config.mjs fe/README.md fe/styles/fonts.ts fe/tailwind.config.ts fe/tsconfig.json README.md ``` ### Dependencies - be/pyproject.toml: chromadb@^0.6.3, elasticsearch@^8.17.1, elevenlabs@^1.51.0, fastapi@^0.115.8, groq@^0.18.0, langchain-core@^0.3.35, langchain-groq@^0.2.4, langchain-openai@^0.3.6, langchain-text-splitters@^0.3.6, langgraph@^0.2.73, python-dotenv@^1.0.1 - fe/package.json: @eslint/eslintrc@^3, @radix-ui/react-avatar@^1.1.3, @radix-ui/react-dialog@^1.1.6, @radix-ui/react-scroll-area@^1.2.3, @radix-ui/react-select@^2.1.6, @radix-ui/react-slot@^1.1.2, @types/node@^20, @types/react@^19, @types/react-dom@^19, class-variance-authority@^0.7.1, clsx@^2.1.1, eslint@^9, eslint-config-next@15.1.7, lucide-react@^0.475.0, next@15.1.7, postcss@^8, react@^19.0.0, react-dom@^19.0.0, react-markdown@^9.0.3, tailwind-merge@^3.0.1, tailwindcss@^3.4.1, tailwindcss-animate@^1.0.7, typescript@^5 ### Recent commits (newest first) - Create README.md - fix bug - api key - smth was changed but i do not remember or know - ok so i think i finished the basic stuff ig - CRAZY PROGRESS????? - commit from last night of coding iaojidosd ## Key source files (fetched from GitHub, selected and truncated for size) ### be/pyproject.toml ``` [tool.poetry] name = "be" version = "0.1.0" description = "" authors = ["Ryan Pham "] readme = "README.md" [tool.poetry.dependencies] python = "^3.12" fastapi = {extras = ["standard"], version = "^0.115.8"} elevenlabs = "^1.51.0" python-dotenv = "^1.0.1" elasticsearch = {extras = ["async"], version = "^8.17.1"} chromadb = "^0.6.3" langchain-text-splitters = "^0.3.6" langchain-openai = "^0.3.6" langchain-groq = "^0.2.4" langgraph = "^0.2.73" groq = "^0.18.0" langchain-core = "^0.3.35" [build-system] requires = ["poetry-core"] build-backend = "poetry.core.masonry.api" ``` ### fe/package.json ``` { "name": "fe", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev --turbopack", "build": "next build", "start": "next start", "lint": "next lint" }, "dependencies": { "@radix-ui/react-avatar": "^1.1.3", "@radix-ui/react-dialog": "^1.1.6", "@radix-ui/react-scroll-area": "^1.2.3", "@radix-ui/react-select": "^2.1.6", "@radix-ui/react-slot": "^1.1.2", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "lucide-react": "^0.475.0", "next": "15.1.7", "react": "^19.0.0", "react-dom": "^19.0.0", "react-markdown": "^9.0.3", "tailwind-merge": "^3.0.1", "tailwindcss-animate": "^1.0.7" }, "devDependencies": { "@eslint/eslintrc": "^3", "@types/node": "^20", "@types/react": "^19", "@types/react-dom": "^19", "eslint": "^9", "eslint-config-next": "15.1.7", "postcss": "^8", "tailwindcss": "^3.4.1", "typescript": "^5" } } ``` ### be/app.py ```python from typing import Union from fastapi import FastAPI from pydantic import BaseModel from chroma import create_canvas_collection, rag_specific_assignment from canvas_api import get_all_assignments, get_all_available_canvas_courses, get_canvas_assignment_info, get_canvas_assignment_names, get_canvas_assignments, get_all_course_with_ids_and_names import os from llm import do_query import json from fastapi.middleware.cors import CORSMiddleware from langgraph_helper import invoke_graph app = FastAPI() origins = ["*"] app.add_middleware( CORSMiddleware, allow_origins=origins, allow_credentials=True, allow_methods=["*"], allow_headers=["*"], ) class Query(BaseModel): query: str class CourseID(BaseModel): course_id: str class CourseIDAndAssignmentID(BaseModel): course_id: str assignment_id: str class Image(BaseModel): base64_image: str @app.post('/canvas') def get_courses(query: Query): return do_query(query.query) @app.get('/delete') def delete_history(): try: os.remove("history.json") return {"message": "History deleted"} except FileNotFoundError: pass @app.get('/message') def get_message_history(): history_file = "history.json" if os.path.exists(history_file): try: with open(history_file, 'r') as f: history = json.load(f) return history except json.JSONDecodeError: history = [] return history @app.get('/courses') def get_courses(): return get_all_course_with_ids_and_names() @app.get('/rag') def rag(): try: os.remove("history.json") except FileNotFoundError: pass create_canvas_collection() return "RAG created" @app.post('/assignments') def get_assignment_ids(obj: CourseID): return get_canvas_assignment_names(obj.course_id) @app.post('/rag-specific') def rag_specific(obj: CourseIDAndAssignmentID): try: os.remove("history.json") except FileNotFoundError: pass rag_specific_assignment(obj.course_id, obj.assignment_id) return "cool" @app.post("/image-query") def image_query(obj: Image): return invoke_graph(obj.base64_image) ``` ### fe/app/layout.tsx ```typescript import type { Metadata } from "next"; import "./globals.css"; export const metadata: Metadata = { title: "Calvin", description: "Your Canvas Assistant", }; export default function RootLayout({ children, }: Readonly<{ children: React.ReactNode; }>) { return ( {children} ); } ``` ### fe/app/page.tsx ```typescript "use client"; import Image from "next/image"; import { useRouter } from "next/navigation"; import { Button } from "@/components/ui/button"; import { Input } from "@/components/ui/input"; import { useState } from "react"; export default function Home() { const router = useRouter(); const [apiKey, setApiKey] = useState(""); const handleSubmit = () => { if (!apiKey.trim()) { alert("Please enter an API key"); return; } // Store API key in localStorage localStorage.setItem("apiKey", apiKey); router.push('/chat'); }; return (
Calvin
setApiKey(e.target.value)} className="text-lg p-6" />
); } ``` ### fe/app/chat/page.tsx ```typescript "use client"; import { useEffect, useState } from "react"; import Markdown from 'react-markdown' import LeftSidebarContainer from "@/components/sidebar/LeftSidebarContainer"; import { Skeleton } from "@/components/ui/skeleton"; import { Pencil } from "lucide-react"; import { Dialog, DialogContent, DialogHeader, DialogTitle, DialogTrigger, } from "@/components/ui/dialog"; import { DrawingCanvas } from "@/components/DrawingCanvas"; import { Avatar, AvatarFallback, AvatarImage } from "@/components/ui/avatar"; interface Message { type: string; message: string; } const isBase64PNG = (str: string): boolean => { const pngBase64Prefix = "data:image/png;base64,"; return str.trim().startsWith(pngBase64Prefix); }; export default function ChatPage() { const initialGreeting = { type: "assistant", message: "How can I help you?" }; const [messages, setMessages] = useState([initialGreeting]); const [input, setInput] = useState(""); const [loading, setLoading] = useState(false); const [isDrawingModalOpen, setIsDrawingModalOpen] = useState(false); const [isImageSaving, setIsImageSaving] = useState(false); const setMessagesWithGreeting = (newMessages: Message[]) => { setMessages([initialGreeting, ...newMessages]); }; const fetchMessages = async () => { const response = await fetch("http://localhost:8000/message"); const data = await response.json(); const messageArray = Array.isArray(data) ? data : []; setMessagesWithGreeting(messageArray); }; useEffect(() => { fetchMessages(); }, []); const handleSubmit = async (e: React.FormEvent) => { e.preventDefault(); if (!input.trim() || loading) return; setLoading(true); try { const response = await fetch("http://localhost:8000/canvas", { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({ query: input }), }); if (!response.ok) throw new Error("Failed to send message"); const data = await response.text(); setInput(""); await fetchMessages(); // Refresh messages after sending } catch (error) { console.error("Error sending message:", error); } finally { setLoading(false); } }; const handleDrawingSave = async (imageData: string) => { setLoading(true); try { const response = await fetch("http://localhost:8000/image-query", { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({ base64_image: imageData }), }); if (!response.ok) { throw new Error("Failed to process image"); } await fetchMessages(); setIsDrawingModalOpen(false); } catch (error) { console.error("Error processing drawing:", error); alert("Failed to process image. Please try again."); } finally { setLoading(false); } }; return (
{loading ? (
) : ( messages && messages.map((message, index) => (
{message.type === "human" ? ( <> U ) : ( <> A )}
{isBase64PNG(message.message) ? ( Generated content ) : ( {message.message} )}
)) )}