HackStack

Project Info

Elevate

Devpost

Inspiration

Frustrated by the steep learning curve of LaTeX—a formatting language used by over 90% of students in scientific fields like math or physics—we, as engineering students, found ourselves spending more time converting handwritten notes into LaTeX than solving the actual problems.

What it does

Upload notes or problem sets as images, and Elevate converts them into LaTeX documents while annotating errors with explanations. This process not only simplifies formatting but also enhances learning by helping you identify and correct mistakes for improved academic performance.

How we built it

Convex helped us link our React frontend to the backend smoothly, making it quicker to change things and catch mistakes thanks to its type-safe coding. We organized our data using Convex's NoSQL database, adding structure and making searches faster with indexing. For security, we hooked up authentication easily with Convex's Clerk Adaptor. Since we needed to store images and LaTeX files, Convex was handy there too. Plus, we used Convex’s vector database to smarten up our chatbot, making everything work together nicely and more efficiently. OCR Preprocessing (Image2Latex): We converted images to grayscale via the following formula: We applied Gaussian Blur, which smooths the image by averaging the pixels based on their spatial closeness and intensity similarity, helping to reduce high-frequency noise for next steps. We utilized the OpenCV Library to compute a (5, 5) kernel with standard deviation 0, via the following formula for a two-dimensional Gaussian: Finally, we employed a custom sharpening kernel, which enhances edges by increasing the contrast between adjacent pixels. The convolution operation of the image with the kernel, I * K, is given by After the preprocessing stage, when then feed our resulting image into OpenAI’s Computer Vision API, to generate text describing what is in the image. We then feed this text into together.AI’s Mixtral-8x7B LLM, which is able to generate appropriate latex code for the text. Finally, once again using together.AI’s LLM and extensive prompt engineering and fine-tuning, we were able to highlight errors and generate corrections. Original Image Image After Gaussian Blur (Reduced Noise) Image After Sharpening Kernel Convolution Latex Code Generated by Together.AI Latex Code Corrected by Together.AI Database Vector Search: Using Intersystem’s Langchain-IRIS vector search technology, we were able to: Generate embedded vectors for latex notes Use cosine similarity to query, for a certain problem, the most relevant notes that contain information to answer this problem By representing each note as an edge and drawing an edge between nodes with high similarity (low vector degree distance), create a connection graph and mind map of all notes in the database For example, querying the question “How can I find a stable matching?” in a database of one of our team member's discrete math course notes results in the following responses from Intersystem’s API: Leveraging Together.AI’s Mixtral LLM, we also created an interactive ChatBot that is able to summarize ideas in the notes and answer questions from users about the notes.

Challenges we ran into

It took a while to figure out how to render LaTeX text in a browser, as no existing packages met our needs for clarity and precision in displaying complex formulas. Creating LLM prompts for annotating student solutions with feedback proved difficult. We had to not only ensure that the feedback was accurate but also that it was formatted and presented properly. The technical complexity of implementing the vector graph for note comparison presented a hard challenge. Perfecting an algorithm that seemlessly integrate Intersystem's Langchain vector embeddings into a graph structure, and generating appropriate adjacency lists required deep dives into ML, optimization, and Data Structures. Additionally, rendering the graph on our live website presented an additional challenge.

Accomplishments we're proud of

We successfully built a tool that we will use for the rest of our lives and are optimistic enough about the quality/market potential to go to market after the Hackathon.

What we learned

We learned the crucial importance of having a diverse team with varied skill sets, enabling us to effectively divide the workload across design, backend development, and machine learning tasks.

What's next

Get 1k users by posting on social media. Use feedback from users to iterate product and add features they would find helpful.

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only
  • CSSIn code
  • FlaskIn code
  • HTMLIn code
  • JavaScriptIn code
  • OpenAIIn code
  • PythonIn code
  • ReactIn code
  • Tailwind CSSIn code
  • TypeScriptIn code
  • LangChainClaimed

9 of 10 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

79 KB

Source files

36

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.

0 stars