# Project export: Tex Tutor

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: UC Berkeley AI Hackathon 2025
- Tagline: A helpful note taking tool that converts handwritten mathematical notation into embedded and rendered LaTex with a built in AI tutor that utilizes local context to give appropriately helpful hints.
- Devpost: https://devpost.com/software/tex-tech
- GitHub: https://github.com/Kourosh-Salahi/UCB-AI-Hackathon-2025?tab=readme-ov-file
- Demo: https://rapid-tutor-880186915241.us-west1.run.app/
- Video: https://www.youtube.com/embed/ayswlWbsoag?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 2 GitHub contributor(s) — Jesse Oh (2 commits), Kourosh Salahi (2 commits)

## Devpost submission (written by the team)

### Inspiration

As EECS Students who go to large public schools, we have noticed two problems that remain significant throughout our education journeys. Firstly, in the many math-heavy courses that we take, professors prefer and even sometimes require work to be submitted in LaTex format. Often, this means doing the work by hand, (on an ipad or similar note taking device) then manually transcribing the notes into LaTex, which is a time consuming, and rather inefficient process. Secondly, when we find ourselves stumped by particularly difficult problems, Office hours becomes a drag. Queues to talk to a TA can take literal hours, and by the time it becomes our turn to ask questions, we have already moved on. We wanted to solve both of these problems in unison by creating a smart note taking app that could act as an intelligent study partner that enables students to be as productive as possible.

### What it does

Through our user interface, students can upload their pdfs which they want to markup. After writing their work by hand, they can use our LaTex select tool to select over the area of work that they want to transcribe into rendered LaTeX. After a quick call to the Gemini API, rendered LaTex appears in place of the handwritten work. By pressing the AI Tutor hint button, a text field opens for students to enter the problem that they specifically are working on, so that the AI knows where to focus. Utilizing the local context as well as the transcribed work on the page, a quick call to Gemini generates informative hints based on current progress through the problem, possible mistakes made throughout the work, and overall problem strategy.

### How we built it

Using google ai studio we created a frontend to house our pdf and markups, then utilized API calls to groq, and gemini to handle the LaTex decoding as well as the instructional hints. This was all displayed through a React App frontend and hosted locally as well as on Google Cloud.

### Challenges we ran into

Frequently we ran into issues with displaying the LaTex on the pdf. In particular, the library we were using KaTex, would display our Latex rendered text as well as raw text side by side, which was not desired. We found that through proper installations and imports of the KaTex CSS, we resolved this issue. Additionally, Our early version of the AI tutor was unoptimized. At first, it did not utilize local contexts, which was an issue in the amount of information that we were sending it (not sending all annotations made to the document). After this, the tutor became way to revealing, often giving the full answer if prompted. Through iterative prompt engineering, we finalized a tutor that highlights mistakes, and gives hints that only inch the user closer to the answer, ensuring that proper learning can be done.

### Accomplishments we're proud of

Sleek UI that allows for page indexing of PDF, multicolored/size variable markups. Accurate and efficient LaTex Translations, as well as appropriately tuned helpfulness of AI Hints

### What we learned

We learned how to implement using Google AI studio for creating seamless frontend environments. We also learned how to use LaTex in React using the Katex Library. Finally, we learned how to utilize groq along with our LLMs to generate realtime responses for our queries.

### What's next

Next, we want to create an iOS app that would have all of this functionality locally (through an open source model such as gemma) so that this can be done end to end on an ipad. Additionally, we want to add voice integration so that the student can talk to the AI tutor in order to have a converstational experience as if they were at Office Hours. Finally, we would want to implement cloud based databased so that students can have access to their notes anywhere, regardless of if they have access to the machine that they made the original notes on.

## README (from the GitHub repository)

# UCB-AI-Hackathon-2025
Repo for Berkeley AI Hackathon

# PDF Annotator & AI Tutor 🎓📝

An intelligent PDF annotation application that allows you to markup PDFs with handwritten and text notes, convert handwriting to LaTeX using AI, and get personalized tutoring hints based on your work.

## ✨ Features

- **📄 PDF Viewer**: Upload and view PDF documents with zoom controls
- **✏️ Annotation Tools**: 
  - Pen tool for freehand drawing
  - Text tool for typed annotations
  - Eraser for removing annotations
  - Select/Move tool for repositioning annotations
- **🤖 AI-Powered Features**:
  - **Handwriting to LaTeX**: Convert handwritten math to LaTeX using Google's Gemini AI
  - **AI Tutoring**: Get personalized hints and guidance based on your work
- **📱 Responsive Design**: Works on desktop and mobile devices
- **💾 Real-time Editing**: All annotations are saved in real-time

## 🚀 Quick Start

### Prerequisites

- **Node.js** 18.0.0 or higher
- **npm** 8.0.0 or higher
- **Gemini API Key** (Get yours from [Google AI Studio](https://aistudio.google.com/app/apikey))

### Installation

1. **Clone the repository**
   ```bash
   git clone https://github.com/yourusername/pdf-annotator-ai-tutor.git
   cd pdf-annotator-ai-tutor
   ```

2. **Install dependencies**
   ```bash
   npm install
   ```

3. **Set up environment variables**
   ```bash
   cp .env.example .env
   ```
   Edit `.env` and add your Gemini API key:
   ```env
   GEMINI_API_KEY=your_actual_gemini_api_key_here
   ```

4. **Start the development server**
   ```bash
   npm run dev
   ```

5. **Open your browser** and navigate to `http://localhost:5173`

## 🎯 How to Use

### Basic PDF Annotation

1. **Upload a PDF**: Click "Upload PDF" button and select your file
2. **Choose a tool**: Select from pen, text, eraser, or select/move tools
3. **Annotate**: Draw, type, or markup your PDF as needed
4. **Navigate**: Use page controls and zoom to navigate your document

### AI Features

#### Convert Handwriting to LaTeX
1. Select the **LaTeX Select** tool (📐 icon)
2. Draw a rectangle around handwritten math
3. The AI will automatically convert it to properly formatted LaTeX

#### Get AI Tutoring Hints
1. Click the **Get Hint** button (💡 icon)
2. Optionally provide a specific question
3. The AI tutor will analyze your work and provide helpful guidance

## 🛠️ Technology Stack

- **Frontend**: React 19, TypeScript, Tailwind CSS
- **PDF Processing**: PDF.js
- **Math Rendering**: KaTeX, react-katex
- **AI Integration**: Google Gemini AI
- **Build Tool**: Vite
- **Styling**: Tailwind CSS

## 📁 Project Structure

```
pdf-annotator-ai-tutor/
├── components/           # React components
│   ├── FileUpload.tsx   # PDF file upload
│   ├── PdfAnnotator.tsx # Main PDF viewer & annotation canvas
│   ├── Toolbar.tsx      # Tool selection and controls
│   ├── Modal.tsx        # Modal dialogs
│   └── LoadingSpinner.tsx # Loading indicators
├── services/            # External service integrations
│   └── geminiService.ts # Google Gemini AI integration
├── App.tsx             # Main application component
├── types.ts            # TypeScript type definitions
├── constants.ts        # Application constants
├── index.tsx          # Application entry point
└── index.html         # HTML template
```

## 🔧 Development

### Available Scripts

- `npm run dev` - Start development server
- `npm run build` - Build for production
- `npm run preview` - Preview production build
- `npm run lint` - Run ESLint
- `npm run lint:fix` - Fix ESLint issues
- `npm run type-check` - Check TypeScript types
- `npm test` - Run tests

### Environment Variables

| Variable | Description | Required |
|----------|-------------|----------|
| `GEMINI_API_KEY` | Google Gemini AI API key | Yes |
| `GROQ_API_KEY` | Groq AI API key | No |
| `GROK_API_KEY` | Grok AI API key | No |
| `VITE_APP_NAME` | Application name | No |
| `VITE_API_BASE_URL` | API base URL | No |

## 🚀 Deployment

### Building for Production

```bash
npm run build
```

The built files will be in the `dist/` directory.

### Deploy to Vercel

1. Push your code to GitHub
2. Connect your repository to [Vercel](https://vercel.com)
3. Add your environment variables in Vercel dashboard
4. Deploy!

### Deploy to Netlify

1. Run `npm run build`
2. Upload the `dist/` folder to [Netlify](https://netlify.com)
3. Add environment variables in Netlify dashboard

## 🤝 Contributing

1. Fork the repository
2. Create a feature branch: `git checkout -b feature/amazing-feature`
3. Commit your changes: `git commit -m 'Add amazing feature'`
4. Push to the branch: `git push origin feature/amazing-feature`
5. Open a Pull Request

## 📝 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## 🙏 Acknowledgments

- [PDF.js](https://mozilla.github.io/pdf.js/) for PDF rendering
- [Google Gemini AI](https://ai.google.dev/) for AI capabilities
- [KaTeX](https://katex.org/) for LaTeX rendering
- [Tailwind CSS](https://tailwindcss.com/) for styling
- [React](https://react.dev/) for the UI framework

## 📞 Support

If you have any questions or run into issues:

1. Check the [Issues](https://github.com/yourusername/pdf-annotator-ai-tutor/issues) page
2. Create a new issue if your problem isn't already reported
3. Provide as much detail as possible including:
   - Steps to reproduce
   - Expected vs actual behavior
   - Browser and OS information
   - Console error messages

---

**Happy Annotating! 🎉**


## Detected evidence (automated analysis)

Indexed codebase: 2 recognized source files, 5 KB.
- Python (language) — detected in the code
- Google Gemini (technology) — claimed on Devpost, not found in the code
- React (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (3 of 3)

```
LICENSE
Main.py
README.md
```

### Dependencies

No dependency index available.

### Recent commits (newest first)

- Update README.md
- Update README.md
- main file empty
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

No suitable source files found.