# Project export: ForMath This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 10.0 - Tagline: Helping you identify mistakes.Helping you correct mistakes.Helping you grow from mistakes. - Devpost: https://devpost.com/software/formath - GitHub: https://github.com/darthvader58/biryani - Demo: https://github.com/sahajrastogi/biryanibackend - Team: 1 GitHub contributor(s) — Shashwat Raj (3 commits) ## Devpost submission (written by the team) ### Inspiration The motivation to start working on "ForMath" was our struggling friends who frequently deal with the frustration of knowing that something is wrong with their problem-solving process but are unable to identify the issue. To solve the same, we developed a website that not only offers answers but also explains the steps involved in reaching the solution. Our goal is to give individuals a comprehensive understanding of the correct problem-solving approach so that they are not just reaching the correct answer, but also following the correct approach to reach it. Through this project, we aim to empower learners with the knowledge and confidence to navigate challenges effectively. ### What it does Our project is a user-friendly website designed to assess image solutions provided by users. It verifies the correctness of answers and evaluates the accuracy of problem-solving steps. Users receive detailed feedback, pinpointing specific mistakes and categorizing errors as either calculational or conceptual, aiding in their learning process. It has two primary target audiences - students and teachers. Teachers can use our web - app to quickly grade test papers which are graded based on math work shown by students. Additionally, students can use our service to ensure that they show all the work; furthermore, they can clarify steps that they might've gotten wrong on previous exams. ### How we built it For the front end, we designed in Figma and coded the UI elements in React. We built the backend using node and express with CockroachDB as the database. Additionally, we leveraged several APIs from Mathpix, WolframAlpha, OpenAI, and an image hosting service. ### Challenges we ran into We faced quite a lot of challenges in terms of implementing and accessing APIs as some of the APIs (Mathpix and ChatGPT), were paid and also they had a limited number of trials (Wolfram). ### Accomplishments we're proud of During the process of trying to give shape to "ForMath" we ran into various challenges, overcame them, and made our way to what it is right now- a beta version of our vision. ### What we learned We explored how to use different databases (Cockroach DB, AWS, etc). Apart from the technical stuff (like environmental setup), we were also able to learn how to work long hours as this was our first hackathon for. We learned how to be organized and manage time efficiently too. ### What's next We look forward to applying our software for the line-by-line checking of works in other subjects as well like chemistry, physics, etc. We further hope to partner with schools to encourage students to follow steps when doing work as showing work and steps is very important in many careers. ## README (from the GitHub repository) # ForMath **Step-by-Step Problem Solving Analysis Helper** ForMath is an educational tool that helps students analyze their math homework by identifying calculation errors, conceptual mistakes, and providing detailed feedback on their solutions. The application uses advanced OCR, OpenAI's GPT-4, and Wolfram Alpha to provide comprehensive mathematical analysis and tutoring help. ## Features ### Core Functionality - **Multi-format Input Support**: Upload images (JPG, PNG, GIF, BMP) or PDF files of math homework - **Camera Integration**: Capture math problems directly using your device's camera - **Text Input**: Enter problems manually or paste from other sources - **Advanced OCR**: Extract text from images and scanned PDFs using Tesseract.js - **PDF Processing**: Direct text extraction from digital PDFs with OCR fallback for scanned documents ### AI-Powered Analysis - **Problem Parsing**: Automatically separates original problems from student solutions - **Error Detection**: Identifies conceptual errors, computational mistakes, or confirms correct solutions - **Solution Verification**: Uses Wolfram Alpha to provide correct solutions for comparison - **Detailed Feedback**: GPT-4 powered analysis with explanations, hints, and improvement suggestions - **Topic Classification**: Categorizes problems by mathematical topic and difficulty level ## Technology Stack ### Frontend - **React 18** - Modern UI framework - **React Router** - Client-side routing - **Axios** - HTTP client for API communication - **Tesseract.js** - Client-side OCR processing - **PDF.js** - PDF rendering and text extraction - **React Webcam** - Camera integration - **React Dropzone** - File upload interface - **Recharts** - Data visualization - **KaTeX** - Mathematical notation rendering ### Backend - **Node.js** with Express - RESTful API server - **PostgreSQL** (CockroachDB) - Database for user data and analytics - **OpenAI GPT-4** - Problem analysis - **Wolfram Alpha API** - Mathematical computation and verification - **Multer** - File upload handling - **PDF-Parse** - Server-side PDF text extraction ### Deployment - **Vercel** - Frontend hosting and deployment - **Docker** - Containerized backend deployment - **Environment Variables** - Secure configuration management ## Getting Started ### Prerequisites - Node.js 16+ and npm - PostgreSQL database (or CockroachDB) - OpenAI API key - Wolfram Alpha App ID - Google OAuth 2.0 credentials ### Installation 1. **Clone the repository** ```bash git clone https://github.com/darthvader58/biryani.git cd biryani ``` 2. **Install frontend dependencies** ```bash npm install ``` 3. **Install backend dependencies** ```bash cd backend npm install cd .. ``` 4. **Configure environment variables** Create `.env` in the root directory: ```env REACT_APP_GOOGLE_CLIENT_ID=your_google_client_id ``` Create `backend/.env`: ```env DATABASE_URL=your_postgresql_connection_string OPENAI_API_KEY=your_openai_api_key WOLFRAM_APP_ID=your_wolfram_alpha_app_id PORT=8080 NODE_ENV=development ``` 5. **Set up the database** The application will automatically create the required tables on first run. 6. **Start the development servers** Backend server: ```bash cd backend npm start ``` Frontend development server: ```bash npm start ``` ### Google OAuth Setup 1. Go to [Google Cloud Console](https://console.cloud.google.com/) 2. Create a new project or select an existing one 3. Enable the Google+ API 4. Create OAuth 2.0 credentials 5. Add authorized redirect URIs: - `http://localhost:3000` (development) - Your production domain (e.g., `https://your-app.vercel.app`) ## Usage ### Analyzing Math Problems 1. **Sign in** with your Google account 2. **Upload your homework** using one of three methods: - **File Upload**: Drag and drop images or PDF files - **Camera**: Take a photo of your math work - **Text Input**: Type or paste the problem directly 3. **Review extracted text** and edit if necessary 4. **Click "Analyze Problem"** to get AI feedback 5. **View detailed results** including: - Original problem identification - Your solution analysis - Correct solution from Wolfram Alpha - Error classification and explanations - Hints for improvement ### Dashboard Features - **Performance Overview**: Total problems, accuracy rates, error distribution - **Progress Charts**: Visual tracking of improvement over time - **Topic Analysis**: Performance breakdown by mathematical topics - **Problem History**: Complete record of analyzed problems ## API Endpoints ### Core Endpoints - `POST /api/analyze-problem` - Analyze a math problem - `GET /api/dashboard/:email` - Get user dashboard data - `POST /api/upload-file` - Upload and process files - `POST /api/feedback` - Submit user feedback ### Authentication The application uses Google OAuth for authentication. Users must be signed in to access analysis features and dashboard. ## File Processing ### Supported Formats - **Images**: JPG, PNG, GIF, BMP (processed with OCR) - **PDFs**: Digital PDFs (direct text extraction) and scanned PDFs (OCR fallback) ### Processing Pipeline 1. **File Upload**: Secure file handling with size limits (10MB) 2. **Text Extraction**: OCR for images, direct extraction for digital PDFs 3. **Content Parsing**: AI separation of problems from solutions 4. **Analysis**: Multi-step AI analysis with external verification 5. **Storage**: Secure database storage with user analytics ## Deployment ### Frontend (Vercel) ```bash npm run build vercel --prod ``` ### Backend (Docker) ```bash cd backend docker build -t formath-backend . docker run -p 8080:8080 formath-backend ``` ## Contributing 1. Fork the repository 2. Create a feature branch (`git checkout -b feature/amazing-feature`) 3. Commit your changes (`git commit -m 'Add amazing feature'`) 4. Push to the branch (`git push origin feature/amazing-feature`) 5. Open a Pull Request ## License This project is licensed under the MIT License - see the LICENSE file for details.
**Made with <3 by [Shashwat Raj](https://github.com/shashwatraj), [Vaibhav Urs](https://github.com/vurs1), [Sahaj Rastogi](https://github.com/sahajrastogi), [Ananya Bhargava](https://github.com/aloobhaalu)** *Because everyone deserves to look good without the stress* [🌟 Star this repo](https://github.com/darthvader58/whatrobe) • [🐛 Report Bug](https://github.com/darthvader58/whatrobe/issues) • [💡 Request Feature](https://github.com/darthvader58/whatrobe/issues)
## Detected evidence (automated analysis) Indexed codebase: 35 recognized source files, 1216 KB. - CSS (language) — detected in the code - Express (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - OpenAI (technology) — detected in the code - PostgreSQL (technology) — detected in the code - React (technology) — detected in the code - AWS (technology) — claimed on Devpost, not found in the code - Node.js (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (46 of 46) ``` .gitignore .vercelignore .vscode/settings.json api/analyze-problem.js api/dashboard/[email].js api/debug-parse.js api/extract-text.js api/fix-db.js api/simple-analyze.js api/simple-test.js api/test-db.js api/test.js backend/Dockerfile backend/package.json backend/server.js docker-compose.yml Dockerfile.frontend LICENSE package.json public/index.html public/manifest.json public/pdf.worker.min.js public/robots.txt README.md scripts/deploy-vercel.sh scripts/deploy.sh src/App.css src/App.js src/components/Footer.js src/components/GoogleAuthDebug.js src/components/Navbar.js src/index.css src/index.js src/pages/dashboard.js src/pages/feedback.js src/pages/home.js src/pages/notFound.js src/pages/signin.js src/styles/dashboard.css src/styles/feedback.css src/styles/footer.css src/styles/home.css src/styles/navbar.css src/styles/notfound.css src/styles/signin.css vercel.json ``` ### Dependencies - backend/package.json: axios@^1.6.0, cors@^2.8.5, dotenv@^16.3.1, express@^4.18.2, multer@^1.4.5-lts.1, nodemon@^3.0.1, openai@^4.20.1, pdf-parse@^2.4.5, pg@^8.11.3 - package.json: @react-oauth/google@^0.11.1, @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^13.5.0, axios@^1.6.0, katex@^0.16.9, mathjs@^12.2.1, openai@^4.20.1, pdf-parse@^2.4.5, pdfjs-dist@^5.4.449, pg@^8.11.3, react@^18.2.0, react-dom@^18.2.0, react-dropzone@^14.3.8, react-hot-toast@^2.6.0, react-katex@^3.0.1, react-latex@^2.0.0, react-router-dom@^5.3.4, react-scripts@5.0.1, react-webcam@^7.2.0, recharts@^2.15.4, tesseract.js@^5.1.1, web-vitals@^2.1.4 ### Recent commits (newest first) - frontend fix - formath footer creator changes - licenseMerge branch 'main' of https://github.com/darthvader58/biryani - footer - Add additional contributors to README - ocr, styling, dashboard, api endpoints all fix - problem analysis part fixed - yes licenseMerge branch 'main' of https://github.com/darthvader58/biryani - hardcoded bug fixed - Add creator info and links to README - Remove support and acknowledgments sections - Add MIT License to the project - readme - logo issues fixed - dashboard fixed - feedback form and footer elements fixeed - footer styling - ui revamp and footer - wtv - fix: improve Vercel deployment and add debugging ## Key source files (fetched from GitHub, selected and truncated for size) ### docker-compose.yml ```yaml version: '3.8' services: frontend: build: context: . dockerfile: Dockerfile.frontend ports: - "3000:3000" environment: - REACT_APP_API_URL=http://localhost:8080 depends_on: - backend backend: build: context: ./backend dockerfile: Dockerfile ports: - "8080:8080" environment: - DATABASE_URL=${DATABASE_URL} - OPENAI_API_KEY=${OPENAI_API_KEY} - WOLFRAM_APP_ID=${WOLFRAM_APP_ID} - NODE_ENV=production depends_on: - db db: image: cockroachdb/cockroach:latest command: start-single-node --insecure ports: - "26257:26257" - "8081:8080" volumes: - cockroach-data:/cockroach/cockroach-data volumes: cockroach-data: ``` ### package.json ``` { "name": "biryani", "version": "0.1.0", "private": true, "dependencies": { "@react-oauth/google": "^0.11.1", "@testing-library/jest-dom": "^5.17.0", "@testing-library/react": "^13.4.0", "@testing-library/user-event": "^13.5.0", "axios": "^1.6.0", "katex": "^0.16.9", "mathjs": "^12.2.1", "openai": "^4.20.1", "pdf-parse": "^2.4.5", "pdfjs-dist": "^5.4.449", "pg": "^8.11.3", "react": "^18.2.0", "react-dom": "^18.2.0", "react-dropzone": "^14.3.8", "react-hot-toast": "^2.6.0", "react-katex": "^3.0.1", "react-latex": "^2.0.0", "react-router-dom": "^5.3.4", "react-scripts": "5.0.1", "react-webcam": "^7.2.0", "recharts": "^2.15.4", "tesseract.js": "^5.1.1", "web-vitals": "^2.1.4" }, "scripts": { "start": "react-scripts start", "build": "react-scripts build", "test": "react-scripts test", "eject": "react-scripts eject" }, "eslintConfig": { "extends": [ "react-app", "react-app/jest" ] }, "browserslist": { "production": [ ">0.2%", "not dead", "not op_mini all" ], "development": [ "last 1 chrome version", "last 1 firefox version", "last 1 safari version" ] } } ``` ### backend/Dockerfile ``` # Backend Dockerfile FROM node:18-alpine WORKDIR /app # Copy package files COPY package*.json ./ # Install dependencies RUN npm ci --only=production # Copy source code COPY . . # Expose port EXPOSE 8080 # Start the server CMD ["npm", "start"] ``` ### backend/package.json ``` { "name": "math-problem-solver-backend", "version": "1.0.0", "description": "Backend for AI-powered math problem solver", "main": "server.js", "scripts": { "start": "node server.js", "dev": "nodemon server.js" }, "dependencies": { "axios": "^1.6.0", "cors": "^2.8.5", "dotenv": "^16.3.1", "express": "^4.18.2", "multer": "^1.4.5-lts.1", "openai": "^4.20.1", "pdf-parse": "^2.4.5", "pg": "^8.11.3" }, "devDependencies": { "nodemon": "^3.0.1" } } ``` ### src/index.js ```javascript import React from 'react'; import ReactDOM from 'react-dom/client'; import './index.css'; import App from './App'; const root = ReactDOM.createRoot(document.getElementById('root')); root.render( ); ``` ### src/App.js ```javascript import React, { useState, useEffect } from 'react'; import { BrowserRouter as Router, Route, Switch } from 'react-router-dom'; import { GoogleOAuthProvider } from '@react-oauth/google'; import Navbar from './components/Navbar'; import Footer from './components/Footer'; import Home from './pages/home'; import Signin from './pages/signin'; import Dashboard from './pages/dashboard'; import Feedback from './pages/feedback'; import NotFound from './pages/notFound'; import './App.css'; // Error Boundary Component class ErrorBoundary extends React.Component { constructor(props) { super(props); this.state = { hasError: false, error: null }; } static getDerivedStateFromError(error) { return { hasError: true, error }; } componentDidCatch(error, errorInfo) { console.error('App Error:', error, errorInfo); } render() { if (this.state.hasError) { return (

Something went wrong

Error: {this.state.error?.message}

); } return this.props.children; } } function App() { const [user, setUser] = useState(null); // Debug: Check if Google Client ID is loaded const googleClientId = process.env.REACT_APP_GOOGLE_CLIENT_ID; console.log('App.js loaded - Environment check:'); console.log('NODE_ENV:', process.env.NODE_ENV); console.log('Google Client ID:', googleClientId); console.log('All env vars:', Object.keys(process.env).filter(key => key.startsWith('REACT_APP'))); if (!googleClientId) { console.error('REACT_APP_GOOGLE_CLIENT_ID is not defined in environment variables'); } // Load user from localStorage on mount useEffect(() => { const savedUser = localStorage.getItem('formath_user'); if (savedUser) { try { setUser(JSON.parse(savedUser)); } catch (e) { console.error('Error loading user:', e); } } }, []); // Handle login const handleLogin = (userData) => { setUser(userData); if (userData) { localStorage.setItem('formath_user', JSON.stringify(userData)); } else { localStorage.removeItem('formath_user'); } }; // Don't render GoogleOAuthProvider if clientId is missing if (!googleClientId) { return (

ForMath - Debug Mode

Environment Variables Missing

Google OAuth Client ID is not configured.

Environment: {process.env.NODE_ENV}

Available REACT_APP vars: {Object.keys(process.env).filter(key => key.startsWith('REACT_APP')).join(', ') || 'None'}

Add environment variables in Vercel dashboard and redeploy.

); } return (
); } export default App; ``` ### backend/server.js ```javascript const express = require('express'); const cors = require('cors'); const multer = require('multer'); const { Pool } = require('pg'); const OpenAI = require('openai'); const axios = require('axios'); const pdfParse = require('pdf-parse'); require('dotenv').config(); const app = express(); const port = process.env.PORT || 8080; // Middleware const corsOptions = { origin: process.env.NODE_ENV === 'production' ? [process.env.FRONTEND_URL, /\.vercel\.app$/] : ['http://localhost:3000', 'http://localhost:3001'], credentials: true, optionsSuccessStatus: 200 }; app.use(cors(corsOptions)); app.use(express.json({ limit: '50mb' })); app.use(express.urlencoded({ extended: true, limit: '50mb' })); // File upload configuration const storage = multer.memoryStorage(); const upload = multer({ storage: storage, limits: { fileSize: 10 * 1024 * 1024 // 10MB limit }, fileFilter: (req, file, cb) => { // Accept images and PDFs if (file.mimetype.startsWith('image/') || file.mimetype === 'application/pdf') { cb(null, true); } else { cb(new Error('Only image files and PDFs are allowed'), false); } } }); // Database connection (CockroachDB) const pool = new Pool({ connectionString: process.env.DATABASE_URL, ssl: { rejectUnauthorized: false } }); // OpenAI configuration const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY }); // Wolfram Alpha configuration const WOLFRAM_APP_ID = process.env.WOLFRAM_APP_ID; // Database initialization async function initializeDatabase() { try { await pool.query(` CREATE TABLE IF NOT EXISTS users ( id SERIAL PRIMARY KEY, email VARCHAR(255) UNIQUE NOT NULL, name VARCHAR(255) NOT NULL, created_at TIMESTAMP DEFAULT NOW() ) `); await pool.query(` CREATE TABLE IF NOT EXISTS problems ( id SERIAL PRIMARY KEY, user_email VARCHAR(255) REFERENCES users(email), problem_text TEXT NOT NULL, user_solution TEXT, correct_solution TEXT, wolfram_solution TEXT, error_type VARCHAR(100), error_description TEXT, confidence_score DECIMAL(3,2), topic VARCHAR(100), difficulty_level VARCHAR(20), time_spent INTEGER, created_at TIMESTAMP DEFAULT NOW() ) `); await pool.query(` CREATE TABLE IF NOT EXISTS user_analytics ( id SERIAL PRIMARY KEY, user_email VARCHAR(255) REFERENCES users(email), topic VARCHAR(100), total_problems INTEGER DEFAULT 0, correct_problems INTEGER DEFAULT 0, conceptual_errors INTEGER DEFAULT 0, computational_errors INTEGER DEFAULT 0, avg_confidence DECIMAL(3,2) DEFAULT 0, last_updated TIMESTAMP DEFAULT NOW() ) `); await pool.query(` CREATE TABLE IF NOT EXISTS feedback ( id SERIAL PRIMARY KEY, user_id VARCHAR(255), name VARCHAR(255) NOT NULL, email VARCHAR(255) NOT NULL, type VARCHAR(50) NOT NULL, message TEXT NOT NULL, status VARCHAR(20) DEFAULT 'new', created_at TIMESTAMP DEFAULT NOW(), updated_at TIMESTAMP DEFAULT NOW() ) `); console.log('Database initialized successfully'); } catch (error) { console.error('Database initialization error:', error); } } // Helper function to query Wolfram Alpha async function queryWolframAlpha(query) { try { const url = `http://api.wolframalpha.com/v2/query?input=${encodeURIComponent(query)}&format=plaintext&output=JSON&appid=${WOLFRAM_APP_ID}`; const response = await axios.get(url); if (response.data.queryresult && response.data.queryresult.pods) { const pods = response.data.queryresult.pods; const solutionPod = pods.find(pod => pod.title.includes('Solution') || pod.title.includes('Result') || pod.title.includes('Answer') ); if (solutionPod && solutionPod.subpods) { return solutionPod.subpods[0].plaintext; } } return null; } catch (error) { console.error('Wolfram Alpha API error:', error); return null; } } // Helper function to parse and separate problem from solution async function parseHomeworkContent(rawText) { try { const parsePrompt = ` You are an AI that helps separate math homework content into distinct parts. Given this text from a student's homework image: "${rawText}" Please identify and separate: 1. The original problem/question being asked 2. The student's solution/work (if any) 3. Any given information or constraints Look for common indicators like: - Problem indicators: "Problem:", "Question:", numbers like "1.", "2.", etc. - Solution indicators: "Solution:", "Answer:", "Work:", or mathematical work/calculations - Given information: "Given:", "Let", initial conditions Respond in JSON format: { "originalProblem": "The math problem or question being asked", "studentSolution": "The student's work/solution attempt (if present)", "givenInformation": "Any given information or constraints", "confidence": 0.95, "notes": "Any additional observations about the content structure" } If you cannot clearly distinguish between problem and solution, put everything in originalProblem and leave studentSolution empty. `; const response = await openai.chat.completions.create({ model: "gpt-4", messages: [{ role: "user", content: parsePrompt }], temperature: 0.2 }); return JSON.parse(response.choices[0].message.content); } catch (error) { console.error('Content parsing error:', error); return { originalProblem: rawText, studentSolution: '', givenInformation: '', confidence: 0, notes: 'Failed to parse content structure' }; } } // Helper function to analyze problem with ChatGPT async function analyzeW [truncated — 12056 more characters] ``` ### scripts/deploy-vercel.sh ```shell #!/bin/bash echo "Deploying ForMath to Vercel..." # Set environment variables in Vercel echo "Setting environment variables..." vercel env add DATABASE_URL vercel env add OPENAI_API_KEY vercel env add WOLFRAM_APP_ID vercel env add NODE_ENV # Deploy to Vercel echo "Deploying..." vercel --prod echo "Deployment complete!" echo "Don't forget to add your environment variables in Vercel dashboard if not set via CLI" ``` ### api/simple-test.js ```javascript export default function handler(req, res) { res.setHeader('Access-Control-Allow-Origin', '*'); res.setHeader('Access-Control-Allow-Methods', 'POST, GET, OPTIONS'); res.setHeader('Access-Control-Allow-Headers', 'Content-Type'); if (req.method === 'OPTIONS') { res.status(200).end(); return; } if (req.method === 'POST') { const { problemText } = req.body; res.status(200).json({ success: true, message: 'Simple test working', receivedText: problemText, env: { hasOpenAI: !!process.env.OPENAI_API_KEY, hasWolfram: !!process.env.WOLFRAM_APP_ID } }); } else { res.status(405).json({ error: 'Method not allowed' }); } } ``` ### api/test.js ```javascript export default function handler(req, res) { // Enable CORS res.setHeader('Access-Control-Allow-Credentials', true); res.setHeader('Access-Control-Allow-Origin', '*'); res.setHeader('Access-Control-Allow-Methods', 'GET,OPTIONS,PATCH,DELETE,POST,PUT'); res.setHeader('Access-Control-Allow-Headers', 'X-CSRF-Token, X-Requested-With, Accept, Accept-Version, Content-Length, Content-MD5, Content-Type, Date, X-Api-Version'); if (req.method === 'OPTIONS') { res.status(200).end(); return; } res.status(200).json({ message: 'Backend is working!', timestamp: new Date().toISOString(), env: { hasOpenAI: !!process.env.OPENAI_API_KEY, hasWolfram: !!process.env.WOLFRAM_APP_ID, hasDatabase: !!process.env.DATABASE_URL } }); } ``` [26 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]