# Project export: Mentra This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 12.0 - Tagline: Learn anything on our software. - Devpost: https://devpost.com/software/mentra-bpuh6w - GitHub: https://github.com/FabianSiswanto/calhacks2025 - Video: https://www.youtube.com/embed/QQZAdpMY0Xk?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 4 GitHub contributor(s) — Fabian Siswanto (20 commits), camero1993 (19 commits), mfon_ (11 commits), dewgong5 (4 commits) ## Devpost submission (written by the team) ### Inspiration A college degree and job security become more loosely correlated every year. One thing that still differentiates workers in all fields is hands on skills, and these can only be learned by doing. Many AI services have tried and failed to personalize learning, by either failing to innovate past chatbots, or simply lacking the long-term context to teach a long-term skill. Meet Mentra. ### What it does Mentra introduces a new approach to learning, where you are learn by doing, directly within your software environment; no context switching necessary. An agentic lesson planner curates lessons, with specificity down to each click. The user is then guided through the curriculum by a translucent text overlay placed on their screen—an agentic mentor challenging and guiding on them through hands-on experience. ### How we built it When discussing on the best way to build Mentra, phrases like AI agents, long context, and AI workflows were brought up then and again. The core problem we wanted to solve with Mentra was to reduce the context switching a user has to make when they have software in one screen, and a lesson in another. By solving this, we also open the door to providing instant feedback relevant to the context at that point in time. With all these goals and ideas in mind, we decided to build Mentra as an AI workflow, window-embedded lecture guide that uses AI agents with long memory to teach software to people more effectively. ### Challenges we ran into One big challenge that nearly killed our idea was configuring agentic flows for multi-step instructions including feedback to the user when they make a mistak, all while keeping AI context drift to zero. We solved this by segmenting Lectures into multiple Steps. The AI only needs two things in its co text at any point in time; the overarching goal of the lesson, and the goal of the specific step (could be a step to fix mistakes from a previous step). We saw massive improvements in AI coherence which is paramount to any educational endeavour. ### Accomplishments we're proud of The small translucent box at the top right corner that shows instructions to the user is not only innovative, but incredibly helpful. We’re proud of solving window-embedded UIs for instructive feedback. ### What we learned Start out with the simplest way to solve a specific problem. Complexity should increase out of necessity, not because you want flashy features. ### What's next We believe enterprise will benefit the most from Mentra. Imagine a world where every employee is a rockstar at each software they learn. Imagine the productivity boost! ## README (from the GitHub repository) # CalHacks 2025 - Electron + Flask + React A desktop application built with Electron, Flask backend, and React frontend for CalHacks 2025. ## Architecture - **Frontend**: React application served by Electron - **Backend**: Flask API server running locally - **Desktop**: Electron wrapper for cross-platform desktop app ## Project Structure ``` calhacks2025/ ├── backend/ # Flask backend │ ├── app.py # Main Flask application │ ├── requirements.txt # Python dependencies │ ├── models/ # Database models │ ├── routes/ # API routes │ └── utils/ # Utility functions ├── frontend/ # React frontend │ ├── src/ # React source code │ ├── public/ # Static assets │ └── package.json # Frontend dependencies ├── scripts/ # Build and utility scripts │ └── start-backend.js # Backend process manager ├── main.js # Electron main process ├── preload.js # Electron preload script └── package.json # Main dependencies and scripts ``` ## Getting Started ### Prerequisites - Node.js (v16 or higher) - Python 3.7 or higher - npm or yarn ### Installation 1. Clone the repository: ```bash git clone https://github.com/FabianSiswanto/calhacks2025.git cd calhacks2025 ``` 2. Install dependencies: ```bash npm install ``` 3. Install Python dependencies: ```bash cd backend pip install -r requirements.txt cd .. ``` ### Development 1. Start the development environment: ```bash npm run dev ``` This will: - Start the Flask backend on http://localhost:5000 - Start the React frontend on http://localhost:3000 - Launch the Electron app 2. Or start components individually: ```bash # Terminal 1: Start backend npm run start-backend # Terminal 2: Start frontend npm run start-frontend # Terminal 3: Start Electron npm start ``` ### Building for Production 1. Build the React frontend: ```bash npm run build ``` 2. Create distributable packages: ```bash # All platforms npm run dist # Specific platforms npm run dist-mac npm run dist-win npm run dist-linux ``` ## API Endpoints The Flask backend provides the following endpoints: - `GET /` - Backend status - `GET /health` - Health check - `GET /api/test` - Test endpoint - `GET /api/data` - Get sample data - `POST /api/data` - Create new data - `GET /api/files` - List project files ## Development Notes - The Flask backend runs on port 5000 - The React frontend runs on port 3000 in development - Electron loads the React app from the built files in production - Backend and frontend communicate via HTTP API calls ## Scripts - `npm start` - Start Electron app - `npm run dev` - Start development environment - `npm run start-backend` - Start Flask backend only - `npm run start-frontend` - Start React frontend only - `npm run build` - Build React frontend - `npm run dist` - Create distributable packages ## License MIT ## Detected evidence (automated analysis) Indexed codebase: 58 recognized source files, 211 KB. - CSS (language) — detected in the code - Flask (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Supabase (technology) — detected in the code ## Codebase structure (from repository index) ### Files (75 of 75) ``` .gitignore backend/app.py backend/generated_course.json backend/lesson_generator.py backend/models/__init__.py backend/pytest.ini backend/requirements.txt backend/routes/__init__.py backend/routes/api_routes.py backend/routes/lesson_plans.py backend/routes/media_routes.py backend/routes/test_db.py backend/scripts/mock_agent_demo.py backend/tests/test_database_context.py backend/tests/test_learning_agent.py backend/tools/bright_data_results.json backend/tools/bright_data_tool.py backend/tools/scraped_results.txt backend/upload_to_supabase_simple.py backend/utils/__init__.py backend/utils/database_context.py backend/utils/learning_agent.py backend/utils/README.md backend/utils/supabase.py connect_to_llm.js electron-builder.json frontend/.gitignore frontend/package.json frontend/public/index.html frontend/public/manifest.json frontend/README.md frontend/src/App.css frontend/src/App.js frontend/src/index.css frontend/src/index.js frontend/src/pages/Home.css frontend/src/pages/Home.js frontend/src/services/apiService.js frontend/src/services/screenshotService.js frontend/src/utils/supabase.js main.js overlay-screen/package.json overlay-screen/public/index.html overlay-screen/src/__tests__/screenshotService.test.js overlay-screen/src/App.css overlay-screen/src/App.js overlay-screen/src/components/OverlayScreen.css overlay-screen/src/components/OverlayScreen.js overlay-screen/src/index.css overlay-screen/src/index.js overlay-screen/src/services/apiService.js overlay-screen/src/services/screenshotService.js overlay-screen/src/services/webSocket.js package.json packages/mac_mouse_hook/binding.gyp packages/mac_mouse_hook/index.js packages/mac_mouse_hook/package.json packages/mac_mouse_hook/src/mousehook.mm preload.js README.md supabase.js test_integration.js TEST_README.md test_screenshot_pipeline.js tests/__mocks__/electron.js tests/.gitignore tests/e2e/overlay-screen.test.js tests/integration/overlay-screen.test.js tests/jest.config.js tests/overlay-simple.test.js tests/package.json tests/README.md tests/run-tests.sh tests/setup.js tests/unit/overlay-screen.test.js ``` ### Dependencies - backend/requirements.txt: beautifulsoup4@==4.12.2, brightdata-sdk@==1.1.3, chromadb, dotenv, Flask@==2.3.3, Flask-CORS@==4.0.0, Flask-SocketIO@==5.3.6, groq, html2text@==2020.1.16, httpx, letta-client, pip-system-certs, python-dotenv@==1.0.0, requests@==2.31.0, supabase - frontend/package.json: @testing-library/jest-dom@^5.16.4, @testing-library/react@^13.3.0, @testing-library/user-event@^13.5.0, axios@^0.27.2, react@^18.2.0, react-dom@^18.2.0, react-router-dom@^6.3.0, react-scripts@5.0.1, styled-components@^5.3.5, web-vitals@^2.1.4 - overlay-screen/package.json: @testing-library/jest-dom@^5.16.4, @testing-library/react@^13.3.0, @testing-library/user-event@^13.5.0, axios@^1.6.0, jest@^27.5.1, react@^18.2.0, react-dom@^18.2.0, react-scripts@5.0.1, socket.io-client@^4.7.2 - package.json: @supabase/supabase-js@^2.76.1, concurrently@^8.2.2, electron@^27.0.0, electron-builder@^24.6.4, mac-mouse-hook@file:./packages/mac_mouse_hook, react-icons@^5.5.0, wait-on@^7.2.0 - packages/mac_mouse_hook/package.json: node-addon-api@^8.5.0, node-gyp@^10.3.1 - tests/package.json: jest@^29.0.0, jest-environment-jsdom@^29.0.0 ### Recent commits (newest first) - Merge branch 'generatelesson' - chore: revert partial generate program frontend - feat: partial generate lesson plan frontend - small functions - feat: generate-lesson-plan return all steps - agent flow - feat: api/lesson_plans endpoint - feat: make bright data dynamic - learning_agent: refactor popup to use step description, add /api/screenshot-event endpoint with boolean 'completed' field - fix compilation issues - Merge pull request #7 from FabianSiswanto/letta - new - Merge pull request #6 from FabianSiswanto/letta - Merge branch 'main' into letta - learning_agent: use step description for popup; add tests and mock demo; scope pytest to backend/tests - minor ui changes to overlay - feat: implement transluscent overlay - Merge pull request #5 from FabianSiswanto/frontend - Add test files and documentation: TEST_README.md, overlay-screen tests, integration and screenshot pipeline tests - Merge: pull origin/letta + restore local changes; add batch-loaded lesson cache, screenshot event handler, WebSocket popup endpoint; resolve requirements deps (supabase, httpx, pip-system-certs, chromadb, dotenv, psycopg2-binary) ## Key source files (fetched from GitHub, selected and truncated for size) ### TEST_README.md ```markdown # Screenshot Pipeline Tests This directory contains comprehensive tests for the screenshot capture and analysis pipeline. ## Test Files ### 1. `test_integration.js` - Quick Integration Test **Purpose**: Simple test to verify the basic pipeline works **Run**: `npm test` or `node test_integration.js` **What it tests**: - ✅ Backend startup and health check - ✅ Screenshot endpoint with mock data - ✅ Basic API functionality ### 2. `test_screenshot_pipeline.js` - Comprehensive Test Suite **Purpose**: Full test suite covering all components **Run**: `npm run test-full` or `node test_screenshot_pipeline.js` **What it tests**: - ✅ File structure and service files exist - ✅ Electron screenshot setup (main.js, preload.js) - ✅ Backend health and all API endpoints - ✅ Screenshot endpoint with mock data - ✅ Lesson plan generation endpoint - ✅ Media endpoints - ✅ Complete error handling ### 3. `overlay-screen/src/__tests__/screenshotService.test.js` - Unit Tests **Purpose**: Unit tests for the screenshot service **Run**: `npm run test-overlay` or `cd overlay-screen && npm test` **What it tests**: - ✅ Screenshot service availability detection - ✅ Screenshot capture functionality - ✅ Data URL generation - ✅ Backend API integration - ✅ Error handling scenarios - ✅ Download functionality ## Running Tests ### Quick Test (Recommended) ```bash npm test ``` This runs the integration test to verify the basic pipeline works. ### Full Test Suite ```bash npm run test-full ``` This runs all tests including file structure, backend endpoints, and error handling. ### Unit Tests Only ```bash npm run test-overlay ``` This runs only the unit tests for the screenshot service. ## Test Requirements ### Prerequisites - Node.js installed - Python installed (for Flask backend) - All dependencies installed (`npm install`) ### Backend Dependencies The tests will automatically start the Flask backend, but ensure you have: - Flask installed (`pip install flask`) - All Python dependencies from `backend/requirements.txt` ## Expected Results ### ✅ Successful Test Run ``` 🧪 Testing Screenshot Pipeline Integration... 1️⃣ Starting Flask backend... ✅ Backend started successfully 2️⃣ Testing health endpoint... ✅ Health check passed 3️⃣ Testing screenshot endpoint... ✅ Screenshot endpoint working 📊 Analysis result: { ... } 🎉 All tests passed! Screenshot pipeline is working correctly. ``` ### ❌ Common Issues 1. **Backend won't start**: Check Python installation and dependencies 2. **Port 5000 in use**: Kill other processes using port 5000 3. **File not found errors**: Ensure you're running from the project root 4. **Timeout errors**: Backend might be slow to start, increase timeout in test files ## Test Coverage - **Backend API**: All endpoints tested - **Screenshot Pipeline**: Complete flow from capture to analysis - **Error Handling**: Various failure scenarios - **File Structure**: All required files present - **Electron Integration**: IPC communication setup - **Se [truncated — 365 more characters] ``` ### package.json ``` { "name": "calhacks2025", "version": "1.0.0", "description": "CalHacks 2025 - Electron + Flask + React Application", "main": "main.js", "homepage": "./", "scripts": { "start": "electron .", "dev": "concurrently \"npm run start-backend\" \"npm run start-frontend\" \"wait-on http://localhost:3000 && NODE_ENV=development electron .\"", "start-backend": "backend/venv/bin/python backend/app.py", "start-frontend": "cd frontend && BROWSER=none npm start", "build": "npm run build-mouse-hook && cd frontend && npm run build", "build-frontend": "cd frontend && npm run build", "build-mouse-hook": "cd packages/mac_mouse_hook && npm run build", "rebuild-mouse-hook": "cd packages/mac_mouse_hook && npm run rebuild", "dist": "npm run build && electron-builder", "dist-mac": "npm run build && electron-builder --mac", "dist-win": "npm run build && electron-builder --win", "dist-linux": "npm run build && electron-builder --linux", "pack": "npm run build && electron-builder --dir", "postinstall": "cd frontend && npm install && cd ../packages/mac_mouse_hook && npm run build", "test": "node test_integration.js", "test-full": "node test_screenshot_pipeline.js", "test-overlay": "cd overlay-screen && npm test" }, "keywords": [ "electron", "flask", "react", "desktop", "calhacks" ], "author": "Your Name", "license": "MIT", "dependencies": { "@supabase/supabase-js": "^2.76.1", "electron": "^27.0.0", "mac-mouse-hook": "file:./packages/mac_mouse_hook", "react-icons": "^5.5.0" }, "devDependencies": { "concurrently": "^8.2.2", "electron-builder": "^24.6.4", "wait-on": "^7.2.0" }, "build": { "appId": "com.calhacks2025.app", "productName": "CalHacks 2025", "directories": { "output": "dist" }, "files": [ "main.js", "preload.js", "backend/**/*", "frontend/build/**/*", "packages/**/*", "scripts/**/*", "node_modules/**/*" ], "mac": { "category": "public.app-category.developer-tools", "target": "dmg" }, "win": { "target": "nsis" }, "linux": { "target": "AppImage" } } } ``` ### backend/requirements.txt ``` Flask==2.3.3 Flask-CORS==4.0.0 Flask-SocketIO==5.3.6 python-dotenv==1.0.0 requests==2.31.0 brightdata-sdk==1.1.3 beautifulsoup4==4.12.2 html2text==2020.1.16 groq letta-client supabase httpx pip-system-certs chromadb dotenv ``` ### tests/package.json ``` { "name": "calhacks2025-tests", "version": "1.0.0", "description": "Test suite for CalHacks 2025 overlay screen functionality", "scripts": { "test": "jest", "test:watch": "jest --watch", "test:coverage": "jest --coverage", "test:e2e": "jest tests/e2e", "test:unit": "jest tests/unit", "test:integration": "jest tests/integration", "test:ci": "jest --ci --coverage --watchAll=false" }, "devDependencies": { "jest": "^29.0.0", "jest-environment-jsdom": "^29.0.0" } } ``` ### overlay-screen/package.json ``` { "name": "overlay-screen", "version": "1.0.0", "description": "React-based overlay screen for CalHacks 2025", "private": true, "dependencies": { "react": "^18.2.0", "react-dom": "^18.2.0", "react-scripts": "5.0.1", "axios": "^1.6.0", "socket.io-client": "^4.7.2" }, "devDependencies": { "@testing-library/jest-dom": "^5.16.4", "@testing-library/react": "^13.3.0", "@testing-library/user-event": "^13.5.0", "jest": "^27.5.1" }, "scripts": { "start": "BROWSER=none PORT=3001 react-scripts start", "build": "react-scripts build", "test": "react-scripts test", "eject": "react-scripts eject" }, "eslintConfig": { "extends": [ "react-app", "react-app/jest" ] }, "browserslist": { "production": [ ">0.2%", "not dead", "not op_mini all" ], "development": [ "last 1 chrome version", "last 1 firefox version", "last 1 safari version" ] } } ``` ### frontend/package.json ``` { "name": "calhacks2025-frontend", "version": "1.0.0", "description": "CalHacks 2025 - React Frontend", "private": true, "dependencies": { "@testing-library/jest-dom": "^5.16.4", "@testing-library/react": "^13.3.0", "@testing-library/user-event": "^13.5.0", "react": "^18.2.0", "react-dom": "^18.2.0", "react-router-dom": "^6.3.0", "react-scripts": "5.0.1", "axios": "^0.27.2", "styled-components": "^5.3.5", "web-vitals": "^2.1.4" }, "scripts": { "start": "react-scripts start", "build": "react-scripts build", "test": "react-scripts test", "eject": "react-scripts eject" }, "eslintConfig": { "extends": [ "react-app", "react-app/jest" ] }, "browserslist": { "production": [ ">0.2%", "not dead", "not op_mini all" ], "development": [ "last 1 chrome version", "last 1 firefox version", "last 1 safari version" ] }, "proxy": "http://localhost:5000" } ``` ### packages/mac_mouse_hook/package.json ``` { "name": "mac-mouse-hook", "version": "1.0.0", "description": "Native macOS mouse hook using CGEventTap", "main": "index.js", "gypfile": true, "scripts": { "build": "node-gyp rebuild", "rebuild": "node-gyp clean && node-gyp rebuild", "clean": "node-gyp clean", "install": "node-gyp rebuild" }, "devDependencies": { "node-addon-api": "^8.5.0", "node-gyp": "^10.3.1" }, "keywords": [ "macos", "mouse", "hook", "native", "addon", "electron" ], "author": "CalHacks 2025", "license": "MIT", "engines": { "node": ">=14.0.0" }, "os": [ "darwin" ] } ``` ### main.js ```javascript const { app, BrowserWindow, ipcMain, globalShortcut, desktopCapturer, screen, } = require("electron"); const path = require("path"); const { spawn } = require("child_process"); const mouseHook = require("mac-mouse-hook"); let mainWindow; let overlayWindow = null; // Track the overlay window // Mouse hook state let isMouseHookActive = false; // Overlay screen function triggered by '/' key - creates a new Electron window function triggerOverlayScreen() { // Check if overlay window already exists if (overlayWindow && !overlayWindow.isDestroyed()) { console.log("Overlay window already exists - closing it"); overlayWindow.close(); overlayWindow = null; return null; } console.log( 'Overlay screen triggered by "/" key press - creating new window' ); // Create a new overlay window - independent from main window overlayWindow = new BrowserWindow({ width: 400, height: 175, minWidth: 200, minHeight: 150, // backgroundColor: "#00000000", // Fully transparent if needed // Remove parent property to make it independent // parent: mainWindow, // Removed - now independent // modal: false, // Not needed for independent window alwaysOnTop: true, // Keep it above other windows frame: false, // Frameless window required for transparency on macOS transparent: true, // Enable window transparency webPreferences: { nodeIntegration: false, contextIsolation: true, preload: path.join(__dirname, "preload.js"), webSecurity: false, }, title: "Overlay Screen", show: false, // Don't show until ready x: 100, // Position on desktop y: 100, }); // Load content for the overlay window const isDev = !app.isPackaged; if (isDev) { // In development, try dev server first; fallback to built files if it fails overlayWindow.loadURL("http://localhost:3001").catch(() => { console.log( "Overlay dev server not available; falling back to built files" ); overlayWindow.loadFile( path.join(__dirname, "overlay-screen/public/index.html") ); }); // Also handle async load failures overlayWindow.webContents.on( "did-fail-load", (_event, _errorCode, _errorDescription, validatedURL) => { if (validatedURL && validatedURL.startsWith("http://localhost:3001")) { console.log( "Overlay failed to load dev server; loading built files instead" ); overlayWindow.loadFile( path.join(__dirname, "overlay-screen/public/index.html") ); } } ); } else { // In production, load the built React app overlayWindow.loadFile( path.join(__dirname, "overlay-screen/public/index.html") ); } // Show and initialize after content is fully loaded overlayWindow.webContents.once("did-finish-load", () => { overlayWindow.show(); console.log("Overlay window opened"); // Send initial content to overlay renderer overlayWindow.webContents.send("overlay-set-content", { header: "Step 1", body: "Using prototyping features to connect frames, add interactions, and create clickable mockups that simulate user flows.", }); }); // Handle overlay window closed overlayWindow.on("closed", () => { console.log("Overlay window closed"); overlayWindow = null; // Reset the reference when closed }); // Send message to main window that overlay was created if (mainWindow) { mainWindow.webContents.send( "child-process-output", "Overlay window created" ); } return overlayWindow; } // Mouse hook functions function startMouseMonitoring() { if (isMouseHookActive) { console.log("Mouse hook already active"); return; } try { mouseHook.start((event) => { // Print coordinates and timestamp console.log( `Mouse click: x=${event.x}, y=${ event.y }, timestamp=${new Date().toISOString()}` ); // Trigger action on every click triggerDebugAction(event); }); isMouseHookActive = true; } catch (error) { console.error("Failed to start mouse hook:", error.message); if (error.message.includes("Accessibility permissions")) { console.log( "Please enable accessibility permissions in System Preferences > Security & Privacy > Privacy > Accessibility" ); } } } function stopMouseMonitoring() { if (!isMouseHookActive) { return; } try { mouseHook.stop(); isMouseHookActive = false; } catch (error) { console.error("Failed to stop mouse hook:", error.message); } } function triggerDebugAction(lastEvent) { console.log( `Action triggered: x=${lastEvent.x}, y=${ lastEvent.y }, timestamp=${new Date().toISOString()}` ); // Send debug message to main window if it exists if (mainWindow && !mainWindow.isDestroyed()) { mainWindow.webContents.send("mouse-hook-debug", { action: "click-detected", clickData: lastEvent, timestamp: new Date().toISOString(), }); } // Send debug message to overlay window if it exists if (overlayWindow && !overlayWindow.isDestroyed()) { overlayWindow.webContents.send("mouse-hook-debug", { action: "click-detected", clickData: lastEvent, timestamp: new Date().toISOString(), }); } } function createWindow() { // Create the browser window mainWindow = new BrowserWindow({ width: 1200, height: 800, minWidth: 800, minHeight: 600, webPreferences: { nodeIntegration: false, contextIsolation: true, preload: path.join(__dirname, "preload.js"), webSecurity: false, // Allow localhost requests }, icon: path.join(__dirname, "assets/icon.png"), // Optional: add an icon show: false, // Don't show until ready }); // Backend is managed externally; no backend process started here // Load the React app const isDev = !app.isPackaged; [truncated — 4558 more characters] ``` ### backend/app.py ```python import os import sys import base64 from datetime import datetime from flask import Flask, jsonify, request from flask_cors import CORS from flask_socketio import SocketIO, emit, join_room from utils.learning_agent import ( analyze_screenshot, handle_screenshot_event, user_state, generate_and_send_popup_message, ) from utils.database_context import db_context # Add the backend directory to Python path sys.path.append(os.path.dirname(os.path.abspath(__file__))) app = Flask(__name__) CORS(app) # Allow React to make requests # Initialize SocketIO socketio = SocketIO(app, cors_allowed_origins="*") # Import routes from routes import api_routes from routes.lesson_plans import lesson_plans_bp from routes.media_routes import media_bp from routes.test_db import test_db_bp # Register blueprints app.register_blueprint(api_routes.bp) app.register_blueprint(test_db_bp, url_prefix="/api") app.register_blueprint(lesson_plans_bp, url_prefix='/api') app.register_blueprint(media_bp, url_prefix='/api') @app.route('/screenshot', methods=['POST']) def screenshot(): try: # Get the request data data = request.get_json() if not data or 'image' not in data: return jsonify({ "message": "No image data provided", "status": "error" }), 400 # Extract the base64 image data from the request base64_image = data['image'] # Optional: Log metadata if provided if 'metadata' in data: print(f"Screenshot metadata: {data['metadata']}") # Determine finish_criteria and lesson_id if provided finish_criteria = data.get('finish_criteria') lesson_id = data.get('lesson_id') step_order = data.get('step_order') user_id = data.get('user_id') # If lesson_id and step_order provided but no explicit finish_criteria, derive via batched fetch if (not finish_criteria) and (lesson_id is not None) and (step_order is not None): try: lesson_id_int = int(lesson_id) step_order_int = int(step_order) lesson_data = db_context.get_lesson_steps_batch(lesson_id_int) if step_order_int in lesson_data: finish_criteria = lesson_data[step_order_int].get('finish_criteria') or "" else: finish_criteria = "" except Exception as derive_err: print(f"Warning: failed to derive finish_criteria from lesson data: {derive_err}") finish_criteria = "" except Exception as e: return jsonify({ "message": f"Error processing request: {str(e)}", "status": "error" }), 400 # Decide flow: default to progression-aware handler. If explicitly stateless, skip progression. stateless = bool(data.get('stateless', False)) if not stateless: # Resolve identifiers from data, then from current user_state, then from defaults resolved_user_id = str(user_id or os.getenv("DEFAULT_USER_ID", "default-user")) resolved_lesson_id = lesson_id resolved_step_order = step_order if resolved_lesson_id is None or resolved_step_order is None: existing = user_state.get(resolved_user_id) if existing: resolved_lesson_id = existing.get("lesson_id") resolved_step_order = existing.get("step_order") if resolved_lesson_id is None or resolved_step_order is None: # Fallback to defaults (lesson 1, step 1) or environment overrides resolved_lesson_id = int(os.getenv("DEFAULT_LESSON_ID", "1")) resolved_step_order = 1 try: progression_result = handle_screenshot_event( resolved_user_id, int(resolved_lesson_id), int(resolved_step_order), base64_image, ) return jsonify({ "status": "success", **progression_result }) except Exception as event_err: return jsonify({ "message": f"Event handling failed: {str(event_err)}", "status": "error" }), 500 # Stateless analysis path: compute completion and return try: analysis = analyze_screenshot(base64_image, finish_criteria or "", lesson_id) completed = str(analysis).strip().upper() == "YES" return jsonify({ "message": "Screenshot analyzed successfully", "status": "success", "analysis": analysis, "completed": completed }) except Exception as analyze_err: return jsonify({ "message": f"Analysis failed: {str(analyze_err)}", "status": "error" }), 500 @app.route('/') def index(): return jsonify({ "message": "Flask backend is running!", "status": "success", "version": "1.0.0" }) @app.route('/health') def health(): return jsonify({ "status": "healthy", "service": "calhacks2025-backend" }) ## Removed consolidated event endpoint; use /screenshot only # Explicit start endpoint to trigger popup and set state before first screenshot @app.route('/api/start-step', methods=['POST']) def start_step(): try: data = request.get_json(silent=True) or {} resolved_user_id = str(data.get('user_id') or os.getenv("DEFAULT_USER_ID", "default-user")) lesson_id = data.get('lesson_id') step_order = data.get('step_order') # If not provided, derive from current state or defaults existing = user_state.get(resolved_user_id) if lesson_id is None: lesson_id = existing.get('lesson_id') if existing else int(os.getenv("DEFAULT_LESSON_ID", "1")) if step_order is None: step_order = existing.get('step_order') if existing else 1 lesson_id = int(lesson_id [truncated — 4169 more characters] ``` ### packages/mac_mouse_hook/index.js ```javascript const addon = require('./build/Release/mousehook.node'); module.exports = { start(callback) { addon.start(callback); }, stop() { addon.stop(); } }; ``` [50 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]