Project Info

StudyWorld

Inspiration

Studying today is fragmented: assignments live in one place, class files in another, and “AI help” usually lives in a separate chat tab. Even when students use AI, the tool typically has no idea what happens after it responds—whether the student actually understood, got distracted, or became dependent on asking for the next step. We wanted a workspace that feels like a place to study, where focus and learning are supported in real time, not just through prompts.

What it does

StudyWorld is a pixel-style student “office” where you can interact with specialized AI teacher agents organized by department (Math, Science, English) and then enter the Focus Room—the core experience—where you lock in on real tasks. In the Focus Room, students can: Choose tasks/assignments and run a timed study session Ask subject specialists for guided help and explanations Get learning supports like structured guidance and visual explanations (including Manim-style visuals) Use a real-time computer-vision attention signal layer to detect engagement drops during study and provide supportive interventions View session insights and AI decisions logged with timestamps It also integrates with Opennote as the long-term learning workspace/memory layer, so studying becomes continuous across sessions rather than isolated chats.

How we built it

Frontend: A web app with a pixel-office hub world and a Focus Room page for sessions, tasks, and specialist interaction AI agents: Department-based “teacher” agents designed to guide learning with subject-appropriate styles Focus Room signals: Integrated a live CV pipeline (OpenCV-based) to derive attention/engagement signals during study sessions Logging & insights: Session events and AI decisions are captured with timestamps and displayed in the UI Opennote integration: Used Opennote as the central workspace for study artifacts and structured session insights so the system can “pick up where you left off”

Challenges we ran into

Not becoming “just another chatbot”: We focused on making StudyWorld a learning environment with a dedicated Focus Room rather than a chat-first app. Designing CV support responsibly: We wanted the focus signals to feel helpful—not punitive—while avoiding storing sensitive raw video. Making adaptive behavior explainable: Logging decisions in a way that’s understandable and tied to timestamps and session context. Meaningful Opennote integration: Ensuring Opennote is used as a real memory/workspace layer, not a simple export button.

Accomplishments we're proud of

Built a gamified office UI that makes studying feel like an interactive space Shipped a working Focus Room that combines tasks, timed sessions, specialist help, and real-time engagement signals Implemented timestamped decision/insight logging so the system isn’t a black box Integrated Opennote as a backbone for continuity across sessions

What we learned

The biggest gap in AI studying tools isn’t generating answers—it’s structure, timing, and follow-through. UI/UX changes how people learn: making it feel like a “place” increases engagement compared to a blank chat box. Explainability matters: if a system adapts in real time, users need to see why it changed course.

What's next

Build a full knowledge graph view that visually shows how concepts connect over time using Opennote’s logged artifacts and relationships Add deeper student/teacher analytics (focus patterns, misconception trends, intervention effectiveness) Improve personalization: adapt focus plans and teaching style based on longer-term patterns and outcomes Expand interactive learning modes (retrieval drills, spaced repetition scheduling, and more visual-first explanations)

OfficeMates / CampusSuite

Multi-Agent Student Productivity Platform with Voice-First AI

CruzHacks 3.0 Project - Pixelated office of AI agents that process student inputs (slides, audio, PDFs, voice) and create polished notes saved to Opennote.

Quick Start

1. Install Dependencies

# Backend
cd backend
pip install -r requirements.txt

# Frontend
cd frontend
npm install

2. Configure Environment

Copy .env and fill in your API keys:

cp .env.example .env

3. Run the Application

# Terminal 1: Backend
cd backend
python main.py
# or: uvicorn main:app --reload --port 8000

# Terminal 2: Frontend
cd frontend
npm run dev

API Keys Required

Service	Environment Variable	Get Key
Google AI (Gemini)	`GOOGLE_API_KEY`	Google AI Studio
LiveKit	`LIVEKIT_API_KEY`, `LIVEKIT_API_SECRET`	LiveKit Cloud
Deepgram	`DEEPGRAM_API_KEY`	Deepgram Console
Opennote	`OPENNOTE_API_KEY`	Opennote Docs
Unwrap.ai	`UNWRAP_API_KEY`	Unwrap.ai

Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                           FRONTEND (React + PixiJS)                      │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐    │
│  │ Pixel Map   │  │ Voice UI    │  │ Gradebook   │  │ Agent View  │    │
│  │ (PixiJS)    │  │ (LiveKit)   │  │ (XP/Badges) │  │ (Progress)  │    │
│  └─────────────┘  └─────────────┘  └─────────────┘  └─────────────┘    │
└─────────────────────────────────────────────────────────────────────────┘
                              │
                              ▼ WebSocket / REST
┌─────────────────────────────────────────────────────────────────────────┐
│                           BACKEND (FastAPI)                              │
│  ┌─────────────────────────────────────────────────────────────────┐   │
│  │                    MULTI-AGENT ORCHESTRATOR                      │   │
│  │  ┌────────┐ ┌──────────┐ ┌────────┐ ┌──────────┐ ┌──────────┐  │   │
│  │  │ Intake │→│ Processor│→│ Editor │→│Researcher│→│ Actioner │  │   │
│  │  └────────┘ └──────────┘ └────────┘ └──────────┘ └──────────┘  │   │
│  └─────────────────────────────────────────────────────────────────┘   │
│                              │                                          │
│  ┌───────────────┬───────────────┬───────────────┬───────────────┐    │
│  │ LiveKit       │ Deepgram      │ Opennote      │ Unwrap.ai     │    │
│  │ (Voice)       │ (STT)         │ (Notes/AI)    │ (Social)      │    │
│  └───────────────┴───────────────┴───────────────┴───────────────┘    │
└─────────────────────────────────────────────────────────────────────────┘

Agent Pipeline

Agent	Role	Opennote	Unwrap
Intake	Parse metadata, extract due dates	-	-
Processor	Heavy LLM work (summaries, OCR, STT)	Feynman-3 model	-
Editor	Clean, normalize, polish content	Feynman-3 model	-
Researcher	Enrich with community context	-	Search API
Actioner	Create gradebook entries, save notes	Create Note API	-
QA	Validate, confidence scoring	-	-
Personality	UI messages, voice responses	-	-

Voice Interaction Flow

User speaks → LiveKit captures audio
                    ↓
            Deepgram STT transcribes
                    ↓
            Intent Detection
            ┌───────┴───────┐
      Question?         Dictation?
           ↓                 ↓
      Search Notes     Run Agent Pipeline
           ↓                 ↓
      Voice Response   Create Note → Opennote
                             ↓
                       Voice Confirmation

API Endpoints

Tasks

POST /api/tasks - Create processing task
GET /api/tasks/{id} - Get task status
GET /api/tasks?user_id=xxx - List user tasks

Voice

POST /api/voice/session - Create voice session (returns LiveKit token)
POST /api/voice/transcript - Process transcript
GET /api/livekit/token - Get LiveKit token directly

Users

GET /api/users/{id}/xp - Get XP and level
POST /api/users/{id}/xp - Update XP

Departments

GET /api/departments - List all departments

WebSocket

WS /ws/{user_id} - Real-time updates

WebSocket Events

Client → Server

{ "event": "transcript", "text": "...", "is_final": true }
{ "event": "ping" }

Server → Client

{ "event": "task_progress", "current_agent": "Processor", "progress": 0.5 }
{ "event": "task_completed", "xp_awarded": 15, "opennote_note_id": "..." }
{ "event": "voice_response", "text": "...", "agent": "Professor Pixel" }
{ "event": "xp_update", "level": 3, "level_up": true }

Department Personalities

Department	Agent	Catchphrase	Color
Math	Professor Pixel	"Let me calculate..."	#4A90D9
Science	Dr. Beaker	"Hypothesis confirmed!"	#5CB85C
English	Scribe McWrite	"A tale worth telling..."	#F0AD4E
Study Hub	Coach Campus	"You've got this!"	#9B59B6

Gamification

XP Rewards

Note created: 10 XP
Voice note: 15 XP
Flashcard correct: 5 XP
Quiz completed: 15 XP
Streak day: 20 XP

Levels

Freshman (0 XP)
Sophomore (100 XP) - Custom avatar color
Junior (300 XP) - Desk plant
Senior (600 XP) - Custom agent name
Graduate (1000 XP) - Gold badge
Professor (5000 XP) - All decorations

Demo Script (3 min)

0:00 - Hook: "This is CampusSuite: your pixel office where agents do the heavy lifting."

0:10 - Drag lecture slides to Math Intake → show agent procession

0:30 - Processor shows summary; Researcher pulls community tips from Unwrap

0:50 - Actioner saves note to Opennote (show success)

1:10 - Open Gradebook, mark item done → XP + decor unlock

1:25 - Voice demo: "Hey, can you explain integrals?" → Agent responds

1:45 - Show Manim video generation for equation

2:00 - Quick flashcard quiz from note

2:20 - Wrap: API log screenshot proving Opennote & Unwrap integration

Project Structure

CruzHacks3.0/
├── .env                          # Environment variables
├── MASTER_CONFIG.py              # All prompts, schemas, configs
├── README.md
│
├── backend/
│   ├── main.py                   # FastAPI server
│   ├── requirements.txt
│   └── manim_scenes/            # Manim video templates
│
└── frontend/
    ├── package.json
    └── src/
        ├── lib/
        │   └── livekit-voice.ts  # LiveKit integration
        ├── hooks/
        │   └── useVoiceSession.ts
        └── components/
            └── VoiceInterface.tsx

Troubleshooting

LiveKit connection fails

Check LIVEKIT_API_KEY and LIVEKIT_API_SECRET
Ensure LiveKit URL starts with wss://

Deepgram not transcribing

Verify DEEPGRAM_API_KEY is valid
Check browser microphone permissions
Ensure sample rate matches (16000 Hz)

Demo mode

Set ENABLE_MOCKS=true in .env for stable demo with mock responses.

Team

CruzHacks 3.0 - Built with LiveKit, Deepgram, Opennote, Unwrap.ai, Manim, and love.

Analysis

Compare with all teams

View

Metric

ShryukGrandhi
2

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

CSSIn code
FastAPIIn code
JavaScriptIn code
Next.jsIn code
PythonIn code
ReactIn code
Tailwind CSSIn code
TypeScriptIn code

8 of 8 appear in the indexed code.

AI coding agents

Claude CodeConfig

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

646 KB

Source files

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.