🧠 Braillience

🎥 Demo Video: https://youtu.be/1WcWxWYDlo8?si=g8aWwUHi0QJ5CZUg

---

🎯 Problem Statement

Blind and visually impaired college students face significant barriers when using traditional learning platforms — most study apps are not screen reader compatible, rely on visual cues, and don’t support voice-only control.
Existing accessibility tools are fragmented, forcing students to switch between multiple apps to study effectively.

The challenge:
How can we create a fully voice-driven, accessible learning experience that allows blind students to study hands-free, generate flashcards automatically, and navigate entirely through speech?

---

💡 Our Solution

Braillience is an accessible learning platform designed specifically for blind and visually impaired learners.
It enables teachers to upload PDFs, automatically generate flashcards using AI, and study completely via voice commands — no mouse or screen required. Students start an in-browser voice session and converse with an AI tutor using their microphone and speakers.

Key Innovations:

• 🎙️ Voice-First Learning Interface: Fully voice-controlled navigation for flashcards, tests, and progress tracking.
• 🧠 AI-Generated Flashcards: Letta Agents analyze uploaded PDFs to create structured study sets automatically.
• 🗣️ Seamless Voice Processing: The browser's built-in Web Speech API handles speech recognition and synthesis for real-time, natural voice interactions, at no per-minute cost.

Result: A hands-free, inclusive learning experience that empowers blind students to study independently.

🎯 Project Structure

Braillience/
├── backend/                 # Node.js API server
│   ├── src/
│   │   ├── routes/         # API endpoints
│   │   │   ├── auth.js     # Authentication
│   │   │   ├── upload.js   # PDF upload & processing
│   │   │   ├── flashcards.js # Flashcard management
│   │   │   └── voice-learning.js # Browser-based voice tutoring sessions
│   │   ├── services/       # Business logic
│   │   │   ├── aiService.js # AI flashcard generation
│   │   │   └── voiceSessionService.js # Voice session logic & answer grading
│   │   └── server.js       # Express server
│   ├── package.json
│   └── env.example
├── frontend/               # React frontend application
│   ├── src/
│   │   ├── components/    # React components
│   │   │   ├── Dashboard.js/css    # Main dashboard
│   │   │   ├── VoiceTutor.js/css   # In-browser AI voice tutoring session
│   │   │   ├── Login.js/css        # Authentication
│   │   │   ├── Header.js/css       # Navigation header
│   │   │   └── Footer.js/css       # Footer
│   │   ├── contexts/      # React contexts
│   │   │   └── AuthContext.js
│   │   ├── hooks/         # Custom hooks
│   │   │   └── useVoiceCommands.js # Web Speech API recognition hook
│   │   ├── App.js/css     # Main app component
│   │   └── index.js/css   # App entry point
│   ├── public/            # Static assets
│   │   ├── index.html
│   │   └── manifest.json
│   └── package.json
├── docs/                  # Documentation
├── tasks/                 # Project requirements
│   └── prd-braillience-flashcard-app.md
├── start-demo.sh         # Demo startup script
└── README.md

---

🛠️ Tech Stack

Frontend

• React 18 + TypeScript — Modern and accessible UI
• React Router — App navigation
• Framer Motion — Subtle motion for transitions

Backend

• Node.js + Express — API and logic layer
• Multer — PDF uploads
• JWT — Authentication
• Letta Agents — AI content generation

Voice & AI

• Web Speech API — In-browser speech recognition and synthesis, free
• Letta File + Agent — Stateful learning assistance
• Gemini AI — Enhanced semantic understanding
• Claude Sonnet 3.5 - AI automation and structured calling

---

🚀 Getting Started

Prerequisites

• Node.js 18+
• npm or yarn

Installation & Running

1. Start Backend:

   cd backend
   npm install
   npm start
   

2. Start Frontend:

   cd frontend
   npm install
   npm start
   

3. Or use the demo script:

   ./start-demo.sh
   

Access Points

• Frontend: http://localhost:3000
• Backend API: http://localhost:3001

🎤 Voice Features

Voice Commands

• Navigation: "Start Learning", "Take Test", "Go Back"
• Learning: "Show Answer", "Next Card", "Previous Card", "Repeat"
• Testing: "Select A", "Select B", "Select C", "Select D"

Voice Integration

• Speech-to-text for commands
• Text-to-speech for content
• Voice navigation throughout the app
• Screen reader compatibility

📱 Features

Core Functionality

• PDF Upload: Upload documents to generate flashcards
• AI Generation: Automatic flashcard creation from content
• Learning Modes: Flashcards, multiple choice, typing
• Testing: Timed tests with scoring
• Progress Tracking: Learning analytics and statistics

User Experience

• Voice Navigation: Complete control via voice in call
• Responsive Design: Works on all phones
• Offline Support: Continue learning without internet
• Multi-language: Support for various languages

🎯 Demo Flow

1. Login → Use demo credentials
2. Dashboard → View progress and quick actions
3. Learn → Practice with flashcards using voice commands
4. Test → Take timed quizzes with voice selection
5. Progress → Track learning statistics

🔧 Development

Project Structure

• Clean separation between frontend and backend
• Modular components for easy maintenance
• Voice-first architecture throughout
• Accessibility-first design principles

Key Files

• frontend/src/App.js - Main React application
• frontend/src/components/Dashboard.js - Main dashboard with voice
• backend/src/server.js - Express server setup
• backend/src/services/aiService.js - AI integration

📚 Documentation

• PRD: tasks/prd-braillience-flashcard-app.md
• Accessibility: docs/ACCESSIBILITY_GUIDELINES.md
• Contributing: CONTRIBUTING.md

📝 License

This project is licensed under the MIT License.

Built with ❤️ by the Braillience Team at Cal Hacks 2025 for accessibility and inclusion