# Project export: Vox

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: UC Berkeley AI Hackathon 2025
- Tagline: Vox is a voice-driven IDE for devs with carpal tunnel, fatigue, or disabilities. Write code, navigate files, and run terminal commands: all hands-free. Code without compromise.
- Devpost: https://devpost.com/software/vox-l2v8iu
- GitHub: https://github.com/PatelPurav05/vox
- Demo: https://drive.google.com/uc?id=1-OG1ovVadaOkK6yhyRG03N_ViZovcTv0
- Video: https://www.youtube.com/embed/kJLmLs8bAXY?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 2 GitHub contributor(s) — Purav Patel (23 commits), Jasmine (2 commits)

## Devpost submission (written by the team)

### Inspiration

It started with pain. One of our teammates, a computer science major, was struggling with carpal tunnel syndrome. Writing even a few lines of code felt painful, and she almost couldn’t attend the hackathon because of it. For developers, our keyboards are our tools, our creative outlets. But what happens when you can’t use them? That question hit close to home, and it made us realize how many others might be quietly dealing with the same problem. We wanted to build something that would make coding accessible, no matter your physical limitations. Something that said: even if your hands can't move, your ideas still can. What It Does That’s why we built Vox, a voice-based IDE designed from scratch to empower developers to code using only their voice. Think Cursor meets speech recognition. You speak, and Vox writes the code. But it goes further: Navigate files and folders Search your codebase Open and close terminals Run shell commands All without touching your keyboard. Core Stack Vapi – Real-time speech-to-text Gemini – Translate natural language into code Rust – Powering the IDE backend Vite (React) – Clean, responsive frontend UI Whether you're prototyping, experiencing fatigue, or simply thinking out loud, Vox turns your spoken ideas into executable code. How We Built It Frontend Vite Tailwind CSS for elegant design Backend Vapi API for accurate voice transcription Gemini LLM for translating voice-to-code logic Rust for high-performance core functionality Features Natural-language code generation for Python Edit and refactor code using voice commands File and folder navigation (e.g., “open main.py”, “go to line 12”) Terminal command execution (e.g., “run npm install”, “git status”) Full support for hands-free coding sessions Challenges Getting microphone access and real-time audio streaming to work on macOS was unexpectedly difficult. Only one of our laptops initially supported it, which slowed early testing. Parsing when a spoken phrase is actual logic (“make a new function”) versus non-coding commentary (“uhh wait what was that”) required extra attention. Accomplishments We’re Proud Of One of the most rewarding moments was getting the full voice-to-code flow working on all our machines. In the beginning, only one laptop could handle audio input correctly, which slowed us down significantly. After rewriting parts of our input pipeline, we got it running smoothly across all four machines. We’re also proud of how well the backend came together. Using Vapi for voice transcription and Gemini for text-to-code generation gave us strong results. The two tools worked better than expected when integrated, and we fine-tuned the interactions to make the experience feel natural. It was exciting to see the LLM understand what we meant and return usable, runnable code. On the frontend, we focused on making the UI clean, intuitive, and approachable. We didn’t just want to build a functional tool: we wanted it to feel good to use. That included everything from layout and styling to the product name itself. We chose “Vox” because it’s simple, memorable, and communicates exactly what the product is about: your voice, translated into code. What We Learned Throughout this project, we learned a lot about building end-to-end systems, especially those relying on voice as the primary input. We gained experience integrating multiple APIs, like Vapi and Gemini, and learned how to fine-tune them to work together smoothly. None of us had used Rust before, so diving into it for the backend was a challenge that pushed us to grow quickly. On the frontend, we learned how much design matters when building developer tools. Making the interface simple, intuitive, and responsive made a huge difference. Most importantly, we learned how powerful good teamwork can be. Each of us brought different skills to the table, and figuring things out together made the process faster, smoother, and a lot more fun. What’s Next for Vox Expand support to multiple languages beyond just JavaScript or Python Enable real-time code generation with streaming output Improve LLM performance and responsiveness Native Git integration Voice-based natural language search Support all dev workflows—using only voice

## README (from the GitHub repository)

# Vox – Voice-Based IDE

Vox is a voice-first coding environment built to make programming more accessible. Whether you're dealing with physical limitations or just want to give your hands a break, Vox lets you build software using only your voice.

## What It Does

- Translate speech to working code
- Navigate through your files and folders by voice
- Search through code without touching the keyboard
- Get AI-powered suggestions and refactors through spoken prompts
- Run terminal commands with your voice

## Why We Built It

One of our teammates struggled with carpal tunnel, and it made us think—what happens when your hands can’t keep up, but your mind still can? Vox was built to bridge that gap. It’s a tool for anyone who wants to write code without needing to type.

## Getting Started

### Requirements

- Node.js (v18 or higher)
- Rust (if compiling natively)
- Tauri CLI
- A decent mic

## Tech Stack

- Tauri for native desktop build
- Vite (React) for UI
- Vapi + Gemini for voice processing and LLM
- Rust for secure system-level operations

## Contributing

If you're passionate about accessible tools or want to bring new ideas to voice-based development, feel free to open an issue or pull request.

---

Built with accessibility and creativity in mind.


## Detected evidence (automated analysis)

Indexed codebase: 27 recognized source files, 215 KB.
- CSS (language) — detected in the code
- Google Gemini (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- React (technology) — detected in the code
- Rust (language) — detected in the code

## Codebase structure (from repository index)

### Files (37 of 37)

```
.gitignore
README.md
tauri-app/.gitignore
tauri-app/.vscode/extensions.json
tauri-app/index.html
tauri-app/package.json
tauri-app/README.md
tauri-app/src-tauri/.gitignore
tauri-app/src-tauri/build.rs
tauri-app/src-tauri/capabilities/default.json
tauri-app/src-tauri/Cargo.lock
tauri-app/src-tauri/Cargo.toml
tauri-app/src-tauri/icons/icon.icns
tauri-app/src-tauri/src/lib.rs
tauri-app/src-tauri/src/main.rs
tauri-app/src-tauri/tauri.conf.json
tauri-app/src/App.css
tauri-app/src/App.jsx
tauri-app/src/components/ContextMenu.css
tauri-app/src/components/ContextMenu.jsx
tauri-app/src/components/FileExplorer.css
tauri-app/src/components/FileExplorer.jsx
tauri-app/src/components/NewFileDialog.css
tauri-app/src/components/NewFileDialog.jsx
tauri-app/src/components/Terminal.css
tauri-app/src/components/Terminal.jsx
tauri-app/src/components/VoiceAssistant.css
tauri-app/src/components/VoiceAssistant.jsx
tauri-app/src/config/voiceAssistant.example.js
tauri-app/src/config/voiceAssistant.js
tauri-app/src/hooks/useVapi.js
tauri-app/src/index.html
tauri-app/src/main.jsx
tauri-app/src/services/geminiService.js
tauri-app/src/styles.css
tauri-app/vite.config.js
tauri-app/VOICE_ASSISTANT_README.md
```

### Dependencies

- tauri-app/package.json: @google/generative-ai@^0.24.1, @monaco-editor/react@^4.6.0, @tauri-apps/api@^2.0.0, @tauri-apps/cli@^2, @tauri-apps/plugin-dialog@^2.0.0, @tauri-apps/plugin-fs@^2.0.0, @tauri-apps/plugin-shell@^2.2.2, @vapi-ai/web@^2.3.6, @vitejs/plugin-react@^4.2.1, react@^18.2.0, react-dom@^18.2.0, vite@^5.1.0, xterm@^5.3.0, xterm-addon-fit@^0.8.0
- tauri-app/src-tauri/Cargo.toml: serde@1, serde_json@1, tauri@2, tauri-build@2, tauri-plugin-dialog@2, tauri-plugin-fs@2, tauri-plugin-opener@2, tauri-plugin-shell@2

### Recent commits (newest first)

- Create README.md
- Merge pull request #4 from PatelPurav05/purav4
- Merge pull request #3 from PatelPurav05/frontend-jasmine2
- Merge branch 'purav4' into frontend-jasmine2
- Refactor GeminiService for Enhanced Action Processing and User Guidance
- frontend fixed more
- Merge pull request #2 from PatelPurav05/frontend-jasmine
- Merge branch 'working' into frontend-jasmine
- Add Terminal Command Execution to Voice Assistant and Terminal Integration
- Enhance File Handling and Editor Focus in Voice Assistant and File Explorer
- Refactor FileExplorer and VoiceAssistant for Enhanced Directory Expansion
- frontend fixed
- Enhance File Explorer and Voice Assistant Integration
- Implement Search Functionality and Navigation in Voice Assistant
- Enhance Voice Assistant with New Cursor Movement and Formatting Features
- Enhance Voice Assistant with Improved Command Processing and Error Handling
- Implement feedback loop prevention for VAPI assistant
- Enhance Voice Assistant with Text-to-Speech and Editor Context Features
- Merge pull request #1 from PatelPurav05/purav
- added voice stuff

## Key source files (fetched from GitHub, selected and truncated for size)

### tauri-app/VOICE_ASSISTANT_README.md

```markdown
# Voice Assistant for Vox IDE

A simple voice-controlled coding assistant that converts speech to code actions using VAPI AI with **automatic fallback to browser Speech Recognition API** for maximum reliability.

## Setup

### Step 1: Create a VAPI Assistant (Optional)

1. **Go to VAPI Dashboard**: Visit https://dashboard.vapi.ai
2. **Create an Assistant**: Click "Create Assistant" and configure:
   - **Name**: "Coding Assistant" 
   - **Model**: Choose Gemini 2.5 Flash or your preferred model
   - **Voice**: Select a voice provider (11Labs, PlayHT, etc.)
   - **System Message**: 
     ```
     You are a coding assistant. Listen to the user's voice commands about code and respond briefly and helpfully. Keep responses under 30 words.
     ```
3. **Save the Assistant**: Copy the Assistant ID from the dashboard

### Step 2: Configure API Keys

1. Copy `src/config/voiceAssistant.example.js` to `src/config/voiceAssistant.js`
2. Update the configuration:
   ```javascript
   export const voiceAssistantConfig = {
     vapi: {
       publicKey: 'your_vapi_public_key',     // From VAPI Dashboard
       assistantId: 'your_assistant_id'       // From Step 1 (optional)
     },
     gemini: {
       apiKey: 'your_gemini_api_key'          // For code processing (required)
     }
   };
   ```

### Required API Keys:
- **VAPI Public Key**: Get from https://dashboard.vapi.ai (optional)
- **Assistant ID**: Created in Step 1 above (optional)
- **Gemini API Key**: Get from https://makersuite.google.com/app/apikey (required)

**Note**: If VAPI setup fails, the system automatically falls back to browser Speech Recognition API.

## How it Works

### **Dual-Mode Speech Recognition:**

#### **Primary Mode (VAPI):**
1. Click the microphone button → Green button indicates VAPI mode
2. Speak your command (e.g., "Go to line 15", "Create function hello")
3. VAPI processes speech + provides voice responses
4. Gemini processes the command and performs editor actions

#### **Fallback Mode (Browser Speech API):**
1. If microphone levels are too low → Automatically switches to Speech API
2. Orange button with "API" badge indicates fallback mode
3. Browser handles speech recognition (no voice responses)
4. Gemini still processes commands and performs editor actions

## Supported Commands

- **"Go to line [number]"** - Moves cursor to specified line
- **"Create function [name]"** - Creates a new function
- **"Delete line [number]"** - Deletes the specified line  
- **"Insert [description]"** - Inserts code at current position

## Smart Fallback System

The voice assistant automatically handles microphone issues:

```
🎤 Low microphone levels detected
     ↓
🔄 Falling back to Web Speech API
     ↓
🎤 Using Browser Speech Recognition (Fallback Mode)
```

**Visual Indicators:**
- **Green Button**: VAPI mode (with voice responses)
- **Orange Button + "API" badge**: Browser Speech API mode
- **Status Messages**: Clear indication of current mode

## Troubleshooting

### **No Speech Detect
[truncated — 4492 more characters]
```

### tauri-app/package.json

```
{
  "name": "vox-ide",
  "private": true,
  "version": "0.1.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "vite build",
    "preview": "vite preview",
    "tauri": "tauri",
    "tauri:dev": "tauri dev",
    "tauri:build": "tauri build"
  },
  "devDependencies": {
    "@tauri-apps/cli": "^2",
    "@vitejs/plugin-react": "^4.2.1",
    "vite": "^5.1.0"
  },
  "dependencies": {
    "@google/generative-ai": "^0.24.1",
    "@monaco-editor/react": "^4.6.0",
    "@tauri-apps/api": "^2.0.0",
    "@tauri-apps/plugin-dialog": "^2.0.0",
    "@tauri-apps/plugin-fs": "^2.0.0",
    "@tauri-apps/plugin-shell": "^2.2.2",
    "@vapi-ai/web": "^2.3.6",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "xterm": "^5.3.0",
    "xterm-addon-fit": "^0.8.0"
  }
}

```

### tauri-app/src-tauri/Cargo.toml

```
[package]
name = "vox-ide"
version = "0.1.0"
description = "A lightweight text editor IDE"
authors = ["you"]
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[lib]
# The `_lib` suffix may seem redundant but it is necessary
# to make the lib name unique and wouldn't conflict with the bin name.
# This seems to be only an issue on Windows, see https://github.com/rust-lang/cargo/issues/8519
name = "vox_ide_lib"
crate-type = ["staticlib", "cdylib", "rlib"]

[build-dependencies]
tauri-build = { version = "2", features = [] }

[dependencies]
tauri = { version = "2", features = [] }
tauri-plugin-opener = "2"
tauri-plugin-fs = "2"
tauri-plugin-dialog = "2"
tauri-plugin-shell= "2"
serde = { version = "1", features = ["derive"] }
serde_json = "1"


```

### tauri-app/src/main.jsx

```javascript
import React from 'react'
import ReactDOM from 'react-dom/client'
import App from './App.jsx'
import './styles.css'
 
ReactDOM.createRoot(document.getElementById('root')).render(
  <React.StrictMode>
    <App />
  </React.StrictMode>,
) 
```

### tauri-app/src-tauri/src/main.rs

```rust
// Prevents additional console window on Windows in release, DO NOT REMOVE!!
#![cfg_attr(not(debug_assertions), windows_subsystem = "windows")]

fn main() {
    vox_ide_lib::run()
}

```

### tauri-app/src/App.jsx

```javascript
// import React, { useState, useEffect, useRef } from 'react'
// import Editor from '@monaco-editor/react'
// import FileExplorer from './components/FileExplorer'
// import { readTextFile, writeTextFile, exists, create, mkdir, remove, rename } from '@tauri-apps/plugin-fs'
// import { open } from '@tauri-apps/plugin-dialog'
// import Terminal from './components/Terminal'
// import ContextMenu from './components/ContextMenu'
// import NewFileDialog from './components/NewFileDialog'
// import VoiceAssistant from './components/VoiceAssistant'
// import './App.css'

// function App() {
//   const [currentFile, setCurrentFile] = useState(null)
//   const [fileContent, setFileContent] = useState('')
//   const [unsavedChanges, setUnsavedChanges] = useState(false)
//   const [isAutoSaving, setIsAutoSaving] = useState(false)
//   const [rootPath, setRootPath] = useState(null)
//   const [showDebug, setShowDebug] = useState(false)
//   const [fileTreeRefresh, setFileTreeRefresh] = useState(0)
//   const [debugLogs, setDebugLogs] = useState([])
//   const [debugPanelWidth, setDebugPanelWidth] = useState(300)
//   const [sidebarWidth, setSidebarWidth] = useState(250)
//   const autoSaveTimeoutRef = useRef(null)
//   const [isTerminalVisible, setIsTerminalVisible] = useState(false)
//   const [contextMenu, setContextMenu] = useState({
//     isVisible: false,
//     position: { x: 0, y: 0 },
//     targetPath: null,
//     isDirectory: false
//   })
//   const [newFileDialog, setNewFileDialog] = useState({
//     isVisible: false,
//     targetPath: null
//   })
//   const [editorInstance, setEditorInstance] = useState(null)

//   const sidebarRef = useRef(null)
//   const debugRef = useRef(null)

//   // Simple debug logging function without console override
//   const addDebugLog = (message, type = 'info') => {
//     const timestamp = new Date().toLocaleTimeString()
//     setDebugLogs(prev => [...prev.slice(-30), { timestamp, message, type }]) // Keep last 30 logs
//     console.log(`[${timestamp}] ${message}`)
//   }

//   // Auto-save functionality
//   useEffect(() => {
//     if (unsavedChanges && currentFile) {
//       // Clear existing timeout
//       if (autoSaveTimeoutRef.current) {
//         clearTimeout(autoSaveTimeoutRef.current)
//       }

//       // Set new timeout for auto-save
//       autoSaveTimeoutRef.current = setTimeout(async () => {
//         await handleSave(true)
//       }, 2000) // Auto-save after 2 seconds of inactivity
//     }

//     return () => {
//       if (autoSaveTimeoutRef.current) {
//         clearTimeout(autoSaveTimeoutRef.current)
//       }
//     }
//   }, [unsavedChanges, fileContent, currentFile])

//   // Resizing functionality for debug panel
//   const handleDebugResize = (e) => {
//     e.preventDefault()
//     const startX = e.clientX
//     const startWidth = debugPanelWidth

//     const doDrag = (e) => {
//       const newWidth = startWidth + (startX - e.clientX) // Reversed direction
//       setDebugPanelWidth(Math.max(200, Math.min(600, newWidth)))
//     }

//     const stopDrag = () => {
//       document.removeEventListener('mousemove', doDrag)
//       document.removeEventListener('mouseup', stopDrag)
//     }

//     document.addEventListener('mousemove', doDrag)
//     document.addEventListener('mouseup', stopDrag)
//   }

//   // Resizing functionality for sidebar
//   const handleSidebarResize = (e) => {
//     e.preventDefault()
//     const startX = e.clientX
//     const startWidth = sidebarWidth

//     const doDrag = (e) => {
//       const newWidth = startWidth + (e.clientX - startX) // This direction is correct
//       setSidebarWidth(Math.max(200, Math.min(500, newWidth)))
//     }

//     const stopDrag = () => {
//       document.removeEventListener('mousemove', doDrag)
//       document.removeEventListener('mouseup', stopDrag)
//     }

//     document.addEventListener('mousemove', doDrag)
//     document.addEventListener('mouseup', stopDrag)
//   }

//   const handleFileSelect = async (filePath) => {
//     try {
//       addDebugLog(`Opening file: ${filePath}`)
      
//       // Normalize the file path to handle mixed separators
//       const normalizedPath = filePath.replace(/\\/g, '/')
      
//       // Check if file exists first
//       const fileExists = await exists(normalizedPath)
//       if (!fileExists) {
//         addDebugLog(`File does not exist: ${normalizedPath}`, 'error')
//         return
//       }
      
//       const content = await readTextFile(normalizedPath)
//       setFileContent(content)
//       setCurrentFile(filePath) // Keep original path for display
//       setUnsavedChanges(false)
      
//       // Clear any existing auto-save timeout
//       if (autoSaveTimeoutRef.current) {
//         clearTimeout(autoSaveTimeoutRef.current)
//         autoSaveTimeoutRef.current = null
//       }
//     } catch (error) {
//       addDebugLog(`Error reading file: ${error.message || error}`, 'error')
//       console.error('Error reading file:', error)
//     }
//   }

//   const handleEditorChange = (value) => {
//     setFileContent(value || '')
//     setUnsavedChanges(true)
//   }

//   const handleSave = async (isAutoSave = false) => {
//     if (!currentFile || !unsavedChanges) {
//       return
//     }

//     try {
//       addDebugLog(`${isAutoSave ? 'Auto-' : 'Manual '}saving: ${currentFile}`)
      
//       if (isAutoSave) {
//         setIsAutoSaving(true)
//       }

//       await writeTextFile(currentFile, fileContent)
//       setUnsavedChanges(false)
      
//       addDebugLog('File saved successfully')

//       if (isAutoSave) {
//         setTimeout(() => setIsAutoSaving(false), 1000)
//       }
//     } catch (error) {
//       addDebugLog(`Error saving file: ${error.message}`, 'error')
//       alert(`Error saving file: ${error.message}`)
      
//       if (isAutoSave) {
//         setIsAutoSaving(false)
//       }
//     }
//   }

//   const handleKeyDown = (event) 
[truncated — 29974 more characters]
```

### tauri-app/vite.config.js

```javascript
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'

export default defineConfig({
  plugins: [react()],
  clearScreen: false,
  server: {
    port: 1420,
    strictPort: true,
    watch: {
      ignored: ['**/src-tauri/**'],
    },
  },
}) 
```

### tauri-app/index.html

```html
<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" type="image/svg+xml" href="/assets/tauri.svg" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Vox IDE</title>
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="/src/main.jsx"></script>
  </body>
</html> 
```

### tauri-app/src-tauri/build.rs

```rust
fn main() {
    tauri_build::build()
}

```

### tauri-app/src/index.html

```html
<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="stylesheet" href="styles.css" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Tauri App</title>
    <script type="module" src="/main.js" defer></script>
  </head>

  <body>
    <main class="container">
      <h1>Welcome to Tauri</h1>

      <div class="row">
        <a href="https://tauri.app" target="_blank">
          <img src="/assets/tauri.svg" class="logo tauri" alt="Tauri logo" />
        </a>
        <a
          href="https://developer.mozilla.org/en-US/docs/Web/JavaScript"
          target="_blank"
        >
          <img
            src="/assets/javascript.svg"
            class="logo vanilla"
            alt="JavaScript logo"
          />
        </a>
      </div>
      <p>Click on the Tauri logo to learn more about the framework</p>

      <form class="row" id="greet-form">
        <input id="greet-input" placeholder="Enter a name..." />
        <button type="submit">Greet</button>
      </form>
      <p id="greet-msg"></p>
    </main>
  </body>
</html>

```

[17 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]