# Project export: WiiWork

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: Everyone is hyped about AI agents that can interact with the world. Problem: Websites weren't built with LLMs in mind What would it look like if a website was built ground up with Agent usage first?
- Devpost: https://devpost.com/software/wiiwork-8u1x62
- GitHub: https://github.com/Alezander9/WiiWork
- Demo: https://wiiwork.dev/
- Video: https://www.youtube.com/embed/rkLqYKocnNc?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner (NVIDIA: Best Accelerated Compute (5090s per team member - 1 GPU JHH signed [1st] & 5080s per team member [2nd]))
- Team: 1 GitHub contributor(s) — Alezander9 (48 commits)

## Devpost submission (written by the team)

### Overview

Why: Everyone's excited about AI agents interacting with the world. Companies are pouring billions into development. But there's a problem: websites weren't built with AI in mind. Over this weekend, I sought to answer: what would it look like to build a website from the ground up to be AI-native? How I built WiiWork solo over the weekend, drawing inspiration from web accessibility patterns. Just like screen readers use ARIA labels, I created a context-rich system where AI agents can understand and interact with web elements naturally. The core innovation is a Higher Order Component (HOC) system that wraps around UI elements and communicates their purpose, appearance, and location to the agent. This "Agentic Web Component" system is paired with a comprehensive global state manager (using Zustand) that synchronizes user and agent interactions in real-time and prevents errors from getting out of control. Agent The agent system is especially interesting. Instead of using a one-size-fits-all approach, I built a multi-model router that classifies user requests into categories: Simple tasks → GPT-4-mini (fast & efficient) Medium → Claude 3.5 Sonnet (more thoughtful) Complex Logic → NVIDIA DeepSeek (specialized reasoning) Conversational → GPT-4 (natural interaction) Malicious messages get flagged and a AI alignment model interferes Each category has its own carefully engineered system prompt, optimizing for both performance and cost. I prompt engineered all the models using Theo, and prompt engineering tool I built over winter break: https://trytheo.dev Cool Features Real-time voice interaction using ElevenLabs for speech synthesis and Open AI whisper for speech regognition Mobile-desktop pairing system for voice input Virtual cursor system with smooth animations Tool calling framework using regex parsing Reading list demo app Oh yeah, and the whole Wii thing. I was just feeling nostalgic. ## Technical Challenges The biggest challenge was maintaining state synchronicity between user and agent interactions. I solved this by creating a central state management system that handles both input streams consistently. Another tricky part was getting the voice interaction just right - dealing with audio streams and buffers, especially managing faulty recording features on mobile phone browsers, was a learning experience for me. ## What I Learned This project pushed my full-stack skills to the limit. I got deep experience with: Building complex AI orchestration systems Swapping between multiple LLM providers and maintaining chat history Real-time audio processing Low level web component code Accessibility-inspired design patterns What's Next I believe this pattern of AI-native web development is the future. In a few years, I expect many websites will use similar patterns to make themselves agent-friendly. WiiWork is just the beginning, a proof of concept showing what's possible when we make websites with Agents in mind from the start. Built solo at TreeHacks 2025 using React, TypeScript, Convex, Zustand, and working about 25/36 possible hours. Check it out at https://WiiWork.dev

## README (from the GitHub repository)

# React + TypeScript + Vite

This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.

Currently, two official plugins are available:

- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react/README.md) uses [Babel](https://babeljs.io/) for Fast Refresh
- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react-swc) uses [SWC](https://swc.rs/) for Fast Refresh

## Expanding the ESLint configuration

If you are developing a production application, we recommend updating the configuration to enable type aware lint rules:

- Configure the top-level `parserOptions` property like this:

```js
export default tseslint.config({
  languageOptions: {
    // other options...
    parserOptions: {
      project: ['./tsconfig.node.json', './tsconfig.app.json'],
      tsconfigRootDir: import.meta.dirname,
    },
  },
})
```

- Replace `tseslint.configs.recommended` to `tseslint.configs.recommendedTypeChecked` or `tseslint.configs.strictTypeChecked`
- Optionally add `...tseslint.configs.stylisticTypeChecked`
- Install [eslint-plugin-react](https://github.com/jsx-eslint/eslint-plugin-react) and update the config:

```js
// eslint.config.js
import react from 'eslint-plugin-react'

export default tseslint.config({
  // Set the react version
  settings: { react: { version: '18.3' } },
  plugins: {
    // Add the react plugin
    react,
  },
  rules: {
    // other rules...
    // Enable its recommended rules
    ...react.configs.recommended.rules,
    ...react.configs['jsx-runtime'].rules,
  },
})
```


## Detected evidence (automated analysis)

Indexed codebase: 80 recognized source files, 172 KB.
- Anthropic (technology) — detected in the code
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- OpenAI (technology) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- Vercel (technology) — claimed on Devpost, not found in the code
- AI coding agent: Cursor — evidence: config files committed to the repository

## Codebase structure (from repository index)

### Files (91 of 91)

```
.cursor/rules/claude-cursor-rules.mdc
.gitignore
components.json
convex/_generated/api.d.ts
convex/_generated/api.js
convex/_generated/dataModel.d.ts
convex/_generated/server.d.ts
convex/_generated/server.js
convex/elevenlabs.ts
convex/llm.ts
convex/mutations.ts
convex/queries.ts
convex/README.md
convex/schema.ts
convex/tsconfig.json
convex/whisper.ts
eslint.config.js
index.html
package.json
postcss.config.js
README.md
src/App.css
src/App.tsx
src/components/agent-ui/AgentBadge.tsx
src/components/agent-ui/AgentBreadcrumb.tsx
src/components/agent-ui/AgentButton.tsx
src/components/agent-ui/AgentCard.tsx
src/components/agent-ui/AgentContext.tsx
src/components/agent-ui/AgentDialog.tsx
src/components/agent-ui/AgentIconButton.tsx
src/components/agent-ui/AgentInput.tsx
src/components/agent-ui/AgentLabel.tsx
src/components/agent-ui/AgentSelect.tsx
src/components/agent-ui/AgentSwitch.tsx
src/components/agent-ui/AgentTextarea.tsx
src/components/agent-ui/index.ts
src/components/agent-ui/withAgentControl.tsx
src/components/agent/AgentBrain.tsx
src/components/agent/AgentChatBox.tsx
src/components/agent/AgentChatCompletion.tsx
src/components/agent/AgentCursor.tsx
src/components/agent/AgentStateDebugPanel.tsx
src/components/agent/AgentUtils.tsx
src/components/agent/AgentVoice.tsx
src/components/agent/MobileInputListener.tsx
src/components/navigation/BreadcrumbNav.tsx
src/components/reading-list/ArticleCard.tsx
src/components/settings/SettingsSidebar.tsx
src/components/ui/badge.tsx
src/components/ui/breadcrumb.tsx
src/components/ui/button.tsx
src/components/ui/card.tsx
src/components/ui/dialog.tsx
src/components/ui/icon-button.tsx
src/components/ui/input.tsx
src/components/ui/label.tsx
src/components/ui/select.tsx
src/components/ui/switch.tsx
src/components/ui/textarea.tsx
src/env.d.ts
src/index.css
src/lib/sounds.ts
src/lib/utils.ts
src/llm/factory.ts
src/llm/providers/anthropic.ts
src/llm/providers/deepseek.ts
src/llm/providers/openai.ts
src/llm/types.ts
src/main.tsx
src/pages/AdminPage.tsx
src/pages/DemoPage.tsx
src/pages/Home.tsx
src/pages/MobileInput.tsx
src/pages/ReadingListPage.tsx
src/stores/agentStore.ts
src/stores/readingListStore.ts
src/stores/settingsStore.ts
src/styles/globals.css
src/types/speech.d.ts
src/vite-env.d.ts
src/whisper/whisperService.ts
tailwind.config.js
tsconfig.app.json
tsconfig.json
tsconfig.node.json
tsconfig.node.tsbuildinfo
tsconfig.tsbuildinfo
vercel.json
vite.config.d.ts
vite.config.js
vite.config.ts
```

### Dependencies

- package.json: @anthropic-ai/sdk@^0.36.3, @eslint/js@^9.19.0, @radix-ui/react-dialog@^1.1.6, @radix-ui/react-label@^2.1.2, @radix-ui/react-select@^2.1.6, @radix-ui/react-slot@^1.1.2, @radix-ui/react-switch@^1.1.3, @types/node@^22.13.4, @types/react@^19.0.8, @types/react-dom@^19.0.3, @vitejs/plugin-react@^4.3.4, autoprefixer@^10.4.20, class-variance-authority@^0.7.1, clsx@^2.1.1, convex@^1.19.2, elevenlabs@^1.51.0, eslint@^9.19.0, eslint-plugin-react-hooks@^5.0.0, eslint-plugin-react-refresh@^0.4.18, globals@^15.14.0, lucide-react@^0.475.0, openai@^4.85.1, postcss@^8.5.2, qrcode.react@^4.2.0, react@^19.0.0, react-dom@^19.0.0, react-router-dom@^7.1.5, tailwind-merge@^3.0.1, tailwindcss@^3.4.17, tailwindcss-animate@^1.0.7, typescript@~5.7.2, typescript-eslint@^8.22.0, vite@^6.1.0, zustand@^5.0.3

### Recent commits (newest first)

- feat: add debug logs to show in FaF demo
- feat: various polishing features for FaF demo day
- smaller audio files to reduce failure percent
- smaller audio files to reduce failure percent
- help im so tired
- feat: try fix audio duplication
- log: add debug logs to find double audio problem
- feat: fix double voices
- feat: added sound design, the hover effect broke but oh well, im tired
- feat: much needed visual upgrade
- feat: use nvidia powered deepseek r-1 for hardest tasks
- feat: make agent into multimodal router + specialists agentic system
- feat: make input text work
- feat: make input text work
- feat: make agent switch work
- feat: make agentic breadcrumb work
- feat: add elevenlabs narration
- feat: add elevenlabs narration
- feat: add breadcrumb navigation
- feat: polishing changes

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "name": "my-app",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "tsc -b && vite build",
    "lint": "eslint .",
    "preview": "vite preview"
  },
  "dependencies": {
    "@anthropic-ai/sdk": "^0.36.3",
    "@radix-ui/react-dialog": "^1.1.6",
    "@radix-ui/react-label": "^2.1.2",
    "@radix-ui/react-select": "^2.1.6",
    "@radix-ui/react-slot": "^1.1.2",
    "@radix-ui/react-switch": "^1.1.3",
    "class-variance-authority": "^0.7.1",
    "clsx": "^2.1.1",
    "convex": "^1.19.2",
    "elevenlabs": "^1.51.0",
    "lucide-react": "^0.475.0",
    "openai": "^4.85.1",
    "qrcode.react": "^4.2.0",
    "react": "^19.0.0",
    "react-dom": "^19.0.0",
    "react-router-dom": "^7.1.5",
    "tailwind-merge": "^3.0.1",
    "tailwindcss-animate": "^1.0.7",
    "zustand": "^5.0.3"
  },
  "devDependencies": {
    "@eslint/js": "^9.19.0",
    "@types/node": "^22.13.4",
    "@types/react": "^19.0.8",
    "@types/react-dom": "^19.0.3",
    "@vitejs/plugin-react": "^4.3.4",
    "autoprefixer": "^10.4.20",
    "eslint": "^9.19.0",
    "eslint-plugin-react-hooks": "^5.0.0",
    "eslint-plugin-react-refresh": "^0.4.18",
    "globals": "^15.14.0",
    "postcss": "^8.5.2",
    "tailwindcss": "^3.4.17",
    "typescript": "~5.7.2",
    "typescript-eslint": "^8.22.0",
    "vite": "^6.1.0"
  }
}

```

### src/main.tsx

```typescript
import { ConvexProvider, ConvexReactClient } from "convex/react";
import { StrictMode } from "react";
import ReactDOM from "react-dom/client";
import App from "./App.tsx";
import "./index.css";
import { AgentBrain } from "./components/agent/AgentBrain.tsx";
import { AgentChatBox } from "./components/agent/AgentChatBox.tsx";
import { AgentCursor } from "./components/agent/AgentCursor.tsx";
import { useAgentStore } from "@/stores/agentStore";
import { MobileInputListener } from "./components/agent/MobileInputListener.tsx";
import "./styles/globals.css";
import { StateDebugPanel } from "./components/agent/AgentStateDebugPanel.tsx";
import { useSettingsStore } from "@/stores/settingsStore";

// Create a Convex client
const convex = new ConvexReactClient(import.meta.env.VITE_CONVEX_URL);

function Root() {
  const cursorVisible = useAgentStore((state) => state.cursor.visible);
  const cursorPosition = useAgentStore((state) => state.cursor.position);
  const showStateDebugPanel = useSettingsStore(
    (state) => state.showStateDebugPanel
  );

  const handleSendMessage = async (message: string) => {
    try {
      await (window as any).__agentBrain.handleUserRequest(message);
    } catch (error) {
      console.error("Error sending message:", error);
    }
  };

  return (
    <StrictMode>
      <ConvexProvider client={convex}>
        {showStateDebugPanel && <StateDebugPanel />}
        <AgentBrain />
        {cursorVisible && <AgentCursor position={cursorPosition} />}
        <AgentChatBox onSendMessage={handleSendMessage} />
        <MobileInputListener onSendMessage={handleSendMessage} />
        <App />
      </ConvexProvider>
    </StrictMode>
  );
}

ReactDOM.createRoot(document.getElementById("root")!).render(<Root />);

```

### src/App.tsx

```typescript
import { BrowserRouter, Routes, Route } from "react-router-dom";
import HomePage from "./pages/Home";
import ReadingListPage from "./pages/ReadingListPage";
import AdminPage from "./pages/AdminPage";
import MobileInput from "./pages/MobileInput";
import DemoPage from "./pages/DemoPage";
import { AgentIconButton } from "@/components/agent-ui";
import { Settings } from "lucide-react";
import { useState } from "react";
import { BreadcrumbNav } from "@/components/navigation/BreadcrumbNav";
import { playButtonSound } from "@/lib/sounds";
import { SettingsSidebar } from "@/components/settings/SettingsSidebar";

export default function App() {
  const [settingsOpen, setSettingsOpen] = useState(false);

  return (
    <div className="relative min-h-screen wii-background">
      {/* Settings Button */}
      <div
        className={`fixed top-4 right-4 z-20 transition-opacity duration-300 ${
          settingsOpen ? "opacity-0 pointer-events-none" : "opacity-100"
        }`}
      >
        <AgentIconButton
          controlId="settings-button"
          onUniversalClick={() => {
            playButtonSound.settingsOpen();
            setSettingsOpen(true);
          }}
          context="This button opens the settings menu"
          className="bg-gray-100 hover:bg-gray-200 text-black border-2 border-blue-200 shadow-sm rounded-xl"
        >
          <Settings className="w-6 h-6" />
        </AgentIconButton>
      </div>

      <SettingsSidebar
        isOpen={settingsOpen}
        onClose={() => setSettingsOpen(false)}
      />

      {/* Main Content */}
      <BrowserRouter>
        <BreadcrumbNav />
        <Routes>
          <Route path="/" element={<HomePage />} />
          <Route path="/reading-list" element={<ReadingListPage />} />
          <Route path="/admin" element={<AdminPage />} />
          <Route path="/mobile-input" element={<MobileInput />} />
          <Route path="/mobile-input/:port" element={<MobileInput />} />
          <Route path="/demo" element={<DemoPage />} />
        </Routes>
      </BrowserRouter>
    </div>
  );
}

```

### convex/_generated/server.js

```javascript
/* eslint-disable */
/**
 * Generated utilities for implementing server-side Convex query and mutation functions.
 *
 * THIS CODE IS AUTOMATICALLY GENERATED.
 *
 * To regenerate, run `npx convex dev`.
 * @module
 */

import {
  actionGeneric,
  httpActionGeneric,
  queryGeneric,
  mutationGeneric,
  internalActionGeneric,
  internalMutationGeneric,
  internalQueryGeneric,
} from "convex/server";

/**
 * Define a query in this Convex app's public API.
 *
 * This function will be allowed to read your Convex database and will be accessible from the client.
 *
 * @param func - The query function. It receives a {@link QueryCtx} as its first argument.
 * @returns The wrapped query. Include this as an `export` to name it and make it accessible.
 */
export const query = queryGeneric;

/**
 * Define a query that is only accessible from other Convex functions (but not from the client).
 *
 * This function will be allowed to read from your Convex database. It will not be accessible from the client.
 *
 * @param func - The query function. It receives a {@link QueryCtx} as its first argument.
 * @returns The wrapped query. Include this as an `export` to name it and make it accessible.
 */
export const internalQuery = internalQueryGeneric;

/**
 * Define a mutation in this Convex app's public API.
 *
 * This function will be allowed to modify your Convex database and will be accessible from the client.
 *
 * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument.
 * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible.
 */
export const mutation = mutationGeneric;

/**
 * Define a mutation that is only accessible from other Convex functions (but not from the client).
 *
 * This function will be allowed to modify your Convex database. It will not be accessible from the client.
 *
 * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument.
 * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible.
 */
export const internalMutation = internalMutationGeneric;

/**
 * Define an action in this Convex app's public API.
 *
 * An action is a function which can execute any JavaScript code, including non-deterministic
 * code and code with side-effects, like calling third-party services.
 * They can be run in Convex's JavaScript environment or in Node.js using the "use node" directive.
 * They can interact with the database indirectly by calling queries and mutations using the {@link ActionCtx}.
 *
 * @param func - The action. It receives an {@link ActionCtx} as its first argument.
 * @returns The wrapped action. Include this as an `export` to name it and make it accessible.
 */
export const action = actionGeneric;

/**
 * Define an action that is only accessible from other Convex functions (but not from the client).
 *
 * @param func - The function. It receives an {@link ActionCtx} as its first argument.
 * @returns The wrapped function. Include this as an `export` to name it and make it accessible.
 */
export const internalAction = internalActionGeneric;

/**
 * Define a Convex HTTP action.
 *
 * @param func - The function. It receives an {@link ActionCtx} as its first argument, and a `Request` object
 * as its second.
 * @returns The wrapped endpoint function. Route a URL path to this function in `convex/http.js`.
 */
export const httpAction = httpActionGeneric;

```

### src/components/agent-ui/index.ts

```typescript
export { AgentButton } from "./AgentButton";
export { AgentIconButton } from "./AgentIconButton";
export { AgentContext } from "./AgentContext";

export { AgentInput } from "./AgentInput";

export {
  AgentSelect,
  AgentSelectContent,
  AgentSelectItem,
  AgentSelectTrigger,
  AgentSelectValue,
} from "./AgentSelect";

export {
  AgentDialog,
  AgentDialogContent,
  AgentDialogHeader,
  AgentDialogTitle,
  AgentDialogTrigger,
} from "./AgentDialog";

export { AgentTextarea } from "./AgentTextarea";
export { AgentBadge } from "./AgentBadge";

export {
  AgentCard,
  AgentCardHeader,
  AgentCardTitle,
  AgentCardDescription,
  AgentCardContent,
  AgentCardFooter,
} from "./AgentCard";

export { AgentLabel } from "./AgentLabel";
export { AgentSwitch } from "./AgentSwitch";

```

### vite.config.d.ts

```typescript
declare const _default: import("vite").UserConfig;
export default _default;

```

### postcss.config.js

```javascript
export default {
  plugins: {
    tailwindcss: {},
    autoprefixer: {},
  },
}

```

### vite.config.ts

```typescript
import path from "path";
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [react()],
  resolve: {
    alias: {
      "@": path.resolve(__dirname, "./src"),
    },
  },
});

```

### vite.config.js

```javascript
import path from "path";
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";
// https://vitejs.dev/config/
export default defineConfig({
    plugins: [react()],
    resolve: {
        alias: {
            "@": path.resolve(__dirname, "./src"),
        },
    },
});

```

### index.html

```html
<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" type="image/png" href="/favicon.png" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>WiiWork</title>
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="/src/main.tsx"></script>
  </body>
</html>

```

[69 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]