# Project export: Kateto
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: OpenAI Build Week
- Tagline: Too busy to organize your projects? Too lazy to handle documentation? Kateto handles all of that for you, evolving at every step, adapting to your way of working.
- Devpost: https://devpost.com/software/kateto
- GitHub: https://github.com/Gonanf/kateto
- Demo: https://huggingface.co/spaces/Gonanf/KatetoSpace
- Video: https://www.youtube.com/embed/IhkamcAkuzk?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 2 GitHub contributor(s) — Chaos (121 commits), Sisyphus (23 commits)
## Devpost submission (written by the team)
### Inspiration
For a while I became obsessed with automating my life, Kateto is an attempt at this. For now, Kateto only automates the project management process, which is something I struggle to do because I usually work with many projects at the same time as a lead developer.
### What it does
It is a system of plugins and events aimed at handling AI models, represented as Voices. Voices are agents with their own personalities, workflows (Which enforces a step by step pipeline with deliverables in a deterministic way) and skills, these agents evolve with the data they obtain and improve their usefulness for each person in particular. The voices that are currently in the system are: Jane: Orchestrator, handles the other voices and is the entry point of interaction with the user. Doktor: Project Manager, Manage risks, backlog, events and the work team. Conquest: SCRUM master, makes sure everyone follows the SCRUM framework.
### How we built it
I used codex to create and test all the initial plugins and system, as well as expand the voice personalities, create the initial workflows and workflow system, the online MVP, fixes and the visual components of my own presentation video.
### Challenges we ran into
A lot of mistakes, regressions, last minute changes, both in problem solving and ideas that were going through my head.
### Accomplishments we're proud of
It is a very extensible system, i wanted to bring more value to this project, and i probably will continue iterating this project.
### What we learned
NEVER LEAVE CODEX ALONE (And also verify you'r ideas), even less with the framework that i used, it was spawning a million agents.
### What's next
Kateto's system actually has +10 voices planned, with defined roles, such as Xavier who is in charge of everything artistic, Germ who is in charge of everything marketing (with Xavier), Greedy Grinner who is in charge of budgets... and more.
## README (from the GitHub repository)
# Kateto — Event-Driven Voice Team for Project Work
  
Kateto is an autonomous, event-driven voice team designed for real-time project management and technical collaboration. Instead of forcing conversations through a brittle, linear request-response chain, Kateto runs an event-driven architecture where specialized voice agents, audio hardware, intent classifiers, and autonomous workflows interact concurrently through a centralized `PluginManager` event bus.
---
## 1. Architectural Overview
```
+----------------------------------------+
| PluginManager (Event Bus) |
+---+--------+--------+--------+-----+---+
| | | | |
+---------------------+ | | | +---------------------+
| | | | |
+-----v-------+ +------v---+ +-v------+ v-------------+ +-------v--------+
| Audio Input | | Whisper | | Intent | | Voice Agent | | Audio Output |
| (Mic + VAD) | | Processor| | Router | | (LLM + MCP) | | (TTS + Player) |
+-------------+ +----------+ +--------+ +-------------+ +----------------+
sounddevice whisper.cpp mmBERT Jane, Doktor, Camb AI / Zonos
Silero VAD Vulkan / CUDA llama.cpp Conquest ALSA PCM Mixer
| | | | |
v v v v v
[audio_chunk] [transcription] [generate] [text_chunk] [audio_output]
|
v
+----------------+
| Visual Overlay |
| (VTuber Avatar |
| + Subtitles) |
+----------------+
```
### The Department & Voice Model
Kateto models organizations as **Departments** populated by domain-specialized **Voices**:
- **Jane (`voice.jane`)** — *Project Coordinator & Orchestrator*. Manages high-level communication, coordinates inter-voice delegation, and interfaces with external stakeholders.
- **Doktor (`voice.doktor`)** — *Technical Planner & Delivery Advisor*. Translates fuzzy requirements into Work Breakdown Structures (WBS), risk matrices, and prioritized backlogs.
- **Conquest (`voice.conquest`)** — *Agile Lead & Scrum Facilitator*. Facilitates standups, sprint retrospectives, process enforcement, and milestone tracking.
---
## 2. Technical Capabilities & Highlights
### ⚡ Hardware-Accelerated Speech-to-Text (Whisper Vulkan / CUDA)
- **Sub-Second Latency**: Transcribes conversational utterances in **~1.5–2.2s** using discrete GPUs (e.g. AMD Radeon RX 6500 XT via Vulkan device 1) instead of 22+ seconds on CPU.
- **Persistent Server Architecture**: Leverages `whisper-server` keeping the quantized model (`ggml-large-v3-turbo-q5_0.bin`) resident in VRAM.
- **Configurable Language**: Supports explicit multilingual target configurations (`audio_processor_whisper.language = "es"`) across HTTP, CLI, and embedded backends.
- **Fault-Tolerant Parsing**: Handles silent VAD frames and background room noise cleanly without throwing validation exceptions.
### 🎙️ Real-Time Token-to-Speech Streaming (Camb AI)
- **Low-Latency Synthesis**: As the LLM generates tokens, `PhraseSegmenter` identifies natural sentence and clause boundaries (configured with punctuation boundaries and token thresholds) and streams them to the TTS engine without waiting for the full response.
- **Streaming Downsampling**: Parses incoming RIFF WAV chunk headers on-the-fly and downsamples 48kHz audio to 24kHz PCM frames directly in memory, yielding audio frames immediately to the output mixer.
- **Glitch-Free ALSA Ring-Buffers**: Eliminates PortAudio ALSA xrun underflows and device contention crashes (`Assertion self->neverDropInput failed`) by maintaining a persistent open output stream with generous buffer sizing (`blocksize=2048`, `latency="high"`).
### 🎭 Visual Overlay & VTuber Avatar Kinematics
- **Audio-Synchronized Subtitles**: Subtitles in the browser overlay are synchronized with **actual audio playback** rather than premature LLM token generation. Subtitles stay on screen for the duration of the spoken sentence plus a 3.5s reading grace period.
- **Reactive Jaw Physics**: Dynamic real-time visemes driven by root-mean-square (RMS) amplitude:
- Vertical displacement up to **-52px**
- Random lateral excursions up to **±28px**
- Rotational tilt up to **±34°** coupled directly to lateral movement direction for lifelike cartoon physics.
### 🧠 LLM KV-Cache Prefilling
- **Zero First-Turn Ingestion Latency**: On runtime startup, Kateto automatically builds each active voice's stable system prompt (personality, tool definitions, durable memories) and dispatches a lightweight 1-token prefill ping with session affinity headers (`x-session-affinity`).
- Inference engines with prefix-caching (such as `llama-server --cache-reuse`, vLLM, or Ollama) ingest and retain the entire prompt prefix in KV memory before the user speaks their first word.
---
## 3. Installation & Dependencies
### Prerequisites
- **Linux** (x86_64) or macOS
- **Python 3.12+**
- **[uv](https://docs.astral.sh/uv/)** package manager
- System libraries for audio: `libportaudio2`, `libasound2`
- For native GPU acceleration: Vulkan drivers (`vulkan-tools`, `mesa-vulkan-drivers`) or NVIDIA CUDA Toolkit
### Setup
```bash
# Clone the repository
git clone https://github.com/Gonanf/Kateto.git
cd Kateto
# Synchronize environment with uv
uv sync
# Optional: Install with in-process backends
uv sync --extra classifier --extra whispercpp
# Or install globally as a standalone CLI tool
uv tool install "kateto[classifier,llamacpp,whispercpp]"
```
---
## 4. Configuration
Bootstrap default configuration files into `~/.config/kateto/`:
```bash
uv run kateto config check
```
Edit `~/.config/kateto/config.toml`:
```toml
[kateto]
name = "Kateto"
language = "es_ar"
log_level = "INFO"
[plugin]
# Microphone Audio Input with Silero VAD
audio_input_mic.enabled = true
audio_input_mic.sample_rate = 16000
audio_input_mic.silence_timeout = 1.0
audio_input_mic.vad_model = "silero"
# Whisper Speech-to-Text Processor
audio_processor_whisper.enabled = true
audio_processor_whisper.backend = "server"
audio_processor_whisper.language = "es"
audio_processor_whisper.model = "/path/to/ggml-large-v3-turbo-q5_0.bin"
# Intent Classification
executor_classifier.enabled = true
executor_classifier.backend = "server"
# Voice LLM Backend (OpenAI-compatible / llama-server / Ollama)
voice_llm.enabled = true
voice_llm.endpoint = "http://127.0.0.1:11434/v1"
voice_llm.model = "qwen2.5-coder:7b"
# Camb AI Streaming TTS
audio_output_camb.enabled = true
audio_output_camb.api_key = "YOUR_CAMB_API_KEY"
# Visual Overlay & HTTP Server
system_http_server.enabled = true
system_http_server.port = 8087
visual_overlay.enabled = true
```
---
## 5. Running the Application
### Start the Runtime
```bash
uv run kateto run
```
This starts:
1. The **Event Bus** and lifecycle manager.
2. Background **LLM system prompt prefill** routines.
3. Microphone listener with **Silero VAD**.
4. The **Vulkan-accelerated Whisper se
[README truncated for size]
## Detected evidence (automated analysis)
Indexed codebase: 234 recognized source files, 1193 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- PyTorch (technology) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- AI coding agent: Codex — evidence: config files committed to the repository
## Codebase structure (from repository index)
### Files (120 of 259)
```
.codex
.dockerignore
.gitignore
.omo/boulder.json
.omo/drafts/kateto-mvp.md
.omo/plans/kateto-mvp.md
.omo/plans/kateto-zerogpu-publishing.md
.omo/start-work/ledger.jsonl
.omo/ulw-loop/019f678c-8a86-7162-9d5b-02ccc74b4949/brief.md
.omo/ulw-loop/019f678c-8a86-7162-9d5b-02ccc74b4949/goals.json
.omo/ulw-loop/019f678c-8a86-7162-9d5b-02ccc74b4949/ledger.jsonl
.omo/ulw-loop/019f6809-7d84-7c63-b441-f90678641f55/brief.md
.omo/ulw-loop/019f6809-7d84-7c63-b441-f90678641f55/goals.json
.omo/ulw-loop/019f6809-7d84-7c63-b441-f90678641f55/ledger.jsonl
.omo/ulw-loop/event-discovery/brief.md
.omo/ulw-loop/event-discovery/goals.json
.omo/ulw-loop/event-discovery/ledger.jsonl
.python-version
AGENTS.md
config/defaults/config.toml
config/defaults/skills/backlog/SKILL.md
config/defaults/skills/orchestrator/SKILL.md
config/defaults/skills/planning-poker/SKILL.md
config/defaults/skills/risk-analysis/SKILL.md
config/defaults/voices/conquest/SOUL.md
config/defaults/voices/conquest/workflows/continuous-improvement/workflow.py
config/defaults/voices/conquest/workflows/daily-standup/workflow.py
config/defaults/voices/conquest/workflows/sprint-execution/workflow.py
config/defaults/voices/conquest/workflows/sprint-retrospective/workflow.py
config/defaults/voices/doktor/SOUL.md
config/defaults/voices/doktor/workflows/backlog-grooming/workflow.py
config/defaults/voices/doktor/workflows/resource-planning/workflow.py
config/defaults/voices/doktor/workflows/risk-management/workflow.py
config/defaults/voices/doktor/workflows/sprint-planning/workflow.py
config/defaults/voices/doktor/workflows/sprint-review/workflow.py
config/defaults/voices/jane/SOUL.md
config/defaults/voices/jane/workflows/project-closure/workflow.py
config/defaults/voices/jane/workflows/project-initiation/workflow.py
config/defaults/voices/jane/workflows/stakeholder-communication/workflow.py
demo-video/.gitignore
demo-video/.prettierrc
demo-video/eslint.config.mjs
demo-video/package.json
demo-video/README.md
demo-video/remotion.config.ts
demo-video/src/Composition.tsx
demo-video/src/index.css
demo-video/src/index.ts
demo-video/src/Root.tsx
demo-video/tsconfig.json
docker/config.toml
docker/entrypoint.sh
Dockerfile
docs/AGENT.md
docs/architecture/config.md
docs/architecture/design-decisions.md
docs/architecture/event-system.md
docs/architecture/hot-reload.md
docs/architecture/overview.md
docs/architecture/plugin-manager.md
docs/architecture/ponytail-review.md
docs/bugs/01-whisper-gpu.md
docs/bugs/02-no-e2e-tests.md
docs/bugs/03-callbackqueue-capacity.md
docs/bugs/04-plugin-error-isolation.md
docs/bugs/05-no-structured-logging.md
docs/bugs/06-hot-reload-test-coverage.md
docs/bugs/07-not-runnable.md
docs/bugs/08-hot-reload-cancels-workers.md
docs/bugs/09-text-chunk-capture.md
docs/bugs/10-runtime-voice-enable.md
docs/bugs/11-backlog-priority-filter.md
docs/bugs/12-todo-md-location.md
docs/bugs/13-duplicated-files.md
docs/bugs/14-no-runtime-tools.md
docs/bugs/15-hot-reload-unnecessary-replacement.md
docs/bugs/16-tui-plugin-toggle.md
docs/bugs/17-tui-plugin-switch-voice-visibility.md
docs/bugs/18-tui-cwd-config-priority.md
docs/bugs/19-agent-loop-no-stream.md
docs/bugs/20-tui-autocomplete-multiline-json.md
docs/bugs/21-tui-conversation-single-bubble.md
docs/bugs/22-default-voices-no-soul.md
docs/bugs/23-plugin-soul-journal-manager.md
docs/bugs/24-tui-voice-name-conflict.md
docs/bugs/25-web-sandbox-presentation.md
docs/bugs/26-smoke-cli-deleted-qa-path.md
docs/bugs/27-tui-freeze-tts-streaming.md
docs/bugs/28-workflow-discovery-and-tui-event-filter.md
docs/bugs/29-workflow-no-owner-generation.md
docs/bugs/30-new-project-workflow-selection.md
docs/bugs/31-active-workflow-switch-and-interrupt.md
docs/bugs/32-workflow-does-not-advance.md
docs/bugs/33-disabled-called-voice.md
docs/bugs/34-agent-silent-after-tool-call.md
docs/bugs/35-dynamic-voice-stays-disabled.md
docs/bugs/36-fixture-tui-no-runtime.md
docs/bugs/37-fixture-voices-identical.md
docs/concepts/avatars.md
docs/concepts/workspaces.md
docs/development/build-week.md
docs/development/final-assessment.md
docs/development/free-publishing-plan.md
docs/development/masks-prompts.md
docs/development/space-zero-gpu.md
docs/development/tdd.md
docs/development/tooling.md
docs/features/post-ponytail-plan.md
docs/known-issues.md
docs/plugins/audio-input.md
docs/plugins/audio-output.md
docs/plugins/audio-processor.md
docs/plugins/connectors.md
docs/plugins/executors.md
docs/plugins/system.md
docs/system-mcp-architecture.md
docs/test-evidence-2026-07-18.md
docs/test-evidence-comprehensive-2026-07-18.md
docs/test-read-tool-2026-07-18.md
docs/voices/backlog.md
[139 more files omitted for size]
```
### Dependencies
- demo-video/package.json: @remotion/cli@4.0.496, @remotion/eslint-config-flat@4.0.496, @remotion/tailwind-v4@4.0.496, @types/react@19.2.7, @types/web@0.0.166, eslint@9.19.0, prettier@3.8.1, react@19.2.3, react-dom@19.2.3, remotion@4.0.496, tailwindcss@4.0.0, typescript@5.9.3
- pyproject.toml: anyio@>=4.0.0, edge-tts@>=7.0.0, google-auth-oauthlib@==1.2.2, gradio@>=6.15.2,<7.0.0, gradio@==6.15.2, httpx@==0.28.1, mcp@==1.28.1, numpy@>=2.5.1, openai@==2.45.0, pydantic@==2.13.4, python-dotenv@==1.2.2, silero-vad@<6.2, sounddevice@==0.5.5, spaces@>=0.39.0, spaces@>=0.39.0, textual@==8.2.8, torch@>=2.13.0, watchdog@==6.0.0
- requirements.txt: anyio@==4.12.0, gradio@==6.15.2, httpx@==0.28.1, pydantic@==2.13.4, python-dotenv@==1.2.2, spaces@>=0.39.0
### Recent commits (newest first)
- Revert "fix: serialize fixture voice responses"
- Revert "fix: ignore non-owner fixture workflow requests"
- fix: ignore non-owner fixture workflow requests
- fix: serialize fixture voice responses
- fix: render dynamic TUI text safely
- fix: escape markup in TUI conversation
- fix: updated readme
- fix: advance fixture workflows to completion
- fix: make fixture TUI demonstrate voice team
- fix: updated readme
- docs: rewrite project README
- fix: expose dynamically enabled voices
- fix: deliver agent responses after tool calls
- fix: enable voices requested by workflows
- fix: expose workflow completion contract
- fix: preserve active workflow context
- Revise README with updated project information
- docs: record new project workflow routing fix
- fix: prioritize initiation for new projects
- fix: generate workflow owner phase prompts
## Key source files (fetched from GitHub, selected and truncated for size)
### AGENTS.md
```markdown
# Kateto — AGENTS.md
## What It Is
Kateto is an event-driven voice team for project work. Jane orchestrates, Doktor handles planning/backlog, Conquest facilitates agile ceremonies. All communication runs through a PluginManager that is simultaneously the event bus and plugin lifecycle manager — no separate mediator.
**Stack:** Python 3.12+ async/await · uv · pytest-asyncio · Pydantic · Textual 8 · MCP 1.28
---
## Entrypoints & Commands
```
uv run kateto config check # validate TOML, bootstrap defaults
uv run kateto run # event runtime (no TUI)
uv run kateto tui # text UI with event stream
uv run kateto tui --fixture # TUI with deterministic fixtures
uv run kateto smoke --fixture # full bounded smoke test
```
--fixture mode is the no-network fallback — supplies transcription, classification, streaming, and PCM via mocks. External servers (whisper.cpp, llama.cpp, Zonos) only needed for live runs.
## Architecture
### PluginManager (Event Bus + Lifecycle)
```python
# Singleton. Scans `on_*` methods for subscriber registration.
# Plugin emits → Manager dispatches to matching subscribers concurrently.
```
**Plugin types:**
- `audio_input/` — mic capture (sounddevice), silence detection with Silero VAD
- `audio_processor/` — Whisper transcription
- `audio_output/` — Zonos TTS, PCM playback
- `connector/` — CLI allowlist, Google Calendar/Meet
- `executor/` — Classifier (intent), Interrupt, TODO List, Backlog
- `system/` — TUI (Textual), internal MCP server
- `work/` — (future)
Each plugin gets its own `asyncio.Queue`. Streaming plugins process events one-by-one as they arrive. Batch plugins (voice agents) accumulate and process on `generate` trigger.
### Event System
- Registration: auto-scanned from `on_*` methods on enabled plugins
- Contracts: every event has a Pydantic model (frozen, strict — `EventModel` in `core/event.py`)
- Envelope: `EventEnvelope(name, data, source, timestamp, target, capabilities, only_once)`
- Dispatch filters: Broadcast · Target · Capabilities (AND) · Only Once
- Self-delivery OFF by default (prevents loops)
Errors are caught per-plugin and emitted as error events. The bus stays up.
### Voices (3 built-in: Jane, Doktor, Conquest)
Defined in `kateto/voices/factory.py` with `VoiceProfile` (voice_id, display_name, role, system_prompt). Config keys in `config.toml` are lowercase (`voice.jane`, `voice.doktor`, `voice.conquest`). Directory names in `config/defaults/voices/` match lowercase.
Voice lifecycle:
1. `create_voice()` reads profile, creates `VoiceAgent` with `OpenAICompatibleProvider`
2. `VoiceAgent` sets up `VoiceMemory` (per-voice file store in `~/.config/kateto/voices/{name}/`)
3. Optional: agent provider + `VoiceToolExecutor` for tool-enabled generation
### Config Bootstrap
```
config/defaults/config.toml + config/defaults/voices/{name}/workflows/
→ _copy_missing_defaults() → user config dir
```
Bootstrap copies
[truncated — 8478 more characters]
```
### docs/test-read-tool-2026-07-18.md
```markdown
# Read Tool Test Evidence — 2026-07-18
> Can an agent access config.toml via the read_file tool?
## Test 1: Direct read_file via VoiceToolExecutor
**Input:** `executor.execute("read_file", {"path": "config.toml"})`
**Result:** PASS
**Output:**
```
[kateto]
name = "Kateto"
language = "es"
debug = true
hot_reload = true
[plugin.audio_input_mic]
enabled = true
device = "pulse"
...
```
**Evidence:** read_file returned 1102 chars from `~/.config/kateto/config.toml` (the user config, since that's the working directory).
## Test 2: Absolute path security check
**Input:** `executor.execute("read_file", {"path": "/home/chaos/proyectos/OpenaiBuildWeek/Kateto/config.toml"})`
**Result:** PASS (security works)
**Output:**
```json
{"error": "path escapes working directory"}
```
**Evidence:** Absolute paths are correctly rejected. The tool only allows relative paths within the working directory (`~/.config/kateto/`).
## Test 3: Jane reads config.toml via generate
**Input:** Generate event asking Jane to use read_file on config.toml
**Result:** FAIL — Timeout 60s
**Root cause:** Audio capture plugin blocks the event loop (bug #8). The LLM call never completes.
**Note:** Tests 1 and 2 prove the tool works correctly. The failure is in the LLM integration layer, not the tool itself.
## Conclusion
- **read_file tool works** — can access files relative to working directory
- **Security works** — absolute paths and path traversal are blocked
- **LLM integration broken** — audio capture blocks event loop, preventing Jane from using tools via generate
```
### requirements.txt
```
# Hugging Face Gradio Space runtime dependencies.
# The application imports the repository in-place; no editable install is required.
anyio==4.12.0
gradio==6.15.2
httpx==0.28.1
pydantic==2.13.4
python-dotenv==1.2.2
spaces>=0.39.0
```
### pyproject.toml
```
[project]
name = "kateto"
version = "0.1.0"
description = "Kateto MVP"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
"anyio>=4.0.0",
"edge-tts>=7.0.0",
"gradio>=6.15.2,<7.0.0",
"google-auth-oauthlib==1.2.2",
"httpx==0.28.1",
"mcp==1.28.1",
"numpy>=2.5.1",
"openai==2.45.0",
"pydantic==2.13.4",
"python-dotenv==1.2.2",
"silero-vad<6.2",
"spaces>=0.39.0",
"sounddevice==0.5.5",
"textual==8.2.8",
"torch>=2.13.0",
"watchdog==6.0.0",
]
[project.scripts]
kateto = "kateto.__main__:main"
[dependency-groups]
dev = [
"pytest==9.1.1",
"pytest-asyncio==1.4.0",
]
[project.optional-dependencies]
space = [
"gradio==6.15.2",
"spaces>=0.39.0",
]
[tool.pytest.ini_options]
testpaths = ["kateto/tests"]
[build-system]
requires = ["uv_build>=0.11.28,<0.12.0"]
build-backend = "uv_build"
[tool.uv.build-backend]
module-root = ""
```
### Dockerfile
```
# syntax=docker/dockerfile:1.7
#
# Build from Kateto with the sibling classifier as a named context:
# docker build --build-context classifier=../classifiers/mmbert -t kateto-local .
ARG LLAMA_REF=846e991ec3c7ccec49112ff2c5b00b710e5f551d
ARG WHISPER_REF=080bbbe85230f624f0b52127f1ae1218247989f9
FROM python:3.12-slim-bookworm AS native-build
ARG LLAMA_REF
ARG WHISPER_REF
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update \
&& apt-get install --no-install-recommends -y \
build-essential \
ca-certificates \
cmake \
git \
libopenblas-dev \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /src
RUN git clone --depth 1 https://github.com/ggml-org/llama.cpp.git llama.cpp \
&& cd llama.cpp \
&& git fetch --depth 1 origin "${LLAMA_REF}" \
&& git checkout "${LLAMA_REF}" \
&& cmake -S . -B build \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_NATIVE=OFF \
-DGGML_OPENMP=ON \
-DLLAMA_BUILD_SERVER=ON \
-DLLAMA_BUILD_TESTS=OFF \
-DLLAMA_BUILD_EXAMPLES=OFF \
&& cmake --build build --config Release --target llama-server -j"$(nproc)"
RUN git clone --depth 1 https://github.com/ggerganov/whisper.cpp.git whisper.cpp \
&& cd whisper.cpp \
&& git fetch --depth 1 origin "${WHISPER_REF}" \
&& git checkout "${WHISPER_REF}" \
&& cmake -S . -B build \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_NATIVE=OFF \
-DGGML_OPENMP=ON \
-DWHISPER_BUILD_SERVER=ON \
-DWHISPER_BUILD_TESTS=OFF \
-DWHISPER_BUILD_EXAMPLES=ON \
&& cmake --build build --config Release --target whisper-server -j"$(nproc)"
FROM python:3.12-slim-bookworm AS runtime
ARG WHISPER_MODEL=ggml-large-v3-turbo.bin
ARG LLAMA_MODEL_REPO=Qwen/Qwen2.5-1.5B-Instruct-GGUF
ARG LLAMA_MODEL_FILE=qwen2.5-1.5b-instruct-q4_k_m.gguf
ENV DEBIAN_FRONTEND=noninteractive \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
HF_HOME=/opt/models/huggingface \
HF_HUB_DISABLE_TELEMETRY=1 \
XDG_CONFIG_HOME=/opt/kateto-config \
WHISPER_MODEL_FILE=${WHISPER_MODEL} \
LLAMA_MODEL_REPO=${LLAMA_MODEL_REPO} \
LLAMA_MODEL_FILE=${LLAMA_MODEL_FILE}
RUN apt-get update \
&& apt-get install --no-install-recommends -y \
ffmpeg \
libgomp1 \
libopenblas0 \
libportaudio2 \
&& rm -rf /var/lib/apt/lists/*
COPY --from=native-build /src/llama.cpp/build/bin/llama-server /usr/local/bin/llama-server
COPY --from=native-build /src/whisper.cpp/build/bin/whisper-server /usr/local/bin/whisper-server
WORKDIR /opt/kateto
COPY pyproject.toml uv.lock README.md ./
COPY kateto ./kateto
COPY config ./config
COPY docs ./docs
COPY public ./public
# BuildKit named contexts avoid vendoring the sibling classifier into Kateto.
COPY --from=classifier server.py pyproject.toml uv.lock /opt/classifier/
RUN python -m pip install --no-cache-dir uv \
&& uv sync --locked --no-dev \
&& uv pip install --system /opt/classifier \
&& python -m pip install --no-cache-dir huggingface_hub
COPY docker/config.toml /opt/kateto-config/kateto/config.toml
COPY docker/entrypoint.sh /usr/local/bin/kateto-entrypoint
RUN chmod 0755 /usr/local/bin/kateto-entrypoint \
&& mkdir -p /opt/kateto-config/kateto/voices \
&& cp -a config/defaults/voices/. /opt/kateto-config/kateto/voices/
RUN python - <<'PY'
import os
from huggingface_hub import hf_hub_download
hf_hub_download(
repo_id="ggerganov/whisper.cpp",
filename=os.environ["WHISPER_MODEL_FILE"],
local_dir="/opt/models/whisper",
)
hf_hub_download(
repo_id="Qdrant/all-MiniLM-L6-v2-onnx",
filename="model.onnx",
local_dir="/opt/models/classifier",
)
hf_hub_download(
repo_id="Qdrant/all-MiniLM-L6-v2-onnx",
filename="tokenizer.json",
)
hf_hub_download(
repo_id=os.environ["LLAMA_MODEL_REPO"],
filename=os.environ["LLAMA_MODEL_FILE"],
local_dir="/opt/models/llama",
)
PY
EXPOSE 8090 8091 8092
HEALTHCHECK --interval=30s --timeout=10s --start-period=90s --retries=5 \
CMD python -c "import urllib.request; [urllib.request.urlopen(url, timeout=5) for url in ('http://127.0.0.1:8090/', 'http://127.0.0.1:8091/health', 'http://127.0.0.1:8092/health')]"
ENTRYPOINT ["/usr/local/bin/kateto-entrypoint"]
```
### demo-video/package.json
```
{
"name": "demo-video",
"version": "1.0.0",
"description": "My Remotion video",
"repository": {},
"license": "UNLICENSED",
"private": true,
"dependencies": {
"@remotion/cli": "4.0.496",
"react": "19.2.3",
"react-dom": "19.2.3",
"remotion": "4.0.496",
"@remotion/tailwind-v4": "4.0.496",
"tailwindcss": "4.0.0"
},
"devDependencies": {
"@remotion/eslint-config-flat": "4.0.496",
"@types/react": "19.2.7",
"@types/web": "0.0.166",
"eslint": "9.19.0",
"prettier": "3.8.1",
"typescript": "5.9.3"
},
"scripts": {
"dev": "remotion studio",
"build": "remotion bundle",
"upgrade": "remotion upgrade",
"lint": "eslint src && tsc"
},
"sideEffects": [
"*.css"
]
}
```
### web/app.js
```javascript
/**
* Kateto Web Sandbox — Dashboard UI
*
* Vanilla JS (ponytail: no framework needed for an info dashboard).
* Connects to a Kateto instance via WebSocket when available;
* falls back to mock data for standalone presentation.
*/
/* ------------------------------------------------------------------ */
/* Data */
/* ------------------------------------------------------------------ */
const CAPABILITIES = [
{
icon: '🎙️',
title: 'Voice Interaction',
desc: 'Speak to Jane, Doktor, or Conquest. Transcription, classification, generation, and TTS loop in real time.',
},
{
icon: '🧩',
title: 'Plugin Architecture',
desc: 'Event-driven plugin system. Audio I/O, executors, connectors, and system plugins communicate via a shared bus.',
},
{
icon: '📋',
title: 'Backlog Management',
desc: 'Create, prioritize, and track work items with Must/Should/Could/Won\'t. Filter by status or priority.',
},
{
icon: '🔄',
title: 'Workflow Engine',
desc: 'Declarative multi-phase workflows with checkpoints. Perfect for sprint planning, standups, and retrospectives.',
},
{
icon: '🔧',
title: 'MCP Integration',
desc: 'Model Context Protocol servers for tools, memory, and external service integration. Extend via config.toml.',
},
{
icon: '⚡',
title: 'Hot Reload',
desc: 'Watchdog-based hot reload of plugins, voices, and config. Change code without restarting the runtime.',
},
];
const VOICES = [
{
name: 'Jane',
role: 'Orchestrator',
status: 'idle',
color: 'var(--accent)',
desc: 'Coordinates the team, delegates tasks, manages conversation flow.',
},
{
name: 'Doktor',
role: 'Analyst & Backlog',
status: 'idle',
color: 'var(--purple)',
desc: 'Planning, risk analysis, backlog grooming, and structured thinking.',
},
{
name: 'Conquest',
role: 'Facilitator',
status: 'idle',
color: 'var(--orange)',
desc: 'Agile ceremonies, sprint execution, standups, and retrospectives.',
},
];
const MOCK_EVENTS = [
{ tag: 'TRANSCRIPTION', text: '"plan tomorrow standup"', highlight: true },
{ tag: 'CLASSIFICATION', text: 'category=EXECUTE confidence=0.94', highlight: false },
{ tag: 'GENERATE', text: 'Conquest · planning standup agenda', highlight: true },
{ tag: 'STREAM_RESPONSE', text: 'Starting standup session for tomorrow…', highlight: false },
{ tag: 'AUDIO_CHUNK', text: 'voice=conquest sequence=3 final=false', highlight: false },
];
/* ------------------------------------------------------------------ */
/* State */
/* ------------------------------------------------------------------ */
const state = {
ws: null,
connected: false,
sandboxReady: false,
};
/* ------------------------------------------------------------------ */
/* DOM refs */
/* ------------------------------------------------------------------ */
const $ = (sel) => document.querySelector(sel);
const $$ = (sel) => document.querySelectorAll(sel);
/* ------------------------------------------------------------------ */
/* Render */
/* ------------------------------------------------------------------ */
function renderCapabilities() {
const grid = document.getElementById('capabilitiesGrid');
grid.innerHTML = CAPABILITIES.map(
(c) => `
${c.icon}
${c.title}
${c.desc}
`
).join('');
}
function renderVoices() {
const grid = document.getElementById('voicesGrid');
grid.innerHTML = VOICES.map(
(v) => `
${v.name}
${v.role}
${v.desc}
`
).join('');
}
function updateSystemStatus(online) {
const dot = document.getElementById('systemDot');
const label = document.getElementById('systemStatus');
dot.className = `status-dot ${online ? 'online' : 'offline'}`;
label.textContent = online ? 'Connected' : 'Disconnected';
}
function updateVoicesCount(active) {
document.getElementById('voicesOnline').textContent = active;
}
function updateSandboxButton(ready) {
const btn = document.getElementById('btnSandbox');
btn.disabled = !ready;
btn.textContent = ready ? '▶ Launch Sandbox' : '⏳ Connecting…';
state.sandboxReady = ready;
}
function updateWSIndicator(mode) {
const el = document.getElementById('wsIndicator');
el.textContent = mode === 'ws'
? '🔗 WebSocket connected'
: '⚡ Standalone mode';
}
function addEventEntry(tag, text, highlight) {
const stream = document.getElementById('eventStream');
const placeholder = stream.querySelector('.placeholder');
if (placeholder) placeholder.remove();
const entry = document.createElement('div');
entry.className = 'event-entry';
entry.innerHTML = `
${tag}
${escapeHtml(text)}
`;
stream.appendChild(entry);
stream.scrollTop = stream.scrollHeight;
// keep last 50 entries
while (stream.children.length > 50) stream.removeChild(stream.firstChild);
}
function escapeHtml(str) {
const d = document.createElement('div');
d.textContent = str;
return d.innerHTML;
}
function loadMockEvents() {
MOCK_EVENTS.forEach((e, i) => {
setTimeout(() => addEventEntry(e.tag, e.text, e.highlight), i * 600);
});
}
/* ------------------------------------------------------------------ */
/* WebSocket */
/* ---------------------------------
[truncated — 2813 more characters]
```
### space/app.py
```python
from collections.abc import Callable
import os
import gradio as gr
from pydantic import JsonValue
from space.contracts import MAX_BYOK_KEY_LENGTH, ProviderChoiceError, ProviderSelection
from space.providers import SpaceProviderError
from space.runtime import JsonRecord, SpaceRuntimeSession, create_runtime_session
def select_provider(raw_provider: str | None, raw_key: str) -> ProviderSelection:
normalized_provider = (raw_provider or "").strip().casefold()
match normalized_provider:
case "byok":
if not raw_key or not raw_key.strip():
raise ProviderChoiceError("A BYOK key is required.")
if raw_key != raw_key.strip():
raise ProviderChoiceError(
"The BYOK key cannot start or end with whitespace."
)
if len(raw_key) > MAX_BYOK_KEY_LENGTH:
raise ProviderChoiceError("The BYOK key exceeds the maximum length.")
return ProviderSelection(provider="byok", session_key=raw_key)
case "bonsai":
return ProviderSelection(provider="bonsai", session_key=None)
case _:
raise ProviderChoiceError("Choose exactly BYOK or Bonsai.")
def _status(selection: ProviderSelection) -> str:
provider_label = "BYOK" if selection.provider == "byok" else "Bonsai"
mode = os.getenv("KATETO_SPACE_MODE", "fixture")
status = "configured live model" if mode == "live" else "fixture fallback"
return f"**{status} · provider: {provider_label} · ready**"
def accept_provider(
raw_provider: str,
raw_key: str,
on_provider_ready: Callable[[ProviderSelection], None] | None = None,
) -> str:
selection = select_provider(raw_provider, raw_key)
if on_provider_ready is not None:
on_provider_ready(selection)
return _status(selection)
def _empty_outputs() -> JsonRecord:
return {
"provider": "",
"model": "",
"mode": "",
"closed": False,
"events": [],
"notifications": [],
"plans": [],
"agent_statuses": [],
"agents": [],
"workflows": [],
"mcp": [],
"plugins": [],
"artifacts": [],
"evolution": [],
}
def _snapshot_outputs(session: SpaceRuntimeSession | None) -> JsonRecord:
if session is None:
return _empty_outputs()
snapshot = session.snapshot()
return snapshot.as_outputs()
def submit_prompt(
session: SpaceRuntimeSession | None, prompt: str
) -> tuple[str, JsonRecord]:
if session is None:
return "**Runtime unavailable:** choose BYOK or Bonsai first.", _empty_outputs()
previous_notification_count = len(session.snapshot().notifications)
try:
snapshot = session.prompt_sync(prompt)
except (RuntimeError, ValueError) as error:
return f"**Prompt error:** {error}", _snapshot_outputs(session)
if snapshot.notifications[previous_notification_count:]:
return (
"**Runtime degraded:** provider error recorded; inspect runtime state.",
snapshot.as_outputs(),
)
return (
"**Runtime ready:** prompt processed through the event bus.",
snapshot.as_outputs(),
)
def _cleanup_session(session: SpaceRuntimeSession | None) -> None:
if session is not None:
session.close_sync()
def create_app(
on_provider_ready: Callable[[ProviderSelection], None] | None = None,
) -> gr.Blocks:
with gr.Blocks(title="Kateto Space") as app:
_ = gr.Markdown("# Kateto\nChoose a provider to enter the demo.")
_ = gr.Markdown(
"**Status:** fixture mode · live runtime not connected · provider choice required"
)
provider = gr.Radio(
choices=["BYOK", "Bonsai"],
label="Provider",
info="Your first action must be exactly one provider choice.",
elem_id="provider-choice",
)
key = gr.Textbox(
label="OpenRouter key",
info=f"Session-only; maximum {MAX_BYOK_KEY_LENGTH} characters.",
type="password",
max_length=MAX_BYOK_KEY_LENGTH,
visible=False,
elem_id="byok-key",
)
submit = gr.Button("Continue", variant="primary", elem_id="provider-submit")
status = gr.Markdown(
"Select BYOK or Bonsai to continue.", elem_id="provider-status"
)
session_state = gr.State(value=None, delete_callback=_cleanup_session)
prompt = gr.Textbox(
label="Prompt",
placeholder="Ask the team to plan work",
visible=False,
elem_id="prompt",
)
prompt_submit = gr.Button("Send", visible=False, elem_id="prompt-submit")
_ = gr.Markdown("## Live orchestration evidence")
provider_model = gr.JSON(
value={}, label="Provider / model", elem_id="provider-model"
)
timeline = gr.JSON(
value=[], label="Event timeline · name + source", elem_id="event-timeline"
)
plans = gr.JSON(value=[], label="Plans produced", elem_id="plans")
agents = gr.JSON(
value=[], label="Agents / voices · status + actions", elem_id="agents"
)
workflows = gr.JSON(
value=[],
label="Workflow tree · phase + task + checkpoints",
elem_id="workflows",
)
integrations = gr.JSON(
value={}, label="MCP / plugin status", elem_id="integrations"
)
evolution = gr.JSON(
value=[], label="Evolution / work ledger", elem_id="evolution"
)
outputs = (
provider_model,
timeline,
plans,
agents,
workflows,
integrations,
evolution,
)
def output_values(
value: JsonRecord,
) -> tuple[
JsonRecord,
list[JsonValue],
list[JsonValue],
list
[truncated — 3236 more characters]
```
### demo-video/src/index.ts
```typescript
import { registerRoot } from "remotion";
import { RemotionRoot } from "./Root";
registerRoot(RemotionRoot);
```
### kateto/plugins/connector/cli.py
```python
from __future__ import annotations
import asyncio # noqa: ANYIO_OK
import hashlib
import os
import shlex
import shutil
import signal
from dataclasses import dataclass
from enum import StrEnum
from pathlib import Path, PurePath, PureWindowsPath
from typing import Final
from pydantic import BaseModel, Field
from kateto.core.config import ConfigError, CliSettings, validate_cli_command
from kateto.core.event import BacklogAddData, BacklogItem, BacklogPriority, BacklogStatus, EventEnvelope, EventModel, InterruptData, TodoItemData
from kateto.core.manager import PluginManager
from kateto.core.plugin import EventHandler, Plugin
from kateto.core.manager import PluginManager
_SHELL_CONTROL_CHARACTERS: Final[frozenset[str]] = frozenset(";&|<>`$!\r\n\x00")
_TODO_TAGS: Final[tuple[str, ...]] = ("todo", "completed")
@dataclass(frozen=True, slots=True)
class CliArgumentRejectedError(Exception):
reason: str
def __str__(self) -> str:
return f"cli argument rejected: {self.reason}"
@dataclass(frozen=True, slots=True)
class CliExecutableMissingError(Exception):
executable: str
def __str__(self) -> str:
return f"configured cli executable is unavailable: {self.executable}"
@dataclass(frozen=True, slots=True)
class CommandResult:
returncode: int
stdout: bytes
stderr: bytes
class CliReplyStatus(StrEnum):
COMPLETED = "completed"
REJECTED = "rejected"
TIMED_OUT = "timed_out"
CANCELLED = "cancelled"
FAILED = "failed"
class CliCommandData(EventModel):
command: str = Field(min_length=1)
reply_to: str | None = Field(default=None, min_length=1)
timeout_seconds: float = Field(default=5.0, gt=0, le=30)
class CliReplyData(EventModel):
argv: tuple[str, ...] = ()
stdout: str = ""
stderr: str = ""
returncode: int | None = None
status: CliReplyStatus
reason: str | None = None
class SubprocessCommandRunner:
async def run(self, argv: tuple[str, ...], *, working_directory: Path) -> CommandResult:
executable = shutil.which(argv[0], path=os.defpath)
if executable is None:
raise CliExecutableMissingError(executable=argv[0])
process = await asyncio.create_subprocess_exec(
executable,
*argv[1:],
cwd=working_directory,
env={"LC_ALL": "C", "PATH": os.defpath},
stdin=asyncio.subprocess.DEVNULL,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
start_new_session=True,
)
try:
stdout, stderr = await process.communicate()
except asyncio.CancelledError:
await _terminate_process(process)
raise
return CommandResult(returncode=await process.wait(), stdout=stdout, stderr=stderr)
class CliConnector(Plugin):
def __init__(
self,
*,
settings: CliSettings,
runner: SubprocessCommandRunner | None = None,
working_directory: Path | None = None,
) -> None:
super().__init__("connector_cli")
self._settings = CliSettings(allowlist=list(settings.allowlist))
self._runner = SubprocessCommandRunner() if runner is None else runner
self._working_directory = (Path.cwd() if working_directory is None else working_directory).resolve()
self._active_task: asyncio.Task[None] | None = None
self._interruption_requested = False
self._synced_todo_ids: set[str] = set()
async def initialize(self) -> None:
manager = self._manager()
manager.register_event("cli_execute", CliCommandData)
manager.register_event("cli_reply", CliReplyData)
manager.register_event("todo_completed", TodoItemData)
manager.register_event("backlog_add", BacklogAddData)
async def _enqueue(self, envelope: EventEnvelope[BaseModel], handler: EventHandler) -> None:
match envelope.name, envelope.data:
case "interrupt", InterruptData() as interrupt:
await self.on_interrupt(interrupt)
case _:
await super()._enqueue(envelope, handler)
async def on_cli_execute(self, data: CliCommandData) -> None:
argv: tuple[str, ...] = ()
active_task = asyncio.current_task()
if active_task is not None:
self._active_task = active_task
try:
try:
argv = normalize_argv(data.command, settings=self._settings, working_directory=self._working_directory)
except (ConfigError, CliArgumentRejectedError) as error:
await self._emit_reply(
CliReplyData(argv=argv, status=CliReplyStatus.REJECTED, reason=str(error)),
reply_to=data.reply_to,
)
return
try:
result = await asyncio.wait_for(
self._runner.run(argv, working_directory=self._working_directory),
timeout=data.timeout_seconds,
)
except TimeoutError:
reply = CliReplyData(argv=argv, status=CliReplyStatus.TIMED_OUT, reason="command exceeded timeout")
except (CliExecutableMissingError, OSError) as error:
reply = CliReplyData(argv=argv, status=CliReplyStatus.FAILED, reason=str(error))
else:
reply = CliReplyData(
argv=argv,
stdout=result.stdout.decode("utf-8", errors="replace"),
stderr=result.stderr.decode("utf-8", errors="replace"),
returncode=result.returncode,
status=CliReplyStatus.COMPLETED,
)
await self._emit_reply(reply, reply_to=data.reply_to)
except asyncio.CancelledError:
await self._emit_reply(
CliReplyData(argv=argv, status=CliReplyStatus.CANCELLED, reason="command interrupted"),
reply_to=data.reply_to,
)
[truncated — 3597 more characters]
```
[225 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]