# Project export: TRIAGRE

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: OpenAI Build Week
- Tagline: AI agents diagnose production outages like a senior SRE — investigating logs, metrics & dependencies, then proposing a fix with evidence, in minutes instead of hours. Built with GPT-5.6 + Codex.
- Devpost: https://devpost.com/software/open-ai-build-week-gk8dim
- GitHub: https://github.com/ABHINAV-SHUKLA-11/OPEN-AI-BUILD-WEEK
- Video: https://www.youtube.com/embed/eTA2iUelytE?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) — abhinav shukla (5 commits)

## Devpost submission (written by the team)

### Inspiration

On-call engineers lose the first 20–30 minutes of every incident just figuring out what's broken — digging through logs, dashboards, and service maps before they can even start fixing anything. We wanted to build something that compresses that window: an AI that investigates an outage the way a senior SRE would, not just answers a support ticket.

### What it does

Incident Commander runs a 4-stage pipeline of specialized AI agents — Triage → Diagnosis → Fix → Postmortem — each one handing its findings to the next. The agents don't just guess: they call real tools (log search, metric queries, service dependency mapping, runbook search, and live host metrics) to gather evidence, the same way a human investigator would. Every conclusion comes with a confidence score and a visible evidence chain, so the engineer can see why the AI reached its diagnosis, not just trust a black box.

### How we built it

The backend is FastAPI with a WebSocket layer that streams each agent's progress to the frontend in real time. Each agent runs a genuine OpenAI function-calling loop on GPT-5.6 — deciding which tool to call, reading the result, and reasoning over it before concluding. Structured outputs (strict JSON schemas) keep every stage's result typed and predictable instead of free-form text. Codex was our development partner throughout: it helped implement the multi-agent pipeline and tool-calling loop, caught and fixed an XSS vulnerability in how the frontend rendered model output, added a live host-metrics tool (real CPU/memory data via psutil) so at least one data source in the demo is genuinely live rather than simulated, and helped tighten our documentation and demo flow for reliability.

### Challenges we ran into

Getting context-chaining right between agents was the hardest part — the Diagnosis agent needs the Triage agent's findings to know which services to investigate, and the Fix agent needs both. We had to carefully design each agent's output schema so it was exactly what the next agent needed as input. We also had to make sure the frontend degrades gracefully — it has a built-in offline demo mode so the experience never breaks even if the backend or API is unavailable.

### What we learned

Multi-agent systems are only as good as the evidence they're grounded in — a single prompt guessing at a root cause is very different from an agent that actually searched the logs and can show its work. We also learned how much a small amount of real data (like live host metrics) does for credibility compared to an entirely simulated demo.

### What's next

Swapping the mock log/metric tools for real integrations (Datadog, Prometheus, PagerDuty), adding a human-approval step before any suggested remediation command can be run, and expanding the tool set so the Diagnosis agent can correlate evidence across services by timestamp automatically.

## README (from the GitHub repository)

#  Incident Commander

**AI-powered incident response agent** — built for OpenAI Build Week 2026.

When a production incident hits, most engineers spend the first 20-30 minutes just figuring out what's going on — digging through logs, checking dashboards, tracing which service actually broke. Incident Commander tries to shrink that window. You describe (or simulate) an outage, and a pipeline of specialized AI agents investigates it the way a senior SRE would: checking health metrics, searching logs, tracing service dependencies, and pulling relevant runbooks — before handing you a diagnosis and a fix.

## How it works

Four agents run in sequence, each one handing its findings to the next:

```
Alert → Triage Agent (health_check, metric_query)
            ↓
       Diagnosis Agent (log_search, service_map, metric_query)
            ↓
       Fix Agent (runbook_search, service_map)
            ↓
       Postmortem Agent (full context)
            ↓
       Complete incident writeup
```

Each agent isn't just prompting an LLM once — it runs a real tool-calling loop. The agent decides what to check, calls a tool, reads the result, and keeps investigating until it has enough evidence to reach a conclusion with a confidence score attached.

## Features

- **Multi-agent pipeline** — triage, diagnosis, fix, and postmortem are handled by separate agents with their own system prompts and tool access, not one giant prompt trying to do everything
- **Real tool use** — 5 function-calling tools (log search, metric query, service map, health check, runbook search) that agents call to actually gather evidence instead of guessing
- **Structured outputs** — every agent returns a strict, typed schema (severity, evidence chain, confidence score, etc.), not free-form text
- **Live streaming** — results stream to the frontend in real time over WebSocket as each agent finishes its stage
- **One-click demo** — a pre-built outage scenario you can trigger instantly for a clean demo run

## Tech stack

| Component | Technology |
|---|---|
| Frontend | Vanilla HTML/CSS/JS (single-page, no build step) |
| Backend | Python, FastAPI, WebSocket |
| AI | OpenAI GPT-5.6 (function calling + structured outputs) |
| Database | SQLite |

## Getting started

### Prerequisites
- Python 3.10+
- An OpenAI API key with access to GPT-5.6

### Backend

```bash
cd backend
python -m venv venv
source venv/bin/activate        # Mac/Linux
# venv\Scripts\activate         # Windows

pip install -r requirements.txt
cp ../.env.example ../.env      # then add your real API key to .env
uvicorn main:app --reload --port 8000
```

### Frontend

No build step needed — it's a single static page. Either:

```bash
cd frontend
python -m http.server 3000
```
then open `http://localhost:3000`, or just open `frontend/index.html` directly in your browser.

> The frontend also has a built-in demo mode that works without the backend running — useful for a quick, guaranteed-not-to-break demo.

### Running the demo

1. Open the frontend
2. Click **"Simulate Outage"**
3. Watch the agents investigate in real time — triage, then diagnosis with evidence, then a fix with commands, then a postmortem

## Project structure

```
incident-commander/
├── backend/
│   ├── agents/          # triage, diagnosis, fix, postmortem agents + orchestrator
│   ├── tools/            # the 5 function-calling tools
│   ├── models/           # Pydantic schemas for structured outputs
│   ├── services/         # vision, voice, incident services
│   ├── routes/           # API endpoints + WebSocket
│   ├── mock_data/        # simulated logs, metrics, and runbooks for the demo
│   └── main.py
├── frontend/
│   ├── index.html
│   └── styles.css
└── README.md
```

## Built for OpenAI Build Week 2026

**Category:** AI Agents

Built with Codex on GPT-5.6 — multi-agent orchestration with real tool calling, structured outputs, and real-time streaming, wrapped around a scenario every engineer has lived through: the 3 AM outage nobody understands yet.

## License

MIT


## Detected evidence (automated analysis)

Indexed codebase: 38 recognized source files, 125 KB.
- CSS (language) — detected in the code
- FastAPI (technology) — detected in the code
- HTML (language) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- SQL (language) — detected in the code
- JavaScript (language) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (48 of 48)

```
.gitignore
backend/agents/__init__.py
backend/agents/diagnosis_agent.py
backend/agents/fix_agent.py
backend/agents/orchestrator.py
backend/agents/postmortem_agent.py
backend/agents/triage_agent.py
backend/config.py
backend/db/__init__.py
backend/db/database.py
backend/db/schema.sql
backend/main.py
backend/mock_data/logs/api_gateway.log
backend/mock_data/logs/database_service.log
backend/mock_data/logs/payment_service.log
backend/mock_data/metrics/cpu_metrics.json
backend/mock_data/metrics/memory_metrics.json
backend/mock_data/metrics/request_latency.json
backend/mock_data/runbooks/database_connection_pool.md
backend/mock_data/runbooks/high_cpu.md
backend/mock_data/runbooks/payment_gateway_timeout.md
backend/mock_data/services.json
backend/models/__init__.py
backend/models/diagnosis.py
backend/models/incident.py
backend/models/postmortem.py
backend/models/triage.py
backend/requirements.txt
backend/routes/__init__.py
backend/routes/chat.py
backend/routes/demo.py
backend/routes/incidents.py
backend/routes/ws.py
backend/services/__init__.py
backend/services/incident_service.py
backend/services/vision_service.py
backend/services/voice_service.py
backend/tools/__init__.py
backend/tools/health_check.py
backend/tools/log_search.py
backend/tools/metric_query.py
backend/tools/runbook_search.py
backend/tools/service_map.py
CODEX_USAGE.md
frontend/index.html
frontend/styles.css
LICENSE
README.md
```

### Dependencies

- backend/requirements.txt: aiosqlite@==0.20.0, fastapi@==0.115.0, httpx@==0.27.2, openai@==1.52.0, psutil@==5.9.8, pydantic@==2.9.0, python-dotenv@==1.0.1, python-multipart@==0.0.9, uvicorn[standard]@==0.30.0, websockets@==12.0

### Recent commits (newest first)

- Add MIT License to the project
- Add README for Incident Commander project
- Add usage guide for Codex with Incident Commander
- Create .gitignore
- Add files via upload

## Key source files (fetched from GitHub, selected and truncated for size)

### CODEX_USAGE.md

```markdown
# Using Codex with Incident Commander

This document is a practical guide for using Codex to develop, review, and demo Incident Commander.

> Note: this is a human-facing usage guide. If you want Codex to automatically follow repository rules in every task, place those rules in an `AGENTS.md` file at the repository root.

## Project map

The application code lives under `incident-commander/`:

```text
incident-commander/
  backend/        FastAPI API, WebSocket stream, agents, tools, and mock data
  frontend/       Static single-page dashboard (`index.html` and `styles.css`)
```

The core incident flow is:

```text
Triage -> Diagnosis -> Fix -> Postmortem
```

Each agent uses the result of the preceding stage. The frontend receives progress over a WebSocket and also includes a simulated demo mode for presentation reliability.

## What I built with Codex

Incident Commander is the product I built. Codex was my development partner: it helped turn the idea into a working multi-agent incident-response application, review implementation details, and improve the code quickly.

For a Build Week submission or demo, describe Codex's contribution accurately like this:

> I used Codex as a development partner to design and implement Incident Commander: a FastAPI and WebSocket backend, the multi-agent incident pipeline, function-calling tools, a realtime dashboard, mock incident data, and the hackathon-ready demo flow. I also used it to review security risks in dynamic frontend rendering and improve the project's developer documentation.

Codex is not the incident-response runtime itself. At runtime, the application uses the OpenAI API, its own agent orchestration, backend tools, and the browser dashboard. This distinction makes the project both credible and easy to explain.

### Concrete areas where Codex helps

- **Build features:** implement backend routes, agent stages, schemas, frontend views, and mock scenarios.
- **Review code:** find XSS risks, missing validation, brittle demo behavior, and unsafe remediation flows.
- **Debug quickly:** trace a broken WebSocket event, malformed agent output, or incorrect mock-data result.
- **Improve the demo:** tighten the story, add visible evidence trails, and keep a reliable offline simulated path.
- **Prepare the submission:** improve the README, architecture explanation, setup instructions, and demo checklist.

## Start a Codex task

Give Codex a concrete outcome and the area of the app that is in scope. Good examples:

- `Review the frontend for XSS and make only safe rendering changes.`
- `Add a second mock incident scenario without changing the payment-outage demo.`
- `Improve the diagnosis agent output schema and add validation.`
- `Make the WebSocket reconnect gracefully when the backend restarts.`
- `Review the demo flow and list only issues that could break a live judging session.`

Include constraints when they matter, for example: `Do not change the UI design`, `Keep demo mode working without an API ke
[truncated — 3555 more characters]
```

### backend/mock_data/runbooks/high_cpu.md

```markdown
# Runbook: High CPU Usage

## Symptoms
- CPU usage exceeds 80% on one or more services
- Response latency increases
- Potential request timeouts

## Diagnosis Steps
1. Identify the service with high CPU: `kubectl top pods -n production`
2. Check for runaway processes: `kubectl exec -it <pod> -- top -b -n 1`
3. Review recent deployments or config changes
4. Check for traffic spikes in metrics dashboard

## Immediate Fix
1. Scale up the affected deployment:
   ```bash
   kubectl scale deployment/<service> --replicas=5 -n production
   ```
2. If caused by a bad deploy, rollback:
   ```bash
   kubectl rollout undo deployment/<service> -n production
   ```

## Escalation
- If CPU stays above 90% after scaling: escalate to Platform Team

```

### backend/requirements.txt

```
fastapi==0.115.0
uvicorn[standard]==0.30.0
openai==1.52.0
pydantic==2.9.0
python-dotenv==1.0.1
aiosqlite==0.20.0
python-multipart==0.0.9
websockets==12.0
httpx==0.27.2
psutil==5.9.8

```

### backend/main.py

```python
"""Incident Commander — FastAPI Backend Entry Point."""
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from db.database import init_db
from config import CORS_ORIGINS
from routes import incidents, chat, ws, demo


@asynccontextmanager
async def lifespan(app: FastAPI):
    """Initialize database on startup."""
    await init_db()
    yield


app = FastAPI(
    title="Incident Commander API",
    description="AI-Powered Multi-Agent Incident Response System",
    version="1.0.0",
    lifespan=lifespan,
)

# CORS
app.add_middleware(
    CORSMiddleware,
    allow_origins=CORS_ORIGINS,
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Include routers
app.include_router(incidents.router)
app.include_router(chat.router)
app.include_router(ws.router)
app.include_router(demo.router)


@app.get("/")
async def health_check():
    """Health check endpoint."""
    return {
        "status": "healthy",
        "service": "Incident Commander API",
        "version": "1.0.0",
        "agents": ["triage", "diagnosis", "fix", "postmortem"],
        "tools": ["log_search", "metric_query", "runbook_search", "service_map", "health_check"]
    }


if __name__ == "__main__":
    import uvicorn
    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)

```

### backend/config.py

```python
"""Configuration for Incident Commander backend."""
import os
from pathlib import Path
from dotenv import load_dotenv

# Load environment variables
env_path = Path(__file__).parent.parent / ".env"
load_dotenv(dotenv_path=env_path)

# OpenAI
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "")
MODEL_NAME = os.getenv("MODEL_NAME", "gpt-5.6")  # Using hackathon GPT-5.6 credits

# Database
BASE_DIR = Path(__file__).parent
DATABASE_URL = str(BASE_DIR / "db" / "incident_commander.db")

# Mock data path
MOCK_DATA_DIR = BASE_DIR / "mock_data"

# CORS
CORS_ORIGINS = [
    "http://localhost:3000",
    "http://127.0.0.1:3000",
]

```

### frontend/styles.css

```css
/* =========================================
   INCIDENT COMMANDER — Design System
   Dark Mode + Glassmorphism + Animations
   ========================================= */

@import url('https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700;800&family=JetBrains+Mono:wght@400;500&display=swap');

:root {
  /* Background */
  --bg-primary: #0a0a0f;
  --bg-secondary: #12121a;
  --bg-tertiary: #1a1a28;
  --bg-glass: rgba(255, 255, 255, 0.03);
  --bg-glass-hover: rgba(255, 255, 255, 0.06);
  --bg-glass-active: rgba(255, 255, 255, 0.08);
  
  /* Borders */
  --border-subtle: rgba(255, 255, 255, 0.06);
  --border-medium: rgba(255, 255, 255, 0.1);
  --border-bright: rgba(255, 255, 255, 0.15);
  
  /* Severity */
  --sev1: #ff4757;
  --sev1-glow: rgba(255, 71, 87, 0.3);
  --sev2: #ffa502;
  --sev2-glow: rgba(255, 165, 2, 0.3);
  --sev3: #ffdd59;
  --sev3-glow: rgba(255, 221, 89, 0.3);
  
  /* Status */
  --healthy: #2ed573;
  --healthy-glow: rgba(46, 213, 115, 0.4);
  --degraded: #ffa502;
  --critical: #ff4757;
  --critical-glow: rgba(255, 71, 87, 0.4);
  
  /* Accent */
  --accent-blue: #3b82f6;
  --accent-blue-glow: rgba(59, 130, 246, 0.3);
  --accent-purple: #a855f7;
  --accent-purple-glow: rgba(168, 85, 247, 0.3);
  --accent-cyan: #06b6d4;
  
  /* Text */
  --text-primary: #e4e4e7;
  --text-secondary: #a1a1aa;
  --text-muted: #71717a;
  --text-dim: #52525b;
  
  /* Spacing */
  --sidebar-width: 260px;
  --header-height: 56px;
  
  /* Shadows */
  --shadow-sm: 0 2px 8px rgba(0, 0, 0, 0.3);
  --shadow-md: 0 4px 16px rgba(0, 0, 0, 0.4);
  --shadow-lg: 0 8px 32px rgba(0, 0, 0, 0.5);
  --shadow-glow-blue: 0 0 20px rgba(59, 130, 246, 0.2);
  --shadow-glow-red: 0 0 20px rgba(255, 71, 87, 0.2);
  
  /* Transitions */
  --transition-fast: 150ms ease;
  --transition-normal: 250ms ease;
  --transition-slow: 400ms ease;
}

/* =================== RESET =================== */
*, *::before, *::after {
  box-sizing: border-box;
  margin: 0;
  padding: 0;
}

html, body {
  height: 100%;
  font-family: 'Inter', -apple-system, BlinkMacSystemFont, sans-serif;
  background: var(--bg-primary);
  color: var(--text-primary);
  -webkit-font-smoothing: antialiased;
  overflow: hidden;
}

/* =================== SCROLLBAR =================== */
::-webkit-scrollbar {
  width: 6px;
}
::-webkit-scrollbar-track {
  background: transparent;
}
::-webkit-scrollbar-thumb {
  background: var(--border-medium);
  border-radius: 3px;
}
::-webkit-scrollbar-thumb:hover {
  background: var(--border-bright);
}

/* =================== LAYOUT =================== */
.app {
  display: flex;
  height: 100vh;
  overflow: hidden;
}

/* =================== SIDEBAR =================== */
.sidebar {
  width: var(--sidebar-width);
  min-width: var(--sidebar-width);
  background: var(--bg-secondary);
  border-right: 1px solid var(--border-subtle);
  display: flex;
  flex-direction: column;
  padding: 0;
  overflow-y: auto;
}

.sidebar-logo {
  display: flex;
  align-items: center;
  gap: 10px;
  padding: 20px;
  border-bottom: 1px solid var(--border-subtle);
}

.sidebar-logo .logo-dot {
  width: 10px;
  height: 10px;
  border-radius: 50%;
  background: var(--sev1);
  animation: pulse-dot 2s ease-in-out infinite;
}

.sidebar-logo h1 {
  font-size: 15px;
  font-weight: 700;
  letter-spacing: -0.3px;
  background: linear-gradient(135deg, var(--text-primary), var(--accent-blue));
  -webkit-background-clip: text;
  -webkit-text-fill-color: transparent;
}

.sidebar-nav {
  padding: 12px 10px;
  display: flex;
  flex-direction: column;
  gap: 2px;
}

.nav-item {
  display: flex;
  align-items: center;
  gap: 10px;
  padding: 10px 14px;
  border-radius: 8px;
  font-size: 13px;
  font-weight: 500;
  color: var(--text-secondary);
  cursor: pointer;
  transition: all var(--transition-fast);
  border: none;
  background: none;
  width: 100%;
  text-align: left;
}

.nav-item:hover {
  background: var(--bg-glass-hover);
  color: var(--text-primary);
}

.nav-item.active {
  background: var(--bg-glass-active);
  color: var(--accent-blue);
}

.sidebar-services {
  padding: 12px 20px;
  flex: 1;
}

.sidebar-services h3 {
  font-size: 10px;
  font-weight: 600;
  text-transform: uppercase;
  letter-spacing: 1px;
  color: var(--text-muted);
  margin-bottom: 12px;
}

.service-item {
  display: flex;
  align-items: center;
  gap: 8px;
  padding: 6px 0;
  font-size: 12px;
  color: var(--text-secondary);
}

.status-dot {
  width: 7px;
  height: 7px;
  border-radius: 50%;
  flex-shrink: 0;
}

.status-dot.healthy {
  background: var(--healthy);
  box-shadow: 0 0 6px var(--healthy-glow);
}

.status-dot.degraded {
  background: var(--degraded);
  animation: pulse-dot 2s ease-in-out infinite;
}

.status-dot.critical {
  background: var(--critical);
  box-shadow: 0 0 6px var(--critical-glow);
  animation: pulse-dot 1.5s ease-in-out infinite;
}

.sidebar-footer {
  padding: 16px;
  border-top: 1px solid var(--border-subtle);
}

.btn-simulate {
  width: 100%;
  padding: 12px;
  border: none;
  border-radius: 10px;
  font-family: 'Inter', sans-serif;
  font-size: 13px;
  font-weight: 600;
  color: white;
  background: linear-gradient(135deg, #ff4757, #ff6b81);
  cursor: pointer;
  transition: all var(--transition-normal);
  box-shadow: var(--shadow-glow-red);
  letter-spacing: 0.3px;
}

.btn-simulate:hover {
  transform: translateY(-1px);
  box-shadow: 0 0 30px rgba(255, 71, 87, 0.35);
}

.btn-simulate:active {
  transform: translateY(0);
}

.btn-simulate:disabled {
  opacity: 0.5;
  cursor: not-allowed;
  transform: none;
}

/* =================== MAIN CONTENT =================== */
.main-content {
  flex: 1;
  display: flex;
  flex-direction: column;
  overflow: hidden;
}

/* =================== HEADE
[truncated — 14059 more characters]
```

### backend/db/__init__.py

```python
# db package

```

### backend/agents/__init__.py

```python
# agents package

```

### backend/routes/__init__.py

```python
# routes package

```

### backend/services/__init__.py

```python
# services package

```

[28 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]