Gallop Defensive Agents

Multi-agent defense framework composed of three cooperating services:

• agent/: Rust + eBPF responder that observes process starts and executes bounded remediation tasks.
• server/: Axum control plane that coordinates detections, queues actions, and brokers AI reasoning for agents.
• frontend/: Next.js 16 command console that renders the current posture and suggests next actions via Jina embeddings.

Plain-English Overview

• Mission: shrink detection-to-containment time with cooperating agents.
• Pattern: sense activity → reason with policy/AI → act → verify.
• Role split: control plane plans, host agents execute, console validates.
• Benefit: closed-loop defense that records protection history instead of alert noise.
• Result: higher containment rates with fewer manual touches.

Quick Start

1. Launch Elasticsearch and note the endpoint plus an API key.
2. Start the control plane: cd server && cargo run. It exposes the APIs on port 8080 and an interactive cmd> prompt for issuing agent instructions.
3. Build and run an agent on a Linux host with eBPF support: cd agent && LIBBPF_SYS_USE_SYSTEM=1 cargo run --release (usually needs sudo).
4. Bring up the console: cd frontend && pnpm install && pnpm dev, then open http://localhost:3000.

Control Plane (server/)

• Purpose: Receives process docs from agents at /v1/ingest/processes, forwards them to Elasticsearch, maintains a command queue (/v1/agent/commands), accepts agent results, and exposes /v1/openai/chat as a reasoning broker.
• Environment:
  • ELASTIC_URL (required) – Elasticsearch node URL
  • ELASTIC_API_KEY (required) – API key for the cluster
  • PROCESS_INDEX (default logs-processstart) – index to write into
  • OPENAI_API_KEY (optional) – enables /v1/openai/chat
  • OPENAI_MODEL / OPENAI_BASE (optional) – override model or base URL
• Run: make run or cargo run. The CLI accepts list (show agent IDs) and <agent_id> <instruction> to enqueue work.

Agent (agent/)

• Purpose: Attaches the bpf/execsnoop.bpf.c program to sys_enter_execve, buffers process metadata, and batches it to the server. It polls /v1/agent/commands, runs scoped tasks (5s timeout), and reports structured outcomes.
• Environment:
  • SERVER_URL (default https://api.joingallop.com) – control plane base URL
  • OPENAI_MODEL (optional) – model name passed to the server’s OpenAI proxy
  • Agent ID is the host name; ensure it matches commands you enqueue.
• Prereqs: Linux with eBPF enabled, kernel headers, clang/llvm, libbpf (build expects LIBBPF_SYS_USE_SYSTEM=1), and root privileges to load the BPF program.
• Run: cd agent && LIBBPF_SYS_USE_SYSTEM=1 cargo run --release (or make run).

Console (frontend/)

• Purpose: Reads Elasticsearch threat data, renders current defenses, lists agents, and provides /api/jina/insight for AI judgment on commands.
• Environment (.env.local):
  • ELASTIC_URL – Elasticsearch endpoint
  • ELASTIC_API_KEY – API key
  • PROCESS_INDEX – index to query (should match server)
  • JINA_API_KEY / JINA_MODEL_ID (optional) – enable Jina-based command analysis
• Run: pnpm install && pnpm dev (or pnpm build && pnpm start).

Coordination Flow

1. Agents capture exec events via eBPF and post batches to POST /v1/ingest/processes.
2. Control plane enriches/forwards events into Elasticsearch and tracks active agent IDs.
3. Console API routes query Elasticsearch for posture summaries, threat charts, and agent status.
4. Use the server CLI to enqueue instructions; agents fetch them, optionally call the OpenAI proxy, execute bounded actions, and report results to /v1/agent/result.

Detection and Response

1. Activity lands in Elasticsearch with host/user/process context.
2. Detection rules, IOC matches, and Jina v3 embeddings (via Elastic Inference Service) label events as malicious/suspicious/benign.
3. A threat score merges AI confidence, rule severity, heuristics (e.g., netcat reverse shell, curl | bash), and asset context.
4. Policy maps score + criticality + allowlist/denylist into actions: investigate, contain, kill process, quarantine host, or log-only.
5. Agents receive scoped plans, execute bounded steps, and emit structured results.
6. Control plane verifies remediation by rechecking Elasticsearch state and closes or escalates.

Workflow Loop

1. Events flow into Elasticsearch
2. Detection rule fires
3. Workflow triggers automatically
4. Agent is invoked with a scoped plan
5. Agent returns structured action/result
6. Workflow executes deterministic response (kill proc, isolate host, notify, ticket)
7. Verification re-reads ES and closes or retries

Why Agents (instead of alert-only)

• Shrinks detection-to-action latency and cuts alert fatigue.
• Runs repeatable, policy-checked remediation without external SOAR glue.
• Consolidates automation in one control plane (Elasticsearch + agents + workflows).
• Focuses on protection history (what was remediated, when, how) instead of just scan history.

Product Narrative

• User signs up and gets a console backed by Elasticsearch; credentials stay in their environment.
• They deploy agents on hosts; each streams exec context and enforces actions locally.
• 14-day Elastic Cloud Serverless trial can power ingestion without new infra.
• Jina models via EIS generate embeddings and rerank search results to improve detections.
• Elastic Workflows and Agent Builder wire detections to actions; custom tools can be added for environment-specific remediation.

Development Notes

• Rust toolchain 1.75+ recommended for agent/ and server/.
• Node 20+ with pnpm for the dashboard.
• Formatting helpers: make fmt in Rust projects; next lint available in frontend/.
• When running agents on production hosts, review and harden command execution policies before enabling remote instructions.