# Project export: engram This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: your agent's next call is already done - Devpost: https://devpost.com/software/engram-81o3xf - GitHub: https://github.com/Bryce-Lim/engram - Team: 1 GitHub contributor(s) — Bryce-Lim (2 commits) ## Devpost submission (written by the team) ### Inspiration Especially in the modern era, agents are handling more and more complex tasks which involves making many tool calls sequentially, potentially taking a long time. But what if your agent already had the information? ### What it does When an agent takes in a prompt, it often has a chain of thought before making a sequence of MCP tool calls. We read that, predict which MCP tool calls it will make, and fetch them all in parallel before execution to put in a cache. When the agent starts executing, it reads the cached responses ,which makes it much faster. ### How we built it A drop-in MCP proxy that can be used with any agent making any types of MCP tool calls. It uses multiple systems like a small classifier AI model, and a mathematical Markov model, to predict the tool calls in advance based on the chain of thought. ### Challenges we ran into Building the system to be effective with any type of MCP tool calls was difficult, as different types of prompts can lead to very different tool calls. ### Accomplishments we're proud of We are proud of building something that generalizes fairly well in testing, and can have very significant speedups (3x or faster) in complex prompts. ### What we learned We learned a lot about the middleware layer between the agent and the tool it is accessing. ### What's next Researching more advanced systems to make the tool call predictions more accurate. ## README (from the GitHub repository) # Engram — speculative execution for AI agents **A performance layer for the Model Context Protocol (MCP). Your tools answer before the model finishes asking.** AI agents are slow because they *wait* — the model reasons for seconds, emits a tool call, fires it, blocks on the network round-trip, then repeats. The downstream API sits idle during the "thinking," and again between every step. Engram is a drop-in proxy that predicts an agent's next tool calls **while it's still thinking**, fires the side-effect-free ones in parallel, and serves the results the instant the model actually asks. Branch prediction, for agents. ``` engram wrap ./your-mcp-server → same agent, lower latency, zero code changes ``` ``` agent / host ⇄ stdio JSON-RPC ⇄ ENGRAM ⇄ stdio JSON-RPC ⇄ your MCP server │ ├─ observes tools/list → learns readOnlyHint ├─ watches reasoning → prefetches during the think ├─ learns tool→tool → prewarms the next call └─ serves warm results → ~0 ms on a hit ``` --- ## The insight CPUs solved this in the 1990s. A processor doesn't wait to learn whether a branch is taken — it predicts, executes speculatively, and commits or squashes. The same pattern maps directly onto agent tool calls: | CPU branch prediction | Engram, for agents | | ------------------------------ | ----------------------------------- | | Predict which branch is taken | Predict the next tool call | | Speculatively execute down it | Fire the API now, in parallel | | Commit the result if right | Serve the warm result on a hit (~0 ms) | | Squash & discard if wrong | Drop the speculation on a miss | --- ## How it predicts — four signals, layered from "always safe" to "genuinely novel" 1. **Eager dispatch** (zero guessing) — when the host signals (via the hint channel) that a fully-formed call is imminent, begin executing it immediately instead of after the request is routed; several such intents overlap. No guessing — the floor of the system. See [`engram/predictors/eager.py`](engram/predictors/eager.py). 2. **Chain-of-thought oracle** (the novel part) — watch the model's reasoning stream. It narrates intent — *"I'll look up their recent orders"* — seconds before the call. Engram parses that and prefetches during the think, even capturing arguments straight out of the narrated intent. See [`engram/predictors/cot_oracle.py`](engram/predictors/cot_oracle.py). 3. **Markov sequence model** — learn tool→tool transitions from traffic. *"After `search`, `fetch` follows 80% of the time."* Gets smarter with every run. See [`engram/predictors/markov.py`](engram/predictors/markov.py). 4. **Safety by protocol** (correctness gate) — speculate **only** on tools MCP marks `readOnlyHint: true`. Never a `send_email` or `charge_card`. Side-effect-free by construction, fail-closed when unsure. See [`engram/safety.py`](engram/safety.py). --- ## Quick start No dependencies — pure Python 3 standard library. ```bash # Run the split-screen demo: race the same agent with and without Engram. python3 demo/run_demo.py # Exaggerate the I/O cost to see the speedup grow toward the parallel ceiling. ENGRAM_DEMO_LATENCY=1.0 python3 demo/run_demo.py # Run the test suite (83 tests). python3 -m unittest discover -s tests # or: ./run_tests.sh ``` ### Web UI — race a prompt in the browser A small web app lets you type what an agent should do and watch the same plan race with and without Engram, with the timings measured live: ```bash ./web/run_web.sh # builds the frontend (first run) and serves on :8765 # open http://127.0.0.1:8765 ``` React + Vite + Tailwind frontend (Inter Tight throughout), served by a stdlib Python backend (`web/server.py`) whose `POST /api/compare` runs the **real** Engram-vs-baseline comparison against a live MCP server subprocess. Because there is no LLM in the loop, your prompt is turned into tool calls by a deterministic planner (`web/planner.py`) standing in for the model — but the with/without timings are genuinely measured, not fabricated. > Note: this host's Node runs the build on **Node 16** (`nvm use 16`); the > system glibc predates Node 18+. `run_web.sh` selects it automatically if nvm > is present. ### Wrap a real MCP server ```bash # Point your agent at `engram` instead of the server. That's the whole change. bin/engram wrap -- ./your-mcp-server --its --args # Unlock argument-capturing chain-of-thought prediction with a rules file: bin/engram wrap --rules demo/rules.example.json -- ./your-mcp-server ``` Flags: `--no-cot`, `--no-markov`, `--no-eager` (toggle signals), `--timeout` (downstream call budget), `--rules FILE`, `--quiet`. Logs go to **stderr**; stdout is reserved for the MCP byte stream. --- ## What's measured by the demo Everything the demo prints is measured at runtime against a real MCP server subprocess — no number is hardcoded. It has four parts: 1. **Race 1 — parallel prefetch.** With the default 400 ms tool latency and 500 ms think time, a three-call plan that a serial agent runs in ~1.7 s returns in ~0.5 s through Engram (**~3×**, 100% hit rate), because the three round-trips collapse into one parallel prefetch during the single think. The speedup grows toward the parallel ceiling as I/O latency dominates the think. *This race uses curated chain-of-thought intent rules* (`--rules` / `install_demo_rules`) whose regexes match the demo prompt and capture its exact arguments. Without rules, the auto-keyword oracle proposes only empty-argument calls, so an argument-bearing scenario warms 0 — argument capture is what makes the headline number, and it requires either rules or a host that forwards reasoning the keyword layer can ground. 2. **Race 2 — Markov learning.** An identical argument-free chain goes from 0/3 warm on the first run to 3/3 on the third as transitions are learned. *The demo lowers `MarkovModel.min_observations` from the shipped default of 2 to 1* so learning surfaces within three runs; the exact 0→2→3 trajectory is a property of these tuned settings, not an intrinsic guarantee. 3. **Race 3 — squash on miss.** The model narrates one intent but calls a different tool; the wrong speculation is squashed and the agent still gets the correct answer. A miss costs a wasted read-only fetch, never a wrong one. 4. **Safety.** A `readOnlyHint: false` tool (`send_email`) is **never** speculated. > The demo's speedup is illustrative of the mechanism on a simulated server; > real numbers depend on your model's think time and your tools' latency. > Measure against your own server before quoting figures. --- ## How it stays correct - **Safety gate.** A tool call is only ever speculated if its tool is known to be `readOnlyHint: true`. Unknown or unannotated tools fail closed. The gate is re-checked at the instant of dispatch, not just when the prediction is made, so a tool that flips to non-read-only mid-session is never fired. - **Squash on miss.** A wrong guess is discarded; the agent's real call falls through to a normal downstream execution. A miss costs a wasted (harmless) read, never a wrong answer. - **Dedup.** A given `(tool, arguments)` signature is fired downstream at most once, no matter how many predictors propose it or how a late real call races it. A real call that arrives mid-flight *attaches* to the running speculation (a "late hit") instead of issuing a duplicate. An in-flight speculation is never evicted from the cache, so this guarantee holds even under load. - **Freshness & error handling.** A cached result older than a TTL is treated as stale and re-fetched. Only *deterministic* errors (e.g. `INVALID_PARAMS`) [README truncated for size] ## Detected evidence (automated analysis) Indexed codebase: 56 recognized source files, 267 KB. - CSS (language) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - Docker (technology) — claimed on Devpost, not found in the code - Next.js (technology) — claimed on Devpost, not found in the code - Node.js (technology) — claimed on Devpost, not found in the code - Redis (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (62 of 62) ``` .gitignore bin/engram DEMO_RUNBOOK.md demo/driver.py demo/harness.py demo/mock_server.py demo/rules.example.json demo/run_demo.py demo/scenario.complex.json demo/scenario.example.json demo/try_prompt.py engram/__init__.py engram/cache.py engram/cli.py engram/config.py engram/downstream.py engram/jsonrpc.py engram/metrics.py engram/predictors/__init__.py engram/predictors/base.py engram/predictors/cot_oracle.py engram/predictors/eager.py engram/predictors/markov.py engram/proxy.py engram/safety.py engram/speculator.py README.md RUN_ON_LAPTOP.md run_tests.sh START_HERE.md tests/__init__.py tests/test_cache.py tests/test_config.py tests/test_integration.py tests/test_jsonrpc.py tests/test_metrics.py tests/test_predictors.py tests/test_review_fixes.py tests/test_safety.py tests/test_speculator.py web/compare.py web/frontend/index.html web/frontend/package.json web/frontend/postcss.config.js web/frontend/src/App.jsx web/frontend/src/components/CallList.jsx web/frontend/src/components/ComparisonCards.jsx web/frontend/src/components/HeroPrompt.jsx web/frontend/src/components/NumberTicker.jsx web/frontend/src/components/RaceTrack.jsx web/frontend/src/components/ShaderBackground.jsx web/frontend/src/components/StatRow.jsx web/frontend/src/index.css web/frontend/src/main.jsx web/frontend/tailwind.config.js web/frontend/vite.config.js web/learning.py web/planner.py web/preflight.py web/run_web.sh web/server.py web/stream.py ``` ### Dependencies - web/frontend/package.json: @vitejs/plugin-react@^4.0.4, autoprefixer@^10.4.16, postcss@^8.4.31, react@^18.2.0, react-dom@^18.2.0, tailwindcss@^3.3.5, vite@^4.5.0 ### Recent commits (newest first) - Rebrand product from precog to engram - Initial commit: precog speculative LLM proxy with demo and web UI ## Key source files (fetched from GitHub, selected and truncated for size) ### RUN_ON_LAPTOP.md ```markdown # Running Engram on your laptop This archive contains the full Engram project. The web UI's frontend is **already built** (in `web/static/`), so you can run everything with just Python 3 — no Node, no build step. ## 1. Unpack ```bash tar -xzf engram.tar.gz cd engram ``` ## 2. Run the test suite (pure Python stdlib) ```bash python3 -m unittest discover -s tests # 83 tests # or: ./run_tests.sh ``` ## 3. Run the CLI demo (terminal split-screen race) ```bash python3 demo/run_demo.py ENGRAM_DEMO_LATENCY=1.0 python3 demo/run_demo.py # exaggerate the I/O cost ``` ## 4. Run the web UI (the live race in a browser) The build output is bundled, so just start the server: ```bash python3 web/server.py --port 8765 # open http://127.0.0.1:8765 ``` ## Rebuilding the frontend (only if you change the React source) `node_modules` was excluded to keep the archive small. To rebuild the UI: ```bash cd web/frontend npm install npm run build # emits to ../static ``` > Node note: this was authored against **Node 16** because the remote host's > glibc was too old for Node 18/20. On your laptop any modern Node (18/20/22) > should build it fine — the pinned versions in `package.json` (Vite 4, React 18, > Tailwind 3) are compatible with current Node. If `npm install` complains, run > `npm install` without the lockfile or bump Vite to 5. ## Wrap a real MCP server (the actual product) ```bash bin/engram wrap -- ./your-mcp-server --args bin/engram wrap --rules demo/rules.example.json -- ./your-mcp-server ``` ## Layout - `engram/` — the proxy engine (jsonrpc, downstream, speculator, predictors, safety) - `demo/` — mock MCP server, scripted agent, CLI demo, prompt tester - `tests/` — 83 unittest tests - `web/` — Python server + planner + comparison/stream + React frontend - `web/static/` — the prebuilt UI (served by web/server.py) - `README.md` — full project documentation ``` ### START_HERE.md ```markdown # Engram — start here **Speculative execution for AI agents.** Engram is a drop-in proxy for the Model Context Protocol (MCP): it predicts an agent's next tool calls *while the model is still thinking*, runs the safe ones in parallel, and serves the results the instant the model asks. Branch prediction, for agents. This package includes a **prebuilt web demo**, so you can run it with just **Python 3 — no Node, no build step, no internet** (other than loading a web font). --- ## Run the demo (30 seconds) ```bash # unpack, then from inside the engram/ folder: python3 web/server.py --port 8765 ``` Open **http://127.0.0.1:8765** in your browser. Type what an agent should do (or click an example), hit **Run the race**, and watch the same plan run twice — once plain, once through Engram — as two liquid-glass bars fill in real time. Everything on screen is measured live. Try this prompt for the most dramatic result: > Investigate a refund dispute for Alice and Bob. Pull their orders and > profiles, check ORD-1001 payment and shipping, review system health, then > email Alice a resolution. --- ## What you're looking at - **With Engram** (orange) fills almost instantly — the calls were prefetched during the model's "think". - **Without Engram** (white) fills one call at a time, each waiting on the network. - **Safety:** anything side-effecting (send email, issue refund) is *never* speculated — look for "side-effecting — never speculated by design" in the per-call list. - **It learns:** run the same prompt twice and Engram recognizes the pattern. ### Honest disclosure The proxy, the MCP protocol over a real server subprocess, the speculation engine, and **every millisecond shown are real**. What's simulated for the demo: per-tool latency (a fixed delay standing in for a real API) and the model's think time — and, since there's no live LLM bundled, your prompt is turned into tool calls by a small deterministic planner standing in for the model. The with/without timings are not fabricated; real-world speedups depend on your model and tools. --- ## Other things you can run ```bash python3 demo/run_demo.py # terminal version of the race python3 web/preflight.py # health-check the whole stack python3 -m unittest discover -s tests # 83 engine tests (pure stdlib) ``` Wrap a real MCP server (the actual product): ```bash bin/engram wrap -- ./your-mcp-server --args ``` --- ## If you want to change the UI The web frontend is React + Vite (already built into `web/static/`). To rebuild after editing `web/frontend/src`: ```bash cd web/frontend npm install npm run build # outputs to ../static ``` (Any modern Node works for the build; `node_modules` was left out of this package to keep it small.) --- ## What's in here - `web/` — the demo: Python server + the prebuilt UI in `web/static/` - `engram/` — the proxy engine (the actual product) - `demo/` — mock MCP server, scripted agen [truncated — 176 more characters] ``` ### web/frontend/package.json ``` { "name": "engram-frontend", "private": true, "version": "0.1.0", "type": "module", "scripts": { "dev": "vite", "build": "vite build", "preview": "vite preview" }, "dependencies": { "react": "^18.2.0", "react-dom": "^18.2.0" }, "devDependencies": { "@vitejs/plugin-react": "^4.0.4", "autoprefixer": "^10.4.16", "postcss": "^8.4.31", "tailwindcss": "^3.3.5", "vite": "^4.5.0" } } ``` ### engram/cli.py ```python """``engram`` command-line interface. Usage:: engram wrap [options] -- [args...] engram wrap ./your-mcp-server ``wrap`` launches ```` as the downstream MCP server and serves the Engram proxy on this process's stdio. Point your agent/host at ``engram`` instead of the server and you get speculative execution with zero changes to the agent. Logs go to stderr (stdout is reserved for the MCP byte stream). """ import argparse import sys from typing import List, Optional from engram import __version__ from engram.proxy import Engram def _build_parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser( prog="engram", description="Speculative execution for AI agents — a performance layer for MCP.") parser.add_argument("--version", action="version", version="engram " + __version__) sub = parser.add_subparsers(dest="command") wrap = sub.add_parser("wrap", help="wrap an MCP server with the Engram proxy") wrap.add_argument("--quiet", action="store_true", help="suppress stderr logging") wrap.add_argument("--no-cot", action="store_true", help="disable the chain-of-thought oracle") wrap.add_argument("--no-markov", action="store_true", help="disable the Markov sequence model") wrap.add_argument("--no-eager", action="store_true", help="disable eager dispatch") wrap.add_argument("--timeout", type=float, default=30.0, help="downstream call timeout in seconds (default: 30)") wrap.add_argument("--rules", metavar="FILE", help="JSON file of chain-of-thought intent rules " "(enables argument-capturing prediction)") wrap.add_argument("server", nargs=argparse.REMAINDER, help="the MCP server command (prefix with -- to be safe)") return parser def _normalize_server_command(server: List[str]) -> List[str]: # argparse.REMAINDER keeps a leading "--" if the user wrote one; drop it. if server and server[0] == "--": return server[1:] return server def main(argv: Optional[List[str]] = None) -> int: parser = _build_parser() args = parser.parse_args(argv) if args.command != "wrap": parser.print_help(sys.stderr) return 2 server_cmd = _normalize_server_command(args.server) if not server_cmd: print("engram wrap: missing server command\n", file=sys.stderr) parser.parse_args(["wrap", "--help"]) return 2 def log(message: str) -> None: if not args.quiet: sys.stderr.write("[engram] " + message + "\n") sys.stderr.flush() intent_rules = None if args.rules: from engram.config import ConfigError, load_intent_rules try: intent_rules = load_intent_rules(args.rules) except ConfigError as exc: print("engram wrap: %s" % exc, file=sys.stderr) return 2 log("loaded %d intent rule(s) from %s" % (len(intent_rules), args.rules)) proxy = Engram( downstream_command=server_cmd, enable_cot=not args.no_cot, enable_markov=not args.no_markov, enable_eager=not args.no_eager, late_hit_timeout=args.timeout, on_log=log, intent_rules=intent_rules, ) try: proxy.serve_forever() except KeyboardInterrupt: proxy.shutdown() return 130 return 0 if __name__ == "__main__": sys.exit(main()) ``` ### web/server.py ```python #!/usr/bin/env python3 """Engram web server — serves the built frontend and the live comparison API. Endpoints: GET / -> the built React app (web/static/) POST /api/compare -> {prompt, latency?, think?} => measured comparison JSON POST /api/plan -> {prompt} => the planned reasoning + calls (preview) Run: python3 web/server.py [--port 8765] Stdlib only. The frontend is built with Vite into web/static; if that folder is missing, the root route explains how to build it. """ import argparse import json import os import sys from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer _HERE = os.path.dirname(os.path.abspath(__file__)) _ROOT = os.path.dirname(_HERE) if _ROOT not in sys.path: sys.path.insert(0, _ROOT) import threading # noqa: E402 from web.compare import run_comparison # noqa: E402 from web.planner import plan as build_plan # noqa: E402 from web.stream import stream_comparison # noqa: E402 from web import learning # noqa: E402 # A comparison run mutates a process-wide env var (ENGRAM_DEMO_LATENCY) that the # mock server reads at call time, so concurrent runs would race on latency. # Serialize comparison runs behind one lock; the UI is single-user anyway, and # this guarantees clean, reproducible numbers during a live demo. _run_lock = threading.Lock() MAX_BODY_BYTES = 64 * 1024 STATIC_DIR = os.path.join(_HERE, "static") _CONTENT_TYPES = { ".html": "text/html; charset=utf-8", ".js": "text/javascript; charset=utf-8", ".css": "text/css; charset=utf-8", ".svg": "image/svg+xml", ".json": "application/json", ".woff2": "font/woff2", ".ico": "image/x-icon", } class Handler(BaseHTTPRequestHandler): server_version = "EngramWeb/0.1" def _send(self, code, body, content_type="application/json"): if isinstance(body, (dict, list)): body = json.dumps(body).encode("utf-8") elif isinstance(body, str): body = body.encode("utf-8") self.send_response(code) self.send_header("Content-Type", content_type) self.send_header("Content-Length", str(len(body))) self.send_header("Cache-Control", "no-store") self.end_headers() self.wfile.write(body) def log_message(self, fmt, *args): # quieter logging sys.stderr.write("[web] " + (fmt % args) + "\n") def _read_json(self): try: length = int(self.headers.get("Content-Length", 0) or 0) except (TypeError, ValueError): length = 0 if length <= 0: return {} raw = self.rfile.read(min(length, MAX_BODY_BYTES)) try: return json.loads(raw or b"{}") except ValueError: return {} def _stream_ndjson(self, plan, latency, think, learn): """Stream race events to the client as newline-delimited JSON. Runs are serialized behind ``_run_lock`` so a second request can't perturb the shared latency env var mid-race. If the lock is held, we wait briefly; the demo is single-user so contention is unexpected. """ self.send_response(200) self.send_header("Content-Type", "application/x-ndjson") self.send_header("Cache-Control", "no-store") self.send_header("X-Accel-Buffering", "no") # disable proxy buffering self.send_header("Connection", "close") self.end_headers() acquired = _run_lock.acquire(timeout=30) if not acquired: try: self.wfile.write((json.dumps( {"ev": "error", "error": "server busy with another run"}) + "\n").encode("utf-8")) self.wfile.flush() except Exception: pass return try: for event in stream_comparison(plan, latency=latency, think=think, learn=learn): line = (json.dumps(event) + "\n").encode("utf-8") self.wfile.write(line) self.wfile.flush() # push each event immediately except (BrokenPipeError, ConnectionResetError): pass # client navigated away mid-stream except Exception as exc: try: self.wfile.write((json.dumps({"ev": "error", "error": str(exc)}) + "\n").encode("utf-8")) self.wfile.flush() except Exception: pass finally: _run_lock.release() def _clamp_params(self, data): try: latency = float(data.get("latency", 0.4)) except (TypeError, ValueError): latency = 0.4 try: think = float(data.get("think", 1.0)) except (TypeError, ValueError): think = 1.0 return max(0.05, min(latency, 2.0)), max(0.0, min(think, 3.0)) def do_POST(self): if self.path == "/api/reset-learning": learning.reset() return self._send(200, {"ok": True, "message": "learning reset"}) if self.path == "/api/compare/stream": data = self._read_json() prompt = (data.get("prompt") or "").strip() if not prompt: return self._send(400, {"error": "prompt is required"}) latency, think = self._clamp_params(data) learn = data.get("learn", True) is not False planned = build_plan(prompt) return self._stream_ndjson(planned, latency, think, learn) if self.path == "/api/compare": data = self._read_json() prompt = (data.get("prompt") or "").strip() if not prompt: return self._send(400, {"error": "prompt is required"}) latency = float(data.get("latency", 0.4)) think = float(data.get("think", 1.0)) # Clamp to keep a single request bounded. latency = max(0.05, min(latency, 2.0)) think = max(0.0, min(think, 3.0)) [truncated — 2958 more characters] ``` ### web/frontend/src/main.jsx ```javascript import React from 'react' import ReactDOM from 'react-dom/client' import App from './App' import './index.css' ReactDOM.createRoot(document.getElementById('root')).render( ) ``` ### web/frontend/src/App.jsx ```javascript import { useRef, useState } from 'react' import ShaderBackground from './components/ShaderBackground' import HeroPrompt from './components/HeroPrompt' import RaceTrack from './components/RaceTrack' import StatRow from './components/StatRow' import ComparisonCards from './components/ComparisonCards' import CallList from './components/CallList' const LATENCY = 0.4 const THINK = 1.0 const EXAMPLES = [ 'Investigate a refund dispute for Alice and Bob — pull their orders and profiles, check invoice ORD-1001, review system health, then email Alice a resolution.', 'Look up the orders for carol, check her profile and tier, then review system status, metrics, and alerts.', 'Pull the recent orders for dave and fetch the invoice for ORD-1002.', ] export default function App() { const [query, setQuery] = useState('') const [loading, setLoading] = useState(false) const [error, setError] = useState('') const [data, setData] = useState(null) const [events, setEvents] = useState([]) const [meta, setMeta] = useState(null) const [racing, setRacing] = useState(false) const [countdown, setCountdown] = useState(null) // 3,2,1,"GO" or null const resultsRef = useRef(null) const started = meta || loading || data || countdown !== null // Kick off a run: reset state, play a 3-2-1-GO countdown, THEN launch the // real race. The countdown is purely client-side, so the measured race // timing is untouched — it only starts once we fire launch(). const run = (prompt) => { const q = (prompt ?? query).trim() if (!q || loading || countdown !== null) return setQuery(q) setError('') setData(null) setEvents([]) setMeta(null) setRacing(false) setTimeout(() => { resultsRef.current?.scrollIntoView({ behavior: 'smooth', block: 'start' }) }, 150) const steps = [3, 2, 1, 'GO'] let i = 0 setCountdown(steps[0]) const tick = () => { i += 1 if (i < steps.length) { setCountdown(steps[i]) setTimeout(tick, 700) } else { // "GO" has shown for one beat — clear it and launch the real race. setCountdown(null) launch(q) } } setTimeout(tick, 700) } const launch = async (q) => { setLoading(true) try { const res = await fetch('/api/compare/stream', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ prompt: q, latency: LATENCY, think: THINK }), }) if (!res.ok || !res.body) { const j = await res.json().catch(() => ({})) throw new Error(j.error || 'request failed') } const reader = res.body.getReader() const decoder = new TextDecoder() let buf = '' const collected = [] // eslint-disable-next-line no-constant-condition while (true) { const { done, value } = await reader.read() if (done) break buf += decoder.decode(value, { stream: true }) let nl while ((nl = buf.indexOf('\n')) >= 0) { const line = buf.slice(0, nl).trim() buf = buf.slice(nl + 1) if (!line) continue const ev = JSON.parse(line) if (ev.ev === 'start') { setMeta(ev) setRacing(true) } else if (ev.ev === 'done') { setData(ev.summary) setRacing(false) } else if (ev.ev === 'error') { throw new Error(ev.error) } else { collected.push(ev) setEvents(collected.slice()) } } } } catch (e) { setError(String(e.message || e)) setRacing(false) } finally { setLoading(false) } } return (
{/* ===== HERO ===== exact match to the source design */}
{/* radial darken so the headline reads */}
{/* logo */}
Engram
{/* centered headline + prompt */}

Your agent's next call is already done.

run()} loading={loading} />
{EXAMPLES.map((ex, i) => ( ))}
{!started && (
Enter a prompt to race it — with and without Engram.
[truncated — 3705 more characters] ``` ### run_tests.sh ```shell #!/usr/bin/env bash # Run the Engram test suite. Keeps the simulated server fast for CI. set -euo pipefail cd "$(dirname "$0")" export ENGRAM_DEMO_LATENCY="${ENGRAM_DEMO_LATENCY:-0.15}" python3 -m unittest discover -s tests "$@" ``` ### engram/__init__.py ```python """Engram — speculative execution for AI agents, as MCP middleware. Engram is a drop-in proxy for the Model Context Protocol (MCP). It sits between an agent (the MCP *client*/host) and a tool server (the MCP *server*), predicts the agent's next tool calls while the model is still thinking, fires the side-effect-free ones in parallel, and serves the results the instant the model actually asks. Branch prediction, for agents. The public entry points are :class:`engram.proxy.Engram` (the proxy itself) and :func:`engram.cli.main` (the ``engram wrap ...`` command-line interface). """ __version__ = "0.1.0" __all__ = ["__version__"] ``` ### web/run_web.sh ```shell #!/usr/bin/env bash # Launch the Engram web UI: builds the frontend if needed, then serves it # together with the live comparison API from one Python process. # # ./web/run_web.sh [PORT] # # Requires Node 16 (this host's glibc is too old for Node 18/20). The script # auto-selects it via nvm if available. set -euo pipefail cd "$(dirname "$0")/.." PORT="${1:-8765}" # Select a Node that runs on this host (16.x). Honor nvm if present. if [ -s "$HOME/.nvm/nvm.sh" ]; then # shellcheck disable=SC1091 export NVM_DIR="$HOME/.nvm" source "$HOME/.nvm/nvm.sh" >/dev/null 2>&1 || true nvm use 16 >/dev/null 2>&1 || true fi if [ ! -f web/static/index.html ]; then echo "[run_web] building frontend (first run)…" ( cd web/frontend && npm install --no-audit --no-fund && npm run build ) fi echo "[run_web] starting server on http://127.0.0.1:${PORT}" exec python3 web/server.py --port "${PORT}" ``` [45 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]