Project Info
Project description Organizations often fail after a problem has already been seen. The problem changes as it moves upward: a production incident becomes a delay, the delay becomes a risk, and the risk becomes a reassuring status update. By the time a decision-maker sees it, the report may sound coherent while no longer matching reality. Many language-agent simulations combine the state generator and the reporter, leaving no independent truth. A transcript can be persuasive while still lacking evidence of distortion. Groundline gives the operational world its own authority. The world engine computes the state; language carries reports about local evidence. The system runs a small simulated company with persistent employees arranged in a configurable reporting tree. The included scenario has Product, Engineering, and QA departments. The configuration also supports deeper, unbalanced trees and gives each employee a role, department, skills, honesty, blame sensitivity, and utility weights that remain stable throughout the run. The deterministic world engine owns the business state. It computes task progress, dependencies, quality, defects, deadlines, reliability, and incident severity from a seed and a scenario. A real shock is injected at a known point in the run. No model response can directly set those values. Default demo The default demo is deliberately small enough to inspect: Each run contains a request, event ledger, decision ledger, distortion metrics, and hash-verifiable manifest. Reporting path Employees receive scoped evidence. Hidden world state stays outside the policy context. A policy may choose a report and a permitted action, while the world engine resolves the action and records its real consequence. Reports travel upward one level at a time, so a manager sees subordinate reports and local evidence, never a privileged copy of the simulator’s truth. The executive dashboard therefore represents a belief assembled through the hierarchy while the simulator keeps an authoritative record beside it. Groundline then measures the difference. Distortion is calculated numerically between reported state and world truth, broken down by time, department, and hierarchy depth. The operator can change incentive pressure and manager attention, hold the seed fixed, rerun the same company, and compare the curves. A no-pressure ablation checks whether distortion shrinks when the proposed mechanism is removed. The evidence is the metric’s response to the intervention; transcripts provide context for the measured path. Measurement For employee $i$ at tick $t$, the report and world state each contain four normalized health dimensions: progress, quality, schedule, and reliability. Groundline computes a weighted vector loss: $$ L_{i,t} = 0.20\lvert p^{r}{i,t}-p^{w}{t}\rvert + 0.30\lvert q^{r}{i,t}-q^{w}{t}\rvert + 0.20\lvert s^{r}{i,t}-s^{w}{t}\rvert + 0.30\lvert r^{r}{i,t}-r^{w}{t}\rvert $$ The optimism bias used for upward amplification is the difference between the report’s aggregate health score and the world score: $$ B_{i,t}=H^{r}{i,t}-H^{w}{t} $$ The configured upward-amplification outcome compares adverse reports from the root executive with adverse contributor reports: $$ A_{\mathrm{exec},\mathrm{contrib}} =\operatorname{wmean}(B_{\mathrm{exec},t}) -\operatorname{wmean}(B_{\mathrm{contrib},t}) $$ The weighted means use each report’s scoped business value, and only rows below the configured adverse-health threshold enter the outcome. A positive $A_{\mathrm{exec},\mathrm{contrib}}$ means the executive report is more optimistic relative to truth. The experiment uses paired seeds across a 2×2 design: incentive pressure crossed with manager attention. The same underlying random world is reused in every treatment cell, making the seed the unit of comparison. Groundline reports seed-level contrasts and uncertainty because messages from one run are correlated. Its statistical layer includes paired randomization tests, bootstrap intervals, multiplicity correction, sensitivity checks, and factorial main-effect and interaction estimates. Fixture-policy results are engineering verification for the instrument; their estimates stop at the configured simulation. Intervention matrix The same seed (s) appears in all four cells. A paired contrast therefore compares organizational conditions while holding the simulated world’s random tape fixed: $$ \Delta_{a\rightarrow b}=\frac{1}{\lvert S\rvert}\sum_{s\in S}\left(Y_{b,s}-Y_{a,s}\right) $$ where $Y_{a,s}$ is the seed-level outcome for treatment $a$. The implementation is Python-first with a React/TypeScript control room. The core modules have deliberately narrow seams: a world engine for truth, an organization model for topology, an observation layer for local evidence, a policy adapter for structured decisions, a simulation runner for the event order, an immutable event ledger, a replay engine, a paired experiment runner, and an analysis layer. The web interface makes the central contrast legible quickly: WORLD TRUTH beside EXECUTIVE BELIEF, a distortion-by-level view, treatment controls, and an evidence inspector that traces a displayed event back to its causal parents. Data path Runtime modes Hosted decisions are treated as a reproducibility problem. In record mode, a language-model response is bound to the complete authorized context and written to a local decision cache. In locked mode, the system refuses an unseen context before making a network call. A finalized run can therefore be replayed without provider credentials. Fresh hosted output remains explicitly nondeterministic until it is recorded. GPT-5.6 and Codex Codex was the primary development environment. It carried the project from research framing and architecture through the Python simulation, statistical design, React interface, regression tests, browser workflows, documentation, and final integration. The important design decisions were made explicit in code and tests: deterministic truth must remain outside the policy, the model must not be prompted to lie, and causal comparisons must reuse paired seeds. During a bounded compatibility and hardening phase immediately after the GPT-5.6 rollout, early Codex client issues temporarily interrupted the workflow. I used GPT-5.6 through Claude Code and CLIProxyAPI for that portion of the debugging work, then returned to Codex for the final integration and validation pass. That history is intentional and transparent; Claude Code and CLIProxyAPI are not runtime dependencies of Groundline. Groundline also supports GPT-5.6 as the live employee-policy model through its OpenAI-compatible structured-output adapter. The adapter sends only the employee’s authorized context and never permits model output to become authoritative business state. The current live-provider regression harness measures the provider-neutral structured-decision and record/locked seams; a GPT-5.6 behavioral benchmark requires a separately labeled runtime configuration. Why this is useful Groundline gives builders and researchers a sharper question about agent organizations: under a controlled intervention, how much information changed before it reached the person making the decision? That distinction matters for multi-agent work systems, where a fluent summary can hide a broken chain of evidence. Groundline makes that chain inspectable and makes the mechanism testable. Scope and limitations This is a focused instrument for one simulated operational setting. Its causal estimates apply to that configured world; extrapolation to human organizations requires separate validation. Future work can add matrix reporting, committees, cross-functional decision rights, dynamic reorganization, and calibration against human organizational studies while preserving the same separation between computed truth and language.
Groundline
A causal evaluation environment for hierarchical language-agent organizations. A deterministic operational world computes business truth. Persistent employees receive scoped evidence, take permitted actions, and decide what to report upward. The repository and Python package retain the working name groundline.
The narrow research claim is not "LLMs can simulate a company." It is that hierarchical reporting can be treated as a causal, replayable measurement problem: keep world randomness fixed, intervene on organizational conditions, and estimate how information changes as it climbs.
Built with GPT-5.6 and Codex
Groundline was developed primarily with Codex, from the initial research and architecture through the deterministic Python engine, statistical design, React control room, regression tests, browser validation, and final integration. GPT-5.6 was also used directly as the coding and reasoning model during a bounded compatibility and hardening phase.
There is one important implementation-history detail. Immediately after the GPT-5.6 rollout, early issues in the Codex client temporarily interrupted this project by consuming excessive amount of credits. For part of that hardening work, GPT-5.6 was accessed through Claude Code using CLIProxyAPI. Development then returned to Codex, where the final fixes, aggregate verification, documentation, and submission pass were completed. Claude Code and CLIProxyAPI are not runtime dependencies of Groundline.
GPT-5.6 is also supported at runtime as the live employee policy. The adapter sends only an employee's authorized context to an OpenAI-compatible chat-completions endpoint and requires a structured JSON decision. It never sends hidden world truth and never allows model output to become authoritative state. Live decisions can be recorded once, replayed without credentials, and rerun in network-disabled locked mode.
The key engineering decisions made with Codex were:
- operational truth belongs exclusively to the deterministic world engine;
- agents are never instructed to lie, conceal, or exaggerate;
- causal comparisons pair every treatment on the same seeds;
- fresh hosted output becomes reproducible only after it is recorded in the decision ledger;
- organizations are validated rooted trees, not a hardcoded pyramid;
- fixture-policy results verify the instrument but are not presented as evidence about GPT-5.6 behavior.
Run the complete offline experiment
Prerequisites: Python 3.13, uv, Node 24, and npm.
uv sync
npm --prefix frontend install
npm --prefix frontend run build
uv run groundline experiment --config configs/demo.yaml --artifacts artifacts
uv run groundline serve --artifacts artifacts
Open http://127.0.0.1:8000. The backend serves the artifact API and compiled interface. The operator controls launch fresh 2×2 paired intervention matrices; SQLite-backed job progress and expired-lease recovery are exposed through the same API.
A normal wheel build runs the frontend build and packages the compiled same-origin application:
uv build --wheel
uv pip install dist/groundline-*.whl
The demo experiment is a general rooted tree with 13 persistent employees: nine contributors, three department directors, and one executive across Product, Engineering, and QA. The topology engine also accepts deeper, unbalanced trees.
Run one company trajectory
uv run groundline run --config configs/demo.yaml --seed 7 --policy fixture
Use GPT-5.6, or another language model exposed through an OpenAI-compatible chat-completions endpoint, as the employee policy. Configure the bare model name, base URL, and API key in a .env file (see .env.example) or export them as environment variables — real environment variables take precedence over the file:
# .env
GROUNDLINE_MODEL=gpt-5.6
GROUNDLINE_API_BASE=https://api.openai.com/v1
GROUNDLINE_API_KEY=your-key
# Record mode captures unseen decisions from the provider:
uv run groundline run --config configs/demo.yaml --seed 7 --policy record --artifacts artifacts
# Any OpenAI-compatible endpoint works the same way — set GROUNDLINE_API_BASE
# to its URL and GROUNDLINE_MODEL to a bare model name. Providers without a
# native OpenAI-compatible surface (e.g. AWS Bedrock) must be reached through
# an OpenAI-compatible proxy.
Live decisions request JSON-object structured output, receive no hidden world state, and are cached by the hash of the complete authorized context. record captures unseen decisions. locked permits only already-captured decisions and fails before making a network call on a cache miss. Every finalized run can be reconstructed exactly without a network call:
# After a record run, replay works with zero provider credentials:
uv run groundline replay artifacts/<run-id>
# Or re-run with locked mode — no network call, fails on unseen context:
uv run groundline run --config configs/demo.yaml --seed 7 --policy locked --model gpt-5.6 --artifacts artifacts
Artifact contract
Each run contains:
request.json: complete seed, organization, scenario, and treatmentevents.jsonl: truth, observations, verifications, reports, decisions, metrics, and consequencesdecisions.jsonl: context hashes and structured policy outputsmetrics.json: tick-level distortion by agent, department, and hierarchy depthmanifest.json: schema, policy, and engine fingerprints plus request, event, decision, and metrics hashes and counts
The manifest is published last. Shared verification rejects malformed, incomplete, tampered, identity-mismatched, or replay-inconsistent artifacts before replay, analysis, resume, or API reads.
Each experiment adds an atomic execution state, paired run index, typed JSON analysis, CSV and Parquet seed-level outcomes, and a Markdown report. Inference includes exact or seeded Monte Carlo sign-flip tests, deterministic BCa intervals with explicit percentile fallback, Holm adjustment within declared confirmatory families, preregistered sensitivities, within-seed 2×2 factorial contrasts, design-resolution diagnostics, and a report-level mixed-effects model with seed and reporting-agent random intercepts. Fixture-policy results are engineering verification, not evidence about language-agent behavior.
Verification
uv run pytest -q
uv run ruff check .
uv run ruff format --check .
uv run mypy
npm --prefix frontend run test
npm --prefix frontend run build
npm --prefix frontend run test:e2e
Architecture and evidence
| Module | Responsibility |
|---|---|
world/engine.py | Deterministic authoritative business state |
organization/models.py | Arbitrary reporting-tree validation and derived topology |
observation/engine.py | Scoped local evidence and manager verification |
openai_compat_policy.py | GPT-5.6/OpenAI-compatible structured decisions, retries, and record/locked behavior |
simulation/runner.py | Tick, report, action, consequence, and metric orchestration |
events/store.py | Canonical event ledger and finalized artifact manifest |
replay/engine.py | Zero-network reconstruction and equivalence checks |
experiments/runner.py | Paired intervention execution, resume, recovery, and exports |
statistics/inference.py | Seed-level causal inference and sensitivity analysis |
api/app.py | Artifact queries, experiment jobs, and compiled control-room hosting |
LICENSE
Apache 2.0
Analysis
View
Metric
- 2
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- CSSIn code
- FastAPIIn code
- HTMLIn code
- PythonIn code
- ReactIn code
- TypeScriptIn code
- Node.jsClaimed
6 of 7 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
646 KB
Source files
91
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
RoyCoding8/Groundline
103 files · 902 KB · @ 82b0500
Structure
Interface
3 files · 3%Screens, components and styles rendered to the user.
API & routing
4 files · 4%Request entry points: routes, handlers and controllers.
Application logic
59 files · 57%Domain rules, services and shared utilities.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- Python70%
- TypeScript20%
- CSS7%
- YAML1%
- Markdown1%
- HTML0%
- Other (1)0%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
frontend/package.json
npm · 15- @material/web
- @phosphor-icons/react
- react
- react-dom
- +11 more
pyproject.toml
pypi · 13- fastapi
- httpx
- numpy
- pandas
- pyarrow
- pydantic
- python-dotenv
- pyyaml
- rich
- scipy
- statsmodels
- typer
- uvicorn
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.