Project Info
MY MOM ANUPAM SINGH MADE A DEVPOST ACCOUNBT AND LET ME, RONIT SINGH, PARTICPATE
Inspiration
As I've gotten more caught up in the developer and AI world, I began to become very interested in the progress of Frontier-Level AI models from OpenAI, Anthropic, and more recently labs like DeepSeek and Moonshot for programming and general use tasks. A big part of this recent interest is monitoring model releases and their performance on AI benchmarks like SWEBenchPro, DeepSWE, and Artificial Analysis. But, I felt a big disconnect from these arbitrary scores I saw on these benchmarks and the performance I saw in my own work. So, I made Ariadne.
What it does
Ariadne is a AI testing and Benchmarking tool that is runs on your code and grades on what YOU value.
How we built it
Ariadne is a local-first Node.js and TypeScript CLI. I used Codex with GPT-5.6 throughout Build Week for the following: exploring the existing architecture and iterating on it, testing edge cases, reviewing changes, making and planning new features, and testing Ariadne against disposable repositories. Commander wires thin commands, Zod and YAML validate inputs, Execa manages processes, Git provides attributable change evidence, and Ink/React powers the TUI. There is no hosted backend or hidden database.
Challenges we ran into
The hardest part was executing the vision for Ariadne, I had a general Idea of what i wanted to do but I had to stumble along the way to really find the niche this product needed to target and what it was meant to function as for devs. Another thing was the TUI, codex is still not the best at designing TUIs, so balancing info and good design for the TUIs using my own taste that codex doesn't have took me a while.
Accomplishments we're proud of
I'm happy I grew this project from a simple runnable tool to a fully built CLI tool that has guardrails for safety and varying degrees of verbosity so anyone can use it how they want I am are especially proud of the safety details: ignored-file checks can still catch paths such as .env, read-only tasks are verified against repository mutations, promotion revalidates repository identity and cleanliness under a lock, and conflicts are preflighted and rolled back rather than left in the user's checkout. \
What we learned
I learned what devs actually need in their tools, as Codex flagged a lot of things along the way that I wouldn't have thought about. I also learned how to more effectively use AI in my day-to-day dev work
Ariadne
Ariadne is a local-first CLI for evaluating coding-agent reliability and orchestrating file-based task workflows. It can run tasks in the shared checkout or detached Git worktrees, capture successful changes as durable local result commits, and promote reviewed result closures explicitly.
Ariadne is an observability and policy tool. It is not an operating-system sandbox, secrets vault, hosted service, or proof that command-like output actually executed.
Requirements and installation
- Node.js 20 or newer
- pnpm 10.34.1 for development
- Git when changed-file/diff policies or worktree isolation are enabled
pnpm install --frozen-lockfile
pnpm build
pnpm link
ariadne --help
The repository is pinned to pnpm 10.34.1. With pnpm 11, use pnpm add --global . instead of the removed global-link behavior. A packed or published npm package needs Node and its production dependencies, not pnpm. Release validation installs the real tarball through npm locally and globally, npm exec, pnpm locally, and the direct package binary.
Quick start
Run these commands in a repository you want to evaluate:
ariadne init
ariadne doctor
ariadne plan --all
ariadne run --all
ariadne list --batches
ariadne report
ariadne tui
Run Ariadne from the project root containing ariadne.yml. Nested invocation fails with project-root guidance instead of searching parent directories or writing state in the wrong folder.
In an interactive terminal, init offers a repository-aware Default setup and a Custom setup. Default detects the project type, package manager, validation script, installed Codex/Claude Code executable, and Git worktree capability; it imports the strongest detected validation command as a task. Custom additionally configures task dependencies, isolation, concurrency, retries, sensitive-file protections, change limits, and timeouts, then provides YAML and file-diff review before writing.
ariadne init --yes accepts detected defaults without prompts. Plain non-interactive ariadne init keeps the portable example-agent behavior used by automation. --custom requires a TTY.
An existing ariadne.yml is never overwritten automatically. Interactive init defaults to validation and offers explicit Default/Custom replacement. Replacement always shows a diff, validates the proposal before touching the original, creates an ignored timestamped backup, and then performs atomic writes. Existing task files are never overwritten.
version: 5
agent:
command:
kind: exec
file: codex
args: [exec, --sandbox, workspace-write, "-"]
timeout_ms: 600000
model_label: gpt-5.6-sol
tasks:
directory: .ariadne/tasks
verification:
commands:
- kind: exec
file: pnpm
args: [test]
timeout_ms: 300000
execution:
termination_grace_ms: 2000
concurrency: 1
failure_mode: continue
isolation: shared
worktree:
retention: on-failure
preparation:
commands: []
timeout_ms: 600000
checks:
forbidden_files: [.env, ".env.*"]
forbidden_commands: ["rm -rf"]
max_changed_files: 20
max_diff_lines: 500
Direct exec specifications preserve argument boundaries. Use { kind: shell, command: "pnpm typecheck && pnpm test" } only when shell syntax is intentional. Versionless and v1–v4 configurations remain readable through compatibility adapters. Version 5 adds declared model provenance and optional professional benchmarking.
Workflow tasks
Tasks are strict YAML files loaded recursively from tasks.directory:
id: package
name: Validate and package
dependsOn: [integration-tests]
workspaceMode: mutable
retry:
attempts: 3
delayMs: 1000
backoff: fixed
verify:
- kind: exec
file: pnpm
args: [build]
metadata:
description: Rebuild and verify the distributable package.
group: release
tags: [package, verification]
issue: 42
prompt: Validate and package the project.
dependsOn is case-insensitive and includes transitive dependencies when a task is selected. workspaceMode defaults to mutable. In shared isolation, mutable tasks are exclusive and only read-only tasks may overlap. In worktree isolation, mutable tasks may overlap because each attempt has a detached checkout. Omitted verify inherits global verification; verify: [] disables it.
Retries in shared mode preserve the current tree. Worktree retries start from a fresh checkout of the recorded source plus successful dependency results; failed-attempt mutations are not inherited. V3 parallelSafe: true adapts to workspaceMode: read-only with a migration warning.
IDs must match [A-Za-z0-9][A-Za-z0-9._-]{0,63}. Filename-derived IDs are supported when valid. Duplicate IDs and dependencies are compared case-insensitively. Missing, self, duplicate, and cyclic dependencies fail before execution.
CLI
Global flags are --verbose, --quiet, --json, and --no-color. Machine-readable modes reserve stdout for the payload; warnings and progress go to stderr.
ariadne plan package --concurrency 2
ariadne run package
ariadne run --task lint --task test
ariadne run --all --failure-mode fail-fast
ariadne run --all --isolation worktree
ariadne benchmark typescript-cli-quality
ariadne benchmark typescript-cli-quality --json
ariadne plan --all --isolation worktree --allow-dirty-base
ariadne resume <batch-id> --concurrency 2
ariadne rerun <batch-id> --failed
ariadne rerun <batch-id> --task package
ariadne list --tasks --format wide
ariadne list --batches --format json
ariadne list --batches --format csv --output exports/batches.csv
ariadne report --run <run-id-or-path>
ariadne report --batch <batch-id-or-path> --output reports/workflow.html
ariadne tui
ariadne changes <run-id>
ariadne diff <run-id> --output exports/result.patch
ariadne diff <run-id> --output exports/result.patch --force
ariadne status <run-id>
ariadne apply <run-id>
ariadne discard <run-id>
ariadne worktree clean --dry-run
run with no selectors remains equivalent to --all. plan is read-only: it creates no run or batch record and launches no processes. benchmark accepts exactly one benchmark-enabled task and is intentionally outside basic onboarding; ordinary run never invokes a judge or incurs judge-model cost. list defaults to child task attempts. report follows .ariadne/latest.json by default. Existing list format flags remain aliases. See Professional benchmarking and the CLI contract.
tui opens a keyboard-first workflow control surface over the same planner, scheduler, review services, compatibility readers, and canonical report models as the CLI. Press p to plan and run work; use Tab on the dashboard to select attention categories and review results or retained workspaces. Result detail supports bounded per-file diffs, retry comparison, safe patch export, eligibility/preflight review, explicit apply/discard confirmation, and conflict diagnostics. Workspace detail supports pure cleanup previews followed by selected or bulk confirmed cleanup. Attached in-process workflows can still be cancelled, resumed, or rerun from history; persisted running/incomplete records from another or restarted process are labeled no active runtime attached. Redirected use exits 2 without ANSI output. --verbose, --no-color, NO_COLOR, ASCII fallback, responsive 100/60/40 layouts, contextual ? help, and r reconciliation remain supported. Remote execution and mouse-first behavior remain out of scope. See Ariadne TUI.
Records and reports
.ariadne/
├── latest.json
├── batches/
│ ├── latest.json
│ └── <batch-id>/
│ ├── batch.json
│ └── report.html
├── worktrees/<workspace-id>/workspace.json
├── promotions/<promotion-id>.json
├── actions/<action-id>.json
├── exports/<task>-<short-run>.patch
└── runs/
├── latest.json
└── <run-id>/
├── run.json
├── report.html
└── artifacts/<task-id>/...
A batch references child attempt manifests instead of duplicating process traces. Run record v5 and batch record v3 add optional structured benchmark results while preserving execution outcome and policy score separately; v4 run and v2 batch records remain readable. Change-artifact v2 stores stable per-file identities, object/mode/symlink metadata, and bounded hashed text-diff artifacts. Promotion-record v2 stores structured conflicts, rollback state, and manual-recovery instructions. Management-action v1 records patch export and workspace cleanup outcomes. Historical adapters do not rewrite old records.
Manifest writes use a same-directory temporary file, sync, atomic rename, and best-effort directory sync. Latest pointers update only after valid terminal manifests exist. Raw stdout/stderr bytes stream to artifacts; manifests retain bounded 4 KiB head and 12 KiB tail previews.
Isolation and promotion
Shared mode is the compatibility default. Mutable tasks run alone; read-only tasks may overlap and fail workspace.read-only if Git-visible mutation occurs. Worktree mode creates detached checkouts from committed HEAD, layers successful dependency result commits, and permits isolated mutable overlap up to execution.concurrency.
Successful safe changes are committed under refs/ariadne/results/<run-id>. changes and diff are inspection commands; patch export is no-clobber unless the CLI receives explicit --force. Only standalone runs and final workflow attempts can be promoted. apply requires the same repository, a clean named branch, an eligible unapplied result, surviving refs, and a fresh preview fingerprint. It preflights the unresolved dependency closure in a temporary worktree, creates one squashed commit, then cherry-picks it into the unchanged primary checkout. Conflicts are aborted and verified; failed abort verification is recorded with manual recovery commands. discard removes only managed refs and eligible retained worktrees; manifests, artifacts, reports, patches, and action history remain.
Git worktrees isolate repository state, not operating-system permissions, external paths, services, caches, network access, credentials, or arbitrary subprocess side effects. Dirty-source acknowledgement uses committed HEAD only and records excluded primary dirt. Secret omission and log redaction are best effort beyond configured forbidden files and tested .env paths.
Policies
| Policy | Failure penalty |
|---|---|
files.forbidden | 40 |
commands.forbidden | 30 |
changes.max-files | 15 |
changes.max-diff-lines | 15 |
workspace.read-only | 100 |
The policy score is 100 - unique penalties, clamped to 0..100. Execution, verification, policy status, numeric score, task outcome, batch status, and CLI exit code remain separate. Command-like process output is warning-only reported evidence.
Development and validation
pnpm check
npm pack --dry-run
pnpm test:package
pnpm check performs no-emit TypeScript validation, a clean build, all tests, a disposable global-command smoke test, package-content assertions, and installed-tarball workflow execution.
Documentation:
Analysis
View
Metric
- 17
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- PythonIn code
- ReactIn code
- TypeScriptIn code
3 of 3 appear in the indexed code.
AI coding agents
- CodexConfig
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
1.2 MB
Source files
145
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
ArkXero/Ariadne
162 files · 1.3 MB · @ 4cab4f4
Structure
Application logic
72 files · 44%Domain rules, services and shared utilities.
+2 moreBackground jobs
4 files · 2%Work run outside a request: tasks, workers and schedules.
Data & schema
9 files · 6%Schema definitions, migrations and data access.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- TypeScript84%
- Markdown11%
- YAML4%
- Python1%
- Shell0%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
package.json
npm · 17- @clack/prompts
- commander
- execa
- fs-extra
- ink
- minimatch
- react
- string-width
- strip-ansi
- yaml
- zod
- +6 more
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.