Project Info
Inspiration
Our teammate Kenil got LASIK. He came out of it with a permanent dark spot in his vision that nobody had told him to expect. He was given statistics beforehand, the way every patient is, but no one who had actually lived through that specific outcome ever got the chance to tell him what it was really like, or whether it fades, or how to think about it. That's the gap we wanted to close. Before an elective or irreversible procedure, people are told probabilities. What they actually want is a real person who's been through it, telling them the truth.
What it does
A patient uploads their pre-surgery consult notes, the actual document their doctor gave them. Pulse AI reads it, pulls out the procedure and the patient's specific concern, and launches a real study on Terac targeting people who've genuinely had that exact procedure. While it waits for a real, verified respondent, the agent doesn't guess or generate a placeholder answer. It pauses, durably, for as long as it takes, then resumes the moment a real account comes in. The response a patient sees is built from that real person's actual words, clearly scoped to how closely their situation matches the patient's own, and never inflated into more confidence than the sample size actually supports. If \( n \) is the number of verified respondents behind a given answer, Pulse AI always shows \( n \) explicitly rather than letting a single account read as a general outcome. Right now \( n = 1 \) for every answer, and the product says so.
How we built it
The core is an agent built on Agentspan, Orkes' durable execution SDK, structured as a sequence of tool calls: parse the intake document, launch a Terac study, pause and wait for a real respondent, check how well that respondent's situation actually matches the patient's question, synthesize an honest answer, then log the run for evaluation. The pause step is the part we cared about most. Real human response time can't be forced into a few seconds, so the workflow has to survive that gap without losing the original question or any prior step's output, which is exactly what Agentspan's approval-gated tools are built for. Every model call in the pipeline runs through PaleBlueDot's TokenRouter rather than a single hardcoded provider, so cheap extraction steps and careful patient-facing writing each get routed to an appropriately sized model, with automatic failover if a provider has an issue. Every completed run is traced through Arize Phoenix and logged for evaluation, with the explicit goal of catching one specific failure mode: the AI claiming something a real respondent never actually said.
Challenges we ran into
The honest one is selection and matching, not infrastructure. Two people who've had the exact same procedure can have completely different experiences, so a system that just hands back any matching respondent's story risks implying relevance that isn't there. We built an explicit match-quality check for this rather than papering over it, and the first time we tested it end to end with a real respondent, the system correctly flagged a partial match. The respondent had the same procedure but a meaningfully different complication than the patient asked about, and it said so honestly instead of presenting it as a clean answer. That moment is the actual proof of the thing we set out to build. We also went through a real architecture correction mid-build. We initially assumed Terac would push completion data to us as a webhook, then learned there were no webhooks available yet, which meant rebuilding the resume mechanism around polling instead of an inbound callback. Catching that early, rather than discovering it the night before judging, mattered.
What we learned
Retrieval is safer than generation when the subject is someone's real medical decision. Early on we considered letting the model infer or extrapolate an answer when no real respondent was available, and we deliberately ruled that out. An AI inventing a plausible-sounding account of a stranger's medical experience is a worse outcome than telling a patient honestly that no one has answered yet.
What's next
A real outcome-distribution view rather than individual anecdotes alone, since showing only verified stories risks skewing toward people motivated to share strong experiences. We'd also want broader procedure coverage and faster respondent recruitment paths so the wait between a question and a real answer keeps shrinking.
Afterward
A backend AI agent that connects patients considering a medical procedure with real people who have actually had it β and never fabricates the answer.
π Overview
Afterward is a hackathon submission. It is a backend agent that answers a patient's pre-procedure question not with generic model output, but with the experience of a real human who has been through the same procedure.
A patient pastes their pre-procedure consult note. The agent extracts the procedure and the patient's real concern, launches a study to recruit someone who has had that procedure, durably pauses until that person responds, and only then writes a patient-facing answer grounded strictly in the real respondent's words.
The orchestration is built on Agentspan, whose durable execution runtime appears to be Conductor/Orkes-backed β the code guards tool state against "breaking Conductor serialization" and runs against a Java runtime server on localhost:6767 (this is also why the default branch is named orkes).
β οΈ This is a hackathon build. It ships with stubs enabled by default so the full flow runs without live external services β the Terac endpoint is an explicit placeholder, and the LLM tools fall back to canned output unless real API keys are provided.
β¨ Features
- Never fabricates an answer. The agent must complete a fixed four-step tool sequence β
parse_intake β launch_terac_study β wait_for_real_response β synthesize_answerβ and is instructed never to skip a step or invent a respondent's answer. - Human-in-the-loop durable pause.
wait_for_real_responseis an approval-gated tool: the execution durably pauses until a real respondent's answer arrives, then resumes exactly where it left off. - Webhook-driven resume. A FastAPI endpoint (
/terac-webhook) receives study-completion callbacks and resumes the paused execution by mappingstudy_id β execution_id. - Multi-model routing by responsibility. Anthropic Claude handles orchestration / tool-calling only; a separate OpenAI-compatible router (TokenRouter) handles mechanical intake parsing and patient-facing writing, each with its own model.
- Failover demo.
SIMULATE_PROVIDER_OUTAGE=1sends a broken model and invalid auth to the router to exercise the provider-failover path. - Stub modes for offline demos. Runs end-to-end without a live Terac API or LLM keys via built-in stubs and fallback respondent answers.
- Structured, secret-redacting logging. Every log line carries a phase and
execution_id; API keys and tokens are redacted, andscripts/tail_execution.shfilters logs down to a single execution.
π οΈ Tech Stack
| Area | Technology |
|---|---|
| Language | Python |
| Agent orchestration | Agentspan (durable executions, approval-gated tools; Conductor/Orkes-backed runtime) |
| Orchestrator LLM | Anthropic Claude via Agentspan (default anthropic/claude-sonnet-4-6) |
| Intake / synthesis LLMs | TokenRouter (OpenAI-compatible), e.g. qwen3.6-flash and deepseek/deepseek-v4-pro |
| Respondent recruiting | Terac studies API (with webhook callback) |
| Webhook server | FastAPI + Uvicorn |
| Utilities | requests, python-dotenv |
π Getting Started
Prerequisites
- Python 3.9+
- The Agentspan runtime server (a Java daemon reachable at
http://localhost:6767). It can be started for you by the helper script below (agentspan server start). - API keys (only required to leave stub mode):
ANTHROPIC_API_KEYβ required for the agent's tool-calling orchestrator.TOKENROUTER_API_KEYβ required for real intake parsing and answer synthesis.
Installation
# Install Python dependencies
pip install -r requirements.txt
# Configure environment
cp .env.example .env
# then edit .env and fill in the keys you need
Usage
The project is built in phases, each a runnable smoke/end-to-end test.
# 0. Ensure the Agentspan runtime server is up (starts it if needed)
python scripts/ensure_server.py
# 1. Agentspan hello-world smoke test (weatherbot)
python phase1/weatherbot.py
# 2. Start the Afterward agent and poll until it pauses for approval
python phase2/run_agent.py
# then, in another terminal, approve with a respondent answer:
python phase2/approve_response.py <execution_id> "I had this procedure. Here's my experience..."
# and poll to completion:
python phase2/wait_complete.py <execution_id>
# 3. Test the LLM-backed tools against the sample consult notes
# (requires TOKENROUTER_API_KEY, or SIMULATE_PROVIDER_OUTAGE=1 to demo failover)
python phase3/test_llm_tools.py
# 4. Full end-to-end via the webhook path
python phase4/run_webhook.py # terminal A: FastAPI webhook receiver
python phase4/run_e2e.py # terminal B: run agent β fire webhook β complete
python phase4/test_webhook.py <study_id> "respondent answer" # post a webhook manually
Key configuration flags (see .env.example):
| Variable | Purpose |
|---|---|
AFTERWARD_LLM_TOOLS | 1 to use real TokenRouter tools; otherwise stubbed output |
TERAC_USE_STUB | 1 to return fake study IDs instead of calling the Terac API |
SIMULATE_PROVIDER_OUTAGE | 1 to exercise the provider-failover path |
AGENTSPAN_MODEL | Orchestrator model (default anthropic/claude-sonnet-4-6) |
AFTERWARD_WEBHOOK_BASE_URL / WEBHOOK_PORT | Webhook receiver location |
π Project Structure
.
βββ afterward/ # Core agent package
β βββ agent.py # Agent definition + enforced 4-step tool sequence
β βββ tools/ # parse_intake, launch_terac_study, wait_for_real_response, synthesize_answer
β βββ terac_client.py # Terac studies API client (+ stub)
β βββ tokenrouter.py # OpenAI-compatible TokenRouter client
β βββ webhook_app.py # FastAPI receiver for Terac completion callbacks
β βββ resume_execution.py # Resume a paused execution with a respondent answer
β βββ study_mapping.py # study_id β execution_id mapping
β βββ pending_responses.py # respondent-answer cache keyed by execution_id
β βββ logging_config.py # structured, secret-redacting logging
βββ phase1/ β¦ phase4/ # Phased smoke / end-to-end test scripts
βββ samples/ # Example pre-procedure consult notes
βββ scripts/ # ensure_server.py, tail_execution.sh
βββ requirements.txt
βββ .env.example
Analysis
View
Metric
- 1
- 1
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- FastAPIIn code
- PythonIn code
- ReactClaimed
- Tailwind CSSClaimed
- TypeScriptClaimed
2 of 5 appear in the indexed code. 3 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
- CursorCommits
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
55 KB
Source files
25
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
2006-sk/AIhack
31 files Β· 58 KB Β· @ 25c7724
Structure
Application logic
18 files Β· 58%Domain rules, services and shared utilities.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here β open the file browser to check anything the diagram implies.
Languages
- Python99%
- Shell1%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
requirements.txt
pypi Β· 5- agentspan
- fastapi
- python-dotenv
- requests
- uvicorn
Declared in the repositoryβs manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
Feature verification
Agentspan-based durable agent architecture (sequence of tool calls)Verified
The core is an agent built on Agentspan, Orkes' durable execution SDK, structured as a sequence of tool calls
Claimed on Devposthigh confidenceafterward/agent.py:19β Imports Agent from agentspan.agents and builds it with an ordered tool list: parse_intake, launch_terac_study, wait_for_real_response, synthesize_answerrequirements.txt:1β agentspan>=0.1.10 declared as a dependency
Anthropic used only for orchestration/tool-calling, not patient-facing generationVerified
The orchestrator model decides which tool to call while patient-facing writing and extraction happen elsewhere
Claimed on Devposthigh confidenceafterward/agent.py:3β Docstring and code confirm Anthropic (via Agentspan) model is used only as the orchestrator; parse_intake and synthesize_answer route through TokenRouter instead
Custom structured logging of every tool call and state transitionVerified
Logging for evaluation (general run logging)
Claimed on Devpostmedium confidenceafterward/logging_config.py:63β setup_logging configures structured logs (phase, execution_id) to stdout and afterward.logafterward/tools/synthesize.py:66β log_tool_start/log_tool_end wrap every tool call with structured logging
Durable pause and resume waiting for a real human respondent (no fabricated placeholder while waiting)Verified
The agent pauses durably and waits for a real, verified respondent instead of guessing or generating a placeholder answer
Claimed on Devposthigh confidenceafterward/tools/wait.py:17β wait_for_real_response is decorated @tool(approval_required=True), an Agentspan approval-gated tool that pauses execution until externally approvedafterward/resume_execution.py:16β resume_with_respondent_answer() calls AgentRuntime().resume() and handle.approve() to resume the paused execution once a real answer arrivesafterward/agent.py:39β Orchestrator instructions explicitly require calling wait_for_real_response and forbid fabricating an answer
PaleBlueDot TokenRouter used for model calls with per-step model routingVerified
Every model call runs through PaleBlueDot's TokenRouter rather than a single hardcoded provider, routing cheap extraction steps and careful patient-facing writing to appropriately sized models
Claimed on Devposthigh confidenceafterward/tokenrouter.py:39β chat_completion() posts to api.tokenrouter.com/v1/chat/completionsafterward/tools/intake.py:18β parse_intake uses TOKENROUTER_PARSE_MODEL (default qwen3.6-flash)afterward/tools/synthesize.py:17β synthesize_answer uses TOKENROUTER_SYNTHESIZE_MODEL (default deepseek/deepseek-v4-pro), a distinct, larger model for patient-facing writing
Sequential mandatory 4-step tool pipeline enforced by orchestrator instructionsVerified
Structured as a sequence of tool calls: parse intake, launch study, pause and wait, synthesize, log
Claimed on Devposthigh confidenceafterward/agent.py:35β INSTRUCTIONS enumerate the exact 4-step sequence and forbid skipping or fabricating steps, and all 4 tools are wired into the Agent
Automatic failover if a TokenRouter provider has an issueCode-supported
TokenRouter provides automatic failover if a provider has an issue
Claimed on Devpostlow confidenceafterward/tokenrouter.py:19β SIMULATE_PROVIDER_OUTAGE is a local demo flag that sends an invalid key/broken model to test failover behavior, but actual failover logic lives on TokenRouter's external service, not in this repo, so it cannot be confirmed from the code alone
Fabricated/stub fallback answer used when no real respondent is cachedCode-supported
The agent never guesses or generates a placeholder answer while waiting for a real respondent
Claimed on Devposthigh confidenceafterward/tools/wait.py:47β If no cached respondent answer exists, wait_for_real_response returns a hardcoded 'STUB: I had this procedure...' string as source=stub_fallback, which contradicts the claim of never fabricating a placeholder; this only fires in dev/demo paths but the code path exists
Honest synthesis that avoids inflating confidence beyond the respondent's actual wordsCode-supported
The response is built from the real person's actual words and never inflated into more confidence than the sample size supports
Claimed on Devpostmedium confidenceafterward/tools/synthesize.py:19β SYNTH_SYSTEM prompt instructs the model to base the answer only on the respondent's real experience and not invent facts, but this is prompt-level guidance, not an enforced/checked constraint, and no n-count is included in the output
Launch a real study on Terac targeting people who've had the exact procedureCode-supported
Pulse AI launches a real study on Terac targeting people who've genuinely had that exact procedure
Claimed on Devpostmedium confidenceafterward/terac_client.py:32β launch_study posts procedure/screening_criteria to a Terac API, but the API URL is a documented placeholder ('https://api.terac.example/...') and TERAC_USE_STUB=1 by default in .env.example, so real Terac integration is unconfirmed/stubbed
Parse pre-surgery consult document to extract procedure and concernCode-supported
A patient uploads their pre-surgery consult notes and Pulse AI reads it, pulling out the procedure and the patient's specific concern
Claimed on Devpostmedium confidenceafterward/tools/intake.py:68β parse_intake extracts procedure, question_focus, and screening_criteria via an LLM call, but it takes raw_question as pasted text, not an uploaded document/file; no file upload or PDF/doc parsing code was found anywhere in the repophase4/run_e2e.py:26β Test harness passes consult text as an inline string, confirming no document upload mechanism exists
Terac webhook-based resume mechanism (contradicts claimed rebuild around polling)Code-supported
The team learned Terac had no webhooks available and rebuilt the resume mechanism around polling instead of an inbound callback
Claimed on Devpostmedium confidenceafterward/webhook_app.py:40β A FastAPI POST /terac-webhook endpoint exists and is the actual mechanism used to resume execution on respondent completion, which is an inbound webhook callback, not polling; this contradicts the narrative that the team abandoned webhooks for polling (the 'poll' references found elsewhere are dev scripts polling Agentspan's own execution status, not Terac)
Answer always shows n (number of verified respondents), currently n=1Claimed only
Pulse AI always shows n explicitly rather than letting a single account read as a general outcome; right now n=1 for every answer
Claimed on Devpostmedium confidenceEvery run traced through Arize Phoenix / OpenTelemetry and logged for evaluationClaimed only
Every completed run is traced through Arize Phoenix and logged for evaluation to catch fabricated respondent claims
Claimed on Devposthigh confidenceExplicit match-quality check scoping the respondent's relevance to the patient's situationClaimed only
The response is clearly scoped to how closely the respondent's situation matches the patient's own, via an explicit match-quality check step
Claimed on Devposthigh confidenceReact/shadcn/Radix/21st.dev patient-facing frontendClaimed only
Built With list includes 21st.dev, react, radix-ui, shadcn/ui, typescript for the product UI
Claimed on Devposthigh confidence
An AI agent derived these features from the projectβs Devpost page and readme, then searched the code for each one. Verified features are backed by cited code; claimed-only features had no supporting code, which is not by itself proof a feature is missing.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.