Project Info
Inspiration
Organ matching is one of the highest-stakes workflows in healthcare, but the process is hard to understand, hard to simulate, and difficult to explain clearly to non-specialists. Hospital staff often deal with fragmented medical documents, imaging files, reports, and patient history while trying to make sense of compatibility, urgency, logistics, and risk. We wanted to build something that makes this workflow more transparent, educational, and easier to reason about. Fetch Health was inspired by the idea of combining agentic AI, medical document understanding, imaging analysis, and explainable scoring into one simple hospital-facing workflow. Our goal was not to replace doctors or clinical systems. Our goal was to create a safe educational and simulation platform that shows how donor-recipient matching could be analyzed, explained, and visualized with modern AI infrastructure to save lives.
What it does
Fetch Health is a hospital-staff-facing donor and recipient intake and matching simulator. The interface is intentionally simple. Staff choose whether they are uploading information for a patient/recipient or a donor, upload medical files, and let the system process the case. For a patient/recipient case, Fetch Health analyzes uploaded documents, imaging, videos, and structured records, then returns the top three donor candidates with: compatibility percentage organ type estimated transport time transportation and medical risk missing information warnings collapsible candidate reports agent-generated reasoning summaries For a donor case, Fetch Health evaluates whether the donor profile can be added to the donor database. If accepted, the system confirms the donor profile was added and shows a respectful donor-status message. If not eligible, it explains that the donor is not currently eligible based on the available information. Behind the scenes, the system uses local agents for document analysis, image/video processing, chunking, profile creation, Redis-backed memory, candidate retrieval, compatibility scoring, ranking, transport risk, benchmarking, reporting, and voice debrief generation. Fetch.ai agents act as the public agent layer, while the heavy processing runs on our local backend infrastructure.
How we built it
We built Fetch Health as a modular agentic healthcare simulation system. The frontend is a simple hospital-staff interface with three screens: intake upload screen loading/processing screen result screen The backend is built around a local HPC-first architecture. Uploaded files are processed by a Master Orchestrator Agent, which coordinates local agents for documents, images, videos, profile building, compatibility scoring, ranking, benchmarking, and reporting. We used Redis as the core state and memory layer. Redis stores profile state, run status, agentic memory, stream events, rankings, document chunks, embeddings, and traceable agent outputs. Every agent writes a structured memory record so the system can explain what happened during a run. We designed a context-compacting layer for agent-to-agent handoff. Instead of passing huge raw context between agents, each agent emits a compact “context capsule” containing hard facts, findings, risks, missing fields, and references. Long narrative context can be compressed before report generation, while critical facts like IDs, scores, rankings, organ type, and risk values are preserved exactly. We used Fetch.ai as the public capability layer. The Fetch agents are lightweight routers that expose capabilities like intake, document intelligence, image intelligence, candidate retrieval, compatibility scoring, ranking, reports, and health checks. These agents route requests to the local backend instead of trying to run heavy medical processing themselves. We also integrated sponsor tools into the workflow: Fetch.ai for public agent orchestration and mailbox-style routing Redis for state, memory, rankings, traces, and retrieval Anthropic for final reasoning and report generation The Token Company conceptually for context compacting and token efficiency Sentry for monitoring backend, agent, and tool failures Deepgram for final voice debriefs MidJourney for UI/UX visual direction and design inspiration The system is designed as an educational and simulation workflow, not a clinical decision engine.
Challenges we ran into
One major challenge was balancing ambition with usability. Organ matching is complex, but hospital staff need a simple interface. We kept the frontend minimal and moved complexity into the backend agents. Another challenge was data structure. We needed profiles that could support documents, images, videos, reports, segmentation outputs, and matching metadata without pretending composite data represented a real patient. We solved this by separating public profile summaries, private artifacts, provenance, and agent memory. Agent orchestration was also difficult. If every agent receives every file and every trace, the system becomes noisy and inefficient. We solved this with Redis-backed context capsules and structured handoffs, so each agent receives only the information it needs. We also had to handle missing or incomplete medical information responsibly. Instead of fabricating data, the system tracks missing fields, lowers confidence, and surfaces warnings in the final report. Finally, coordinating Fetch.ai agents with local HPC processing required a clean separation: Fetch agents are public routers, while local agents do the heavy processing.
Accomplishments we're proud of
We are proud that Fetch Health turns a complex healthcare workflow into a simple three-screen hospital-facing experience. We built a system where every step is traceable. Each agent writes memory, each run has events, and each result can be explained through structured reports. We are also proud of the context-compacting design. Instead of blindly dumping large documents and traces into an LLM, we separate hard facts from compressible context. This makes the agent pipeline more efficient, more reliable, and easier to debug. Another accomplishment is the modular architecture. Document analysis, imaging, video, retrieval, scoring, ranking, reporting, monitoring, and voice output are all separated into agents that can improve independently. Most importantly, we kept the system safe. Fetch Health is framed as an educational simulation and decision-support prototype, not a replacement for doctors or transplant allocation systems.
What we learned
We learned that good agent systems are not just about having many agents. They are about having the right boundaries between agents. We learned that Redis is extremely useful as an agent memory and state layer because it can store run status, events, rankings, vectors, chunks, traces, and summaries in one fast system. We also learned that healthcare AI needs transparency. A compatibility score alone is not enough. Users need to know why a profile ranked highly, what information was missing, and where the system had uncertainty. We learned that frontend simplicity matters. A powerful backend means nothing if hospital staff cannot use the product quickly and confidently. Finally, we learned that context management is one of the hardest parts of agentic AI. Passing less context, but better context, leads to better outputs.
What's next
Next, we want to expand Fetch Health in several directions. First, we want to improve the profile database with more real open-source medical imaging, reports, and benchmark cases across brain, lung, heart, kidney, and liver workflows. Second, we want to strengthen the compatibility scoring system with better baselines, better evaluation, and clearer confidence scoring. Third, we want to improve the Fetch.ai agent layer so hospital staff and educators can interact with the system through ASI:One-style agent workflows, not only the web interface. Fourth, we want to expand the voice debrief system so doctors, educators, and trainees can ask follow-up questions about a report. Finally, we want to make Fetch Health a stronger educational simulator for medical students and transplant teams, where users can explore “what-if” cases, compare candidate rankings, and understand how different compatibility factors affect the final result. Fetch Health is not trying to replace clinical judgment. It is trying to make complex medical matching workflows more understandable, traceable, and teachable.
Fetch Health
Educational organ donor/recipient matching simulator. A swarm of public Fetch.ai capability agents fronts a private FastAPI "HPC" gateway that does the real work (document/image/video analysis, retrieval, transparent rule-based matching, reporting). Six sponsor technologies are integrated honestly — no faked clinical output, ever.
Educational simulation and research workflow only. Not for clinical decision-making. No real patient data.
Sponsor integration map
| Sponsor | Where | What it does |
|---|---|---|
| Fetch.ai | swarm/ | 80 generated uAgents (mailbox, publish_agent_details) + a selective-activation Master Orchestrator harness, discoverable on ASI:One. |
| Redis | backend/app/repositories/candidate_store.py | Local redis-stack RediSearch vector KNN for candidate retrieval + edu semantic cache; Python-cosine fallback when the module is absent. |
| The Token Company | backend/app/integrations/llm.py | Compresses every prompt before Claude; exposes tokens_saved / compression_ratio. |
| Anthropic Claude | backend/app/integrations/llm.py | Real field extraction + edu explanations + report narration. Default claude-opus-4-8 (no sampling params — they 400 on Opus 4.8). |
| Deepgram | backend/app/integrations/audio.py | Transcribes uploaded audio / video narration into the document pipeline. |
| Midjourney | backend/app/integrations/imagegen.py | Edu-mode diagrams via a proxy API; pre-generate + cache; degrades to none without a key. |
| Sentry | backend/app/integrations/observability.py | Error monitoring + tracing across gateway, bureau, harness. |
Every integration degrades gracefully without its key — it returns a truthful "not configured" signal rather than fabricating output.
Architecture (5 layers)
- Public Fetch agents —
swarm/generated/(80 thin routers, generated from YAML). - Harness —
swarm/harness.py+swarm/activation_manager.py: the single agent ASI:One talks to; activates only the agents a task needs (resource allocation). - HPC gateway —
backend/FastAPI: run lifecycle, uploads,/api/agents/{name},/api/edu/explain. Returns truthful statuses (not_implementednames the missing worker). - Local Bureau —
local_agents/: ~15 internal workers (document_parserreal, rest truthful stubs). - Shared integrations —
backend/app/integrations/(llm/audio/imagegen/observability/redis_search).
Setup
cd backend && python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cd .. && cp .env.example .env # fill in keys you have (all optional to start)
docker compose up -d redis # local Redis Stack with RediSearch/vector KNN
Commands
scripts/start_gateway.sh # FastAPI gateway (:8000)
scripts/start_local_bureau.sh # local Bureau (needs uagents)
scripts/start_harness.sh # Master Orchestrator harness (needs uagents)
scripts/start_fetch_agents.sh # run a demo subset of public agents
scripts/smoke_test.sh # validate -> generate -> compile -> route (no keys needed)
python configs/_build_specs.py # regenerate agent_specs.yaml
python swarm/validate_agent_specs.py # validate specs
python swarm/generate_agents.py # (re)generate the 80 agents
cd backend && pytest # 24 tests
Example
Request → gateway:
curl -X POST localhost:8000/api/agents/master-orchestrator \
-H 'content-type: application/json' \
-d '{"request_id":"r1","task_type":"orchestration.master_orchestrator"}'
Response (truthful — a run is queued, not faked complete):
{"status":"queued","request_id":"r1","run_id":"run_…","agent_name":"master-orchestrator",
"message":"Run run_… queued. Start it via POST /api/runs/run_…/start.",
"next_status_url":"/api/runs/run_…/status"}
Edu slice (returns not_implemented until ANTHROPIC_API_KEY is set):
curl -X POST localhost:8000/api/edu/explain -H 'content-type: application/json' \
-d '{"question":"How does HLA matching work?","with_diagram":true}'
Generated agents
80 agents across 8 categories (intake, document, image, video, retrieval, matching,
benchmark, orchestration). See swarm/generated/manifest.json and per-agent
Agentverse READMEs in swarm/generated/*.md.
Missing implementation (truthful TODO)
- Image/video workers return
not_implementeduntil the local analysis models are wired. - Midjourney proxy provider/endpoint to confirm (
_submit_and_waithas a TODO). - Matching beyond blood/organ is rule-based scaffolding; extend in
services/retrieval.py. - Bureau stub workers (
upload_manager,chunking_agent, …) returnnot_implemented. - uAgents runtime not installed in the default venv —
pip install -r requirements.txtto run the swarm/bureau/harness (the gateway + factory + smoke test run without it).
Analysis
View
Metric
- 8
- 3
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- AnthropicIn code
- FastAPIIn code
- PythonIn code
- RedisIn code
- ReactClaimed
4 of 5 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
- Claude CodeCommits
- CodexConfig
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
1.4 MB
Source files
1,421
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
abhinavprkash/Fetch-Health
4,000 files · 4.5 MB · @ 69ad33d
Structure
Interface
24 files · 1%Screens, components and styles rendered to the user.
API & routing
7 files · 0%Request entry points: routes, handlers and controllers.
Application logic
2,972 files · 74%Domain rules, services and shared utilities.
Data & schema
4 files · 0%Schema definitions, migrations and data access.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- Markdown45%
- Python39%
- YAML16%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
backend/requirements.txt
pypi · 17- anthropic
- fastapi
- httpx
- jinja2
- pydantic
- pydantic-settings
- pypdf
- pytest
- pytest-asyncio
- python-docx
- python-multipart
- PyYAML
- redis
- sentry-sdk
- the-token-company
- uagents
- uvicorn[standard]
data/data_pipeline/requirements.txt
pypi · 2- pytest
- PyYAML
data/transplant_data_factory/requirements.txt
pypi · 2- pytest
- PyYAML
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This repository has more files than are indexed here, so the diagram and browser show a partial tree. Open it on GitHub for the complete structure.
Feature verification
80 generated uAgents discoverable on ASI:OneVerified
80 generated uAgents (mailbox, publish_agent_details) discoverable on ASI:One
Claimed on readmehigh confidenceswarm/generated/manifest.json— Manifest lists 80 agent specs; 81 .py files found in swarm/generated (80 agents + generate script output), each with mailbox/publish_agent_details set per template
Anthropic Claude for reasoning and report generationVerified
Anthropic is used for final reasoning and report generation, default claude-opus-4-8, no sampling params
Claimed on readmehigh confidencebackend/app/integrations/llm.py:178— _complete_anthropic calls client.messages.create with no temperature/top_p/top_k params, matching the 'they 400 on Opus 4.8' claim
De-identification / redacted profile summariesVerified
System separates public profile summaries, private artifacts, provenance, and agent memory; avoids fabricating data and tracks missing fields
Claimed on Devposthigh confidencebackend/app/services/deidentify.py:8— PII_KEYS list and redact()/age_range() implement identifier stripping and coarsening into a RedactedSummary
Donor/recipient document, image, video upload and processingVerified
Staff upload medical files (documents, imaging, video) and the system processes the case
Claimed on Devposthigh confidencebackend/app/routes/uploads.py:20— PROCESSOR_MAP dispatches uploads by extension to document/image/video/structured processorsbackend/app/processors/document.py:58— process_document reads pdf/docx/txt and extracts fields via Claude
Edu explain endpoint with optional diagramVerified
Educational explanation endpoint that returns not_implemented until ANTHROPIC_API_KEY is set
Claimed on readmehigh confidencebackend/app/routes/edu.py:20— /api/edu/explain calls edu_service.explain and returns status='not_implemented' on LLMNotConfigured
Fetch.ai public agent layer (intake, document, image, retrieval, scoring, ranking, reports, health checks)Verified
Fetch.ai agents act as a lightweight public router layer exposing these capabilities and routing to the local backend
Claimed on Devposthigh confidenceswarm/generated/manifest.json— 801-line manifest cataloguing 80 generated agent specs across categoriesswarm/templates/public_agent_template.py.j2:50— Generated agents use uAgents with mailbox=True and publish_agent_details=True, matching the 'thin router, Agentverse-discoverable' claim
Real document field extraction via ClaudeVerified
Local agents perform document analysis and profile creation
Claimed on Devposthigh confidencebackend/app/processors/document.py:74— Calls llm.complete with an extraction system prompt and parses structured JSON fields
Redis-backed state, memory, run status, rankings, chunks, embeddings, tracesVerified
Redis stores profile state, run status, agentic memory, stream events, rankings, document chunks, embeddings, and traceable agent outputs
Claimed on Devposthigh confidencebackend/app/repositories/candidate_store.py:166— RedisCandidateStore stores candidate hashes with embeddings and uses RediSearch KNN with a Python-cosine fallback, matching the claimed Redis vector storebackend/app/services/run_service.py:24— _write_memory persists AgentMemory records via a memory_store (Redis-backed profile store) for traceability
Rule-based ABO/organ/HLA compatibility scoringVerified
Transparent rule-based matching for compatibility scoring, ranking, transport risk, benchmarking
Claimed on readmehigh confidencebackend/app/services/donor_matching.py:45— ABO_COMPATIBLE_DONORS table and _score_candidate implement explicit organ/blood/HLA/logistics/vector/completeness scoring
Selective-activation Master Orchestrator harness (resource allocation)Verified
A selective-activation Master Orchestrator harness that activates only the agents a task needs
Claimed on readmehigh confidenceswarm/activation_manager.py:34— ActivationManager.ensure()/start()/reap_idle() lazily starts/stops generated agent subprocesses by id with an idle timeout, matching the claim
Sentry error monitoring across gateway, bureau, harnessVerified
Sentry for monitoring backend, agent, and tool failures
Claimed on readmehigh confidencebackend/app/integrations/observability.py:15— init_sentry initializes sentry_sdk with DSN/env tag; capture_exception forwards errors, no-op if unconfigured
The Token Company prompt compression with tokens_saved / compression_ratioVerified
Compresses every prompt before Claude; exposes tokens_saved / compression_ratio
Claimed on readmehigh confidencebackend/app/integrations/llm.py:65— compress_prompt uses thetokencompany.TheTokenCompany and returns a Compression dataclass with tokens_saved and ratio fields
Top-3 donor candidates with compatibility %, organ type, transport time, risk, missing-info warningsVerified
For a recipient case, returns top three donor candidates with compatibility percentage, organ type, transport time, risk, and missing information warnings
Claimed on Devposthigh confidencebackend/app/services/donor_matching.py:599— match_donors scores and ranks top_k (3-4) candidates with compatibility breakdown, strengths, risks, and missing_fieldsbackend/app/services/donor_matching.py:590— _top_k clamps result count to between 3 and 4 candidates
Truthful not_implemented status instead of fabricated outputVerified
Every integration degrades gracefully without its key, returning a truthful 'not_implemented'/'not configured' signal rather than fabricating output
Claimed on readmehigh confidencebackend/app/processors/document.py:79— LLMNotConfigured caught and recorded as extraction_status='not_implemented' with the real reason, no fake data writtenlocal_agents/agents/document_parser.py:22— Bureau worker reports status='not_implemented' when ANTHROPIC_API_KEY is unset
Vector KNN candidate retrieval with RediSearch + Python-cosine fallbackVerified
Local redis-stack RediSearch vector KNN for candidate retrieval; Python-cosine fallback when the module is absent
Claimed on readmehigh confidencebackend/app/repositories/candidate_store.py:255— search_by_vector tries knn_search via RediSearch and falls back to cosine_similarity over all records on failure
24 backend testsCode-supported
cd backend && pytest runs 24 tests
Claimed on readmemedium confidencebackend/tests/test_donor_matching.py— Test suite exists (test_candidate_store, test_donor_matching, test_edu_llm_failures, test_profiles, test_runs, test_uploads) covering the claimed areas; exact test count of 24 not independently confirmed without running pytest
Benchmarking against baseline matching algorithmCode-supported
Local agents perform benchmarking
Claimed on Devpostmedium confidencebackend/app/services/run_service.py:172— BenchmarkResult is populated with a hardcoded baseline_score=78.4 constant rather than a computed baseline algorithm
Collapsible candidate reports / agent reasoning summariesCode-supported
Results include collapsible candidate reports and agent-generated reasoning summaries
Claimed on Devpostmedium confidencebackend/app/services/run_service.py:24— _write_memory stores per-agent reasoning_summary records (backend data exists); no frontend to confirm 'collapsible' UI rendering since no frontend code exists in repo
Context-compacting / context capsule layer for agent handoffCode-supported
Each agent emits a compact 'context capsule' with hard facts preserved and narrative context compressed before handoff
Claimed on Devpostlow confidencebackend/app/integrations/llm.py:65— compress_prompt performs prompt compression via The Token Company before LLM calls, but no explicit 'context capsule' structure (hard facts vs. narrative separation) was found elsewhere in the codebase
Deepgram voice/audio transcription integrated into document pipelineCode-supported
Deepgram transcribes uploaded audio/video narration into the document pipeline; voice debrief generation
Claimed on readmelow confidencebackend/app/integrations/audio.py:32— A working Deepgram REST transcribe() function exists, but no caller of audio.transcribe was found anywhere in backend/app routes or processors, so it is not wired into the upload/document pipeline
Estimated transport time / transportation risk scoringCode-supported
Results include estimated transport time and transportation/medical risk
Claimed on Devpostmedium confidencebackend/app/services/donor_matching.py:368— region_score/logistics compatibility is computed from region string match only; no explicit transport-time estimate or distance/logistics risk model found
Image/video analysisCode-supported
Behind the scenes, the system uses local agents for image/video processing
Claimed on Devposthigh confidencebackend/app/processors/image.py:12— process_image is an explicit stub returning placeholder fields (dimensions: 'placeholder', detected_objects: ['placeholder'])backend/app/processors/video.py:12— process_video is an explicit stub returning zeroed/placeholder frame data, not real video analysis
Master Orchestrator Agent coordinating local agentsCode-supported
Uploaded files are processed by a Master Orchestrator Agent, which coordinates local agents for documents, images, videos, profile building, scoring, ranking, benchmarking, and reporting
Claimed on readmemedium confidenceswarm/generated/master_orchestrator.py:50— A master-orchestrator public agent exists (mailbox=True, publish_agent_details=True) as a thin routerswarm/harness.py:46— harness.py describes itself as the single public agent that fronts a selective-activation swarm, but actual heavy local-agent coordination beyond document parsing is largely stubbed (see local_agents/agents which mostly return not_implemented)
Midjourney edu-mode diagram generation with cachingCode-supported
Edu-mode diagrams via a proxy API, pre-generate + cache, degrades to none without a key
Claimed on readmemedium confidencebackend/app/integrations/imagegen.py:35— generate() implements cache-first disk lookup and graceful None fallback, but _submit_and_wait has an explicit TODO noting the exact proxy request/response shape is unconfirmed
Truthful gateway status for missing local workersCode-supported
The HPC gateway returns truthful statuses (not_implemented names the missing worker) for bureau stub workers like upload_manager, chunking_agent
Claimed on readmemedium confidencelocal_agents/agents/document_parser.py:1— Only one real local Bureau worker (document_parser) exists in local_agents/agents/; other named stub workers (upload_manager, chunking_agent) referenced in README were not found as separate files to confirm their individual not_implemented behavior
Donor eligibility evaluation and donor-status messagingClaimed only
For a donor case, the system evaluates whether the donor profile can be added, confirms acceptance, or explains ineligibility
Claimed on Devposthigh confidenceReact frontend with three-screen hospital workflowClaimed only
The frontend is a simple hospital-staff interface with three screens: intake upload, loading/processing, and result screens, built in React
Claimed on readmehigh confidence
An AI agent derived these features from the project’s Devpost page and readme, then searched the code for each one. Verified features are backed by cited code; claimed-only features had no supporting code, which is not by itself proof a feature is missing.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.