Project Info
[Elastic] Best end-to-end Agentic system on Elasticsearch (1st Place: $2,000 (split across team members) 2nd Place: $1,000 (split across team members))
CareLink
Inspiration
1 in 5 Medicare recipients are readmitted within just a month of discharge from the hospital due to post-surgery complications - complications which would have been highly avoidable had proactive and sufficient follow-up been conducted on discharged patients. Readmissions cost the U.S. hospital system 26 billion dollars a year, exerting stress upon understaffed and busy hospital systems without the bandwidth to ensure patients get consistent care after walking through the doors of the operating room.
What it does
CareLink aims to take the burden off of doctors, hospital systems, and even patients, by automating the post-surgical follow-up process while keeping medical staff in the loop - resulting in safe and efficient decision-making. The platform accomplishes two main purposes: 1) Automate patient post-surgical follow-up through agentic and adaptive voice call 2) Provide hospitals and doctors with digestible patient care recommendations through RAG-powered clinical reasoning. 48 hours after a patient’s discharge date, our voice AI agent automatically calls the patient for a check-in to ask the patient about any discomfort they’ve experienced and any concerns they have. With access to the patient’s surgery information, preexisting risk factors, and medical documentation on potential surgery complications, the agent dynamically reasons about follow-up questions to extract more information from the patient and determine the possibility of major surgery complications. If the agent believes the patient may be at risk, the hospital-facing dashboard flags the patient as requiring urgent care, alerting doctors to the need for follow-up appointments - preventing escalation of the patient condition and reducing the chances of patient readmission for increasingly severe surgical complications.
How we built it
We built CareLink with React for the frontend, Express.js for the backend, and a demo patient database stored using Elasticsearch’s powerful vector embedding and retrieval capabilities. 1) Pre-call information retrieval: We use Perplexity API to gather contextual information from credible medical documentation (including PubMed, NCBI, and FDA publications) on the patient’s surgery and potential complications. Next, we use Elasticsearch’s Jina embedding model to convert this doc to embedding vectors for more efficient comparison with the patient’s true symptoms. 2) Automated phone call pipeline: We use the Twilio API to provide the communication infrastructure via a direct phone call 48 hours after the discharge date. We use a combination of Claude and ElevenLabs API to support dynamic, natural conversation with the patient. We leverage Claude's reasoning capabilities to generate personalized questions for the patient and use ElevenLabs for high quality text-to-speech conversion for the phone call. 3) Transcription and summarization: The agent-patient conversation is transcribed and summarized for clinician records. We compare the patient’s symptoms with the expected results retrieved using Perplexity via the vector evaluation system in Elasticsearch, allowing us to present the appropriate actionables in the hospital dashboard. 4) Hospital chatbot: On the hospital UI, we used Elastic Agent Builder via Kibana as part of Elastic Cloud. Specifically, we developed an agent that has access to customized workflows and tools to investigate specific data through semantic reasoning. The chatbot feature allows hospital clinicians to receive information about patients through retrieval-augmented generation that draws directly from patient records and call transcripts from the Elasticsearch database. 5) Clinician dashboards: We also incorporated an analytics dashboard to present various statistics about the patients and surgeries stored in the database. Finally, a full patient list allows clinicians to see individual patients data, call transcripts/summaries, and recommended actionables. In addition, they can manually initiate communication through phone call or email with the patient if needed.
Challenges we ran into
1) TTS pipeline: The agent voice generated using the Twilio API for the full phone call pipeline was highly robotic. We integrated the ElevenLabs API for TTS to improve the flow of the agent’s speech; we definitely had to explore a lot of new tools to accomplish our goals for this project. 2) Performance obstacles: Our use of Elastic agents for the hospital chatbot resulted in timing bottlenecks due to the customized nature of the agent to focus on preplanning and RAG using patient information as opposed to immediate content synthesis. We restructured parts of our database pretty late into the hackathon in order to take advantage of faster ES|QL queries. 3) Multiple APIs: The integration of multiple APIs resulted in access and permission issues in the pipeline that required careful debugging to resolve. 4) Integration: Generally, connecting the various parts of the web app was difficult; we weren’t sure at first how to combine all the individual parts that everyone had worked on, but we were ultimately successful through a lot of trial and error.
Accomplishments we're proud of
1) Successfully implementing an autonomous, dynamically reasoning clinical voice AI agent 2) Integrating dynamic reasoning abilities into agent-patient calls with context retrieval through Perplexity 3) Creating a chatbot using RAG for querying patient information and data 4) Managing and updating Elastic database and agents
What we learned
We definitely learned a lot as a team. None of us had experience in designing and implementing agentic workflows, so diving into that process headfirst was very rewarding. We got to combine multiple state-of-the-art APIs and technologies to build an autonomous voice AI system for realistic patient follow-ups in healthcare, developing a full-stack agentic application using React and Express.js. We also began to think in a very product-design-oriented way, as we had to consider the very specific pain points of our target audience—like designing analytics dashboards with clinician UI/UX in mind and streamlining the patient and clinician experience in sensitive medical settings—which weren't necessarily pain points we'd have thought about before we started building. We found it incredibly cool to experiment with the voice AI and its capabilities; APIs are functional and applicable in ways we couldn’t have imagined, and seeing them work in action was really exciting.
What's next
1) Semantic filtering in patient dashboard: Allow for more flexible and robust search of patient cards in the patient information tab 2) Computer vision integration: Incorporate CV to analyze images in patients’ medical reports and user-provided images of their symptoms 3) Dynamic monitoring: Initiate repeated follow-ups on a timeline personalized for each patient’s medical conditions and history 4) Performance improvements: Reduce latency of voice agent reasoning in real-time calls
Inspiration
1 in 5 Medicare recipients are readmitted within just a month of discharge from the hospital due to post-surgery complications - complications which would have been highly avoidable had proactive and sufficient follow-up been conducted on discharged patients. Readmissions cost the U.S. hospital system 26 billion dollars a year, exerting stress upon understaffed and busy hospital systems without the bandwidth to ensure patients get consistent care after walking through the doors of the operating room.
What it does
CareLink aims to take the burden off of doctors, hospital systems, and even patients, by automating the post-surgical follow-up process while keeping medical staff in the loop - resulting in safe and efficient decision-making. The platform accomplishes two main purposes:
Automate patient post-surgical follow-up through agentic and adaptive voice call, Provide hospitals and doctors with digestible patient care recommendations through RAG-powered clinical reasoning.
48 hours after a patient's discharge date, our voice AI agent automatically calls the patient for a check-in to ask the patient about any discomfort they've experienced and any concerns they have. With access to the patient's surgery information, preexisting risk factors, and medical documentation on potential surgery complications, the agent dynamically reasons about follow-up questions to extract more information from the patient and determine the possibility of major surgery complications.
If the agent believes the patient may be at risk, the hospital-facing dashboard flags the patient as requiring urgent care, alerting doctors to the need for follow-up appointments - preventing escalation of the patient condition and reducing the chances of patient readmission for increasingly severe surgical complications.
How we built it
We built CareLink with React for the frontend, Express.js for the backend, and a demo patient database stored using Elasticsearch's powerful vector embedding and retrieval capabilities.
-
Pre-call information retrieval: We use Perplexity API to gather contextual information from credible medical documentation (including PubMed, NCBI, and FDA publications) on the patient's surgery and potential complications. Next, we use Elasticsearch's Jina embedding model to convert this doc to embedding vectors for more efficient comparison with the patient's true symptoms.
-
Automated phone call pipeline: We use the Twilio API to provide the communication infrastructure via a direct phone call 48 hours after the discharge date. We use a combination of Claude and ElevenLabs API to support dynamic, natural conversation with the patient. We leverage Claude's reasoning capabilities to generate personalized questions for the patient and use ElevenLabs for high quality text-to-speech conversion for the phone call.
-
Transcription and summarization: The agent-patient conversation is transcribed and summarized for clinician records. We compare the patient's symptoms with the expected results retrieved using Perplexity via the vector evaluation system in Elasticsearch, allowing us to present the appropriate actionables in the hospital dashboard.
-
Hospital chatbot: On the hospital UI, we used Elastic Agent Builder via Kibana as part of Elastic Cloud. Specifically, we developed an agent that has access to customized workflows and tools to investigate specific data through semantic reasoning. The chatbot feature allows hospital clinicians to receive information about patients through retrieval-augmented generation that draws directly from patient records and call transcripts from the Elasticsearch database.
-
Clinician dashboards: We also incorporated an analytics dashboard to present various statistics about the patients and surgeries stored in the database. Finally, a full patient list allows clinicians to see individual patients data, call transcripts/summaries, and recommended actionables. In addition, they can manually initiate communication through phone call or email with the patient if needed.
Architecture
┌──────────────────────────────────────────────────────────┐
│ PATIENT LAYER │
└──────────────────────────────────────────────────────────┘
▲
│
┌──────────────────────────────────────────────────────────┐
│ Conversational Interface Layer │
│ - Web / Mobile App │
│ - Text-to-speech (ElevenLabs) │
│ - Real-time streaming (Twilio) │
└──────────────────────────────────────────────────────────┘
▲
│
┌──────────────────────────────────────────────────────────┐
│ Primary Clinical Agent │
│ (Reasoning + Conversation Engine) │
│ │
│ - Perplexity deep web research (PubMed, FDA, CDA) │
│ - Jina API embedding similarity match │
│ - LLM (Elasticsearch-integrated Claude) reasoning │
└──────────────────────────────────────────────────────────┘
│ ▲
▼ │
┌──────────────────────────────────────────────────────────┐
│ Data & Retrieval Layer │
│ │
│ - Elastic Cloud database │
│ - Jina embedding for semantic search │
│ - Interactive clinician decision interface │
└──────────────────────────────────────────────────────────┘
│ ▲
▼ │
┌──────────────────────────────────────────────────────────┐
│ Agent Orchestration Layer │
│ (Multi-Agent Router + Controller) │
│ │
│ - Elastic Agent Builder + Workflow Orchestrator │
│ - Guardrails & Safety Policies │
│ - Escalation Rules + Clinical guidelines RAG │
└────┬────────────┬────────────┬────────────┬──────────────┘
▼ ▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ Symptom │ │ Action │ │Escalation│ │ General │
│ Tool │ │ Tool │ │ Tool │ │ Reasoning│
│(Triage) │ │(Follow-up│ │(Risk │ │(FAQ / │
│ │ │ Logic) │ │Detector) │ │ Support) │
└────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘
▼ ▼ ▼ ▼
┌──────────────────────────────────────────────────────────┐
│ Clinical Intelligence Layer │
│ │
│ - Medical NLP Model │
│ - Structured clinical reasoning prompts │
│ - Interactive decision support interface │
└──────────────────────────────────────────────────────────┘
Project Structure
carelink/
├── client/ # React frontend (Vite + Tailwind)
│ └── src/
│ └── components/
│ ├── layouts/ # SidebarLayout (shared shell)
│ ├── modules/ # PatientCard, PatientProfile, CallSummary, etc.
│ ├── pages/ # Dashboard, Chatbot, Analytics
│ └── utils/ # Shared helpers (patientUtils)
├── server/ # Express server
│ ├── server.js # Main entry — routes, SSE chat, Socket.IO
│ └── services/
│ ├── callAgent.js # Elastic Agent Builder client (SSE streaming)
│ ├── chatFallback.js # Fallback chat when Agent Builder unavailable
│ ├── elasticService.js # Elasticsearch helpers
│ └── emailService.js # Email notifications (Nodemailer)
├── patient-followup/ # Follow-up call system
│ ├── routes/
│ │ ├── analytics.js # Analytics aggregation API
│ │ ├── patients.js # Patient CRUD + search API
│ │ └── twilio.js # Twilio call webhooks
│ ├── services/
│ │ ├── analyticsService.js # Population-level analytics computation
│ │ ├── claudeService.js # Claude LLM for call triage
│ │ ├── twilioService.js # Twilio call orchestration
│ │ ├── elevenLabsService.js # Text-to-speech voice generation
│ │ ├── embeddingService.js # Jina embedding for semantic search
│ │ ├── perplexityService.js # Medical context research
│ │ ├── patientService.js # Elasticsearch patient queries
│ │ └── schedulerService.js # Cron-based follow-up scheduler
│ └── scripts/ # Database seeding scripts
├── data/
│ ├── agent_setup.md # Agent Builder setup instructions
│ └── patients.json # Sample patient data
├── package.json
├── SETUP.md # Installation and setup guide
└── .env.example
Getting Started
See SETUP.md for prerequisites, installation, environment configuration, and running instructions.
Analysis
View
Metric
- 24
- 11
- 10
- 7
- 3
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- AnthropicIn code
- CSSIn code
- ExpressIn code
- HTMLIn code
- JavaScriptIn code
- ReactIn code
- Tailwind CSSIn code
7 of 7 appear in the indexed code.
AI coding agents
- Claude CodeCommits
- CursorCommits
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
365 KB
Source files
84
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
JoyZhuoz/CareLink
101 files · 808 KB · @ 09e0964
Structure
Interface
29 files · 29%Screens, components and styles rendered to the user.
API & routing
18 files · 18%Request entry points: routes, handlers and controllers.
Application logic
15 files · 15%Domain rules, services and shared utilities.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- JavaScript85%
- Markdown9%
- CSS6%
- HTML0%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
package.json
npm · 28- @anthropic-ai/sdk
- @elastic/elasticsearch
- axios
- core-js
- dotenv
- express
- node-cron
- nodemailer
- nodemon
- pdf-parse
- react
- react-dom
- react-markdown
- react-router
- react-router-dom
- recharts
- remark-gfm
- socket.io
- +10 more
patient-followup/package.json
npm · 7- @anthropic-ai/sdk
- @elastic/elasticsearch
- axios
- dotenv
- express
- node-cron
- twilio
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.