Project Info
Inspiration
Growing up in an Indian household, I watched my grandparents manage 4 to 8 daily prescription pills. It was a constant struggle: (Which pill is for blood pressure? Which one needs to be taken before food? Are these two medicines safe together?) Every pill-reminder app on the market assumes the user is a tech-savvy adult who can read English, navigate complex app menus, and respond to smartphone push notifications. But the highest-risk users β elderly parents, rural families, and non-literate/ less-literate patients β who don't use smartphone apps. I asked myself: (What if we split the roles?) Let the tech-savvy caregiver (an adult child or nurse) snap a photo of the medicine strips using AI, while the elderly patient simply receives a warm, automated phone call in their mother tongue (Hindi/English) and presses 1 to confirm. That inspiration became -DawaiSaathi ("Medicine Companion").
What it does
DawaiSaathi simplifies daily medication setup and safety: -Photo Strip Scanner: Caregivers upload up to 5 medicine strip photos. The system extracts the brand name, active salt composition (e.g. Telmisartan 40mg), price, expiry date, batch number, and manufacturer. -Multilingual Voice Reminders: Places automated phone calls using Twilio and Gemini TTS (Sulafat voice profile). The patient hears a clear Hindi or English voice prompt and presses 1 on any keypad to confirm the dose. -openFDA Safety Check: Cross-checks active salts against openFDA label data to warn caregivers about dangerous drug interactions like Warfarin + Aspirin. -Jan Aushadhi Generic Savings: Pairs branded drugs with identical generic substitutes under India's Jan Aushadhi scheme, showing exact monthly savings. -Household Roster: Lets co-caregivers manage patient schedules together with owner, caregiver, or viewer roles.
How we built it
-App & PWA: Next.js 15 App Router and Tailwind CSS, running as a PWA and packaged into signed Android APK releases on GitHub. -AI Vision & Extraction: Google Gemini 2.0 Flash (gemini-flash-latest) and Groq (llama-4-scout Vision & llama-3.3-70b) with dual-verify cross-checking. -Voice Generation: Google Gemini 2.5 Flash Native TTS for natural Hindi/English audio with automatic retry backoff. -Infrastructure: Cloudflare Workers via OpenNext, Cloudflare D1 (SQLite), R2 private storage, and Supabase Auth + RLS Postgres database. -Telephony: Twilio Programmable Voice with TwiML webhooks.
Challenges we ran into
-Voice Generation Quotas: Free inference APIs often served robotic audio or hit rate limits. We switched to streaming 24kHz raw PCM directly from Gemini's native audio model with automatic exponential backoff retries (1.5s, 4s, 8s) for 429 errors. -API Key Header Formats: Gemini's newer AQ. key format rejected standard Bearer headers. Rewriting our Gemini client to use Google's native REST endpoints with x-goog-api-key headers fixed authentication across Cloudflare edge workers. -Blister Pack Glare: Foil packaging reflects daylight heavily. We tuned the vision prompts to reliably target printed batch details, expiry dates, and salt names despite reflections.
Accomplishments we're proud of
-Voice Generation Quotas: Free inference APIs often served robotic audio or hit rate limits. We switched to streaming 24kHz raw PCM directly from Gemini's native audio model with automatic exponential backoff retries (1.5s, 4s, 8s) for 429 errors. -API Key Header Formats: Gemini's newer AQ. key format rejected standard Bearer headers. Rewriting our Gemini client to use Google's native REST endpoints with x-goog-api-key headers fixed authentication across Cloudflare edge workers. -Blister Pack Glare: Foil packaging reflects daylight heavily. We tuned the vision prompts to reliably target printed batch details, expiry dates, and salt names despite reflections. -Passing 176 automated unit tests covering validation, gateway proxying, D1 fallback, and dual-verify logic. -Building an automated GitHub Actions pipeline that compiles Android APKs on every release tag. -Making something an elderly grandparent can use without learning how to navigate a smartphone.
What we learned
-Voice calls remain the most accessible interface for elderly and non-literate patients. -Grounding vision model outputs with openFDA label data prevents dangerous AI hallucinations. -Building on edge workers (Cloudflare Workers + OpenNext) keeps multi-model AI workflows fast and cheap.
What's next
-WhatsApp Missed-Dose Alerts: Sending text updates to caregivers when a patient misses a call. -Jan Aushadhi Pharmacy Ordering: Direct re-ordering of low-stock generic medicines. -Regional Language Expansion: Adding native spoken voice support for Tamil, Telugu, Marathi, and Gujarati.
π The Cause
Elderly patients commonly take 4β8 daily medications. Roughly 50% of patients with chronic diseases do not take medicines as prescribed (WHO estimate). Confusion about which pill, when, and with/without food causes hospitalizations that are largely preventable. Furthermore, dangerous interactions go unnoticed across multiple prescriptions, and people overpay for branded molecules because they don't know about generic equivalents like India's Jan Aushadhi program.
Every existing solution assumes a smartphone-literate patient. Pill-reminder apps require the patient to read English, install an app, and respond to push notifications. The highest-risk users (rural, elderly, low-literacy) are excluded.
DawaiSaathi ("Medicine Companion") splits the roles. The caregiver (e.g., an adult child) snaps photos of the medicine strips to set up the schedule. The patient receives an automated phone call in their own language at dose time and simply presses 1 to confirm. No app, no reading required.
β¨ What it does
- πΈ Scan & Extract: Photograph up to 5 medicine strips at once. The AI extracts brand name, salt composition, form, MRP, expiry, and manufacturer in seconds.
- π Caregiver UI languages: Reviewed interface dictionaries are available in English, Hindi, and Spanish. The app only exposes a UI language after the full dictionary is checked in and tested.
- π£οΈ Spoken Reminders (IVR): Places outbound voice calls using Twilio. Hindi and English remain first-class; Bengali, Arabic, French, Portuguese, Afrikaans, Amharic, Swahili, Hausa, Yoruba, and Spanish are available for call setup. Languages without a matching Twilio
<Say>locale use generated audio rather than an unrelated fallback voice. - π¨ Safety Checks: Cross-references medications against openFDA label data to detect and clearly explain dangerous drug-drug interactions (e.g., Warfarin + Aspirin) without hallucinating.
- π° Generic Savings: Identifies identical generic medicines from India's Jan Aushadhi program and shows exactly how much money the patient could save each month.
- π Caregiver Dashboard: A beautiful, accessible Next.js web app that tracks adherence, upcoming doses, and alerts the caregiver if a dose is missed.
ποΈ Deployment and data architecture
The clean public origin is dawaisaathi.pages.dev. A lightweight Cloudflare Pages gateway forwards requests privately to the OpenNext Worker, so the stable Pages URL and Worker-only bindings can coexist.
The live deployment currently uses the isolated D1/R2 preview runtime. The Supabase Postgres + Auth + RLS migration is present in this repository but deliberately not claimed as live until a Supabase project has been linked, the migration applied, and tenant/duplicate-delivery tests have passed. This is a safety gate for health information, not a cosmetic switch.
graph TD
subgraph Data Ingestion
C[Caregiver browser / Android TWA] --> P[Cloudflare Pages gateway<br/>dawaisaathi.pages.dev]
P --> A[Next.js App Worker<br/>OpenNext on Cloudflare Workers]
T[Twilio webhooks] --> A
end
subgraph AI & Verification
A --> B[Groq API<br/>Llama-4-Scout Vision]
A --> HF[Hugging Face Inference API<br/>VITS / MMS SOTA Voice Engine]
A --> C2[Gemini / OpenAI TTS<br/>Voice Generation Fallbacks]
A --> D[openFDA API<br/>Label Grounding]
end
subgraph State & Storage
A --> E[(D1 β current isolated runtime)]
A --> F[(Private Cloudflare R2<br/>photos + generated audio)]
A --> I[(Dedicated R2 + Durable Object<br/>OpenNext incremental cache)]
end
subgraph Production data cutover
A -.- S[(Supabase Postgres + Auth)]
S -.- H[Households / members / medication records]
end
Read the production architecture and rollout gates before inviting real households.
π Getting Started
The easiest way to try DawaiSaathi is the live dashboard at dawaisaathi.pages.dev.
If you want to run it locally or deploy it yourself:
Prerequisites
- Node.js 22+
- Groq API Key: For fast Llama-4-Scout Vision extraction and LLM routing.
- OpenAI API Key: For GPT-4o-mini-tts generation.
- Twilio Account: For live IVR phone calls (simulated calls work in the browser without Twilio).
- Supabase Project: Required only for the gated Postgres/Auth cutover described in
supabase/README.md.
Running Locally
git clone https://github.com/vksh1cool/DawaiSaathi.git
cd DawaiSaathi
npm install
# Setup your .env file
cp .env.example .env
# Edit .env with your Groq, OpenAI, and (optionally) Twilio keys
# Initialise the local D1 database used by `next dev` through OpenNext
npm run d1:migrate:local
# Start the dev server
npm run dev
# In a separate terminal, run the local background worker for reminders
npm run worker
Head over to http://localhost:3000 to access the dashboard.
With DEMO_MODE=true, you can optionally seed the local D1 demo after the app is running:
npm run demo:seed
npm run db:push is still useful for standalone Prisma/SQLite scripts, but it does not initialise the D1 binding that OpenNext uses during next dev.
Deploying to Cloudflare
The production setup has two deployables on purpose: the OpenNext Worker holds the application and private bindings; the Pages project is a minimal gateway that owns the clean public URL.
- Create the Worker bindings in
wrangler.jsonc: D1 (isolated runtime), private R2 media/cache buckets, and the OpenNext cache Durable Object. - Set secrets with
wrangler secret put(AI keys, Twilio, access-gate secrets, and later Supabase credentials). Never commit them or add them tovars. - Deploy the application Worker:
npm run cf:deploy. - Create a Pages project named
dawaisaathi, then deploypages-gateway:npx wrangler pages deploy pages-gateway --project-name dawaisaathi. - Onboard launchpixel.in in Cloudflare Email Sending (SPF/DKIM) and authorize feedback@launchpixel.in. The in-app feedback button sends appreciation or feature requests to contact@launchpixel.in; until this binding is live it fails closed rather than pretending an email was sent.
- Verify https://dawaisaathi.pages.dev/api/app-info and a real, consented Twilio test call. The live build reports whether telephony is configured.
For the Supabase cutover, follow supabase/README.md first. Do not change the runtime database flag merely because credentials exist.
Twilio trial-mode rule
For this proof of concept, one Twilio sender number may serve every household, while each household's consent and reminder records remain isolated. A Twilio trial account can only contact verified destination numbers: test only with explicit consent and verified recipients. Keep caregiver sign-in OTP delivery separate from medication-reminder traffic, send no medicine names or doses by SMS, and honor STOP across the shared sender before any future message is queued.
π¬ Product feedback
The floating feedback button asks for one short thing to improve or one short thing the caregiver appreciated and why. It warns people not to include medicine names, prescriptions, phone numbers, or other health details. Messages are sent server-to-server to contact@launchpixel.in; no browser mailto link or email credential is exposed.
π οΈ Tech Stack
- Framework: Next.js 15 (App Router) running on Cloudflare Workers (via OpenNext).
- Language: TypeScript (strict).
- Current isolated data runtime: SQLite locally / Cloudflare D1 in the deployed demo.
- Production data target: Supabase Postgres + Auth + RLS, with a migration and rollout gates in
supabase/. - Storage: Private Cloudflare R2 for photos and generated TTS audio; private routes use
no-storerather than public object URLs. - Styling: Tailwind CSS 4.x with a custom accessibility-first design system.
- Framework: Next.js 15 (App Router) running on Cloudflare Workers (via OpenNext).
- Language: TypeScript (strict).
- Data runtime: SQLite locally / Supabase Postgres + Auth + RLS / Cloudflare D1.
- Storage: Private Cloudflare R2 for photos and generated TTS audio; private routes use
no-storerather than public object URLs. - Styling: Tailwind CSS 4.x with a custom accessibility-first design system.
- AI & Voice Engines: Groq API (
llama-4-scoutfor structured extraction,llama-3.3-70bfor logic), Hugging Face Inference API (espnet/kan-bayashi_ljspeech_vits&facebook/mms-tts-engSOTA voice models), Gemini Native TTS & OpenAI TTS fallbacks. - Telephony: Twilio Programmable Voice (TwiML).
- Android: Trusted Web Activity generated with Bubblewrap; automated signed APK releases (
arm64-v8a&armeabi-v7a).
π± Android APK and GitHub Releases
The Android app is versioned from android/twa-manifest.json and opens the production origin https://dawaisaathi.pages.dev. The release workflow produces signed APKs (arm64-v8a & armeabi-v7a) and an Android App Bundle (AAB), attaches SHA-256 checksums, and creates a GitHub Release for each vX.Y.Z tag (current release: v1.1.0).
π― The Demo
DawaiSaathi includes a built-in demo persona: Kamla Devi. To experience the app as a caregiver setting up medicines for an elderly patient:
- The UI will guide you through scanning a demo medicine strip, checking interactions, and initiating a simulated phone call.
- You can hear exactly what Kamla Devi hears in Hindi with warm humanizer voice instructions and confirm the dose by pressing 1.
πΊοΈ Roadmap & Status
| Feature | Status | Details |
|---|---|---|
| Android APK (TWA) | β
Live (v1.1.0) | Automated signed release pipeline building arm64-v8a & armeabi-v7a APKs on GitHub Releases. |
| Supabase Tenant Architecture | β Live | Supabase Auth, household RLS security policies, caregiver invitations, and Postgres data layer fully integrated. |
| Hugging Face SOTA Voice Engine | β Live | Free open-source TTS voice models (espnet/kan-bayashi_ljspeech_vits & facebook/mms-tts-eng) with automatic cold-boot retry. |
| Jan Aushadhi Kendra Locator | β Live | Direct integrated store locator linking to https://janaushadhi.gov.in. |
| WhatsApp Bot Integration | β³ Planned | Send text-based alerts to the caregiver if a dose is missed. |
| Pharmacy Ordering | β³ Planned | Direct integration to re-order medicines when a strip is running low. |
Have an idea to improve DawaiSaathi? Open an issue or submit a pull request!
Analysis
View
Metric
- 37
- 4
- 1
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- CSSIn code
- HTMLIn code
- JavaIn code
- JavaScriptIn code
- Next.jsIn code
- OpenAIIn code
- PostgreSQLIn code
- ReactIn code
- SQLIn code
- SupabaseIn code
- Tailwind CSSIn code
- TypeScriptIn code
- Google GeminiClaimed
12 of 13 appear in the indexed code. 1 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
- Claude CodeCommits
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
5.2 MB
Source files
464
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
vksh1cool/DawaiSaathi
725 files Β· 15.8 MB Β· @ eee6f70
Structure
Interface
76 files Β· 10%Screens, components and styles rendered to the user.
API & routing
204 files Β· 28%Request entry points: routes, handlers and controllers.
Application logic
94 files Β· 13%Domain rules, services and shared utilities.
+5 moreBackground jobs
3 files Β· 0%Work run outside a request: tasks, workers and schedules.
Data & schema
26 files Β· 4%Schema definitions, migrations and data access.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here β open the file browser to check anything the diagram implies.
Languages
- JavaScript50%
- TypeScript37%
- HTML5%
- Markdown3%
- CSS2%
- SQL2%
- Other (3)1%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
package.json
npm Β· 33- @opennextjs/cloudflare
- @prisma/adapter-d1
- @prisma/adapter-pg
- @prisma/client
- @supabase/ssr
- @supabase/supabase-js
- dotenv
- fastest-levenshtein
- lucide-react
- luxon
- next
- openai
- papaparse
- pg
- pino
- react
- react-dom
- server-only
- +15 more
Declared in the repositoryβs manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This projectβs features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.