scanOTiC

A digital patient advocate for the pharmacy aisle — point your phone at any OTC medication and get a personalized, plain-English safety verdict in seconds.

scanOTiC onboards your health profile once (name, age, allergies, medications, conditions, lifestyle), then turns any OTC barcode into a Red / Yellow / Green verdict tailored to you — with a plain second-person note, active ingredients, common side effects, and safer same-class alternatives when you should pick something else. A deterministic safety layer runs underneath a local LLM, and everything stays on your device.

---

The problem

As a person stands in a pharmacy aisle trying to determine the safety of an OTC medication, the information they need already exists — but it's obscured by:

• Microscopic typeface on FDA labeling, effectively useless for the aged and visually impaired.
• Jargon-laden medical language written for liability rather than human comprehension.
• Confusing brand-name labeling that hides common active ingredients.
• The danger of polypharmacy, where interaction risk compounds as the number of medications grows.

Why does this matter?

This is not a small problem. It is a systemic problem of health equity:

• ~1.3 million ER visits each year in the US are due to adverse drug events, which often involve OTC products that patients thought were safe.
• Unintentional acetaminophen overdose is the most common cause of acute liver failure, often from inadvertently combining several brands of the same active ingredient.
• Seniors suffer the most — 4+ medications per day, the weakest vision, the highest interaction potential, and the poorest label readability.
• Health literacy correlates with demographics. Non-native English speakers and lower-income patients pay the price of jargon-filled labels.

All the information needed to avoid these issues already exists in public databases. The problem is not a lack of data, but accessibility, translation, and customization. So we built a digital patient advocate.

---

What does our app do?

Three taps:

1. Onboard once — name, age, allergies, medications, conditions, lifestyle factors.
2. Point at any OTC barcode (or search by name).
3. Get a Red / Yellow / Green verdict in seconds — plus a personal note in plain second-person English, the active ingredients, common side effects, and same-class alternatives if you should pick something else.

Every result is personalized. A scan of the same Advil box returns:

• GREEN for a healthy adult.
• YELLOW for a warfarin patient, with the bleeding risk explained by name.
• RED for someone allergic to ibuprofen — and the LLM is not allowed to override it.

---

Software requirements

• Python: 3.11 or newer (Flask backend + analysis pipeline).
• Node.js: 18 or newer (Expo / React Native mobile app).
• Ollama: for the local Gemma 3 model (setup.sh installs it if missing).
• Expo Go on your phone, on the same Wi-Fi as your computer.
• Permissions requested at runtime: Camera (barcode scanning).
• Optional: a free openFDA API key — without one, anonymous traffic is capped at 1,000 requests/day per IP.

No cloud, no API keys for the AI, and no health data ever leaves the device.

---

Setup from scratch

Prerequisites (Zero to Running Demo)

To run this project from absolute scratch, you'll need the following on your machine:

1. Git — for version control (e.g. brew install git on macOS).
2. Python 3.11+ — the backend and analysis pipeline.
3. Node.js 18+ — the Expo mobile app.
4. Ollama — host for the local Gemma 3 model. setup.sh installs it automatically if it isn't already present.

Step 1: Clone the repository

git clone https://github.com/stevenliii/scanotic.git
cd scanotic

Step 2: One-command setup

Install every dependency and pull the model with a single script:

./setup.sh

This installs the Python package (pip install -e .), the mobile npm dependencies (npm install --legacy-peer-deps), installs Ollama if needed, and pulls the gemma3 model locally. Note: the first model pull downloads several GB and can take a few minutes; subsequent runs reuse it.

(Optional, recommended) Register a free openFDA key and export it before the demo to raise the rate limit from 1,000 to 120,000 requests/day:

export OPENFDA_API_KEY=...

Step 3: Run everything

./run.sh

This starts Ollama (if not already running), launches Flask on port 5050, and starts the Expo dev server. Your computer's LAN IP is auto-detected — no manual config needed. Scan the QR code with Expo Go to open the app on your phone.

---

Run & usage

1. Onboard. On first launch, enter your name, age, allergies, current medications, conditions, and lifestyle factors. Everything is stored on-device in AsyncStorage — nothing is uploaded.
2. Scan or search. Point the camera at an OTC barcode (UPC/NDC), or tap the floating search button to type a drug name or NDC. Prefix-matched suggestions appear as you type.
3. Read the verdict. You get a Red / Yellow / Green hero verdict plus a personal note in plain second-person English, naming the specific profile items at risk.
4. Review the details. Active ingredients, common side effects, and per-finding interaction cards explain why the verdict is what it is.
5. Pick a safer option. When a scan isn't GREEN, scanOTiC surfaces same-class alternatives that deterministically clear the safety gate for your profile — never a hallucinated one.
6. Check history. The History tab lists your recent scans with their verdicts.
7. Manage your profile. Edit identity, medications, allergies, conditions, and lifestyle in the Profile tab. Tap your name to edit it, or Reset & start over to clear the device and return to onboarding.

---

Features

Personalized verdicts

• Three-tier Red / Yellow / Green verdict. Every scan is judged against the user's specific allergies, medications, conditions, lifestyle, and age — not a generic label reading. The same product yields different verdicts for different people.
• Plain second-person personal note. A local LLM writes a 2–3 sentence note that names the user's actual conditions/medications/allergies and the concrete risk, then closes with one actionable next step — no jargon, no headers, no filler.
• Per-finding interaction cards. Each concern is shown as its own severity-coded card (Allergy / Severe / Moderate / Mild), so the user sees exactly which ingredient conflicts with which part of their profile.
• Deterministic safety floor the LLM cannot soften. A Python max() over the deterministic verdict and the LLM verdict guarantees the model can only escalate risk, never downgrade it.

Scanning & lookup

• Live barcode scanning. Code 128 / Code 39 (NDC-on-box) and UPC-A / EAN-13 (retail) barcodes are both supported, with automatic UPC→NDC resolution.
• Barcode-format reconciliation. iPhones return 12-digit UPC-A; openFDA stores 13-digit EAN-13 with a leading zero. The resolver tries both forms (and a leading-zero strip) so a real-world scan resolves.
• Name & NDC search with prefix autocomplete. A single floating search control expands into a text box with live, prefix-matched suggestions backed by openFDA's catalogue.

Recommendations

• Hallucination-proof alternatives. Same-class candidates must clear a fully deterministic safety gate (allergy + known-interaction + Rx-only filter + age rules + label-text checks) before the LLM ever sees them. The LLM only ranks and writes rationale; its output is hard-filtered against that vetted set, so it can never introduce or upgrade a candidate.
• Curated interaction knowledge. A small, auditable class-based interaction list (anticoagulants/antiplatelets + NSAIDs, MAOIs + decongestants/dextromethorphan, SSRIs/SNRIs + NSAIDs, lithium/methotrexate + NSAIDs, sedation stacking, …) catches dangerous pairs even when a product's openFDA label has no drug_interactions field.

Privacy & profile

• On-device profile. Name, age, allergies, medications, conditions, and lifestyle live in AsyncStorage. The profile is sent with each scan request but never persisted server-side beyond the live session.
• One-tap reset. "Reset & start over" clears the on-device profile and scan history and returns to the welcome flow.

Engineering

• Deterministic layer below the LLM. Allergy intersection, known-interaction pairs, and label-text matches produce a hard verdict ceiling before Gemma 3 is consulted; the model layers nuance on top but cannot lower the floor.
• Typed openFDA boundary. Every openFDA response is parsed into typed dataclasses (ProductRecord, Substance, DrugLabel) at the boundary, taming arrays-posing-as-strings and wandering parallel arrays before any business logic runs.
• Context-window discipline. Each Drug Facts section is capped at 1,200 characters so verbose labels (5 KB+ warnings blocks) never blow Gemma's context window.
• Output hardening. LLM JSON is tolerantly parsed (fenced or bare), verdicts default conservatively, and notes are sanitized to strip leaked HTML/markup and trailing fragments before display.
• Local-first, offline-tolerant cache. openFDA responses are cached in SQLite, and the demo bottles' UPC→NDC mappings are pre-seeded so a demo survives flaky conference Wi-Fi.

---

How we built it

Layer	Stack
Mobile	Expo / React Native; on-device profile in AsyncStorage
Backend	Flask + openFDA drug catalog (/drug/ndc.json) + Drug Facts labels (/drug/label.json)
AI	Gemma 3 running locally via Ollama — no cloud, no API keys, no health data leaving the device
Cache	SQLite + ~80 pre-seeded UPC mappings → the demo works on flaky Wi-Fi

The analysis pipeline is layered:

1. Deterministic checks run first → allergy intersection + known-interaction pairs + label-text matches produce a hard verdict ceiling the LLM cannot soften.
2. Gemma 3 then layers nuance → condition- and interaction-specific concerns drawn from the actual label text, plus the plain-English note.

---

Architecture (brief)

scan input (UPC / NDC / drug name)
    │
    ├──► resolve via openFDA (/drug/ndc.json + /drug/label.json, SQLite-cached)
    │         └──► typed dataclasses (ProductRecord · Substance · DrugLabel)
    │
    ├──► deterministic rule engine
    │         ├── allergy intersection (name + RxCUI)
    │         ├── known-interaction pairs (curated, class-based)
    │         └── label-text condition / lifestyle / interaction matches
    │                   └──► HARD verdict ceiling
    │
    └──► Gemma 3 (local, via Ollama)
              ├── nuanced concerns from label text
              └── plain second-person personal note
                        │
                        └──► final verdict = max(deterministic, LLM)   ← LLM can only escalate
                                  │
                                  ├──► Red / Yellow / Green hero + interaction cards
                                  ├──► active ingredients + common side effects
                                  └──► alternatives gate (deterministic-safe candidates → LLM ranks only)

---

Data source & configuration

scanOTiC queries the openFDA Drug API live for every product (/drug/ndc.json for catalogue info and /drug/label.json for the consumer Drug Facts content). Responses are cached in a small SQLite file at data/db/scanotic.db, which is regenerated automatically on first launch — there is no build step.

The demo bottles' UPC→NDC mappings (~80 SKUs) are pre-seeded so they resolve without a live call. Re-seed with:

python scripts/seed_demo_upcs.py

Environment variables

Variable	Description
OPENFDA_API_KEY	Optional openFDA API key. Without one you get 1,000 req/day; with one, 120,000/day.
OPENFDA_CACHE_TTL_DAYS	How long cached openFDA responses stay fresh. Default: 7.
GEMMA_MODEL	Gemma model variant to use. Default: gemma3.
LOG_LEVEL	Backend log verbosity (INFO default; DEBUG dumps full prompts and raw model output).

Gemma runs fully locally — no API key needed. If it's unavailable, the app falls back to template-generated notes.

---

Challenges we ran into

• openFDA's actual data is messy. Arrays posing as strings, wandering parallel arrays, and four overlapping pharmacological classes all had to be normalized before any logic could trust the input.
• Barcode format mismatch. iPhones return 12-digit UPC-A; openFDA keeps 13-digit EAN-13. The resolver reconciles both.
• Keeping a 4B-parameter local model honest. Gemma will conjure problems for itself, exhaust its context window with 5 KB warnings, and wrap JSON in defensive prose unless reined in completely.

What we learned

• Put a deterministic layer below the LLM. A Python max() over two risk levels beats a "be careful" instruction to the model.
• Respect the context window. Each Drug Facts section is now capped at 1,200 characters.
• Translate APIs at the boundary. Everything from openFDA is parsed into typed dataclasses before it touches business logic.
• Healthcare reinterprets every default. An unknown verdict is YELLOW, not GREEN — you optimize against being confidently wrong.

What's next

• Supplements + prescriptions to broaden the corpus beyond OTC.
• RxNav drug-drug interactions as a second deterministic layer.
• Full accessibility — VoiceOver / TalkBack, TTS read-aloud, dynamic type.
• Multilingual — Spanish, Mandarin, Vietnamese.
• Household profile sharing for caregivers managing elderly parents.
• "Share this with my pharmacist" button that closes the loop honestly.

---

Tests

The project ships 62 Python unit tests covering the pure-Python analysis pipeline — parsers, rule engine, alternatives gate, LLM helpers, and cache. No device, emulator, or running Ollama is required (LLM-dependent paths are mocked).

# Run the full unit test suite
python -m pytest tests/ -q

# Expected output: 62 passed

What's covered

Test file	Component tested
test_rule_engine.py	Deterministic verdict engine — allergy intersection by name & RxCUI, case-insensitivity, multiple hits, empty-profile GREEN
test_alternative_safety.py	Alternatives safety gate — allergy/known-interaction exclusion, same-class NSAID rejection, label-text interaction catch, condition match, self-match dedupe, LLM fallback & hallucinated-NDC drop
test_app_helpers.py	LLM response parsing & card building — fenced/bare JSON tolerance, verdict normalization, concern parsing, note sanitization, interaction-card dedupe
test_openfda_parse.py	openFDA parsers — substance/parallel-array parsing, pharm-class concatenation, identifier-array helpers, salt-form handling
test_profile_llm.py	Profile → LLM wiring — profile flows into the prompt, deterministic findings passed through, verdict escalation via max(), GREEN skips alternatives
test_cache.py	SQLite openFDA response cache — set/get round-trip, TTL freshness/expiry

---

References

• openFDA Drug API — drug catalogue and labeling endpoints
• openFDA /drug/ndc.json — National Drug Code directory
• openFDA /drug/label.json — consumer Drug Facts labeling
• openFDA API authentication — free API keys & rate limits
• Ollama — local LLM runtime
• Gemma 3 model card — the local model behind the personalized notes
• Expo / React Native — the mobile app framework
• RxNorm / RxNav — drug naming and interaction data (planned second deterministic layer)

---

Team

Name	Email
John Doan	johndoaneo@gmail.com
Steven Li	stevenli45678@gmail.com
Ronald Zhang	ronaldarezhang@gmail.com