# Project export: scanOTiC This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2026 - Tagline: scanOTiC turns your phone into a personal pharmacist: scan/search OTCs, get a personalized interface for usage against your meds, allergies, and conditions. Powered by openFDA and on-device AI. - Devpost: https://devpost.com/software/scanotic - GitHub: https://github.com/stevenliii/scanotic - Video: https://www.youtube.com/embed/IJo2KHMNZIc?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 4 GitHub contributor(s) — rrhzhang (22 commits), johndoan09 (9 commits), stevenliii (6 commits), Claude Sonnet 4.6 (5 commits) ## Devpost submission (written by the team) ### Overview The Problem As a person stands in a pharmacy aisle trying to determine the safety of an OTC medication, the medical information required is available; however, it is obscured by: Microscopic typeface on FDA labeling, which is effectively useless for the aged and visually challenged Jargon-laden medical language written for liability, rather than comprehension by humans Confusing brand name labeling that obscures common active ingredients Danger of polypharmacy, where drug interactions increase with use of multiple medications Why Does This Matter? This is not a small problem. It is a systemic problem of health equity: ~1.3 million visits to emergency rooms each year in the US are due to adverse drug events, which often involve OTC products that the patients thought were safe. Unintentional overdose of acetaminophen is the most common cause of acute liver failure, often resulting from inadvertently using several brands of the same medication. Seniors suffer the most. 4+ medications per day, the weakest vision, the highest potential for interaction and the poorest readability of labels. Differences in health literacy correlate with demographics. Non-native speakers of English and poorer people pay the price of jargon-filled labels. All the necessary information to avoid all these issues already exists in public databases. The problem is not the lack of data, but accessibility, translation, and customization. And so we developed a digital patient advocate. What Does Our App Do? Three taps: Onboard once: name, age, allergies, medications, conditions, lifestyle factors Point at any OTC barcode Get a Red / Yellow / Green verdict in seconds, plus a personal note in plain second-person English, active ingredients, common side effects, and same-class alternatives if you should pick something else Every result is personalized. A scan of the same Advil box returns: GREEN for a healthy adult YELLOW for a warfarin patient, with bleeding risk explained by name RED for someone allergic to ibuprofen, and the LLM is not allowed to override it How We Built It The analysis pipeline is layered: Deterministic allergy check runs first → produces a hard verdict ceiling the LLM cannot soften Gemma 3 then layers nuanced condition and interaction concerns from the label Challenges We Ran Into Messiness in openFDA’s actual data: Arrays posing as strings, wandering parallel arrays, four overlapping pharmacological classes Format problem with Barcode: iPhones give 12 digits; openFDA keeps 13 Maintaining a 4B-parameter local model’s honesty: It will conjure problems for itself, exhaust its context window with 5KB warnings, and wrap JSON in defensive language unless you reign it in completely What We Learned Put a deterministic layer below the LLM. Using a Python max() on two risk levels is better than a "be careful" instruction to the LLM. Observe the context window limitations. Now each Drug Facts is limited to 1200 characters. APIs should be translated when they cross the boundary. Everything from openFDA is parsed to typed dataclasses. Healthcare reinterprets every default setting. Unknown verdict equals YELLOW and not GREEN because you optimize for being confidently wrong. What's Next Supplements + prescriptions broaden the corpus beyond OTC RxNav drug-drug interactions as a second deterministic layer Full accessibility: VoiceOver / TalkBack, TTS read-aloud, dynamic type Multilingual: Spanish, Mandarin, Vietnamese Household profile sharing for caregivers managing elderly parents "Share this with my pharmacist" button that closes the loop honestly ## README (from the GitHub repository) # scanOTiC **A digital patient advocate for the pharmacy aisle — point your phone at any OTC medication and get a personalized, plain-English safety verdict in seconds.** scanOTiC onboards your health profile once (name, age, allergies, medications, conditions, lifestyle), then turns any OTC barcode into a Red / Yellow / Green verdict tailored to *you* — with a plain second-person note, active ingredients, common side effects, and safer same-class alternatives when you should pick something else. A deterministic safety layer runs underneath a local LLM, and everything stays on your device. --- ## The problem As a person stands in a pharmacy aisle trying to determine the safety of an OTC medication, the information they need already exists — but it's obscured by: - **Microscopic typeface** on FDA labeling, effectively useless for the aged and visually impaired. - **Jargon-laden medical language** written for liability rather than human comprehension. - **Confusing brand-name labeling** that hides common active ingredients. - **The danger of polypharmacy**, where interaction risk compounds as the number of medications grows. ### Why does this matter? This is not a small problem. It is a systemic problem of health equity: - **~1.3 million ER visits each year** in the US are due to adverse drug events, which often involve OTC products that patients thought were safe. - **Unintentional acetaminophen overdose is the most common cause of acute liver failure**, often from inadvertently combining several brands of the same active ingredient. - **Seniors suffer the most** — 4+ medications per day, the weakest vision, the highest interaction potential, and the poorest label readability. - **Health literacy correlates with demographics.** Non-native English speakers and lower-income patients pay the price of jargon-filled labels. All the information needed to avoid these issues already exists in public databases. The problem is not a lack of data, but **accessibility, translation, and customization**. So we built a digital patient advocate. --- ## What does our app do? Three taps: 1. **Onboard once** — name, age, allergies, medications, conditions, lifestyle factors. 2. **Point at any OTC barcode** (or search by name). 3. **Get a Red / Yellow / Green verdict in seconds** — plus a personal note in plain second-person English, the active ingredients, common side effects, and same-class alternatives if you should pick something else. Every result is personalized. A scan of the *same* Advil box returns: - **GREEN** for a healthy adult. - **YELLOW** for a warfarin patient, with the bleeding risk explained by name. - **RED** for someone allergic to ibuprofen — and the LLM is *not allowed* to override it. --- ## Software requirements - **Python:** 3.11 or newer (Flask backend + analysis pipeline). - **Node.js:** 18 or newer (Expo / React Native mobile app). - **Ollama:** for the local Gemma 3 model (`setup.sh` installs it if missing). - **Expo Go** on your phone, on the same Wi-Fi as your computer. - **Permissions requested at runtime:** Camera (barcode scanning). - **Optional:** a free [openFDA API key](https://open.fda.gov/apis/authentication/) — without one, anonymous traffic is capped at 1,000 requests/day per IP. No cloud, no API keys for the AI, and no health data ever leaves the device. --- ## Setup from scratch ### Prerequisites (Zero to Running Demo) To run this project from absolute scratch, you'll need the following on your machine: 1. **Git** — for version control (e.g. `brew install git` on macOS). 2. **Python 3.11+** — the backend and analysis pipeline. 3. **Node.js 18+** — the Expo mobile app. 4. **Ollama** — host for the local Gemma 3 model. `setup.sh` installs it automatically if it isn't already present. ### Step 1: Clone the repository ```sh git clone https://github.com/stevenliii/scanotic.git cd scanotic ``` ### Step 2: One-command setup Install every dependency and pull the model with a single script: ```sh ./setup.sh ``` This installs the Python package (`pip install -e .`), the mobile npm dependencies (`npm install --legacy-peer-deps`), installs Ollama if needed, and pulls the `gemma3` model locally. *Note: the first model pull downloads several GB and can take a few minutes; subsequent runs reuse it.* *(Optional, recommended)* Register a free openFDA key and export it before the demo to raise the rate limit from 1,000 to 120,000 requests/day: ```sh export OPENFDA_API_KEY=... ``` ### Step 3: Run everything ```sh ./run.sh ``` This starts Ollama (if not already running), launches Flask on port `5050`, and starts the Expo dev server. Your computer's LAN IP is auto-detected — no manual config needed. Scan the QR code with **Expo Go** to open the app on your phone. --- ## Run & usage 1. **Onboard.** On first launch, enter your name, age, allergies, current medications, conditions, and lifestyle factors. Everything is stored on-device in `AsyncStorage` — nothing is uploaded. 2. **Scan or search.** Point the camera at an OTC barcode (UPC/NDC), or tap the floating search button to type a drug name or NDC. Prefix-matched suggestions appear as you type. 3. **Read the verdict.** You get a **Red / Yellow / Green** hero verdict plus a personal note in plain second-person English, naming the specific profile items at risk. 4. **Review the details.** Active ingredients, common side effects, and per-finding interaction cards explain *why* the verdict is what it is. 5. **Pick a safer option.** When a scan isn't GREEN, scanOTiC surfaces same-class alternatives that deterministically clear the safety gate for *your* profile — never a hallucinated one. 6. **Check history.** The History tab lists your recent scans with their verdicts. 7. **Manage your profile.** Edit identity, medications, allergies, conditions, and lifestyle in the Profile tab. Tap your name to edit it, or **Reset & start over** to clear the device and return to onboarding. --- ## Features ### Personalized verdicts - **Three-tier Red / Yellow / Green verdict.** Every scan is judged against the user's specific allergies, medications, conditions, lifestyle, and age — not a generic label reading. The same product yields different verdicts for different people. - **Plain second-person personal note.** A local LLM writes a 2–3 sentence note that names the user's actual conditions/medications/allergies and the concrete risk, then closes with one actionable next step — no jargon, no headers, no filler. - **Per-finding interaction cards.** Each concern is shown as its own severity-coded card (Allergy / Severe / Moderate / Mild), so the user sees exactly which ingredient conflicts with which part of their profile. - **Deterministic safety floor the LLM cannot soften.** A Python `max()` over the deterministic verdict and the LLM verdict guarantees the model can only *escalate* risk, never downgrade it. ### Scanning & lookup - **Live barcode scanning.** Code 128 / Code 39 (NDC-on-box) and UPC-A / EAN-13 (retail) barcodes are both supported, with automatic UPC→NDC resolution. - **Barcode-format reconciliation.** iPhones return 12-digit UPC-A; openFDA stores 13-digit EAN-13 with a leading zero. The resolver tries both forms (and a leading-zero strip) so a real-world scan resolves. - **Name & NDC search with prefix autocomplete.** A single floating search control expands into a text box with live, prefix-matched suggestions backed by openFDA's catalogue. ### Recommendations - **Hallucination-proof alternatives.** Same-class candidates must clear a fully deterministic safety gate (allergy + known-interaction + Rx-only filter + age rules + label-text checks) *before* the LLM ever sees them. The LLM only ranks and writes rationale; its output is hard-filtered against that vetted set, so it can never introduce or upgrade a candidate. - **Curated interaction knowledge.** A small, auditable class-based interaction list (anticoagulants/antiplatelets + NSAIDs, MAOIs + decongestants/de [README truncated for size] ## Detected evidence (automated analysis) Indexed codebase: 45 recognized source files, 245 KB. - Flask (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - AI coding agent: Claude Code — evidence: commit authorship or trailers ## Codebase structure (from repository index) ### Files (52 of 52) ``` .gitignore app.py data/db/scanotic.db mobile/App.js mobile/app.json mobile/babel.config.js mobile/components/icons.js mobile/config.js mobile/metro.config.js mobile/package.json mobile/screens/HistoryScreen.js mobile/screens/OnboardingScreen.js mobile/screens/ProfileScreen.js mobile/screens/ResultScreen.js mobile/screens/ScannerScreen.js mobile/storage.js pyproject.toml README.md run.sh scanotic/__init__.py scanotic/cache.py scanotic/config.py scanotic/db.py scanotic/lookup/__init__.py scanotic/lookup/known_interactions.py scanotic/lookup/medication_resolver.py scanotic/lookup/product.py scanotic/lookup/rule_engine.py scanotic/lookup/side_effects.py scanotic/lookup/upc_resolver.py scanotic/openfda/__init__.py scanotic/openfda/client.py scanotic/openfda/label.py scanotic/openfda/models.py scanotic/openfda/ndc.py scripts/__init__.py scripts/seed_demo_upcs.py setup.sh templates/base.html templates/history.html templates/profile.html templates/result.html templates/scanner.html tests/__init__.py tests/fixtures/label_0573-0169.json tests/fixtures/ndc_0573-0169.json tests/test_alternative_safety.py tests/test_app_helpers.py tests/test_cache.py tests/test_openfda_parse.py tests/test_profile_llm.py tests/test_rule_engine.py ``` ### Dependencies - mobile/package.json: @babel/core@^7.20.0, @react-native-async-storage/async-storage@2.2.0, @react-navigation/bottom-tabs@^7.18.2, @react-navigation/native@^7.3.3, @react-navigation/native-stack@^7.17.5, babel-preset-expo@~54.0.11, expo@~54.0.0, expo-camera@~17.0.10, expo-constants@~18.0.13, expo-status-bar@~3.0.9, react@19.1.0, react-native@0.81.5, react-native-safe-area-context@~5.6.0, react-native-screens@~4.16.0, react-native-svg@15.12.1 - pyproject.toml: click@>=8.0, flask@>=3.0, httpx@>=0.27, tenacity@>=8.0 ### Recent commits (newest first) - prompt - deterministic - Update README.md - update readme - changed ui and updated logic - tuning model - Merge pull request #6 from stevenliii/recs - Add personalized, hallucination-proof alternative recommendations - adding profile context - Merge pull request #5 from stevenliii/design - Rebrand Verdict to Scanotic and redesign Profile screen to match Profile.dc.html - ui changes - fixed ui bugs and updated prompts - unit and integration testing - side effects - merge - updated searching - Merge pull request #4 from stevenliii/profile - Fix missing suggesting state in ScannerScreen - updated scanning ## Key source files (fetched from GitHub, selected and truncated for size) ### pyproject.toml ``` [build-system] requires = ["hatchling"] build-backend = "hatchling.build" [project] name = "scanotic" version = "0.1.0" description = "OTC Medicine Analyzer & Recommender — data layer" readme = "README.md" requires-python = ">=3.11" dependencies = [ "flask>=3.0", "httpx>=0.27", "tenacity>=8.0", "click>=8.0", ] [tool.hatch.build.targets.wheel] packages = ["scanotic"] [tool.pytest.ini_options] markers = [ "integration: requires Ollama running locally (deselect with -m 'not integration')", ] ``` ### mobile/package.json ``` { "name": "scanotic-mobile", "version": "1.0.0", "main": "node_modules/expo/AppEntry.js", "scripts": { "start": "expo start", "android": "expo start --android", "ios": "expo start --ios" }, "dependencies": { "@react-native-async-storage/async-storage": "2.2.0", "@react-navigation/bottom-tabs": "^7.18.2", "@react-navigation/native": "^7.3.3", "@react-navigation/native-stack": "^7.17.5", "expo": "~54.0.0", "expo-camera": "~17.0.10", "expo-constants": "~18.0.13", "expo-status-bar": "~3.0.9", "react": "19.1.0", "react-native": "0.81.5", "react-native-safe-area-context": "~5.6.0", "react-native-screens": "~4.16.0", "react-native-svg": "15.12.1" }, "devDependencies": { "@babel/core": "^7.20.0", "babel-preset-expo": "~54.0.11" } } ``` ### mobile/App.js ```javascript import { useEffect, useState } from 'react'; import { View } from 'react-native'; import { NavigationContainer } from '@react-navigation/native'; import { createNativeStackNavigator } from '@react-navigation/native-stack'; import { createBottomTabNavigator } from '@react-navigation/bottom-tabs'; import ScannerScreen from './screens/ScannerScreen'; import ResultScreen from './screens/ResultScreen'; import ProfileScreen from './screens/ProfileScreen'; import HistoryScreen from './screens/HistoryScreen'; import OnboardingScreen from './screens/OnboardingScreen'; import { ScanIcon, ProfileIcon, HistoryIcon } from './components/icons'; import { getProfile } from './storage'; const LIGHT_TAB_BAR = { tabBarActiveTintColor: '#14181a', tabBarInactiveTintColor: '#b3b8bd', tabBarStyle: { backgroundColor: '#fff', borderTopColor: '#ecedf0' }, }; const Stack = createNativeStackNavigator(); const Tab = createBottomTabNavigator(); function MainTabs() { return ( , }} /> , }} /> , }} /> ); } export default function App() { const [ready, setReady] = useState(false); const [hasProfile, setHasProfile] = useState(false); useEffect(() => { getProfile().then((p) => { setHasProfile(!!p); setReady(true); }); }, []); if (!ready) { return ; } return ( ); } ``` ### app.py ```python import json import logging import os import re import sys import sqlite3 from dataclasses import asdict from datetime import datetime from typing import Any, Optional from flask import Flask, render_template, request, session, redirect, url_for, jsonify sys.path.insert(0, os.path.dirname(__file__)) from scanotic.db import get_connection from scanotic.config import DB_PATH from scanotic.lookup.product import get_product, find_same_class_alternatives from scanotic.lookup.rule_engine import evaluate, evaluate_alternative_safety, ingredient_set, UserProfile, RiskLevel, Finding from scanotic.lookup.upc_resolver import resolve as resolve_upc from scanotic.lookup.medication_resolver import resolve as resolve_medication from scanotic.lookup.side_effects import get_side_effects from scanotic.openfda import ndc as openfda_ndc from scanotic.openfda.label import fetch_label_for_ndc from scanotic.openfda.models import DrugLabel, ProductRecord # LLM_LOG_LEVEL controls verbosity of the LLM call logs below: INFO (default) # logs one line per request/response with the verdict or ranking outcome; # DEBUG also dumps the full prompt and raw model output, which is what you # want when a response looks wrong and you need to see exactly what the # model was given and what it said back. logging.basicConfig( level=os.environ.get("LOG_LEVEL", "INFO").upper(), format="%(asctime)s %(levelname)s %(name)s: %(message)s", ) logger = logging.getLogger("scanotic.llm") app = Flask(__name__) app.secret_key = "scanotic-demo-2026" # --------------------------------------------------------------------------- # Hardcoded demo profile (Alex Rivera) # --------------------------------------------------------------------------- DEFAULT_PROFILE = { "name": "", "age": "", "medications": [], "allergies": [], "allergy_rxcuis": [], "conditions": [], "lifestyle": [], } # Pre-seeded demo products DEMO_PRODUCTS = [ {"ndc": "50090-1695", "label": "Tylenol Extra Strength"}, {"ndc": "0573-0169", "label": "Advil"}, {"ndc": "0067-0101", "label": "Theraflu Nighttime"}, {"ndc": "0113-0612", "label": "Claritin"}, {"ndc": "0363-0073", "label": "Pepto-Bismol"}, ] def get_profile() -> dict: return session.get("profile", DEFAULT_PROFILE) def _db() -> sqlite3.Connection: return get_connection(DB_PATH) def _build_user_profile(profile: dict) -> UserProfile: age_raw = profile.get("age", "") age = int(age_raw) if str(age_raw).isdigit() else None return UserProfile( medication_rxcuis=[m["rxcui"] for m in profile.get("medications", []) if m.get("rxcui")], medication_names=[m["name"] for m in profile.get("medications", []) if m.get("name")], allergy_rxcuis=profile.get("allergy_rxcuis", []) or [], allergy_names=profile.get("allergies", []) or [], condition_names=profile.get("conditions", []) or [], lifestyle_names=profile.get("lifestyle", []) or [], age=age, ) # --------------------------------------------------------------------------- # LLM: personalized note + structured concerns # --------------------------------------------------------------------------- # Each Drug Facts section is capped so the full payload comfortably fits in # Gemma's default 8K-token context even when the label is verbose (warnings # blocks on some OTCs run to 5KB+). 1200 chars × 14 sections ≈ 4K tokens, # leaving room for the product block, profile, instructions, and JSON reply. _MAX_LABEL_FIELD_CHARS = 1200 def _truncate(text: str, max_chars: int = _MAX_LABEL_FIELD_CHARS) -> str: if len(text) <= max_chars: return text head = text[:max_chars].rsplit(" ", 1)[0] return f"{head} …" def _label_to_payload(label: Optional[DrugLabel]) -> dict[str, Any]: if label is None: return {} out: dict[str, Any] = {} for k, v in asdict(label).items(): if not v: continue out[k] = _truncate(v) if isinstance(v, str) else v return out def _build_llm_prompt( product: ProductRecord, profile: dict, findings: list[Finding], ) -> str: meds = [m["name"] for m in profile.get("medications", [])] allergies = profile.get("allergies", []) or [] lifestyle = profile.get("lifestyle", []) or [] conditions = profile.get("conditions", []) or [] age = profile.get("age", "") or "" user_block: dict = { "medications": meds, "allergies": allergies, "conditions": conditions, "lifestyle": lifestyle, } if age: user_block["age"] = int(age) if str(age).isdigit() else age payload = { "product": { "brand_name": product.brand_name, "generic_name": product.generic_name, "dosage_form": product.dosage_form, "active_ingredients": [ {"name": s.name, "strength": s.strength} for s in product.substances ], "pharm_classes": product.pharm_classes, }, "drug_facts_label": _label_to_payload(product.label), "user": user_block, "deterministic_findings": [ {"kind": f.kind, "level": f.level.value, "description": f.description} for f in findings ], } return ( "You are a careful, plain-spoken pharmacist assistant for an OTC medication " "scanner. Personalize the verdict for THIS user using THIS product's Drug " "Facts label.\n\n" "Hard rules:\n" "- Treat the deterministic_findings list as already-confirmed risks. Don't repeat " "them as new concerns, but reflect them in your verdict.\n" "- Only flag concerns that are actually supported by the label text. Do not invent.\n" "- ONLY flag something as a concern if the label warns about something the user " "actually has: a medication they take, an allergy they have, a condition they have, " "a lifestyle factor they have, or an age-related restr [truncated — 37027 more characters] ``` ### run.sh ```shell #!/usr/bin/env bash cd "$(dirname "$0")" # Start Ollama if not already running if ! pgrep -x ollama &>/dev/null; then echo "Starting Ollama..." ollama serve &>/dev/null & OLLAMA_PID=$! sleep 2 fi LOCAL_IP=$(ipconfig getifaddr en0 2>/dev/null || echo "localhost") echo "" echo " Flask → http://$LOCAL_IP:5050" echo " Expo auto-detects this IP — no manual config needed" echo "" python app.py & FLASK_PID=$! cleanup() { kill "$FLASK_PID" 2>/dev/null [ -n "$OLLAMA_PID" ] && kill "$OLLAMA_PID" 2>/dev/null } trap cleanup EXIT INT TERM cd mobile && npx expo start ``` ### setup.sh ```shell #!/usr/bin/env bash set -e cd "$(dirname "$0")" echo ">>> Python backend" pip install -e . echo "" echo ">>> Expo mobile app" cd mobile && npm install --legacy-peer-deps cd .. echo "" echo ">>> Ollama" if command -v ollama &>/dev/null; then echo "Ollama already installed." else echo "Installing Ollama..." curl -fsSL https://ollama.com/install.sh | sh fi if ollama list 2>/dev/null | grep -q "gemma3"; then echo "gemma3 already pulled." else echo "Pulling gemma3 (this may take a few minutes)..." ollama pull gemma3 fi echo "" echo ">>> openFDA API key" if [ -n "$OPENFDA_API_KEY" ]; then echo "OPENFDA_API_KEY is set in the environment ($(echo "$OPENFDA_API_KEY" | cut -c1-6)…)." else cat <<'EOF' No OPENFDA_API_KEY found in the environment. The app will still work, but openFDA caps anonymous traffic at 1,000 requests/day per IP. Register a free key at https://open.fda.gov/apis/authentication/ and add it to your shell profile (or run.sh) before the demo: export OPENFDA_API_KEY=... EOF fi echo "" echo "Done. Run ./run.sh to start Flask + Expo." ``` ### scanotic/__init__.py ```python """scanOTiC — OTC Medicine Analyzer & Recommender data layer.""" ``` ### mobile/babel.config.js ```javascript module.exports = function (api) { api.cache(true); return { presets: ['babel-preset-expo'] }; }; ``` ### mobile/metro.config.js ```javascript const { getDefaultConfig } = require('expo/metro-config'); module.exports = getDefaultConfig(__dirname); ``` ### mobile/config.js ```javascript import Constants from 'expo-constants'; // Auto-detects the Mac's IP from Expo's dev server — no manual config needed. // Falls back to localhost for simulators. const host = Constants.expoConfig?.hostUri?.split(':').shift() ?? 'localhost'; export const API_BASE = `http://${host}:5050`; ``` [33 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]