# Project export: Iatric AI

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: UC Berkeley AI Hackathon 2025
- Tagline: Bringing clarity and truth to every checkup.
- Devpost: https://devpost.com/software/iatric-ai
- GitHub: https://github.com/Nosp10/IactricAI-CalHacks
- Team: 2 GitHub contributor(s) — no_sp (10 commits), Sanjith Shanmugavel (3 commits)

## Devpost submission (written by the team)

### Inspiration

The inspiration for Iatric AI came from one of our teammates' grandfather in India, who, during his final years under Medicare, was subjected to excessive screenings and placed on unnecessary medications. As his health declined, he watched him go through procedures that felt more routine than necessary, often without clear explanations. That experience pushed him to explore the broader issue of medical overreach — specifically how misdiagnosis and vague communication can lead patients to worry about conditions they may not even have. Through his research, he learned that nearly 800,000 people suffer serious harm from diagnostic errors each year, with 50 to 100 million Americans affected, and over 371,000 deaths and 424,000 permanent disabilities tied to misdiagnosis. We built Iatric AI to give patients a way to understand, verify, and reflect on what’s actually said during medical visits — so they’re not just passive listeners, but informed participants in their care. This tool is for anyone, young or old, who deserves clarity in moments that deeply impact their health and peace of mind.

### What it does

This app empowers patients to take control of their health by understanding and evaluating the accuracy of medical advice. It analyzes the claims a doctor makes during an appointment, flagging them as Verified, Unverified, or Vague based on trusted medical papers and resources to help prevent misdiagnosis and medical malpractice. The app also records the visit, provides a full transcript, summarizes key information, and translates everything into the patient’s preferred language.

### How we built it

Frontend: Typescript, CSS, Tailwind CSS Framer for aniomations Figma for design and. color pallette selection Backend: Websockets, FastAPI and Python for audio retrieval, processing, and sending claim data. Groq-Whisper transcriber for low-latency Speech to Text Custom NLP pipeline with spaCy to extract medical claims from transcript Groq LLaMA-3 model to identify medical specialty from conversation context PubMed via PyMed to verify claims with relevant scientific research Google Gemini AI SDK & Anthropic SDK for claim verification Presidio Analyzer & Anonymizer for patient confidentiality

### Challenges we ran into

The biggest challenge we faced was integrating the backend with the frontend. We developed both sides independently, building out all the components needed for a complete backend and a complete frontend. However, when it came time to connect them into a full-stack application, we ran into issues transferring information between them in real time. Despite our efforts, we couldn’t establish seamless communication, and ultimately had to present the backend and frontend as separate, non-interacting parts of the project.

### Accomplishments we're proud of

We’re proud to have performed thorough sanitization and anonymization of the data to protect sensitive information when using LLM APIs, while also developing a strong algorithm that leverages prompt engineering and voice transcription to accurately verify the validity of a doctor’s claims.

### What we learned

We learned that planning the communication between the frontend and backend should come before building them independently. Establishing a clear framework for how data would be transferred between the two would have saved time and prevented integration issues later on. In future projects, we'll prioritize setting up this structure early to ensure a more seamless full-stack development process.

### What's next

We want to continue developing Iatric AI into a complete health assistant that truly advocates for the patient. Our goal is to create a tool that serves only the individual, with no obligations to private insurers, healthcare managers, or anyone whose priorities might conflict with the patient’s best interests. We envision Iatric AI as a trusted companion that empowers people to make informed, confident decisions about their health.

## README (from the GitHub repository)

# IactricAI-CalHacks
Building a real-time...


## Detected evidence (automated analysis)

Indexed codebase: 34 recognized source files, 97 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code

## Codebase structure (from repository index)

### Files (37 of 37)

```
.gitignore
index.html
main.py
meeting_summary.py
opinion_query.py
package.json
postcss.config.js
README copy.md
README.md
src/App.tsx
src/components/Button.tsx
src/components/ClaimAnalysis.tsx
src/components/ClaimCard.tsx
src/components/DoctorCard.tsx
src/components/HistoryCard.tsx
src/components/LanguageIndicator.tsx
src/components/LanguageToggle.tsx
src/components/Layout.tsx
src/components/NavBar.tsx
src/components/RecordButton.tsx
src/components/SecondOpinion.tsx
src/components/ShareButton.tsx
src/components/TranscriptBubble.tsx
src/components/TranslatingHeader.tsx
src/index.css
src/main.tsx
src/pages/HistoryPage.tsx
src/pages/LandingPage.tsx
src/pages/RecordingPage.tsx
src/pages/SettingsPage.tsx
src/pages/SummaryPage.tsx
src/vite-env.d.ts
tailwind.config.js
test_client.py
tsconfig.json
verify.py
vite.config.ts
```

### Dependencies

- package.json: @getmocha/vite-plugins@latest, @types/react@19.0.10, @types/react-dom@19.0.4, @vitejs/plugin-react@^4.3.1, autoprefixer@^10.4.20, framer-motion@^12.18.1, globals@^15.9.0, lucide-react@^0.510.0, postcss@^8.4.47, react@19.0.0, react-dom@19.0.0, react-icons@^5.5.0, react-router-dom@^7.6.2, tailwindcss@^3.4.14, typescript@^5.5.3, vite@^6.2.1

### Recent commits (newest first)

- unable to decode verify
- add summary - need to display
- Meeting/appointment summary tool
- received audio via websocket
- commit 6
- fixed tel
- added frontend
- commit 5
- commit 4
- removed env
- smt
- new fixed - version
- commit 3
- added main arch - need to link front and backend
- verify2
- Modifying doctor query to include specialty-based search
- Doctor Location API Req Test
- verify1
- initial
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### README copy.md

```markdown
## Iatric AI

This app was created using https://getmocha.com.
Need help or want to join the community? Join our [Discord](https://discord.gg/shDEGBSe2d).

To run the devserver:
```
npm install
npm run dev
```

```

### package.json

```
{
  "name": "mocha-app",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "tsc -b && vite build",
    "lint": "eslint .",
    "preview": "vite preview"
  },
  "dependencies": {
    "framer-motion": "^12.18.1",
    "lucide-react": "^0.510.0",
    "react": "19.0.0",
    "react-dom": "19.0.0",
    "react-icons": "^5.5.0",
    "react-router-dom": "^7.6.2"
  },
  "devDependencies": {
    "@getmocha/vite-plugins": "latest",
    "@types/react": "19.0.10",
    "@types/react-dom": "19.0.4",
    "@vitejs/plugin-react": "^4.3.1",
    "autoprefixer": "^10.4.20",
    "globals": "^15.9.0",
    "postcss": "^8.4.47",
    "tailwindcss": "^3.4.14",
    "typescript": "^5.5.3",
    "vite": "^6.2.1"
  }
}

```

### main.py

```python
import os
import json
import base64
from groq import Groq
import websockets
from fastapi import FastAPI, WebSocket, Request, WebSocketDisconnect
from fastapi.responses import HTMLResponse, JSONResponse, PlainTextResponse
from fastapi.websockets import WebSocketDisconnect
import asyncio
import tempfile
from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine
import anthropic
import spacy
from typing import List
import google.generativeai as genai
from pymed import PubMed
from verify import get_results, get_verification_info, generate_validation
from opinion_query import get_doctors
from meeting_summary import summarize_transcript


nlp = spacy.load("en_core_web_sm")
from dotenv import load_dotenv
load_dotenv()

ant_api_key = os.getenv("ANTHROPIC_API_KEY")
ant_client = anthropic.Anthropic(api_key=ant_api_key)
api_key = os.getenv("GOOGLE_API_KEY")
pubmed = PubMed(tool="MyTool", email="disispavank@gmail.come")
client = Groq(api_key=os.getenv("GROQ_API_KEY"))
TRANSCRIPTS=[]

app = FastAPI()

# async def anonymize(text) -> str:
#     analyzer = AnalyzerEngine()
#     anonymizer = AnonymizerEngine()
#     results = analyzer.analyze(text=text, language='en')
#     anonymized_result = anonymizer.anonymize(text=text, analyzer_results=results)
#     return anonymized_result.text



async def get_claims(transcript: str) -> List[str]:
    LINK_PHRASES = [
        "is a sign of", "is related to", "may indicate", "could suggest",
        "consistent with", "might be due to", "is due to", "is because of"
    ]
    doc = nlp(transcript.lower())
    claims = []

    for sent in doc.sents:
        for phrase in LINK_PHRASES:
            if phrase in sent.text:
                claims.append(sent.text.strip())
                break

    return claims


async def find_doctors(transcripts: list, zipcode: str):
    """Get_specialty and pass that into get_doctors -> return json format and send back"""
        # {
        #     "name": name,
        #     "specialty": specialty,
        #     "address": address_str
        # }
    seperator = " "
    conversation_history = seperator.join(transcripts)

    chat_completion = client.chat.completions.create(
        messages=[
            {
                "role": "system",
                "content": "You are a clinical language expert assisting in specialist referrals."
            },
            {
                "role": "user",
                "content": f"""Given the following conversation between a doctor and a patient, analyze the medical content and determine the most relevant medical specialty associated with the core issue discussed. Your output should be a single medical specialty term (e.g., "cardiology", "neurology", etc.). Do not include explanations, punctuation, or formatting.

    ---

    {conversation_history}"""
            }
        ],
        model="llama-3.3-70b-versatile"
    )

    # to do - > zipcode retrival

    specialty = chat_completion.choices[0].message.content.strip()
    doctor_info = get_doctors("60540", specialty)

    # to do -> send to front end here json.dumps



async def transcribe_audio(audio_chunk: bytes) -> str:
    with tempfile.NamedTemporaryFile(suffix=".wav", delete=True) as temp_audio:
        temp_audio.write(audio_chunk)
        temp_audio.flush()

        with open(temp_audio.name, "rb") as f:
            translation = client.audio.translations.create(
                file=(temp_audio.name, f),
                model="whisper-large-v3",
                prompt="use clinical terminology and correct spelling for conditions, medications, and procedures in a medical context",
                response_format="json",
                temperature=0.0 
            )

        print(translation.text)
        TRANSCRIPTS.append(translation.text)
        return translation.text

@app.websocket("/ws/audio")
async def audio_stream(websocket: WebSocket):
    await websocket.accept()
    try:
        while True:
            audio_chunk = await websocket.receive_bytes()
            print("RECEIVED AUDIO***************")
            transcription = await transcribe_audio(audio_chunk)

            print(transcription)
            claims = await get_claims(transcription)
            verification_data = []
            for claim in claims:
                verif_data = await get_verification_info(claim)
                await websocket.send_json({
                    "type": "verification",
                    "data": verif_data
                })

    except WebSocketDisconnect:
        print("Client disconnected")
    except Exception as e:
        print("Error:", e)
        await websocket.close()
    # finally:
        # sep = " "
        # transcript_str = sep.join(TRANSCRIPTS)
        # bullet_pt_str = await summarize_transcript(transcript_str)

        # bullet_pt_dict = {}

        # bullet_pt_dict["summary"] = bullet_pt_str
        # bullet_pt_dict["type"] = "summary"

        # await websocket.send_json({
        #     bullet_pt_dict
        # })

        # await find_doctors(TRANSCRIPTS)


```

### src/main.tsx

```typescript
import { StrictMode } from 'react'
import { createRoot } from 'react-dom/client'
import App from './App.tsx'
import './index.css'

createRoot(document.getElementById('root')!).render(
  <StrictMode>
    <App />
  </StrictMode>,
)

```

### src/App.tsx

```typescript
import { useEffect } from 'react';
import { BrowserRouter, Routes, Route, Navigate } from 'react-router-dom';
import { Layout } from './components/Layout';
import { LandingPage } from './pages/LandingPage';
import { RecordingPage } from './pages/RecordingPage';
import { SummaryPage } from './pages/SummaryPage';
import './index.css';

export function App() {
  useEffect(() => {
    // Add Google Fonts
    const link = document.createElement('link');
    link.href = 'https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=Poppins:wght@400;500;600;700&display=swap';
    link.rel = 'stylesheet';
    document.head.appendChild(link);

    return () => {
      // Clean up
      if (document.head.contains(link)) {
        document.head.removeChild(link);
      }
    };
  }, []);

  return (
    <BrowserRouter>
      <Layout>
        <Routes>
          <Route path="/" element={<LandingPage />} />
          <Route path="/record" element={<RecordingPage />} />
          <Route path="/summary" element={<SummaryPage />} />
          <Route path="*" element={<Navigate to="/" replace />} />
        </Routes>
      </Layout>
    </BrowserRouter>
  );
}

export default App;

```

### postcss.config.js

```javascript
export default {
  plugins: {
    tailwindcss: {},
    autoprefixer: {},
  },
}

```

### vite.config.ts

```typescript
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";
import { mochaPlugins } from "@getmocha/vite-plugins";

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [...mochaPlugins(process.env), react()],
  server: {
    allowedHosts: true,
  },
  build: {
    chunkSizeWarningLimit: 5000,
  },
});

```

### tailwind.config.js

```javascript
/** @type {import('tailwindcss').Config} */
export default {
  content: [
    "./index.html",
    "./src/**/*.{js,ts,jsx,tsx}",
  ],
  theme: {
    extend: {
      colors: {
        'indigo': {
          50: '#eef2ff',
          100: '#e0e7ff',
          200: '#c7d2fe',
          300: '#a5b4fc',
          400: '#818cf8',
          500: '#6366f1',
          600: '#4f46e5',
          700: '#4338ca',
          800: '#3730a3',
          900: '#312e81',
          950: '#1e1b4b',
        },
      },
      fontFamily: {
        sans: ['Inter', 'system-ui', 'sans-serif'],
        display: ['Poppins', 'Inter', 'system-ui', 'sans-serif'],
      },
      boxShadow: {
        'soft': '0 2px 15px -3px rgba(0, 0, 0, 0.07), 0 10px 20px -2px rgba(0, 0, 0, 0.04)',
      },
      animation: {
        'bounce-slow': 'bounce 2s infinite',
      },
    },
  },
  plugins: [],
}

```

### opinion_query.py

```python
import requests


async def get_doctors(zip_cde, specialty):
    url = "https://npiregistry.cms.hhs.gov/api/"
    params = {
        "version": "2.1",
        "postal_code": zip_cde,
        "taxonomy_description": specialty,
        "pretty": "on"
    }

    response = requests.get(url, params=params)
    data = response.json()
    results = data.get("results", [])

    for provider in results:
        basic = provider.get("basic", {})
        name = f"{basic.get('first_name', '')} {basic.get('last_name', '')}".strip()
        if not name:
            name = basic.get("organization_name", "Unknown Provider")
            taxonomy_list = provider.get("taxonomies", [])
            specialty = taxonomy_list[0]["desc"] if taxonomy_list else "Unknown Specialty"

        addresses = provider.get("addresses", [])
        loc_address = next((a for a in addresses if a["address_purpose"] == "LOCATION"), {})
        address_str = f"{loc_address.get('address_1', '')}, {loc_address.get('city', '')}, {loc_address.get('state', '')} {loc_address.get('postal_code', '')}"
        print(f"{name} ({specialty}) — {address_str}")

        return_dict = {
            "name": name,
            "specialty": specialty,
            "address": address_str
        }
        
        return return_dict
```

### meeting_summary.py

```python
import os
from dotenv import load_dotenv
import anthropic

load_dotenv()

api_key = os.getenv("ANTHROPIC_API_KEY")

client = anthropic.Anthropic(api_key=api_key)

async def summarize_transcript(text: str) -> str:
    prompt = (
        "The following is a transcript of a conversation between a doctor and a patient.\n"
        "Please write a clear, concise summary of the key medical information discussed, using layman's terms where possible but without omitting important clinical details.\n\n"
        "Your summary should include:\n"
        "- The patient's main concern or symptoms\n"
        "- Any diagnosis or suspected condition the doctor mentioned\n"
        "- Any tests or procedures that were ordered or discussed\n"
        "- Treatment plans, prescriptions, or instructions given by the doctor\n"
        "- Follow-up recommendations (e.g., next visit, warning signs to watch for)\n\n"
        "Format the summary in 3 to 5 bullet points. Prioritize clarity and accuracy.\n\n"
        f"{text}\n\n"
        "Summary:"
    )


    response = client.messages.create(
        model="claude-3-haiku-20240307",
        max_tokens=150,
        temperature=0.5,
        messages=[
            {"role": "user", "content": prompt}
        ]
    )

    return response.content[0].text.strip()

```

[24 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]