# Project export: BridgeMed-AI

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: UC Berkeley AI Hackathon 2025
- Tagline: BridgeMed-AI empowers healthcare with AI-driven translation, transcription, and 3D anatomy, which bridges communication gaps for patients and providers in an all in one intuitive website.
- Devpost: https://devpost.com/software/bridgemed-ai
- GitHub: https://github.com/Htetty/BridgeMed-AI
- Team: 2 GitHub contributor(s) — Htetty (4 commits), GiankyleL (1 commits)

## Devpost submission (written by the team)

### Inspiration

As immigrant children, we're aware of the difficulties our parents faced when communicating in English. This is especially apparent when talking about more complicated information, like their health. According to National Polling results, there are over 26 million people who have limited English proficiency. Additionally, about a third of adults with limited English proficiency say that they have faced language barriers when seeking healthcare. And during doctor's appointments, some of these non-English speaking patients are stuck with either inadequate understandings of translations or even no translations. They would have a hard time grasping words or phrases that are too technical for them to understand. Thus, this inspired us to create something that can bridge that communication gap and allow doctors and patients to understand each other easily.

### What it does

Voice-to-Text Transcriptions: Doctors can report on the patient's situation by speaking to them in English. This report will be saved in the patient's appointment tab, which is translated to their native language. This report will be saved in the patient's appointment tab, which is translated to their native language. Voice-to-Voice Transcriptions: As the doctor finishes their sentences, their voice is immediately translated to the patient's native language. 3D Anatomy Visualizations: Allows the patient to actively see what part of the body that doctor is talking about, as the 3D model dynamically changes through the doctor's responses.

### How we built it

The project is primarily rooted within the React.js framework in the frontend, and Python in the backend. We used Vapi's API in combination with OpenAI's models to enable both voice-to-voice interactions and voice-to-text transcription. We used Gemini API to simplify and produce text-to-text translations. Also, we referred to BioDigital to procure and implement a 3D human anatomy model.

### Challenges we ran into

Correctly implementing APIs and understanding their respective documentations Integrating the frontend and the backend to work flawlessly Keeping the UI/UX intuitive for both medical professionals and patients, as well as making it highly accessible for those who are less tech-savvy

### Accomplishments we're proud of

We successfully created a working prototype that can handle voice input, translate it in real-time, and present it both as text and voice in another language Implementing a dynamic, real-time anatomy visualization based on real-time medical discussions

### What we learned

Implementing smooth and accurate voice interactions Balancing features we wanted to add and what was more practical for our users

### What's next

Expanding our translation features to support more dialects, particularly underrepresented ones, will help reach more communities. Encourage more open and personal communication between patients and their doctor.

## README (from the GitHub repository)

# Getting Started with Create React App

This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).

## Available Scripts

In the project directory, you can run:

### `npm start`

Runs the app in the development mode.\
Open [http://localhost:3000](http://localhost:3000) to view it in your browser.

The page will reload when you make changes.\
You may also see any lint errors in the console.

### `npm test`

Launches the test runner in the interactive watch mode.\
See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.

### `npm run build`

Builds the app for production to the `build` folder.\
It correctly bundles React in production mode and optimizes the build for the best performance.

The build is minified and the filenames include the hashes.\
Your app is ready to be deployed!

See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.

### `npm run eject`

**Note: this is a one-way operation. Once you `eject`, you can't go back!**

If you aren't satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project.

Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you're on your own.

You don't have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn't feel obligated to use this feature. However we understand that this tool wouldn't be useful if you couldn't customize it when you are ready for it.

## Learn More

You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).

To learn React, check out the [React documentation](https://reactjs.org/).

### Code Splitting

This section has moved here: [https://facebook.github.io/create-react-app/docs/code-splitting](https://facebook.github.io/create-react-app/docs/code-splitting)

### Analyzing the Bundle Size

This section has moved here: [https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size](https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size)

### Making a Progressive Web App

This section has moved here: [https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app](https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app)

### Advanced Configuration

This section has moved here: [https://facebook.github.io/create-react-app/docs/advanced-configuration](https://facebook.github.io/create-react-app/docs/advanced-configuration)

### Deployment

This section has moved here: [https://facebook.github.io/create-react-app/docs/deployment](https://facebook.github.io/create-react-app/docs/deployment)

### `npm run build` fails to minify

This section has moved here: [https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify](https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify)


## Detected evidence (automated analysis)

Indexed codebase: 24 recognized source files, 69 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Supabase (technology) — detected in the code
- Google Gemini (technology) — claimed on Devpost, not found in the code
- OpenAI (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (29 of 29)

```
.gitignore
flask-backend/anatomyJSON.json
flask-backend/app.py
package.json
public/index.html
public/manifest.json
public/robots.txt
README.md
src/App.css
src/App.js
src/App.test.js
src/components/Appointments.js
src/components/Background.css
src/components/Background.js
src/components/BioDigitalViewer.js
src/components/Dashboard.js
src/components/Documents.js
src/components/Main.css
src/components/Main.js
src/components/Notifications.js
src/components/Recordings.js
src/components/Settings.js
src/components/Sidebar.js
src/components/translation.assistant.js
src/components/vapi.js
src/index.css
src/index.js
src/reportWebVitals.js
src/setupTests.js
```

### Dependencies

- package.json: @supabase/supabase-js@^2.50.0, @vapi-ai/web@^2.3.6, lucide-react@^0.522.0, react@^18.3.1, react-audio-visualize@^1.2.0, react-dom@^18.3.1, react-router-dom@^6.30.1, react-scripts@5.0.1

### Recent commits (newest first)

- added a homepage picture
- added emails saving automation, fixed some ui
- saving appointments data, fixed some ui
- added 3d anatomy, worked on sidebar, added google auth with supabase, worked on saving appointments
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "name": "bridgemed-ai-react",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@supabase/supabase-js": "^2.50.0",
    "@vapi-ai/web": "^2.3.6",
    "lucide-react": "^0.522.0",
    "react": "^18.3.1",
    "react-audio-visualize": "^1.2.0",
    "react-dom": "^18.3.1",
    "react-router-dom": "^6.30.1",
    "react-scripts": "5.0.1"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  }
}

```

### src/index.js

```javascript
import React from 'react';
import { createRoot } from 'react-dom/client';
import App from './App';

const container = document.getElementById('root');
const root = createRoot(container);

root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
); 
```

### src/App.js

```javascript
// src/App.js
import React, { useEffect, useState } from 'react';
import { BrowserRouter as Router, Routes, Route, Navigate, useLocation } from 'react-router-dom';
import { createClient } from '@supabase/supabase-js';

import Sidebar from './components/Sidebar';
import Background from './components/Background';
import Main from './components/Main';
import Appointments from './components/Appointments';
import Notifications from './components/Notifications';
import Settings from './components/Settings';
import Documents from './components/Documents';
import Recordings from './components/Recordings';

import './App.css';

const supabase = createClient(
  process.env.REACT_APP_SUPABASE_URL,
  process.env.REACT_APP_SUPABASE_ANON_KEY
);

// Wrapper to pass user to Appointments
const AppointmentsWrapper = ({ user }) => {
  return <Appointments user={user} />;
};

function AppContent() {
  const [user, setUser] = useState(null);
  const location = useLocation();
  const isMainPage = location.pathname === '/';

  useEffect(() => {
    const ensureInitialAppointment = async (userId) => {
      const { data, error } = await supabase
        .from('appointments')
        .select('*')
        .eq('user_id', userId);

      if (!error && data.length === 0) {
        await supabase.from('appointments').insert([
          {
            user_id: userId,
            notes: null,
            dates: new Date().toISOString()
          }
        ]);
      }
    };

    const restoreSession = async () => {
      const { data, error } = await supabase.auth.getSession();
      if (error) {
        console.error("Session fetch error:", error.message);
        return;
      }

      const user = data.session?.user;
      if (user) {
        setUser(user);
      }
    };

    restoreSession();

    const { data: authListener } = supabase.auth.onAuthStateChange((_event, session) => {
      if (session?.user) {
        setUser(session.user);
        (async () => {
          await ensureInitialAppointment(session.user.id);
        })();
      } else {
        setUser(null);
      }
    });

    return () => {
      authListener.subscription.unsubscribe();
    };
  }, []);

  return (
    <div className={`App ${isMainPage ? 'no-sidebar' : ''}`}>
      {!isMainPage && <Sidebar />}
      <div className="main-content">
        <Routes>
          <Route path="/" element={<Main />} />
          <Route path="/appointments" element={<AppointmentsWrapper user={user} />} />
          <Route path="/notifications" element={<Notifications />} />
          <Route path="/settings" element={<Settings />} />
          <Route path="/documents" element={<Documents />} />
          <Route path="/recordings" element={<Recordings />} />
          <Route path="*" element={<Navigate to="/" replace />} />
        </Routes>
      </div>
    </div>
  );
}

function App() {
  return (
      <Router>
        <AppContent />
      </Router>
  );
}

export default App;

```

### flask-backend/app.py

```python
import anthropic
import os
import json
from fuzzywuzzy import fuzz
from flask import Flask, request, jsonify
from flask_cors import CORS
import difflib
from dotenv import load_dotenv
from google import generativeai as genai
load_dotenv()
genai.configure(api_key=os.getenv("GEMINI_API_KEY"))

app = Flask(__name__)
CORS(app)  # Specify your React app's origin

# Remove the custom CORS headers function since flask-cors handles it
# @app.after_request
# def add_cors_headers(response):
#     response.headers.add("Access-Control-Allow-Origin", "*")
#     response.headers.add("Access-Control-Allow-Headers", "Content-Type")
#     response.headers.add("Access-Control-Allow-Methods", "GET, POST, OPTIONS")
#     return response

client = anthropic.Anthropic(
    api_key=os.getenv("CLAUDE_API_KEY")
)

summaryModel = genai.GenerativeModel("gemini-1.5-pro")
summaryChat = summaryModel.start_chat(history=[
    {
        "role": "user",
        "parts": [
            "You are a helpful chatbot, designed to summarize the doctor's responses to the patient"
            "Provide an accurate, but concise summmary that hits every main point"
            "Do not leave out any potential important information"

            "The first word of your response should be the langauge chosen. For example, if the langauge is Spanish, before you summarize the rest of the text, put `Spanish. (summary starts)`"
            "Do not ever translate the given data, your only role is to keep the name of the language before summarizing the text!"
            "At the end leave a two word response that will act as the title for this appointment"
        ]
    },
    {
        "role": "model",
        "parts": ["Understood! I will provide an accurate summary of the doctor's responses!"]
    }
])

translationModel = genai.GenerativeModel("gemini-1.5-pro")
translationChat = translationModel.start_chat(history=[
    {
        "role": "user",
        "parts": [
            "You are a helpful translator, designed to translate english into a designated language"
            "Provide accurate responses that allow foreigners to easily read in their native language"
            "Do not give any additional information, or two different translations"
            "Solely translate the given response with one solid answer"
        ]
    },
    {
        "role": "model",
        "parts": ["Understood! I will provide an accurate translation to the designated language!"]
    }
])

@app.route("/summarize", methods=["POST"])
def summarize_response():
    try:
        data = request.get_json()
        print("Incoming data:", data)

        # Extract the 'data' field from the JSON
        text_to_summarize = data.get('data', '')
        
        prompt = f"Summarize the following text: {text_to_summarize}"
        response = summaryChat.send_message(prompt)

        output = response.text.strip()
        return {"response": output}
        
    except Exception as e:
        print("ERROR:", e)
        return jsonify({"error": str(e)}), 500

@app.route("/translate", methods=["POST"])
def translate_transcription():
    try:
        data = request.get_json()

        text = data.get("text", "")

        if not text.strip():
            return jsonify({"error": "No text provided"}), 400

        parts = text.strip().split(" ", 1)

        if len(parts) < 2:
            return jsonify({"error": "Text must start with a target language followed by the content to translate"}), 400

        target_language = parts[0]
        content_to_translate = parts[1]

        prompt = f"Translate the following text to {target_language}:\n\n{content_to_translate}"

        response = translationChat.send_message(prompt)
        output = response.text.strip()

        return jsonify({"response": output})
    
    except Exception as e:
        print("ERROR:", e)
        return jsonify({"error": str(e)}), 500

def load_anatomy_json():
    path = os.path.join(os.path.dirname(__file__), "anatomyJSON.json")
    try:
        with open(path, "r") as f:
            return json.load(f)
    except Exception as e:
        print(f"Error loading anatomy JSON: {e}")
        return []

def match_part_to_object_id(part_name, anatomy, original_input=""):
    best_match = None
    highest_score = 0

    candidates = [p.strip() for p in part_name.split(";")]
    original_input = original_input.lower()

    # Infer left/right side preference
    preferred_side = None
    if "left" in original_input:
        preferred_side = "left"
    elif "right" in original_input:
        preferred_side = "right"

    for candidate in candidates:
        candidate_keywords = candidate.lower().split()

        for entry in anatomy:
            name = entry.get("name", "").lower()

            # Only proceed if any keyword is in the entry name
            if not any(kw in name for kw in candidate_keywords):
                continue

            # Optional: check side preference
            score = fuzz.token_sort_ratio(name, candidate)
            if preferred_side and preferred_side in name:
                score += 10

            if score > highest_score:
                highest_score = score
                best_match = entry

    return best_match if highest_score > 70 else None

latest_result = {"objectId": None, "part_name": "", "name": ""}

@app.route("/anatomy", methods=["POST"])
def ask():
    try:
        user_input = request.json.get("question", "")
        if not user_input:
            return jsonify({"error": "No question provided"}), 400
        
        message = client.messages.create(
            model="claude-opus-4-20250514",
            max_tokens=100,
            temperature=0,
            system="You are a medical assistant. Extract the *specific muscle or anatomical structure* mentioned in the sentence. If the language is vague, guess the most likely muscles based on typical anatomy. Output just the names, separated by semicolons.",
            messages=[
                {
            
[truncated — 969 more characters]
```

### src/components/Main.js

```javascript
import React, { useState, useEffect, useRef } from 'react';
import { useNavigate } from 'react-router-dom';
import Background from './Background';
import './Main.css';
import { createClient } from '@supabase/supabase-js';

const supabase = createClient(
  process.env.REACT_APP_SUPABASE_URL,
  process.env.REACT_APP_SUPABASE_ANON_KEY
);


const Main = () => {
  const navigate = useNavigate();
  const [isSticky, setIsSticky] = useState(false);
  const aboutRef = useRef(null);
  const featuresRef = useRef(null);

  const handleGoToDashboard = () => {
    navigate('/recordings');
  };

  const handleLogin = async () => {
    const { error } = await supabase.auth.signInWithOAuth({
      provider: 'google',
    });
  
    if (error) {
      console.error('Login failed:', error.message);
    }
  };  

  useEffect(() => {
    const handleScroll = () => {
      setIsSticky(window.scrollY > window.innerHeight * 0.9);
    };

    const observer = new IntersectionObserver(
      (entries) => {
        entries.forEach((entry) => {
          if (entry.isIntersecting) {
            entry.target.classList.add('visible');
            observer.unobserve(entry.target);
          }
        });
      },
      { threshold: 0.1 }
    );

    window.addEventListener('scroll', handleScroll);
    
    const sectionsToObserve = [aboutRef.current, featuresRef.current].filter(Boolean);
    sectionsToObserve.forEach(section => observer.observe(section));

    if (featuresRef.current) {
      const featureCards = featuresRef.current.querySelectorAll('.feature-card');
      featureCards.forEach(card => observer.observe(card));
    }

    return () => {
      window.removeEventListener('scroll', handleScroll);
      sectionsToObserve.forEach(section => {
        if (section) {
          observer.unobserve(section);
        }
      });
      if (featuresRef.current) {
        const featureCards = featuresRef.current.querySelectorAll('.feature-card');
        featureCards.forEach(card => observer.unobserve(card));

      }
    };
  }, []);

  return (
    <div className="main-page">
      <Background />
      <header className={`main-header ${isSticky ? 'sticky' : ''}`}>
        <div className="logo">
          <span>BridgeMed-AI</span>
        </div>
        <nav className="main-nav">
          <a href="#home">Home</a>
          <a href="#features">Features</a>
          <a href="#about">About Us</a>
        </nav>
        <div className="header-buttons">
        <button className="login-button" onClick={handleLogin}>Log in</button>

          <button className="try-now-button" onClick={handleGoToDashboard}>Try Now</button>
        </div>
      </header>

      <div id="home" className="hero-section">
  <div className="hero-content">
    <div className="hero-text">
      <p className="hero-subheading">AI-Powered Healthcare, Redefined.</p>
      <h1 className="hero-heading">Introducing BridgeMed-AI</h1>
      <div className="hero-buttons">
        <button className="try-button" onClick={handleGoToDashboard}>
          Try BridgeMed-AI <i className="fa-solid fa-arrow-right"></i>
        </button>
      </div>
    </div>

    <div className="hero-image">
      <img src="mainbridgemedai.png" alt="BridgeMed-AI Illustration" />
    </div>
  </div>
</div>

      
      <main className="page-content">
        <section id="about" className="content-section" ref={aboutRef}>
          <h2>About BridgeMed-AI</h2>
          <p>
            At the intersection of technology and healthcare, BridgeMed-AI is dedicated to revolutionizing patient care. Our platform leverages the power of artificial intelligence to provide accessible, reliable, and personalized medical information. We aim to empower both patients and healthcare professionals by bridging the communication gap and providing tools that enhance understanding and decision-making.
          </p>
        </section>

        <section id="features" className="content-section" ref={featuresRef}>
          <h2>Our Core Features</h2>
          <div className="features-grid">
            <div className="feature-card" style={{'--delay': '0s'}}>
              <h3>AI-Powered Diagnosis</h3>
              <p>Get instant, AI-driven insights into your symptoms. Our advanced algorithms analyze your information to provide potential diagnoses and recommend next steps.</p>
            </div>
            <div className="feature-card" style={{'--delay': '0.1s'}}>
              <h3>Appointment Scheduling</h3>
              <p>Seamlessly schedule appointments with top-rated doctors and specialists in your area. Our smart system helps you find the perfect match for your needs.</p>
            </div>
            <div className="feature-card" style={{'--delay': '0.2s'}}>
              <h3>Voice-to-Text Transcription</h3>
              <p>Never miss a detail from your doctor's visit. Our voice transcription service accurately records and transcribes your conversations for easy reference.</p>
            </div>
            <div className="feature-card" style={{'--delay': '0.3s'}}>
              <h3>3D Anatomical Visualization</h3>
              <p>Explore the human body like never before with our interactive 3D anatomical models. Understand your condition better with stunning, detailed visuals.</p>
            </div>
          </div>
        </section>
      </main>

      <footer>
        <p>&copy; 2025 BridgeMed-AI. All rights reserved.</p>
      </footer>
    </div>
  );
};

export default Main; 
```

### src/setupTests.js

```javascript
// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';

```

### src/App.test.js

```javascript
import { render, screen } from '@testing-library/react';
import App from './App';

test('renders learn react link', () => {
  render(<App />);
  const linkElement = screen.getByText(/learn react/i);
  expect(linkElement).toBeInTheDocument();
});

```

### src/reportWebVitals.js

```javascript
const reportWebVitals = onPerfEntry => {
  if (onPerfEntry && onPerfEntry instanceof Function) {
    import('web-vitals').then(({ getCLS, getFID, getFCP, getLCP, getTTFB }) => {
      getCLS(onPerfEntry);
      getFID(onPerfEntry);
      getFCP(onPerfEntry);
      getLCP(onPerfEntry);
      getTTFB(onPerfEntry);
    });
  }
};

export default reportWebVitals;

```

### src/index.css

```css
body {
  margin: 0;
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
    'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
    sans-serif;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

code {
  font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New',
    monospace;
}

```

### public/index.html

```html
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <link rel="icon" href="%PUBLIC_URL%/favicon.ico" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <meta name="theme-color" content="#000000" />
    <meta
      name="description"
      content="Web site created using create-react-app"
    />
    <link rel="apple-touch-icon" href="%PUBLIC_URL%/logo192.png" />
    <link rel="manifest" href="%PUBLIC_URL%/manifest.json" />
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.7.2/css/all.min.css">
    <title>BridgeMed-AI</title>
    <script src="https://developer.biodigital.com/builds/api/human-api-3.0.0.min.js"></script>
  </head>
  <body>
    <noscript>You need to enable JavaScript to run this app.</noscript>
    <div id="root"></div>
  </body>
</html> 
```

[14 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]