# Project export: MedMentor

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: Your personalized healthcare companion to bridge the communication gap in medicine
- Devpost: https://devpost.com/software/medmentor-o7lh9r
- GitHub: https://github.com/nishashastry/treehacks2025
- Team: 3 GitHub contributor(s) — GabrielMissael (25 commits), Alina (15 commits), Nisha Shastry (4 commits)

## Devpost submission (written by the team)

### Overview

Overview This project is a Health Companion Assistant built for TreeHacks 🌳💡. Our goal is to empower patients during and after their consultations by providing an intelligent companion that records, transcribes, and summarizes their doctor visits. In its beta development phase, the app focuses on diabetes 🍬, while future iterations will expand support to broader chronic disease management and personalized symptom tracking. What Inspired Us Our team wanted to work on a project that has real impact on a group of people, and we brainstormed different fields that could benefit most from AI developments. We determined that even though diabetes is a relatively common health condition, it requires great effort for individuals with diabetes to manage their health and blood sugar levels all the time. At the same time, there is not a perfect solution available on the market to meet the specific needs and features for diabetic patients. We saw an opportunity to better the daily experience of people through utilization of AI and took this opportunity to build upon this idea. What It Does Consultation Recording & Transcription 🎤➡️📝: The app records doctor visits and automatically transcribes the audio using advanced AI-powered speech-to-text technology. This ensures that every detail of the consultation is captured for future reference. Actionable Insights 💡✅: After transcription, the app processes the transcript to generate a clear summary and a list of recommended action items. This helps patients understand their treatment plan and know the next steps to take. Context-Aware Chatbot Support 🤖💬: Our chatbot is truly context aware! It uses the transcripts, past consultation notes, and additional patient data as a dynamic knowledge base to provide personalized, empathetic, and medically informed advice. Whether you’re asking about blood sugar management, insulin, or lifestyle changes, the assistant tailors its responses to your unique context. Patient Management 🔐👤: Patients can register and log in securely. Their details—such as name, email, date of birth, gender, and information about diabetes—are stored securely. This helps provide personalized insights and improves the overall experience. Why It Matters ❤️ Our Health Companion Assistant is designed to bridge the gap between patients and their healthcare providers by ensuring no detail is missed during consultations. By automating transcription and generating actionable insights, the app empowers patients to take control of their health and follow up effectively on doctor visits. With a context-aware chatbot that leverages transcripts and patient data as a knowledge base, users receive personalized advice tailored to their individual needs. While initially focused on diabetes in beta, our roadmap includes broader support for chronic diseases, paving the way for a healthier future. How We Built It Jeewon: front-end, full stack development Alina: glucose prediction back-end Nisha: back-end, diabetes ML model training, speech to text Missa: back-end, full stack development Technologies Used Frontend: Next.JS to build a web app Backend: Flask for the web server and API endpoints. Firebase (Firestore & Storage) for real-time data storage and scalable file storage. Celery for asynchronous processing. Flask for the web server and API endpoints. Firebase (Firestore & Storage) for real-time data storage and scalable file storage. Celery for asynchronous processing. AI & Speech Processing: ElevenLabs API for high-quality text-to-speech conversion. OpenAI API (Whisper) for speech-to-text transcription and for generating context-aware chatbot responses. Perplexity API as part of our chatbot backend to ensure responses are tailored and reliable. ElevenLabs API for high-quality text-to-speech conversion. OpenAI API (Whisper) for speech-to-text transcription and for generating context-aware chatbot responses. Perplexity API as part of our chatbot backend to ensure responses are tailored and reliable. Challenges We each chose to challenge ourselves by learning technologies that we were unfamiliar with, so we each experienced a learning curve. Our development process required us to integrate each of our parts near the end of the time, and ensuring that each part worked well together was a challenge that we debugged through together. Future Development 🚀 Extended Chronic Disease Support: While our beta version focuses on diabetes, future updates will expand support to other chronic diseases. The app will offer detailed symptom tracking, personalized treatment recommendations, and more comprehensive health management features. Advanced Symptom Analysis: We plan to integrate tools for real-time symptom analysis and more dynamic insights based on a wider range of health data.

## README (from the GitHub repository)

# MedMentor 🤝
## Overview
This project is a Health Companion Assistant built for TreeHacks 🌳💡. Our goal is to empower patients during and after their consultations by providing an intelligent companion that records, transcribes, and summarizes their doctor visits. In its beta development phase, the app focuses on diabetes 🍬, while future iterations will expand support to broader chronic disease management and personalized symptom tracking.

## What It Does
- **Consultation Recording & Transcription** 🎤➡️📝: The app records doctor visits and automatically transcribes the audio using advanced AI-powered speech-to-text technology. This ensures that every detail of the consultation is captured for future reference.
- **Actionable Insights** 💡✅: After transcription, the app processes the transcript to generate a clear summary and a list of recommended action items. This helps patients understand their treatment plan and know the next steps to take.
- **Context-Aware Chatbot Support** 🤖💬: Our chatbot is truly context aware! It uses the transcripts, past consultation notes, and additional patient data as a dynamic knowledge base to provide personalized, empathetic, and medically informed advice. Whether you’re asking about blood sugar management, insulin, or lifestyle changes, the assistant tailors its responses to your unique context.
- **Patient Management** 🔐👤: Patients can register and log in securely. Their details—such as name, email, date of birth, gender, and information about diabetes—are stored securely. This helps provide personalized insights and improves the overall experience.

## Future Development 🚀
- **Extended Chronic Disease Support**: While our beta version focuses on diabetes, future updates will expand support to other chronic diseases. The app will offer detailed symptom tracking, personalized treatment recommendations, and more comprehensive health management features.
- **Advanced Symptom Analysis**: We plan to integrate tools for real-time symptom analysis and more dynamic insights based on a wider range of health data.

## Technologies Used
- **Frontend**: To-add
- **Backend**:
    - Flask for the web server and API endpoints.
    - Firebase (Firestore & Storage) for real-time data storage and scalable file storage.
    - Celery for asynchronous processing.
- **AI & Speech Processing**:
    - ElevenLabs API for high-quality text-to-speech conversion.
    - OpenAI API for speech-to-text transcription and for generating context-aware chatbot responses.
    - Perplexity API as part of our chatbot backend to ensure responses are tailored and reliable.

## Why It Matters ❤️
Our Health Companion Assistant is designed to bridge the gap between patients and their healthcare providers by ensuring no detail is missed during consultations. By automating transcription and generating actionable insights, the app empowers patients to take control of their health and follow up effectively on doctor visits. With a context-aware chatbot that leverages transcripts and patient data as a knowledge base, users receive personalized advice tailored to their individual needs. While initially focused on diabetes in beta, our roadmap includes broader support for chronic diseases, paving the way for a healthier future.


## Detected evidence (automated analysis)

Indexed codebase: 31 recognized source files, 103 KB.
- Firebase (technology) — detected in the code
- Flask (technology) — detected in the code
- JavaScript (language) — detected in the code
- Next.js (technology) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Redis (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- CSS (language) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (120 of 2261)

```
.gitignore
all_test_data.csv
clinilcal_notetaking.ipynb
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-11-30.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-01.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-02.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-03.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-04.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-05.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-06.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-07.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-08.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-09.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-10.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-11.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-12.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-13.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-14.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-15.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-16.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-17.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-18.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-19.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-20.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-21.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-22.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-23.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-24.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-25.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-26.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-27.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-28.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-29.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-30.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2021-12-31.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-01.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-02.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-03.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-04.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-05.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-06.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-07.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-08.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-09.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-10.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-11.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-12.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-13.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-14.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-15.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-16.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-17.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-18.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-19.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-20.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-21.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-22.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-23.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-24.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-25.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-26.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-27.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-28.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-29.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-30.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-01-31.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-01.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-02.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-03.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-04.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-05.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-06.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-07.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-08.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-09.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-10.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-11.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-12.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-13.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-14.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-15.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-16.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-17.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-18.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-19.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-20.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-21.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-22.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-23.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-24.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-25.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-26.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-27.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-02-28.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-01.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-02.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-03.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-04.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-05.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-06.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-07.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-08.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-09.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-10.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-11.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-12.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-13.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-14.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-15.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-16.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-17.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-18.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-19.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-20.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-21.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-22.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-23.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-24.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-25.csv
data/fitbit_data/2022_04_25_all_time_export/Other/estimated_oxygen_variation-2022-03-26.csv
[2141 more files omitted for size]
```

### Dependencies

- src/backend/requirements.txt: celery, elevenlabs, firebase-admin, Flask, Flask-Cors, openai, python-dotenv, redis, textblob
- src/frontend/package.json: @eslint/eslintrc@^3, @firebase/auth@^1.9.0, @firebase/firestore@^4.7.8, @types/node@^20, @types/react@^19, @types/react-dom@^19, eslint@^9, eslint-config-next@15.1.7, firebase@^11.3.1, next@^15.1.7, postcss@^8, react@^19.0.0, react-dom@^19.0.0, react-icons@^5.4.0, tailwindcss@^3.4.1, typescript@^5

### Recent commits (newest first)

- Update README.md
- small edits to settings page
- pushed to main
- fixed pop up delay
- delay
- pop up added correct text
- pop upfixed
- merge conflicts
- made changes for demo
- pop up
- Fix merge conflicts
- Fix chatbot and login
- Merge branch 'main' of github.com:nishashastry/treehacks2025
- combined chatbot and simplified navbar
- added model inference
- Merge pull request #6 from nishashastry/feature-conversation
- Merge branch 'main' into feature-conversation
- model deploy
- made design frontend changes
- Notif generator and text to speech

## Key source files (fetched from GitHub, selected and truncated for size)

### src/backend/requirements.txt

```
Flask
python-dotenv
elevenlabs
celery
redis
firebase-admin
openai
Flask-Cors
textblob
```

### src/frontend/package.json

```
{
  "name": "disease-management",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev --turbopack",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "@firebase/auth": "^1.9.0",
    "@firebase/firestore": "^4.7.8",
    "firebase": "^11.3.1",
    "next": "^15.1.7",
    "react": "^19.0.0",
    "react-dom": "^19.0.0",
    "react-icons": "^5.4.0"
  },
  "devDependencies": {
    "@eslint/eslintrc": "^3",
    "@types/node": "^20",
    "@types/react": "^19",
    "@types/react-dom": "^19",
    "eslint": "^9",
    "eslint-config-next": "15.1.7",
    "postcss": "^8",
    "tailwindcss": "^3.4.1",
    "typescript": "^5"
  }
}

```

### src/frontend/pages/index.js

```javascript
import '../components/global';
import Layout from '../components/layout';

import { useEffect, useState } from 'react';
import { auth } from '../firebase/config';
import { useRouter } from 'next/router';

export default function LandingPage() {
  const [loading, setLoading] = useState(true);
  const router = useRouter();

  useEffect(() => {
    if (auth.currentUser) {
      router.push('/dashboard');  // Redirect to dashboard if logged in
    } else {
      setLoading(false);  // Stop loading if not logged in
    }
  }, [router]);

  if (loading) {
    return <div className="loading-screen">Loading...</div>;
  }

  return (
    <Layout>
      <div className="landing-page">
        <div className="landing-container">
          <img src="/logo.png" alt="App Logo" className="logo" />
          <h1 className="app-title">Welcome to MedMentor</h1>
          <div className="action-buttons">
            <button
              onClick={() => router.push('/login')}
              className="btn"
            >
              Login
            </button>
            <button
              onClick={() => router.push('/signup')}
              className="btn"
            >
              Sign Up
            </button>
          </div>
        </div>
      </div>
    </Layout>
  );
}

```

### visit_notetaking.py

```python
from openai import OpenAI
client = OpenAI()
PATH = "test_audio_files\doctor_visit1.mp3"

# possibly include prompt to enhance quality of transcription
def transcription(audio_path) :

    audio_file= open(audio_path, "rb")
    transcription = client.audio.transcriptions.create(
        model="whisper-1", 
        file=audio_file
    )
    return transcription.text
def action_items(transcipt):
    prompt = "You are going to get an audio transcript of a doctor's visit for diabetes.In the audio transcript, there is audio of both the doctor and patient. you are basically a medically educated scribe assistant for the doctor. the patient wants a summary and a list of action items based on the doctor's visits."
    completion = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "developer", "content": prompt},
            {
                "role": "user",
                "content": "Reccomend action items for this visit: " + str(transcipt)
            }
        ]
    )
    return (completion.choices[0].message.content)

transcript_visit = (transcription(PATH))
actions = action_items(transcript_visit)
print(actions)
```

### test_model_inference.py

```python
import os
os.environ['TF_ENABLE_ONEDNN_OPTS'] = '0'
import tensorflow as tf
import pandas as pd
import random


# In another Python script (e.g., `another_script.py`)
# from my_functions import bg_denormalize
import numpy as np
# Call the function

# print(test_data)
# # Load the model
model = tf.keras.models.load_model("models/lstm_2h_Glucose_Rapid Insulin IOB_Carbohydrates.keras")
# # Use it for predictions
# input_data = test_data

# print("shape")
# print(test_data_df.shape)

# import pandas as pd
# import numpy as np
MMOL_TO_MGDL = 18.016

# Load the CSV
df = pd.read_csv("all_test_data.csv")

# Create a dictionary to store reshaped examples
reshaped_data = {}
def bg_denormalize(norm_val, unit_to="mgdl"):
    orig = norm_val * 23.979
    if unit_to == "mgdl":
        orig *= MMOL_TO_MGDL
    return orig


random_number = random.randint(0, 10)
# Extract data for each Example_ID and reshape
for example_id in df["Example_ID"].unique():
    example_data = df[df["Example_ID"] == example_id].drop(columns=["Example_ID"]).values  # Remove ID column
    reshaped_data[example_id] = example_data.reshape(1, 20, 3)  # Reshape to (1, 20, 3)

# Example: Predict using the first example
inputs = reshaped_data[random_number]  # Select Example_ID = 1
lstm_predictions = model.predict(inputs)

print("Prediction for Example 1:", (lstm_predictions)[0][0])
print(bg_denormalize(lstm_predictions)[0][0])

```

### dataset.py

```python
import os
import pandas as pd
import numpy as np
import json

def load_reader_dataset():
    reader_data = "data/reader_data"
    exports = [os.path.join(reader_data, export) for export in os.listdir(reader_data)]
    exports.sort(reverse=False)

    df = pd.read_csv(exports[0], sep="\t")
    for export in exports[1:]:
        df = pd.concat([df, pd.read_csv(export, sep="\t")])

    # Exports usually overlap, so drop the duplicates
    df.drop_duplicates(inplace=True)

    # Dropping column which aren't used
    df.drop(['Non-numeric Food', 'Non-numeric Long-Acting Insulin', 'Non-numeric Rapid-Acting Insulin',
             'Notes', 'Ketone (mmol/L)', 'User Change Insulin (units)', 'Correction Insulin (units)', 'ID', 'Meal Insulin (units)'],
            axis=1, inplace=True)

    df.rename(columns={'Rapid-Acting Insulin (units)': 'Rapid Insulin',
                       'Carbohydrates (grams)': 'Carbohydrates',
                       'Long-Acting Insulin (units)': 'Long Insulin'},
              inplace=True)

    df['Time'] = pd.to_datetime(df['Time'], format="%Y/%m/%d %H:%M")
    df.sort_values(by='Time', inplace=True)

    # Only considering Historical Glucose (reliable 15min measurements) - we're not using Scan Glucose column
    df.rename(columns={'Historic Glucose (mmol/L)': 'Glucose'}, inplace=True)

    # Since 2022/01/09 00:00 we use mySugr to keep track of insulin and Carbohydrates, so drop any data after this date
    df.loc[df['Time'] >= "2022/01/09 00:00", ['Rapid Insulin', 'Long Insulin', 'Carbohydrates']] = np.nan

    return df


def load_mySugr_dataset():
    mySugr_data = "data/mySugr_data/2022_01_09-2022_04_25_export.csv"
    mysugr_df = pd.read_csv(mySugr_data, sep=",")

    # Keeping only the relevant columns
    mysugr_df = mysugr_df[['Date', 'Time', 'Tags', 'Basal Injection Units', 'Insulin (Meal)', 'Insulin (Correction)',
                           'Meal Carbohydrates (Grams, Factor 1)', 'Meal Descriptions', 'Body weight (kg)', 'Food type']]

    mysugr_df["Rapid Insulin"] = mysugr_df[["Insulin (Meal)", "Insulin (Correction)"]].apply(
        lambda x: x["Insulin (Correction)"] if np.isnan(x["Insulin (Meal)"]) else x["Insulin (Meal)"], axis=1
    )

    mysugr_df.rename(columns={'Basal Injection Units': 'Long Insulin',
                              'Meal Carbohydrates (Grams, Factor 1)': 'Carbohydrates',
                              'Meal Descriptions': 'GI'},
                     inplace=True)

    mysugr_df["Time"] = pd.to_datetime(mysugr_df["Date"] + " " + mysugr_df["Time"], format="%b %d, %Y %I:%M:%S %p")
    mysugr_df.sort_values(by='Time', inplace=True)
    return mysugr_df


def read_fitbit_json_export(export_file, export_type):
    with open(export_file, "r") as f:
        j = json.load(f)
    df_read = pd.json_normalize(j)
    df_read["dateTime"] = pd.to_datetime(df_read["dateTime"], format="%m/%d/%y %H:%M:%S")

    if export_type == "heart":
        df_read.rename(columns={"value.bpm": "bpm"}, inplace=True)
        df_read.drop("value.confidence", axis=1, inplace=True)
    elif export_type == "calories":
        df_read.rename(columns={"value": "calories"}, inplace=True)
        df_read["calories"] = df_read["calories"].astype(float)
    elif export_type == "distance":
        df_read.rename(columns={"value": "distance"}, inplace=True)
        df_read["distance"] = df_read["distance"].astype(int)
        # Convert from centimeters to meters
        df_read["distance"] = df_read["distance"] / 100
    else:
        raise Exception("Export type not recognized")
    return df_read


def load_fitbit_dataset():
    fitbit_data = "data/fitbit_data/2022_04_25_all_time_export/Physical Activity/"
    calories_exports = sorted([os.path.join(fitbit_data, export) for export in os.listdir(fitbit_data) if "calories" in export])

    distance_exports = sorted([os.path.join(fitbit_data, export) for export in os.listdir(fitbit_data) if "distance" in export])

    heart_rate_exports = sorted([os.path.join(fitbit_data, export) for export in os.listdir(fitbit_data)
                                 if "heart_rate-" in export and not "resting" in export])

    df_fitbit = read_fitbit_json_export(calories_exports[0], "calories")
    for export in calories_exports[1:]:
        df_fitbit = pd.concat([df_fitbit, read_fitbit_json_export(export, "calories")], ignore_index=True)
    for export in distance_exports:
        df_fitbit = pd.concat([df_fitbit, read_fitbit_json_export(export, "distance")], ignore_index=True)
    for export in heart_rate_exports:
        df_fitbit = pd.concat([df_fitbit, read_fitbit_json_export(export, "heart")], ignore_index=True)

    # Change to 1 minute frequency, to lower amount of rows
    df_fitbit = df_fitbit.set_index('dateTime').resample('1T').agg(
        {
            'bpm': pd.Series.mean,
            'distance': pd.Series.sum,
            'calories': pd.Series.sum
        }
    ).reset_index()

    df_fitbit.rename(columns={"dateTime": "Time"}, inplace=True)
    df_fitbit.sort_values(by='Time', inplace=True)
    return df_fitbit
import os
import pandas as pd

```

### my_functions.py

```python
import os
import os
os.environ['TF_ENABLE_ONEDNN_OPTS'] = '0'  # Disable oneDNN custom operations
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import math
import pandas as pd
import numpy as np
import tensorflow as tf
import sklearn
from sklearn import preprocessing
from tensorflow.keras import layers
import keras
import seaborn as sns
import matplotlib.pyplot as plt

# Plotting style

from dataset import load_reader_dataset, load_mySugr_dataset, load_fitbit_dataset

# ## Load data from data sources

df = load_reader_dataset()

mysugr_df = load_mySugr_dataset()

fitbit_df = load_fitbit_dataset()
df = pd.concat([df, mysugr_df], ignore_index=True)

df = pd.concat([df, fitbit_df], ignore_index=True)

df.sort_values(by='Time', inplace=True)

# Keeping relevant features
using_features = ['Time', 'Glucose', 'Rapid Insulin', 'Long Insulin', 'Carbohydrates', 'GI', 'calories', 'bpm', 'distance']
glucose_df = df[using_features]
glucose_df = glucose_df.dropna(subset=using_features, how='all')

# Filtering date range
start_date = "2021/12/01 00:00"
end_date = "2022/04/08 00:00"
glucose_df = glucose_df[(glucose_df["Time"] <= end_date) & (glucose_df["Time"] >= start_date)]


glucose_only = glucose_df[["Glucose", "Time"]].dropna()
deltas = glucose_only['Time'].diff()
gaps = deltas[deltas > pd.Timedelta(minutes=20)]



ph_only = glucose_df[["Time", "distance", "calories", "bpm"]].copy().set_index('Time').resample('15T').agg(
    {
        'bpm':pd.Series.mean,
        'distance':pd.Series.mean,
        'calories':pd.Series.mean,
    }).reset_index()
total_ph_not_null = len(ph_only)
glucose_df["Hour"] = glucose_df.apply(lambda row: row["Time"].hour, axis=1)
using_features.append("Hour")

GI_valus = ["Low", "Medium", "High", "Very High"]
glucose_df["GI"] = glucose_df.apply(lambda row: row["GI"] if row["GI"] in GI_valus else np.nan, axis=1)

# Encode as ordinal features
enc = {
    "Low": 15,
    "Medium": 45,
    "High": 75,
    "Very High": 95,
    np.nan: 0
}
glucose_df["GI"] = glucose_df["GI"].map(enc)

# What we realy want to use is Glycemic load, calculated as (GI * grams_of_carbohydrates) / 100
glucose_df["Glycemic Load"] = glucose_df["GI"] * glucose_df["Carbohydrates"] / 100

using_features.remove("GI")
using_features.append("Glycemic Load")


def resample_data(in_df, min_freq=15):
    resampl_df = in_df.copy()
    resampl_df = resampl_df.set_index('Time').resample(f'{min_freq}T').agg(
        {
            'Glucose':pd.Series.mean,
            'Rapid Insulin':pd.Series.sum,
            'Long Insulin':pd.Series.sum,
            'Carbohydrates':pd.Series.sum,
            'Glycemic Load':pd.Series.sum,
            'bpm':pd.Series.mean,
            'distance':pd.Series.sum,
            'calories':pd.Series.sum,
            'Hour':pd.Series.mean
        }).reset_index()
    return resampl_df

ORIG_FREQ = 15
SAMPL_FREQ = 15
FREQ_CORRECTION = ORIG_FREQ // SAMPL_FREQ

glucose_df_resampled = resample_data(glucose_df, SAMPL_FREQ)

days = 6
roll_window_width = days * 24 * 60 // SAMPL_FREQ
glucose_df_resampled['Rapid Insulin 6d'] = (glucose_df_resampled['Rapid Insulin'].rolling(roll_window_width).sum() / days)
glucose_df_resampled['Rapid Insulin 6d'].replace(to_replace=np.nan, method='bfill', inplace=True)

using_features.append('Rapid Insulin 6d')

# Long insulin acts for approximately 24 hours, stretch the data across this period
glucose_df_resampled['Long Insulin'].replace(to_replace=0, method='ffill', inplace=True)


td = 5*60
# Peak activity time [minutes], 45-85 minutes
tp = 55
# Time constant of exponential decay
tau = tp*(1-tp/td)/(1-2*tp/td)
# Rise time factor
a = 2*tau/td
# Auxiliary scale factor
S = 1/(1-a+(1+a)*math.exp(-td/tau))

# Insulin activity function of time
# Ia(t) = (S/tau^2)*t*(1-t/td)*exp(-t/tau)
def insulin_activity(t):
    return (S/tau**2)*t*(1-t/td)*math.exp(-t/tau)

# Insulin on board function
# IOB(t) = 1-S*(1-a)*((t^2/(tau*td*(1-a)) - t/tau - 1)*exp(-t/tau)+1)
def insulin_on_board(t):
    return 1-S*(1-a)*( (t**2/(tau*td*(1-a)) - t/tau - 1)*math.exp(-t/tau)+1)

# 
iob = [insulin_on_board(x) for x in range(0, td)]
ia = [insulin_activity(x) for x in range(0, td)]


def insulin_on_board_vector(iob_window, dose):
    w_len = len(iob_window)
    result = [current_iob for current_iob in iob_window]
    for i in range(w_len):
        result[i] += dose * insulin_on_board(i * SAMPL_FREQ)
    return result

glucose_df_resampled["Rapid Insulin IOB"] = 0

max_n = len(glucose_df_resampled)

duration_samples = td // SAMPL_FREQ
for idx, data in glucose_df_resampled.iterrows():
    # Insulin injected
    if data["Rapid Insulin"] != 0:
        idx = int(idx)
        dur_end = int(min(idx + duration_samples, max_n))
        ins_dose = data["Rapid Insulin"]
        glucose_df_resampled.loc[slice(int(idx), dur_end), "Rapid Insulin IOB"] = \
            insulin_on_board_vector(glucose_df_resampled.loc[slice(int(idx), dur_end), "Rapid Insulin IOB"], ins_dose)

using_features.append("Rapid Insulin IOB")
MMOL_TO_MGDL = 18.016
def train_val_test_split(df_in):
    n = len(df_in)
    return df_in[0:int(n*0.7)].copy(), df_in[int(n*0.7):int(n*0.9)].copy(), df_in[int(n*0.9):].copy(), n

train_df, val_df, test_df, n = train_val_test_split(glucose_df_resampled)
num_features = glucose_df_resampled.shape[1]

def interpolate_gaps(in_df, method="linear"):
    if method in ["spline", "polynomial"]:
        in_df["Glucose"] = in_df["Glucose"].interpolate(method=method, order=2)
        if "bpm" in in_df.columns:
            in_df["bpm"] = in_df["bpm"].interpolate(method=method, order=2)
    else:
        in_df["Glucose"] = in_df["Glucose"].interpolate(method=method)
        if "bpm" in in_df.columns:
            in_df["bpm"] = in_df["bpm"].interpolate(method=method)

interpolate_gaps(train_df)
interpolate_gaps(val_df)

# Drop any nans from test dataset (we're not interpolating it to keep original data)
test_df = test_df.dropna(subset=["Glucose"])

def min_max_normalize(train_df, val_df, test_df, features):
    min_max_s
[truncated — 535 more characters]
```

### src/frontend/next.config.ts

```typescript
import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  /* config options here */
};

export default nextConfig;

```

### src/backend/run.py

```python
# run.py
from app import create_app

# Create an instance of the Flask application using our factory function.
app = create_app()

if __name__ == '__main__':
    # Start the Flask development server
    app.run(host = '0.0.0.0', port = 5000, debug=True)

```

### src/backend/config.py

```python
# config.py
import os

class Config:
    # Secret key used for session security. Should be a long, random string.
    SECRET_KEY = os.getenv("SECRET_KEY", "defaultsecret")

    # URL for the Celery broker (Redis in this case)
    CELERY_BROKER_URL = os.getenv("CELERY_BROKER_URL", "redis://localhost:6379/0")

    # Additional configuration variables can be added here.

```

[22 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]