# Project export: MedicAI

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 11.0
- Tagline: What if getting healthcare advice was as easy as the ABCs? Our AI companion transcribes visits, summarizes them, and answers questions anytime—making healthcare easy and accessible for everyone.
- Devpost: https://devpost.com/software/medicai-t3na1o
- GitHub: https://github.com/sudo-omar/Cal-Hacks
- Demo: http://www.medicallyempoweringdataintegratingcompanionaiinsights.co/
- Team: 4 GitHub contributor(s) — sudo-omar (32 commits), Blade6264 (23 commits), slvo-loading (11 commits), benkhong27 (6 commits)

## Devpost submission (written by the team)

### Inspiration

We've all left a doctor's office feeling more confused than when we arrived. This common experience highlights a critical issue: over 80% of Americans say access to their complete health records is crucial, yet 63% lack their medical history and vaccination records since birth. Recognizing this gap, we developed our app to empower patients with real-time transcriptions of doctor visits, easy access to health records, and instant answers from our AI doctor avatar. Our goal is to ensure EVERYONE has the tools to manage their health confidently and effectively.

### What it does

Our app provides real-time transcription of doctor visits, easy access to personal health records, and an AI doctor for instant follow-up questions, empowering patients to manage their health effectively.

### How we built it

We used Node.js, Next.js, webRTC, React, Figma, Spline, Firebase, Gemini, Deepgram.

### Challenges we ran into

One of the primary challenges we faced was navigating the extensive documentation associated with new technologies. Learning to implement these tools effectively required us to read closely and understand how to integrate them in unique ways to ensure seamless functionality within our website. Balancing these complexities while maintaining a cohesive user experience tested our problem-solving skills and adaptability. Along the way, we struggled with Git and debugging.

### Accomplishments we're proud of

Our proudest achievement is developing the AI avatar, as there was very little documentation available on how to build it. This project required us to navigate through various coding languages and integrate the demo effectively, which presented significant challenges. Overcoming these obstacles not only showcased our technical skills but also demonstrated our determination and creativity in bringing a unique feature to life within our application.

### What we learned

We learned the importance of breaking problems down into smaller, manageable pieces to construct something big and impactful. This approach not only made complex challenges more approachable but also fostered collaboration and innovation within our team. By focusing on individual components, we were able to create a cohesive and effective solution that truly enhances patient care. Also, learned a valuable lesson on the importance of sleep!

### What's next

With the AI medical industry projected to exceed $188 billion, we plan to scale our website to accommodate a growing number of users. Our next steps include partnering with hospitals to enhance patient access to our services, ensuring that individuals can seamlessly utilize our platform during their healthcare journey. By expanding our reach, we aim to empower more patients with the tools they need to manage their health effectively.

## README (from the GitHub repository)

No README available.

## Detected evidence (automated analysis)

Indexed codebase: 25 recognized source files, 66 KB.
- CSS (language) — detected in the code
- Express (technology) — detected in the code
- Firebase (technology) — detected in the code
- Google Gemini (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- OpenAI (technology) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- Next.js (technology) — claimed on Devpost, not found in the code
- Node.js (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (30 of 30)

```
cal-hacks/.env
cal-hacks/.gitignore
cal-hacks/package.json
cal-hacks/proxyServer.js
cal-hacks/public/index.html
cal-hacks/public/manifest.json
cal-hacks/public/robots.txt
cal-hacks/README.md
cal-hacks/src/App.css
cal-hacks/src/App.js
cal-hacks/src/App.test.js
cal-hacks/src/components/About.jsx
cal-hacks/src/components/appointments.jsx
cal-hacks/src/components/AuthContext.js
cal-hacks/src/components/Contact.jsx
cal-hacks/src/components/firebase.js
cal-hacks/src/components/FirstAppoitment.jsx
cal-hacks/src/components/Gemini.jsx
cal-hacks/src/components/Home.jsx
cal-hacks/src/components/Login.jsx
cal-hacks/src/components/Logo.jsx
cal-hacks/src/components/Navbar.jsx
cal-hacks/src/components/ReportAndTalk.jsx
cal-hacks/src/components/SignUp.jsx
cal-hacks/src/components/SpeechAI.jsx
cal-hacks/src/components/useWebSockets.js
cal-hacks/src/index.css
cal-hacks/src/index.js
cal-hacks/src/reportWebVitals.js
cal-hacks/src/setupTests.js
```

### Dependencies

- cal-hacks/package.json: @deepgram/sdk@^3.8.1, @emotion/react@^11.13.3, @emotion/styled@^11.13.0, @google/generative-ai@^0.21.0, @mui/icons-material@^6.1.4, @mui/material@^6.1.4, @splinetool/react-spline@^4.0.0, @splinetool/runtime@^1.9.32, @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^13.5.0, axios@^1.7.7, cors@^2.8.5, crypto-browserify@^3.12.0, dotenv@^16.4.5, eslint@^8, express@^4.21.1, firebase@^10.14.1, force@^0.0.3, lucide-react@^0.453.0, openai@^4.52.7, os-browserify@^0.3.0, path-browserify@^1.0.1, postcss@^8, react@^18.3.1, react-dom@^18.3.1, react-router-dom@^6.27.0, react-scripts@5.0.1, simli-client@^1.1.2, styled-components@^6.1.13, tailwindcss@^3.4.1, web-vitals@^2.1.4, ws@^8.18.0, wscat@^6.0.1

### Recent commits (newest first)

- Merge pull request #15 from sudo-omar/khoi-ai
- Merge branch 'main' into khoi-ai
- Merge pull request #14 from sudo-omar/sydney
- id fix
- Updated UI for Deepgram
- Added AI to the UI
- Merge pull request #13 from sudo-omar/sydney
- fixed the parsing messed with id passing to components
- chnages
- Merge pull request #11 from sudo-omar/khoi-fixed-ui
- Merge branch 'main' into khoi-fixed-ui
- ai wokring
- Merge pull request #12 from sudo-omar/khoi-simli
- merge conflcits resolved
- Got the AI to work
- Ben's changes to ui
- test
- Merge pull request #10 from sudo-omar/ben-signup
- updates + json
- fixed json formation of generated summary

## Key source files (fetched from GitHub, selected and truncated for size)

### cal-hacks/package.json

```
{
  "name": "cal-hacks",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@deepgram/sdk": "^3.8.1",
    "@emotion/react": "^11.13.3",
    "@emotion/styled": "^11.13.0",
    "@google/generative-ai": "^0.21.0",
    "@mui/icons-material": "^6.1.4",
    "@mui/material": "^6.1.4",
    "@splinetool/react-spline": "^4.0.0",
    "@splinetool/runtime": "^1.9.32",
    "@testing-library/jest-dom": "^5.17.0",
    "@testing-library/react": "^13.4.0",
    "@testing-library/user-event": "^13.5.0",
    "axios": "^1.7.7",
    "cors": "^2.8.5",
    "crypto-browserify": "^3.12.0",
    "dotenv": "^16.4.5",
    "express": "^4.21.1",
    "firebase": "^10.14.1",
    "force": "^0.0.3",
    "lucide-react": "^0.453.0",
    "openai": "^4.52.7",
    "os-browserify": "^0.3.0",
    "path-browserify": "^1.0.1",
    "react": "^18.3.1",
    "react-dom": "^18.3.1",
    "react-router-dom": "^6.27.0",
    "react-scripts": "5.0.1",
    "simli-client": "^1.1.2",
    "styled-components": "^6.1.13",
    "web-vitals": "^2.1.4",
    "ws": "^8.18.0",
    "wscat": "^6.0.1"
  },
  "devDependencies": {
    "eslint": "^8",
    "postcss": "^8",
    "tailwindcss": "^3.4.1"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject",
    "server": "node ./server/server.js"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  }
}

```

### cal-hacks/src/index.js

```javascript
import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import App from './App';
import reportWebVitals from './reportWebVitals';

const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();

```

### cal-hacks/src/App.js

```javascript
// src/App.js
import React from "react";
import { BrowserRouter as Router, Route, Routes } from "react-router-dom"; // Change to use Routes
import Navbar from "./components/Navbar";
import Home from "./components/Home";
import About from "./components/About";
import Contact from "./components/Contact";
import ReportAndTalk from "./components/ReportAndTalk";
import Appointment from "./components/appointments";
import Login from "./components/Login"; // Import the Login component
import { useState, useEffect } from 'react';


import { Link } from "react-router-dom";
import FirstAppointment from "./components/FirstAppoitment";
import SignUp from "./components/SignUp";
import "./App.css";

const App = () => {
  return (

    
    <Router>
      <Navbar /> {/* Navbar always displayed */}
      <Routes>
        {" "}
        {/* Use Routes instead of Switch */}
        <Route path="/" element={<Home />} />{" "}
        {/* Use element prop instead of component */}
        <Route path="/report-and-talk" element={<ReportAndTalk />} />
        <Route path="/about" element={<About />} />
        <Route path="/contact" element={<Contact />} />
        <Route path="/appointments/:id" element={<Appointment />} />
        
        <Route path="/appointments" element={<Appointment />} />

        <Route path="/login" element={<Login />} /> {/* Add this line */}
        <Route path="/appointments/1" element={<FirstAppointment />} />
        <Route path="/signup" element={<SignUp />} />
      </Routes>
    </Router>
  );
};

export default App;

```

### cal-hacks/proxyServer.js

```javascript
//Import required modules
const express = require("express");
const http = require("http");
const WebSocket = require("ws");
const dotenv = require("dotenv");
dotenv.config();


const app = express();
const server = http.createServer(app);
const wss = new WebSocket.Server({ server });


const INPUT_SAMPLE_RATE = 16000;
const OUTPUT_SAMPLE_RATE = 16000;


const CONFIG = {
    type: "SettingsConfiguration",
    audio: {
        input: {
            encoding: "linear16",
            sample_rate: INPUT_SAMPLE_RATE
        },
        output: {
            encoding: "linear16",
            sample_rate: OUTPUT_SAMPLE_RATE,
            container: "none",
        }
    },
    agent: {
        listen: {
            model: "nova-2"
        },
        think: {
            provider: {
                type: "open_ai"
            },
            model: "gpt-4o-mini",
            instructions: "You are a highly experienced and knowledgeable doctor. Your role is to assess the patient's condition and give them advice or instructions for care. Focus entirely on the patient's health, diagnosis, and treatment. Avoid discussing non-medical topics, inventions, or ideas unrelated to healthcare. Act professional, empathetic, and patient-centered."
        },
        speak: {
            model: "aura-helios-en"
        }
    }

}


/*
 Deepgram Streaming Text to Speech
*/
const setupDeepgramWebsocket = (client_ws) => {
   const ws = new WebSocket('wss://agent.deepgram.com/agent', {
       headers: { authorization: `token ${process.env.REACT_APP_DEEPGRAM_API_KEY}` }
   });
   ws.binaryType = 'arraybuffer';
   ws.on("open", function open() {
       console.log('deepgram Voice Agent API: Connected');
       ws.send(JSON.stringify(CONFIG));
   });


   ws.on("message", function message(data, isBinary) {
       client_ws.send(data, { binary: isBinary });
   });


   ws.on('close', function close() {
       console.log('deepgram Voice Agent API: Disconnected from the WebSocket server');
   });


   ws.on('error', function error(error) {
       console.log("deepgram Voice Agent API: error received");
       console.error(error);
   });
   return ws;
}


wss.on("connection", (ws) => {
   console.log("socket: client connected");
   let deepgramWebsocket = setupDeepgramWebsocket(ws);


   ws.on("message", (message) => {
       if (deepgramWebsocket.readyState === WebSocket.OPEN) {
           deepgramWebsocket.send(message);
       }
   });


   ws.on("close", () => {
       console.log("socket: client disconnected");
   });
});


server.listen(3001, 'localhost', () => {
   console.log("Server is listening on port 3001");
});

```

### cal-hacks/src/setupTests.js

```javascript
// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';

```

### cal-hacks/src/App.test.js

```javascript
import { render, screen } from '@testing-library/react';
import App from './App';

test('renders learn react link', () => {
  render(<App />);
  const linkElement = screen.getByText(/learn react/i);
  expect(linkElement).toBeInTheDocument();
});

```

### cal-hacks/src/reportWebVitals.js

```javascript
const reportWebVitals = onPerfEntry => {
  if (onPerfEntry && onPerfEntry instanceof Function) {
    import('web-vitals').then(({ getCLS, getFID, getFCP, getLCP, getTTFB }) => {
      getCLS(onPerfEntry);
      getFID(onPerfEntry);
      getFCP(onPerfEntry);
      getLCP(onPerfEntry);
      getTTFB(onPerfEntry);
    });
  }
};

export default reportWebVitals;

```

### cal-hacks/src/index.css

```css
body {
  margin: 0;
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
    'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
    sans-serif;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

code {
  font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New',
    monospace;
}

```

### cal-hacks/public/index.html

```html
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <link rel="icon" href="%PUBLIC_URL%/favicon.ico" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <meta name="theme-color" content="#000000" />
    <meta
      name="description"
      content="Web site created using create-react-app"
    />
    <link rel="apple-touch-icon" href="%PUBLIC_URL%/logo192.png" />
    <!--
      manifest.json provides metadata used when your web app is installed on a
      user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
    -->
    <link rel="manifest" href="%PUBLIC_URL%/manifest.json" />
    <!--
      Notice the use of %PUBLIC_URL% in the tags above.
      It will be replaced with the URL of the `public` folder during the build.
      Only files inside the `public` folder can be referenced from the HTML.

      Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
      work correctly both with client-side routing and a non-root public URL.
      Learn how to configure a non-root public URL by running `npm run build`.
    -->
    <title>React App</title>
  </head>
  <body>
    <noscript>You need to enable JavaScript to run this app.</noscript>
    <div id="root"></div>
    <!--
      This HTML file is a template.
      If you open it directly in the browser, you will see an empty page.

      You can add webfonts, meta tags, or analytics to this file.
      The build step will place the bundled scripts into the <body> tag.

      To begin the development, run `npm start` or `yarn start`.
      To create a production bundle, use `npm run build` or `yarn build`.
    -->
  </body>
</html>

```

### cal-hacks/src/components/About.jsx

```javascript
const About = () => {
    return (
        <div>   
            about dead link
        </div>
    )
}

export default About;
```

[14 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]