# Project export: Conslt.AI This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 11.0 - Tagline: Conslt.AI: AI virtual consults connect patients to doctors instantly. We cut wait times, automate admin tasks, save doctors time. Revolutionizing healthcare: faster, efficient, accessible. - Devpost: https://devpost.com/software/docbuddy-9bfc4l - GitHub: https://github.com/G-Samarth/DocTalk-CalHacks-2024/ - Team: 1 GitHub contributor(s) — G-Samarth (11 commits) ## Devpost submission (written by the team) ### Inspiration The inspiration for Conslt.AI stems from the ongoing physician shortage in the U.S., which is predicted to reach a shortfall of up to 124,000 doctors by 2034, according to the Association of American Medical Colleges. With an aging population—projected to see a 42% increase in people aged 65 and older by 2034—the demand for healthcare services continues to rise. Additionally, over 25% of current physicians are approaching retirement age, further straining the healthcare system. These challenges result in long wait times, with some specialties averaging over 24 days for an appointment, and underserved communities facing even greater delays. This shortage not only impacts patient care but also increases the workload on physicians, contributing to burnout. Conslt.AI was created to address these systemic issues by leveraging AI to optimize doctors' time, improve patient access, and ultimately provide timely, and VERIFIED healthcare no matter where you are! Besides high demand, if we narrow the scope to a simple doctor and patient interaction, a doctor once performs their diagnosis after which it usually takes the doctor 15-20 minutes to write up a prescription, schedule future appointments with the patients and note down the tests required and so on. Usually so far, this either involves a human behind the scenes responsible for connecting this web of tasks or sometimes the doctors are smart about using transcribers. Even so, this would take about a day for all of the info to reach the patient. ### What it does Conslt.AI is an AI-powered platform that: Connects patients with doctors instantly through virtual consultations Transcribes and summarizes doctor-patient interactions in almost real-time Automatically generate prescriptions and session summaries based on AI analysis Suggests diagnoses based on the conversation and patient history Schedule follow-up appointments and manage future consultations Simplifies medical jargon for patients via an algorithm that is designed to understand medical jargon and mitigate it's difficulty to a layman's understanding With Conslt.AI, we aim to streamline the patient experience, save doctors valuable time, and help alleviate the healthcare burden brought on by the shortage of physicians. ### How we built it Conslt.AI was built using: Deepgram's audio analysis AI trained on medical data for transcribing audio to text Groq's LPUs with the Mixtral-turbo model for processing the transcribed text A python-flask backend system to manage patient information and doctor schedules A user-friendly React.JS interface for both doctors and patients Features Secure video chatting rooms for patient-doctors with real-time, consented session transcribing Prescriptions and doctor notes available almost immediately A network of doctors of all specialities is readily available to anyone in need In-state network insurance support Fast and reliable documentation and database querying ### Challenges we ran into Ensuring accuracy in complex medical cases and diverse specializations Maintaining data privacy and security in compliance with HIPAA regulations Scaling the solution to handle large volumes of patient interactions Securely hosting a monitored video chat without violating privacy and ensuring no data leaks by isolating data centers to only the hospital network ### Accomplishments we're proud of Creating a system that saves doctors 15-20 minutes per patient and a patient days of waiting for just a diagnosis Developing real-time processing capabilities for instant summaries and prescriptions Building a platform that addresses the critical issue of doctor shortages Designing a solution that improves healthcare efficiency and accessibility ### What we learned The complexities of integrating AI into healthcare systems The importance of balancing automation with human expertise in medical settings The critical nature of data security and privacy in healthcare applications The potential of AI to significantly impact and improve healthcare delivery Go-to market plan Our go-to-market strategy for Conslt.AI focuses on creating a monetization model that not only serves the healthcare providers efficiently but also delivers immense value to patients and institutions alike. Here's how we plan to achieve that: 1. Subscription Model for Healthcare Providers We will offer a tiered subscription plan for hospitals, clinics, and private practices based on the size of the institution and the number of doctors utilizing the platform. This ensures that larger institutions benefit from economies of scale, while smaller practices have affordable entry points. Basic Plan: For small clinics, offering basic consultation, transcription, and appointment scheduling services. Pro Plan: For medium to large healthcare providers, adding advanced diagnostic suggestions and EHR integration. Enterprise Plan: For hospital networks with fully customized solutions, priority support, and AI-driven decision support for complex cases. 2. Per-Consultation Fee for Patients While doctors and institutions subscribe to the platform, patients can be charged a nominal fee per virtual consultation. This pay-per-use model ensures affordability while making high-quality, AI-supported consultations accessible to a broad audience. 3. AI-Driven Prescription & EHR Integration Charging healthcare providers for premium AI features such as automatic prescription generation, integration with existing Electronic Health Record (EHR) systems, and advanced reporting tools that offer data-driven insights into patient care and operational efficiency. These add-ons will help institutions save time, reduce administrative overhead, and improve patient outcomes, making Conslt.AI a critical part of their workflow. 4. Strategic Partnerships with Healthcare Networks By partnering with large healthcare providers and insurance companies, we can offer bulk licenses at discounted rates, bundling our service with other telemedicine solutions. This drives adoption and positions Conslt.AI as a comprehensive tool that complements existing systems. 5. Data Insights & Analytics for Healthcare Institutions Aggregated, anonymized data from patient interactions (in compliance with HIPAA) can provide hospitals and clinics with insights into operational efficiencies, patient care trends, and areas for improvement. Institutions can subscribe to our analytics service to access these valuable insights and improve decision-making. Ethical and moral implications that need to be taken care of before the platform goes to market Before Conslt.AI can go to market, several critical ethical and moral considerations must be addressed, particularly given the sensitive nature of healthcare data and AI's involvement in patient care. Here are the key areas we are focusing on to ensure our platform operates responsibly: 1. Data Privacy and Security Healthcare data is highly sensitive, and any mishandling can lead to serious consequences for both patients and healthcare providers. Conslt.AI will strictly comply with HIPAA (Health Insurance Portability and Accountability Act) regulations to ensure the highest levels of privacy and security for patient data. We will implement strong encryption protocols for both data at rest and data in transit, ensuring that no unauthorized access occurs at any point in the data lifecycle. 2. Verification of Medical Information One of the most critical aspects of Conslt.AI is ensuring that all medical information provided is accurate and comes from verified healthcare professionals. The platform will implement rigorous vetting processes for doctors, requiring licenses, certifications, and background checks before granting access to our system. Additionally, AI-generated summaries and diagnoses will be reviewed by licensed physicians to ensure that only accurate and trustworthy information is provided to patients. 3. AI Bias and Fairness AI systems can unintentionally perpetuate biases if they are trained on skewed or unrepresentative data. In healthcare, this could result in misdiagnoses or unequal treatment based on race, gender, or socioeconomic status. To combat this, Conslt.AI will undergo continuous AI bias audits to identify and address any discrepancies in the algorithms. Our models will be trained on diverse datasets to ensure fair and accurate recommendations across different patient demographics. 4. AI Moderation and Decision-Making Limits While AI can assist with diagnoses and summarizing consultations, it must not overstep its bounds. Conslt.AI will maintain a strict policy that all final medical decisions are made by human doctors, not by AI. This ensures that the platform remains a tool to assist physicians, not to replace their judgment. Clear AI moderation mechanisms will be implemented, flagging any ambiguous or potentially harmful recommendations for further review by human professionals. ### What's next Expanding the platform to support more medical specialties Integrating with existing Electronic Health Record (EHR) systems Developing features for telemedicine and emergency room support Enhancing the AI's capability to handle more complex medical scenarios Exploring partnerships with healthcare providers to implement the system at scale ## README (from the GitHub repository) # DocTalk-CalHacks-2024 **Doctor's Assistant AI** is a real-time assistant designed to automate and accelerate doctors’ daily administrative tasks. This project aims to simplify and speed up patient consultations by instantly summarizing doctor-patient interactions, generating prescriptions, scheduling follow-ups, and providing all necessary patient information in one place. The goal is to save doctors time and improve patient care, especially in high-demand healthcare systems. --- ## **Features** - **Real-time Summarization**: Automatically transcribes and summarizes patient consultations. - **Instant Prescriptions**: Generates prescriptions immediately after the consultation. - **Appointment Scheduling**: Schedules follow-up appointments in real time. - **Simplified Medical Information**: Converts complex medical jargon into patient-friendly language. - **Centralized Patient Information**: All patient details are stored in one place for easy access by healthcare providers. --- ## **Project Structure** The project is divided into two main parts: - **Backend**: Handles data processing, AI-powered transcription, and data storage. - **Frontend**: User interface for doctors to review summaries, manage patient information, and schedule appointments. --- ## **Tech Stack** - **Backend**: Node.js, Express.js - **Frontend**: React.js - **Database**: MongoDB - **AI Processing**: Machine Learning models (e.g., NLP for summarization) --- ## **Installation and Running the Application** Follow these steps to get the application up and running: ### **1. Backend Setup** 1. Open your terminal and navigate to the backend folder: ```bash cd backend ``` 2. Install the necessary dependencies: ```bash npm install ``` 3. Start the server: ```bash node server.js ``` The backend server should now be running at `http://localhost:4000`. ### **2. Frontend Setup** 1. In a new terminal window, navigate to the frontend folder: ```bash cd frontend ``` 2. Install the necessary dependencies: ```bash npm install ``` 3. Start the frontend application: ```bash npm start ``` The frontend should now be running at `http://localhost:3000`. --- ## **Usage** 1. **Start Consultation**: Doctors start a patient consultation, which is automatically recorded. 2. **Real-time Processing**: The AI processes the audio and generates a consultation summary, prescription, and appointment scheduling. 3. **Review**: Doctors can review, edit, and confirm the AI-generated information before sending it to the patient. --- ## **Contributing** If you would like to contribute to this project, please fork the repository and submit a pull request. We welcome improvements, bug fixes, and suggestions! --- ## **License** This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for details. ## Detected evidence (automated analysis) Indexed codebase: 33 recognized source files, 89 KB. - CSS (language) — detected in the code - Express (technology) — detected in the code - Firebase (technology) — detected in the code - Flask (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - LlamaIndex (technology) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code ## Codebase structure (from repository index) ### Files (41 of 41) ``` .gitignore backend/.gitignore backend/package.json backend/server.js frontend/.gitignore frontend/package.json frontend/pdf.d.ts frontend/postcss.config.js frontend/public/index.html frontend/public/manifest.json frontend/public/robots.txt frontend/README.md frontend/src/App.css frontend/src/App.js frontend/src/App.test.js frontend/src/components/Button/Button.js frontend/src/components/CalendarPage/CalendarPage.js frontend/src/components/Card/Card.js frontend/src/components/DoctorsPage/DoctorsPage.js frontend/src/components/IncomingCallNotification/IncomingCallNotification.js frontend/src/components/LandingPage/LandingPage.css frontend/src/components/LandingPage/LandingPage.js frontend/src/components/Layout/Layout.js frontend/src/components/PatientCalendarPage/PatientCalendarPage.js frontend/src/components/ReviewPDF/ReviewPDF.js frontend/src/components/VideoCall/VideoCall.js frontend/src/components/VideoChat/VideoChat.css frontend/src/components/VideoChat/VideoChat.js frontend/src/components/VideoChatNew/VideoChat.css frontend/src/components/VideoChatNew/VideoChat.js frontend/src/firebase.js frontend/src/index.css frontend/src/index.js frontend/src/reportWebVitals.js frontend/src/setupTests.js frontend/tailwind.config.js ml/audio_to_text.py ml/firstApi.py ml/pdf.py ml/requirements.txt README.md ``` ### Dependencies - backend/package.json: axios@^1.7.7, cors@^2.8.5, dotenv@^16.4.5, express@^4.21.1, firebase-admin@^12.6.0, form-data@^4.0.1, multer@^1.4.5-lts.1, socket.io@^4.8.0 - frontend/package.json: @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^13.5.0, autoprefixer@^10.4.20, date-fns@^4.1.0, firebase@^10.14.1, lucide-react@^0.453.0, pdfjs-dist@^4.7.76, postcss@^8.4.47, react@^18.3.1, react-day-picker@^9.1.4, react-dom@^18.3.1, react-pdf@^9.1.1, react-router-dom@^6.27.0, react-scripts@5.0.1, simple-peer@^9.11.1, socket.io-client@^4.8.0, tailwindcss@^3.4.14, web-vitals@^2.1.4 - ml/requirements.txt: aenum@==3.1.15, aiofiles@==24.1.0, aiohappyeyeballs@==2.4.3, aiohttp@==3.10.10, aiosignal@==1.3.1, annotated-types@==0.7.0, anyio@==4.6.2.post1, attrs@==24.2.0, beautifulsoup4@==4.12.3, blinker@==1.8.2, CacheControl@==0.14.0, cachetools@==5.5.0, certifi@==2024.8.30, cffi@==1.17.1, chardet@==5.2.0, charset-normalizer@==3.4.0, click@==8.1.7, cryptography@==43.0.3, dataclasses-json@==0.6.7, deepgram-sdk@==3.7.4, Deprecated@==1.2.14, deprecation@==2.1.0, dirtyjson@==1.0.8, distro@==1.9.0, firebase-admin@==6.5.0, Flask@==3.0.3, frozenlist@==1.4.1, fsspec@==2024.9.0, google-api-core@==2.21.0, google-api-python-client@==2.149.0, google-auth@==2.35.0, google-auth-httplib2@==0.2.0, google-cloud-core@==2.4.1, google-cloud-firestore@==2.19.0, google-cloud-storage@==2.18.2, google-crc32c@==1.6.0, google-resumable-media@==2.7.2, googleapis-common-protos@==1.65.0, greenlet@==3.1.1, groq@==0.11.0, grpcio@==1.67.0, grpcio-status@==1.67.0, h11@==0.14.0, httpcore@==1.0.6, httplib2@==0.22.0, httpx@==0.27.2, idna@==3.10, itsdangerous@==2.2.0, Jinja2@==3.1.4, jiter@==0.6.1, joblib@==1.4.2, llama-cloud@==0.1.4, llama-index@==0.11.18, llama-index-agent-openai@==0.3.4, llama-index-cli@==0.3.1, llama-index-core@==0.11.19, llama-index-embeddings-openai@==0.2.5, llama-index-indices-managed-llama-cloud@==0.4.0, llama-index-legacy@==0.9.48.post3, llama-index-llms-openai@==0.2.15, llama-index-multi-modal-llms-openai@==0.2.2, llama-index-program-openai@==0.2.0, llama-index-question-gen-openai@==0.2.0, llama-index-readers-file@==0.2.2, llama-index-readers-llama-parse@==0.3.0, llama-parse@==0.5.10, MarkupSafe@==3.0.2, marshmallow@==3.23.0, msgpack@==1.1.0, multidict@==6.1.0, mypy-extensions@==1.0.0, nest-asyncio@==1.6.0, networkx@==3.4.1, nltk@==3.9.1, numpy@==1.26.4, openai@==1.52.0, packaging@==24.1, pandas@==2.2.3, pdfkit@==1.0.0, pdfrw@==0.4, pillow@==11.0.0, propcache@==0.2.0, proto-plus@==1.24.0, protobuf@==5.28.2, pyasn1@==0.6.1, pyasn1_modules@==0.4.1, pycparser@==2.22, pydantic@==2.9.2, pydantic_core@==2.23.4, PyJWT@==2.9.0, pyparsing@==3.2.0, pypdf@==4.3.1, python-dateutil@==2.9.0.post0, python-dotenv@==1.0.1, pytz@==2024.2, PyYAML@==6.0.2, regex@==2024.9.11, reportlab@==4.2.5, requests@==2.32.3, rsa@==4.9, six@==1.16.0, sniffio@==1.3.1, soupsieve@==2.6, SQLAlchemy@==2.0.36, striprtf@==0.0.26, tenacity@==8.5.0, tiktoken@==0.8.0, tqdm@==4.66.5, typing_extensions@==4.12.2, typing-inspect@==0.9.0, tzdata@==2024.2, uritemplate@==4.1.1, urllib3@==2.2.3, websockets@==13.1, Werkzeug@==3.0.4, wrapt@==1.16.0, yarl@==1.15.5 ### Recent commits (newest first) - Update pdf.py to remove secrets - removing secrets - AI scripts added - final project - footer back - added patient calendar page - firebase connection - dynamic support for both userTypes - video chat updates - ui enhancements - Update README.md - base project - Initial commit ## Key source files (fetched from GitHub, selected and truncated for size) ### backend/package.json ``` { "name": "backend", "version": "1.0.0", "main": "index.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, "keywords": [], "author": "", "license": "ISC", "description": "", "dependencies": { "axios": "^1.7.7", "cors": "^2.8.5", "dotenv": "^16.4.5", "express": "^4.21.1", "firebase-admin": "^12.6.0", "form-data": "^4.0.1", "multer": "^1.4.5-lts.1", "socket.io": "^4.8.0" } } ``` ### frontend/package.json ``` { "name": "frontend", "version": "0.1.0", "proxy": "http://localhost:4000", "private": true, "dependencies": { "@testing-library/jest-dom": "^5.17.0", "@testing-library/react": "^13.4.0", "@testing-library/user-event": "^13.5.0", "date-fns": "^4.1.0", "firebase": "^10.14.1", "lucide-react": "^0.453.0", "pdfjs-dist": "^4.7.76", "react": "^18.3.1", "react-day-picker": "^9.1.4", "react-dom": "^18.3.1", "react-pdf": "^9.1.1", "react-router-dom": "^6.27.0", "react-scripts": "5.0.1", "simple-peer": "^9.11.1", "socket.io-client": "^4.8.0", "web-vitals": "^2.1.4" }, "scripts": { "start": "react-scripts start", "build": "react-scripts build", "test": "react-scripts test", "eject": "react-scripts eject" }, "eslintConfig": { "extends": [ "react-app", "react-app/jest" ] }, "browserslist": { "production": [ ">0.2%", "not dead", "not op_mini all" ], "development": [ "last 1 chrome version", "last 1 firefox version", "last 1 safari version" ] }, "devDependencies": { "autoprefixer": "^10.4.20", "postcss": "^8.4.47", "tailwindcss": "^3.4.14" } } ``` ### ml/requirements.txt ``` aenum==3.1.15 aiofiles==24.1.0 aiohappyeyeballs==2.4.3 aiohttp==3.10.10 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.6.2.post1 attrs==24.2.0 beautifulsoup4==4.12.3 blinker==1.8.2 CacheControl==0.14.0 cachetools==5.5.0 certifi==2024.8.30 cffi==1.17.1 chardet==5.2.0 charset-normalizer==3.4.0 click==8.1.7 cryptography==43.0.3 dataclasses-json==0.6.7 deepgram-sdk==3.7.4 Deprecated==1.2.14 deprecation==2.1.0 dirtyjson==1.0.8 distro==1.9.0 firebase-admin==6.5.0 Flask==3.0.3 frozenlist==1.4.1 fsspec==2024.9.0 google-api-core==2.21.0 google-api-python-client==2.149.0 google-auth==2.35.0 google-auth-httplib2==0.2.0 google-cloud-core==2.4.1 google-cloud-firestore==2.19.0 google-cloud-storage==2.18.2 google-crc32c==1.6.0 google-resumable-media==2.7.2 googleapis-common-protos==1.65.0 greenlet==3.1.1 groq==0.11.0 grpcio==1.67.0 grpcio-status==1.67.0 h11==0.14.0 httpcore==1.0.6 httplib2==0.22.0 httpx==0.27.2 idna==3.10 itsdangerous==2.2.0 Jinja2==3.1.4 jiter==0.6.1 joblib==1.4.2 llama-cloud==0.1.4 llama-index==0.11.18 llama-index-agent-openai==0.3.4 llama-index-cli==0.3.1 llama-index-core==0.11.19 llama-index-embeddings-openai==0.2.5 llama-index-indices-managed-llama-cloud==0.4.0 llama-index-legacy==0.9.48.post3 llama-index-llms-openai==0.2.15 llama-index-multi-modal-llms-openai==0.2.2 llama-index-program-openai==0.2.0 llama-index-question-gen-openai==0.2.0 llama-index-readers-file==0.2.2 llama-index-readers-llama-parse==0.3.0 llama-parse==0.5.10 MarkupSafe==3.0.2 marshmallow==3.23.0 msgpack==1.1.0 multidict==6.1.0 mypy-extensions==1.0.0 nest-asyncio==1.6.0 networkx==3.4.1 nltk==3.9.1 numpy==1.26.4 openai==1.52.0 packaging==24.1 pandas==2.2.3 pdfkit==1.0.0 pdfrw==0.4 pillow==11.0.0 propcache==0.2.0 proto-plus==1.24.0 protobuf==5.28.2 pyasn1==0.6.1 pyasn1_modules==0.4.1 pycparser==2.22 pydantic==2.9.2 pydantic_core==2.23.4 PyJWT==2.9.0 pyparsing==3.2.0 pypdf==4.3.1 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 pytz==2024.2 PyYAML==6.0.2 regex==2024.9.11 reportlab==4.2.5 requests==2.32.3 rsa==4.9 six==1.16.0 sniffio==1.3.1 soupsieve==2.6 SQLAlchemy==2.0.36 striprtf==0.0.26 tenacity==8.5.0 tiktoken==0.8.0 tqdm==4.66.5 typing-inspect==0.9.0 typing_extensions==4.12.2 tzdata==2024.2 uritemplate==4.1.1 urllib3==2.2.3 websockets==13.1 Werkzeug==3.0.4 wrapt==1.16.0 yarl==1.15.5 ``` ### backend/server.js ```javascript const express = require("express"); const http = require("http"); const https = require("https"); const socketIo = require("socket.io"); const cors = require("cors"); const multer = require("multer"); const admin = require("firebase-admin"); const serviceAccount = require("./firebase-adminsdk.json"); const axios = require("axios"); const fs = require("fs"); const path = require("path"); const FormData = require("form-data"); require("dotenv").config(); const app = express(); const server = http.createServer(app); const io = socketIo(server, { cors: { origin: "http://localhost:3000", methods: ["GET", "POST"], }, }); app.use(cors()); app.use(express.json()); const apiKey = process.env.DAILY_API_KEY; // Initialize Firebase admin.initializeApp({ credential: admin.credential.cert(serviceAccount), storageBucket: "doctalk-9e28b.appspot.com", }); const bucket = admin.storage().bucket(); // Function to check Firebase connection async function checkFirebaseConnection() { try { await bucket.exists(); console.log("Successfully connected to Firebase Storage"); return true; } catch (error) { console.error("Failed to connect to Firebase Storage:", error); return false; } } // Use this function in your server startup checkFirebaseConnection().then((isConnected) => { if (isConnected) { // Start your server or proceed with other initializations console.log("Server is ready to use Firebase Storage"); } else { console.log("Please check your Firebase configuration"); } }); async function setupFolders() { const folders = ["audio_recordings", "pdfs"]; for (const folder of folders) { try { // Check if folder exists const [files] = await bucket.getFiles({ prefix: `${folder}/` }); if (files.length === 0) { // If folder doesn't exist, create an empty file to establish the folder await bucket.file(`${folder}/.placeholder`).save(""); console.log(`Created folder: ${folder}`); } else { console.log(`Folder already exists: ${folder}`); } } catch (error) { console.error(`Error setting up folder ${folder}:`, error); } } } // Call this function during your server initialization setupFolders(); // Set up multer for handling file uploads const multerStorage = multer.memoryStorage(); const upload = multer({ storage: multerStorage }); // Helper function to generate a unique filename const generateUniqueFilename = (originalName, prefix) => { const timestamp = Date.now(); const randomString = Math.random().toString(36).substring(2, 15); const extension = originalName.split(".").pop(); return `${prefix}_${timestamp}_${randomString}.${extension}`; }; app.post("/save-audio", upload.single("audio"), async (req, res) => { if (!req.file) { console.log("No file received in the request"); return res.status(400).send("No file uploaded."); } const file = req.file; const fileName = generateUniqueFilename(file.originalname, "audio"); const filePath = `audio_recordings/${fileName}`; console.log(`Attempting to upload file: ${filePath}`); const fileUpload = bucket.file(filePath); const blobStream = fileUpload.createWriteStream({ metadata: { contentType: file.mimetype, }, }); blobStream.on("error", (error) => { console.error("Error uploading to Firebase:", error); res.status(500).send("An error occurred during file upload."); }); blobStream.on("finish", () => { console.log(`File uploaded successfully: ${filePath}`); res.status(200).json({ message: "File uploaded successfully to Firebase", fileName: filePath, }); }); blobStream.end(file.buffer); }); // Add a new route to handle other file uploads app.post("/save-file", upload.single("file"), async (req, res) => { if (!req.file) { return res.status(400).send("No file uploaded."); } const file = req.file; const fileName = generateUniqueFilename(file.originalname, "file"); const filePath = `pdfs/${fileName}`; const fileUpload = bucket.file(filePath); const blobStream = fileUpload.createWriteStream({ metadata: { contentType: file.mimetype, }, }); blobStream.on("error", (error) => { console.error("Error uploading to Firebase:", error); res.status(500).send("An error occurred during file upload."); }); blobStream.on("finish", () => { res.status(200).json({ message: "File uploaded successfully to Firebase", fileName: filePath, }); }); blobStream.end(file.buffer); }); const PORT = process.env.PORT || 4000; let doctorSocket = null; let patientSocket = null; io.on("connection", (socket) => { socket.on("request_id", (role) => { if (role === "doctor") { if (doctorSocket) { socket.emit("error", "A doctor is already connected"); return; } doctorSocket = socket; socket.emit("assigned_id", "doctor"); } else if (role === "patient") { if (patientSocket) { socket.emit("error", "A patient is already connected"); return; } patientSocket = socket; socket.emit("assigned_id", "patient"); } else { socket.emit("error", "Invalid role"); } }); socket.on("disconnect", () => { if (socket === doctorSocket) { doctorSocket = null; } else if (socket === patientSocket) { patientSocket = null; } socket.broadcast.emit("callEnded"); }); socket.on("callUser", ({ userToCall, signalData, from, name }) => { const targetSocket = userToCall === "doctor" ? doctorSocket : patientSocket; if (targetSocket) { targetSocket.emit("callUser", { signal: signalData, from, name }); } }); socket.on("answerCall", (data) => { const targetSocket = data.to === "doctor" ? doctorSocket : patientSocket; if (targetSocket) { targetSocket.emit("callAccepted", data.signal); } }); }); const recordingsDir = path.join(__dirname, "recordings"); if (! [truncated — 5097 more characters] ``` ### frontend/src/index.js ```javascript import React from "react"; import ReactDOM from "react-dom"; import "./index.css"; import App from "./App"; import "./index.css"; ReactDOM.render( , document.getElementById("root") ); ``` ### frontend/src/App.js ```javascript import React from "react"; import { BrowserRouter as Router, Route, Routes, Navigate, } from "react-router-dom"; import LandingPage from "./components/LandingPage/LandingPage"; import CalendarPage from "./components/CalendarPage/CalendarPage"; import PatientCalendarPage from "./components/PatientCalendarPage/PatientCalendarPage"; import VideoChat from "./components/VideoChat/VideoChat"; import VideoChatNew from "./components/VideoChatNew/VideoChat"; import ReviewPDF from "./components/ReviewPDF/ReviewPDF"; import DoctorsPage from "./components/DoctorsPage/DoctorsPage"; import VideoCallPage from "./components/VideoCall/VideoCall"; import "./index.css"; const App = () => { return ( } /> } /> } /> } /> } /> {/* } /> } /> */} } /> } /> } /> } /> } /> } /> ); }; export default App; ``` ### frontend/pdf.d.ts ```typescript declare module "pdfjs-dist/build/pdf.worker.entry"; ``` ### frontend/postcss.config.js ```javascript module.exports = { plugins: { tailwindcss: {}, autoprefixer: {}, }, }; ``` ### frontend/tailwind.config.js ```javascript module.exports = { content: ["./src/**/*.{js,jsx,ts,tsx}"], theme: { extend: {}, }, plugins: [], }; ``` ### ml/audio_to_text.py ```python import os from deepgram import Deepgram from dotenv import load_dotenv import json from deepgram import( DeepgramClient, PrerecordedOptions, FileSource, ) # Load env variable load_dotenv() # Get the current file's directory # current_dir = os.path.dirname(os.path.abspath(_file_)) # Access a file in the same directory # AUDIO_FILE_PATH = os.path.join(current_dir, 'audiofile.wav') # Fetch API key from env file API_KEY = os.getenv("DG_API_KEY") def main(audio_file_path): try: # STEP 1 Create a Deepgram client using the API key deepgram = DeepgramClient(API_KEY) with open(audio_file_path, "rb") as file: buffer_data = file.read() payload: FileSource = { "buffer": buffer_data, } #STEP 2: Configure Deepgram options for audio analysis options = PrerecordedOptions( model="nova-2-medical", punctuate= True, smart_format=True, diarize= True, language= 'en-US', ) # STEP 3: Call the transcribe_file method with the text payload and options response = deepgram.listen.rest.v("1").transcribe_file(payload, options, timeout = 300) # STEP 4: return the response response = response.to_json(indent=4) transcript = json.loads(response) return transcript['results']['channels'][0]['alternatives'][0]['paragraphs']['transcript'] # Error handling for no file found except FileNotFoundError: print(f"Audio file not found at path: {audio_file_path}") except Exception as e: print(f"Exception: {e}") if __name__ == "_main_": main() ``` [22 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]