Project Info
This project did not submit a demo video on Devpost.
Inspiration
The inspiration for Conslt.AI stems from the ongoing physician shortage in the U.S., which is predicted to reach a shortfall of up to 124,000 doctors by 2034, according to the Association of American Medical Colleges. With an aging population—projected to see a 42% increase in people aged 65 and older by 2034—the demand for healthcare services continues to rise. Additionally, over 25% of current physicians are approaching retirement age, further straining the healthcare system. These challenges result in long wait times, with some specialties averaging over 24 days for an appointment, and underserved communities facing even greater delays. This shortage not only impacts patient care but also increases the workload on physicians, contributing to burnout. Conslt.AI was created to address these systemic issues by leveraging AI to optimize doctors' time, improve patient access, and ultimately provide timely, and VERIFIED healthcare no matter where you are! Besides high demand, if we narrow the scope to a simple doctor and patient interaction, a doctor once performs their diagnosis after which it usually takes the doctor 15-20 minutes to write up a prescription, schedule future appointments with the patients and note down the tests required and so on. Usually so far, this either involves a human behind the scenes responsible for connecting this web of tasks or sometimes the doctors are smart about using transcribers. Even so, this would take about a day for all of the info to reach the patient.
What it does
Conslt.AI is an AI-powered platform that: Connects patients with doctors instantly through virtual consultations Transcribes and summarizes doctor-patient interactions in almost real-time Automatically generate prescriptions and session summaries based on AI analysis Suggests diagnoses based on the conversation and patient history Schedule follow-up appointments and manage future consultations Simplifies medical jargon for patients via an algorithm that is designed to understand medical jargon and mitigate it's difficulty to a layman's understanding With Conslt.AI, we aim to streamline the patient experience, save doctors valuable time, and help alleviate the healthcare burden brought on by the shortage of physicians.
How we built it
Conslt.AI was built using: Deepgram's audio analysis AI trained on medical data for transcribing audio to text Groq's LPUs with the Mixtral-turbo model for processing the transcribed text A python-flask backend system to manage patient information and doctor schedules A user-friendly React.JS interface for both doctors and patients Features Secure video chatting rooms for patient-doctors with real-time, consented session transcribing Prescriptions and doctor notes available almost immediately A network of doctors of all specialities is readily available to anyone in need In-state network insurance support Fast and reliable documentation and database querying
Challenges we ran into
Ensuring accuracy in complex medical cases and diverse specializations Maintaining data privacy and security in compliance with HIPAA regulations Scaling the solution to handle large volumes of patient interactions Securely hosting a monitored video chat without violating privacy and ensuring no data leaks by isolating data centers to only the hospital network
Accomplishments we're proud of
Creating a system that saves doctors 15-20 minutes per patient and a patient days of waiting for just a diagnosis Developing real-time processing capabilities for instant summaries and prescriptions Building a platform that addresses the critical issue of doctor shortages Designing a solution that improves healthcare efficiency and accessibility
What we learned
The complexities of integrating AI into healthcare systems The importance of balancing automation with human expertise in medical settings The critical nature of data security and privacy in healthcare applications The potential of AI to significantly impact and improve healthcare delivery Go-to market plan Our go-to-market strategy for Conslt.AI focuses on creating a monetization model that not only serves the healthcare providers efficiently but also delivers immense value to patients and institutions alike. Here's how we plan to achieve that: 1. Subscription Model for Healthcare Providers We will offer a tiered subscription plan for hospitals, clinics, and private practices based on the size of the institution and the number of doctors utilizing the platform. This ensures that larger institutions benefit from economies of scale, while smaller practices have affordable entry points. Basic Plan: For small clinics, offering basic consultation, transcription, and appointment scheduling services. Pro Plan: For medium to large healthcare providers, adding advanced diagnostic suggestions and EHR integration. Enterprise Plan: For hospital networks with fully customized solutions, priority support, and AI-driven decision support for complex cases. 2. Per-Consultation Fee for Patients While doctors and institutions subscribe to the platform, patients can be charged a nominal fee per virtual consultation. This pay-per-use model ensures affordability while making high-quality, AI-supported consultations accessible to a broad audience. 3. AI-Driven Prescription & EHR Integration Charging healthcare providers for premium AI features such as automatic prescription generation, integration with existing Electronic Health Record (EHR) systems, and advanced reporting tools that offer data-driven insights into patient care and operational efficiency. These add-ons will help institutions save time, reduce administrative overhead, and improve patient outcomes, making Conslt.AI a critical part of their workflow. 4. Strategic Partnerships with Healthcare Networks By partnering with large healthcare providers and insurance companies, we can offer bulk licenses at discounted rates, bundling our service with other telemedicine solutions. This drives adoption and positions Conslt.AI as a comprehensive tool that complements existing systems. 5. Data Insights & Analytics for Healthcare Institutions Aggregated, anonymized data from patient interactions (in compliance with HIPAA) can provide hospitals and clinics with insights into operational efficiencies, patient care trends, and areas for improvement. Institutions can subscribe to our analytics service to access these valuable insights and improve decision-making. Ethical and moral implications that need to be taken care of before the platform goes to market Before Conslt.AI can go to market, several critical ethical and moral considerations must be addressed, particularly given the sensitive nature of healthcare data and AI's involvement in patient care. Here are the key areas we are focusing on to ensure our platform operates responsibly: 1. Data Privacy and Security Healthcare data is highly sensitive, and any mishandling can lead to serious consequences for both patients and healthcare providers. Conslt.AI will strictly comply with HIPAA (Health Insurance Portability and Accountability Act) regulations to ensure the highest levels of privacy and security for patient data. We will implement strong encryption protocols for both data at rest and data in transit, ensuring that no unauthorized access occurs at any point in the data lifecycle. 2. Verification of Medical Information One of the most critical aspects of Conslt.AI is ensuring that all medical information provided is accurate and comes from verified healthcare professionals. The platform will implement rigorous vetting processes for doctors, requiring licenses, certifications, and background checks before granting access to our system. Additionally, AI-generated summaries and diagnoses will be reviewed by licensed physicians to ensure that only accurate and trustworthy information is provided to patients. 3. AI Bias and Fairness AI systems can unintentionally perpetuate biases if they are trained on skewed or unrepresentative data. In healthcare, this could result in misdiagnoses or unequal treatment based on race, gender, or socioeconomic status. To combat this, Conslt.AI will undergo continuous AI bias audits to identify and address any discrepancies in the algorithms. Our models will be trained on diverse datasets to ensure fair and accurate recommendations across different patient demographics. 4. AI Moderation and Decision-Making Limits While AI can assist with diagnoses and summarizing consultations, it must not overstep its bounds. Conslt.AI will maintain a strict policy that all final medical decisions are made by human doctors, not by AI. This ensures that the platform remains a tool to assist physicians, not to replace their judgment. Clear AI moderation mechanisms will be implemented, flagging any ambiguous or potentially harmful recommendations for further review by human professionals.
What's next
Expanding the platform to support more medical specialties Integrating with existing Electronic Health Record (EHR) systems Developing features for telemedicine and emergency room support Enhancing the AI's capability to handle more complex medical scenarios Exploring partnerships with healthcare providers to implement the system at scale
DocTalk-CalHacks-2024
Doctor's Assistant AI is a real-time assistant designed to automate and accelerate doctors’ daily administrative tasks. This project aims to simplify and speed up patient consultations by instantly summarizing doctor-patient interactions, generating prescriptions, scheduling follow-ups, and providing all necessary patient information in one place. The goal is to save doctors time and improve patient care, especially in high-demand healthcare systems.
Features
- Real-time Summarization: Automatically transcribes and summarizes patient consultations.
- Instant Prescriptions: Generates prescriptions immediately after the consultation.
- Appointment Scheduling: Schedules follow-up appointments in real time.
- Simplified Medical Information: Converts complex medical jargon into patient-friendly language.
- Centralized Patient Information: All patient details are stored in one place for easy access by healthcare providers.
Project Structure
The project is divided into two main parts:
- Backend: Handles data processing, AI-powered transcription, and data storage.
- Frontend: User interface for doctors to review summaries, manage patient information, and schedule appointments.
Tech Stack
- Backend: Node.js, Express.js
- Frontend: React.js
- Database: MongoDB
- AI Processing: Machine Learning models (e.g., NLP for summarization)
Installation and Running the Application
Follow these steps to get the application up and running:
1. Backend Setup
- Open your terminal and navigate to the backend folder:
cd backend - Install the necessary dependencies:
npm install - Start the server:
The backend server should now be running atnode server.jshttp://localhost:4000.
2. Frontend Setup
- In a new terminal window, navigate to the frontend folder:
cd frontend - Install the necessary dependencies:
npm install - Start the frontend application:
The frontend should now be running atnpm starthttp://localhost:3000.
Usage
- Start Consultation: Doctors start a patient consultation, which is automatically recorded.
- Real-time Processing: The AI processes the audio and generates a consultation summary, prescription, and appointment scheduling.
- Review: Doctors can review, edit, and confirm the AI-generated information before sending it to the patient.
Contributing
If you would like to contribute to this project, please fork the repository and submit a pull request. We welcome improvements, bug fixes, and suggestions!
License
This project is licensed under the MIT License. See the LICENSE file for details.
Analysis
View
Metric
- 11
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- CSSIn code
- ExpressIn code
- FirebaseIn code
- FlaskIn code
- HTMLIn code
- JavaScriptIn code
- LlamaIndexIn code
- OpenAIIn code
- PythonIn code
- ReactIn code
- Tailwind CSSIn code
- TypeScriptIn code
12 of 12 appear in the indexed code.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
89 KB
Source files
33
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
G-Samarth/DocTalk-CalHacks-2024
47 files · 3.6 MB · @ ae11ec2
Structure
Interface
15 files · 32%Screens, components and styles rendered to the user.
Application logic
12 files · 26%Domain rules, services and shared utilities.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- JavaScript77%
- Python12%
- Markdown7%
- HTML2%
- CSS2%
- TypeScript0%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
ml/requirements.txt
pypi · 117- aenum
- aiofiles
- aiohappyeyeballs
- aiohttp
- aiosignal
- annotated-types
- anyio
- attrs
- beautifulsoup4
- blinker
- CacheControl
- cachetools
- certifi
- cffi
- chardet
- charset-normalizer
- click
- cryptography
- +99 more
frontend/package.json
npm · 19- @testing-library/jest-dom
- @testing-library/react
- @testing-library/user-event
- date-fns
- firebase
- lucide-react
- pdfjs-dist
- react
- react-day-picker
- react-dom
- react-pdf
- react-router-dom
- react-scripts
- simple-peer
- socket.io-client
- web-vitals
- +3 more
backend/package.json
npm · 8- axios
- cors
- dotenv
- express
- firebase-admin
- form-data
- multer
- socket.io
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
Feature verification
Automatic prescription and session summary generation via AI (Groq)Verified
Automatically generate prescriptions and session summaries based on AI analysis using Groq's Mixtral model
Claimed on Devposthigh confidenceml/firstApi.py:58— Sends transcription to Groq chat completion (model mixtral-8x7b-32768) with a prompt asking for medicines, exercises, diagnosis, next appointment, tests, and summary in JSONml/pdf.py:39— generate_pdf reads the Groq JSON output and renders it into an HTML/PDF prescription template, then uploads to Firebase Storage
AI-suggested diagnosis based on conversation and patient historyCode-supported
Suggests diagnoses based on the conversation and patient history
Claimed on readmelow confidenceml/firstApi.py:44— The Groq prompt asks the model to extract a 'diagnosis by the doctor' field from the transcript, but this only extracts what the doctor already stated in conversation rather than suggesting a diagnosis from patient history; no patient history data is fed into the prompt
Audio recording and transcription of doctor-patient conversationCode-supported
Transcribes and summarizes doctor-patient interactions in almost real-time using Deepgram audio analysis AI
Claimed on Devpostmedium confidenceml/audio_to_text.py:24— Uses Deepgram SDK (nova-2-medical model) to transcribe an audio file to text with diarizationml/firstApi.py:30— Flask endpoint /process_transcription calls get_transcription(temp_path) on an uploaded audio filebackend/server.js:392— sendFileToFlaskServer forwards a downloaded Daily.co recording to the Flask /process_transcription endpoint, wiring recording to transcription
Automatic follow-up appointment schedulingCode-supported
Schedule follow-up appointments and manage future consultations
Claimed on Devpostlow confidenceml/pdf.py:18— generate_random_appointment() produces a randomly generated follow-up date/time inserted into the PDF, not a real scheduling based on doctor/patient availabilityfrontend/src/components/CalendarPage/CalendarPage.js:20— Appointments shown on the calendar page are a hardcoded allAppointments object with fixed dates/names, not dynamically created by AI or backed by a database
Centralized patient information storage (documentation and database querying)Code-supported
Fast and reliable documentation and database querying; centralized patient information stored in one place
Claimed on readmemedium confidencebackend/server.js:30— Firebase Admin Storage is used to store audio recordings and generated PDF prescriptions in named folders, functioning as centralized file storagefrontend/src/components/CalendarPage/CalendarPage.js:48— CalendarPage lists PDF files pulled from Firebase Storage, giving doctors/patients access to stored documents; there is no structured patient database (e.g. MongoDB per README) actually present, only file storage
Network of doctors across specialties available for bookingCode-supported
A network of doctors of all specialities is readily available to anyone in need
Claimed on Devpostlow confidencefrontend/src/components/DoctorsPage/DoctorsPage.js:35— Doctors list is a hardcoded static array of 5 doctors with specialty and price, not a real network/database of doctors
Real-time/live video consultations between doctor and patientCode-supported
Secure video chatting rooms for patient-doctors with real-time, consented session transcribing; connects patients with doctors instantly through virtual consultations
Claimed on Devpostmedium confidencefrontend/src/components/VideoChat/VideoChat.js:112— Implements WebRTC peer-to-peer video calling with simple-peer and socket.io, including call/answer signaling via backend/server.jsbackend/server.js:169— Socket.io server manages doctor/patient pairing and WebRTC signal relay (callUser/answerCall events)frontend/src/App.js:27— The real peer-to-peer VideoChat route is commented out in the router; the active /video-chat route instead points to VideoChatNew.js, a simple audio recorder page with a webcam preview but no peer connection, so the full live video-call flow is not actually wired up end-to-end
AI fallback virtual doctor (Hume AI) when no doctors availableClaimed only
Not explicitly in Devpost text but present in UI as a feature
Claimed on Devpostmedium confidencefrontend/src/components/DoctorsPage/DoctorsPage.js:6— HumeAISection renders a 'Consult Hume AI' button linking to /patient/hume-ai-consultation, but that route is not defined in App.js and no Hume AI integration code exists anywhere in the repo
In-state network insurance supportClaimed only
In-state network insurance support
Claimed on Devposthigh confidenceSimplifying medical jargon into layman's languageClaimed only
Simplifies medical jargon for patients via an algorithm that is designed to understand medical jargon and mitigate its difficulty to a layman's understanding
Claimed on readmehigh confidence
An AI agent derived these features from the project’s Devpost page and readme, then searched the code for each one. Verified features are backed by cited code; claimed-only features had no supporting code, which is not by itself proof a feature is missing.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.