# Project export: IntelliCare
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: TreeHacks 2025
- Tagline: Patient Monitoring for Crisis Prevention
- Devpost: https://devpost.com/software/intellicare-hr3q2j
- GitHub: https://github.com/anakhag07/intelli-care
- Video: https://www.youtube.com/embed/5ZrN0cIUnh4?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 2 GitHub contributor(s) — Emily (18 commits), anakhag07 (16 commits)
## Devpost submission (written by the team)
### Overview
Every second counts in a medical emergency. When a patient experiences a heart attack, stroke, or other life-threatening event, early intervention is critical. However, identifying these crises before they escalate and understanding how to prioritize remains a challenge in hospitals. Medical data shows: Every 40 seconds, someone in the U.S. has a heart attack or stroke. One in four hospital deaths are linked to sepsis, which often presents with subtle warning signs before becoming critical. AI-driven hospital command centers have reduced patient mortality rates by up to 20% by predicting crises before they occur. Inspired by this, we built IntelliCare, an AI-powered system that flags high-risk patients based on a combination of real-time vitals, medical history, and AI-generated insights. IntelliCare detects life-threatening conditions before they happen by analyzing patient vitals and electronic health records (EHR). Our system: Flags patients at risk based on a dangerous combination of vitals (e.g., rapidly dropping blood pressure + irregular heart rhythm). Generates real-time summaries of a patient’s medical history when a critical alert is raised, making it easier for nurses and doctors to respond quickly with the context needed to treat patients well. Incorporates vision-based assessments to detect facial anomalies that may occur related to medical emergencies, such as facial asymmetry (stroke), swelling (anaphylactic shock), and sudden nosebleeds (hypertensive crisis). Market Size Global AI in healthcare market is projected to reach $187.9 billion by 2030 Predictive analytics in healthcare is rapidly growing as a submarket, projected to exceed $20 billion by 2030 Hospital Costs: Sepsis costs hospitals ~$62 billion annually, and AI powered early sepsis detection models like Epic's have shown a $5000+ in patient savings per patient Stroke and cardiac treatment cost the US $500 billion annually Total Addressable Market Size: Globally, the AI in healthcare market could be $200 billion If early intervention solves even 20% of sepsis/cardiac arrest cases, Intellicare could be looking at a $17 billion total market size
### How we built it
Data Collection & Processing Used publicly available MIMIC-IV-ED vitals data Generated synthetic crisis scenarios based on EHR data (e.g., rapid heart rate increase, oxygen drop). Data Collection & Processing Used publicly available MIMIC-IV-ED vitals data Generated synthetic crisis scenarios based on EHR data (e.g., rapid heart rate increase, oxygen drop). AI-Powered Detection Leveraged a gradient-boosted decision tree model to analyze patient history and detect risk patterns. Used OpenAI’s API to summarize patient history related to a flagged crisis. Utilized Cursor to provide valuable insights into bugs or other useful actionables. AI-Powered Detection Leveraged a gradient-boosted decision tree model to analyze patient history and detect risk patterns. Used OpenAI’s API to summarize patient history related to a flagged crisis. Utilized Cursor to provide valuable insights into bugs or other useful actionables. Vision-Based Tracking Adopted a vision-language model for detecting crises. Incorporated facial analysis for stroke and anaphylaxis detection. Vision-Based Tracking Adopted a vision-language model for detecting crises. Incorporated facial analysis for stroke and anaphylaxis detection.
### Challenges we ran into
One major challenge we faced was the lack of publicly accessible medical data, especially video datasets of patients. Given the highly sensitive nature of medical information, there are strict privacy regulations that limit access to patient videos, particularly those depicting symptoms such as facial asymmetry in stroke patients or swelling in anaphylaxis cases. Another challenge was balancing false positives and false negatives. While we wanted to ensure that every critical case was flagged, excessive false alarms could overwhelm healthcare providers and lead to alert fatigue.
### What we learned
Through this project, we learned that AI in healthcare requires explainability. Clinicians need to understand why a patient is flagged as high-risk in order to trust and act on AI-generated recommendations. Simply providing a risk score is not enough; clear, interpretable insights are essential. Additionally, we gained valuable perspectives into the real-world challenges of AI deployment in healthcare. Hospitals have strict compliance and regulatory requirements, making AI integration more complex than initially expected. Understanding these constraints is crucial for developing AI solutions that can be successfully implemented in clinical settings. Moving forward, we aim to give doctors and nurses a head start in preventing medical crises. Because every second counts.
## README (from the GitHub repository)
# intelli-care
Preventing Patient Crises using Large Language Models for Vitals Synthesis
This codebase contains 2 components. The first utilizes Open AI API's to summarize vitals that are associated with a single patient for a faster alert and classification on the severity of the alert. The second utilizes visual data to alert on a patient crisis before it happens. These include signs such as sudden drooping of the face prior to a stroke or nosebleeds prior to a hypertensive crisis.
Here is the link to the devpost with more context: https://devpost.com/software/intellicare-hr3q2j?ref_content=my-projects-tab&ref_feature=my_projects
## Detected evidence (automated analysis)
Indexed codebase: 5 recognized source files, 20 KB.
- Python (language) — detected in the code
- OpenAI (technology) — claimed on Devpost, not found in the code
- Streamlit (technology) — claimed on Devpost, not found in the code
## Codebase structure (from repository index)
### Files (10 of 10)
```
.gitignore
best_hrvarrest_model.txt
heart_attack_vitals_10.csv
hrvarrest_train.py
normal_vitals_10.csv
openai_script.py
openface_symmetry.py
README.md
vitals_100.csv
vitals_processor.py
```
### Dependencies
No dependency index available.
### Recent commits (newest first)
- Update README.md
- commit new changes
- Added .gitignore to exclude unnecessary files
- Ignore __pycache__ folder
- Merge branch 'main' of https://github.com/anakhag07/intelli-care
- added stroke features
- added stroke features
- updated secret key
- updated secret key
- updated ai model to include risk score, resolved bug with risk score not changing
- updated ai model to include risk score, resolved bug with risk score not changing
- changed vitals_processor.py api key
- changed vitals_processor.py api key
- implemented AI vitals monitoring
- implemented AI vitals monitoring
- updated vitals and openai
- updated vitals and openai
- added messaging portal with doctor
- added messaging portal with doctor
- added open ai script
## Key source files (fetched from GitHub, selected and truncated for size)
### openface_symmetry.py
```python
import pandas as pd
import numpy as np
def process_facial_symmetry(uploaded_file):
# Load OpenFace CSV file
csv_path = uploaded_file # Change this to your file path
df = pd.read_csv(csv_path)
# Extract key facial action units related to stroke detection
au_relevant = ["AU12_c", "AU15_c", "AU17_c", "AU20_c", "AU28_c"]
au_data = df[au_relevant]
# Compute overall AU activation score (higher may indicate stroke-like symptoms)
asymmetry_score = np.mean(au_data.values)
# Extract left and right facial landmarks
landmark_cols = [col for col in df.columns if col.startswith("X_") or col.startswith("Y_")]
landmarks = df[landmark_cols].values.reshape(-1, 2, 34) # Assuming 68 landmarks
# Compute left vs right asymmetry by mirroring half the face
left_side = landmarks[:, :, :17] # First 17 points (left face)
right_side = landmarks[:, :, 17:] # Last 17 points (right face, flipped)
# Calculate Euclidean distance between mirrored points
asymmetry_diffs = np.linalg.norm(left_side - np.flip(right_side, axis=2), axis=1)
asymmetry_index = np.mean(asymmetry_diffs) # Average asymmetry
# Print results
print(f"Facial Asymmetry Index: {asymmetry_index:.2f}")
print(f"Action Unit (AU) Score: {asymmetry_score:.2f}")
# Stroke detection decision
if asymmetry_index > 75 and asymmetry_score > 0.5:
# print("⚠️ Possible stroke detected: High facial asymmetry and muscle drooping.")
return True
else:
# print("✅ No strong stroke indicators detected.")
return False
```
### openai_script.py
```python
import pandas as pd
from pydantic import BaseModel
from openai import OpenAI
from datetime import datetime
OPEN_API_KEY = ""
# Define structured response format
class PatientSummary(BaseModel):
patient_id: str
# date: str
# time: str
temperature: float
heart_rate: float
respiratory_rate: float
oxygen_saturation: float
sys_blood_pressure: str
dia_blood_pressure: str
heart_rhythm: str
past_history: list[str]
summary: str # Summary of the patient's vitals and history
def generate_vitals_summary(api_key, patient_data):
client = OpenAI(api_key=api_key)
# Read the CSV file
# df = pd.read_csv(patient_data_file)
# Convert patient data to DataFrame format
df = pd.DataFrame([patient_data])
# Extract data from the first row since we're processing a single record
row = df.iloc[0]
# Parse timestamp
# timestamp = row['timestamp']
# dt_obj = datetime.strptime(timestamp, "%Y-%m-%d %H:%M:%S")
# date_part = dt_obj.strftime("%Y-%m-%d")
# time_part = dt_obj.strftime("%H:%M")
# Extract vitals from nested dictionary
vitals = row['vitals']
# changes = row['changes_detected']
past_history = ["heart attack"]
patient_info = f"""
Patient ID: {row['patient_id'] if 'patient_id' in row else 'Unknown'}
Temperature: {vitals['temperature']} °F
Heart Rate: {vitals['heart_rate']} bpm
Respiratory Rate: {vitals['respiratory_rate']} breaths/min
Oxygen Saturation: {vitals['oxygen_saturation']}%
Systolic Blood Pressure: {vitals['blood_pressure']['systolic']}
Diastolic Blood Pressure: {vitals['blood_pressure']['diastolic']}
Heart Rhythm: {vitals['heart_rhythm']}
Past Medical History: {past_history}
"""
# Call OpenAI API to summarize patient vitals
completion = client.beta.chat.completions.parse(
model="gpt-4o-2024-08-06",
messages=[
{"role": "system", "content": "You are a medical assistant. Generate a concise summary of the patient's vital signs and health status."},
{"role": "user", "content": patient_info},
],
response_format=PatientSummary,
)
# Extract structured response
patient_summary = completion.choices[0].message.parsed
return patient_summary.summary
```
### hrvarrest_train.py
```python
import pandas as pd
import numpy as np
import os
import sklearn
from sklearn.model_selection import train_test_split, StratifiedKFold
from sklearn.metrics import confusion_matrix, roc_auc_score, roc_curve, auc, precision_recall_curve, average_precision_score, mean_squared_error, r2_score
import lightgbm as lgb
from lightgbm import LGBMClassifier
from bayes_opt import BayesianOptimization
import json
import optuna
def preprocess_data(file_path='vitals_100.csv'):
"""
Preprocess vital signs data for heart rhythm classification.
"""
# Read the CSV file
df = pd.read_csv(file_path)
# Select only numeric columns for features
feature_columns = [
'temperature', 'heartrate', 'resprate', 'o2sat',
'sbp', 'dbp', 'pain', 'hour', 'day_of_week',
'hr_change', 'rr_change', 'o2_change'
]
# Ensure all feature columns are numeric
for col in feature_columns:
df[col] = pd.to_numeric(df[col], errors='coerce')
# Fill any NaN values with 0
df[feature_columns] = df[feature_columns].fillna(0)
# Create binary labels based on rhythm
# Now any rhythm other than 'Sinus Rhythm' is considered high risk
rhythm_to_label = {
'Sinus Rhythm': 0, # Normal rhythm - low risk
'Bradycardia': 0.8, # High risk
'Tachycardia': 0.85, # High risk
'Atrial Fibrillation': 0.9, # Very high risk
'Ventricular Tachycardia': 0.95, # Very high risk
'Ventricular Fibrillation': 1.0, # Extreme risk
'Asystole': 1.0, # Extreme risk
'Unknown': 0.7 # Default high risk for unknown rhythms
}
# Convert predicted_rhythm to risk scores
df['label'] = df['predicted_rhythm'].map(rhythm_to_label)
# Fill any NaN labels with high risk score (0.7)
df['label'] = df['label'].fillna(0.7)
# Save the processed data with the new label column
output_path = file_path
df.to_csv(output_path, index=False)
print(f"\nProcessed data saved to: {output_path}")
# Keep only the features we want and the label for training
df_train = df[feature_columns + ['label']]
# Display the processed dataset info
print("\nFirst few rows of the processed dataset:")
print(df_train.head())
print("\nDataset shape:", df_train.shape)
print("\nLabel distribution:")
print(df['label'].value_counts())
print("\nRhythm distribution:")
print(df['predicted_rhythm'].value_counts())
return df_train
def train_model(df, seed=42, opt_inits=3, opt_iters=50):
"""
Train a LightGBM model on the preprocessed vital signs data.
Args:
df (pd.DataFrame): Preprocessed dataframe from preprocess_data()
seed (int): Random seed for reproducibility
opt_inits (int): Number of initial points for Bayesian optimization
opt_iters (int): Number of optimization iterations
Returns:
lgb.Booster: Trained model
dict: Performance metrics
"""
# Prepare the data
labels = df['label']
inputs = df.drop(columns=['label'])
# Split the data without stratification
X_train, X_test, y_train, y_test = train_test_split(
inputs, labels, test_size=0.2, random_state=seed
)
train_data = lgb.Dataset(X_train, label=y_train)
test_data = lgb.Dataset(X_test, label=y_test, reference=train_data)
# Define base parameters that won't be optimized
base_params = {
'objective': 'regression',
'metric': 'mse',
'boosting_type': 'gbdt',
'verbose': -1,
'feature_pre_filter': False
}
# Define the optimization objective
def objective(trial):
params = {
**base_params,
'num_leaves': trial.suggest_int('num_leaves', 20, 100),
'learning_rate': trial.suggest_float('learning_rate', 0.01, 0.1),
'feature_fraction': trial.suggest_float('feature_fraction', 0.5, 1.0),
'bagging_fraction': trial.suggest_float('bagging_fraction', 0.5, 1.0),
'bagging_freq': trial.suggest_int('bagging_freq', 1, 10),
'min_child_samples': trial.suggest_int('min_child_samples', 10, 50)
}
# Train the model with current parameters
model = lgb.train(
params,
train_data,
num_boost_round=100,
valid_sets=[test_data],
)
return model.best_score['valid_0']['l2']
# Run the optimization
study = optuna.create_study(direction='minimize')
study.optimize(objective, n_trials=opt_iters)
# Train the final model with the best parameters
best_params = {**base_params, **study.best_params}
final_model = lgb.train(
best_params,
train_data,
num_boost_round=100,
valid_sets=[test_data],
)
# Calculate performance metrics
y_pred = final_model.predict(X_test)
mse = mean_squared_error(y_test, y_pred)
rmse = np.sqrt(mse)
r2 = r2_score(y_test, y_pred)
metrics = {
'mse': mse,
'rmse': rmse,
'r2': r2,
'best_params': best_params
}
print("\nModel Performance:")
print(f"MSE: {mse:.4f}")
print(f"RMSE: {rmse:.4f}")
print(f"R2 Score: {r2:.4f}")
return final_model, metrics
def main():
# Preprocess the data
df = preprocess_data('vitals_100.csv')
# Train the model
model, metrics = train_model(df)
# Print results
print("\nModel Performance:")
print(f"MSE: {metrics['mse']:.4f}")
print(f"RMSE: {metrics['rmse']:.4f}")
print(f"R2 Score: {metrics['r2']:.4f}")
# Save the model
model.save_model('best_hrvarrest_model.txt')
print("\nModel saved as 'best_hrvarrest_model.txt'")
if __name__ == "__main__":
main()
```
### vitals_processor.py
```python
import streamlit as st
import time
import pandas as pd
from datetime import datetime, timedelta
from openai_script import generate_vitals_summary
import lightgbm as lgb
from openface_symmetry import process_facial_symmetry
OPEN_API_KEY = ""
def process_vitals(patient_data):
"""Process vitals using AI model with dynamic risk scoring"""
# Load the trained model
model = lgb.Booster(model_file='best_hrvarrest_model.txt')
# Create placeholders for vital signs display
vitals_display = st.empty()
warning_display = st.empty()
chat_display = st.sidebar.empty()
risk_display = st.empty() # New display for risk score
# Add a progress bar
progress_bar = st.progress(0)
for i in range(len(patient_data)):
curr = patient_data[i]
# Display current vitals in a formatted way
vitals_display.markdown(f"""
### Current Vital Signs
- Heart Rate: {curr['heartrate']} bpm
- Respiratory Rate: {curr['resprate']} breaths/min
- O2 Saturation: {curr['o2sat']}%
- Blood Pressure: {curr['sbp']}/{curr['dbp']} mmHg
- Temperature: {curr['temperature']}°F
""")
# Calculate changes from previous timestep
if i > 0:
prev = patient_data[i-1]
hr_change = curr['heartrate'] - prev['heartrate']
rr_change = curr['resprate'] - prev['resprate']
o2_change = curr['o2sat'] - prev['o2sat']
else:
hr_change = 0
rr_change = 0
o2_change = 0
# Create feature DataFrame for prediction
features = pd.DataFrame({
'temperature': [curr['temperature']],
'heartrate': [curr['heartrate']],
'resprate': [curr['resprate']],
'o2sat': [curr['o2sat']],
'sbp': [curr['sbp']],
'dbp': [curr['dbp']],
'pain': [curr['pain']],
'hour': [-1],
'day_of_week': [-1],
'hr_change': [hr_change],
'rr_change': [rr_change],
'o2_change': [o2_change]
})
# Get model prediction for current timestep
risk_score = float(model.predict(features))
# Display current risk score with color coding
risk_color = "green" if risk_score < 0.3 else "orange" if risk_score < 0.7 else "red"
risk_display.markdown(f"""
### Current Risk Assessment
Risk Score: {risk_score:.3f}
""", unsafe_allow_html=True)
# Check if risk score exceeds threshold
print(risk_score)
if risk_score > 0.5: # High risk threshold
warning_display.markdown("⚠️ CARDIAC EVENT WARNING ⚠️
", unsafe_allow_html=True)
# Calculate changes for the summary
changes_detected = {
"heart_rate": f"Current: {curr['heartrate']} bpm",
"respiratory_rate": f"Current: {curr['resprate']} breaths/min",
"oxygen_saturation": f"Current: {curr['o2sat']}%",
"blood_pressure": f"Current: {curr['sbp']}/{curr['dbp']} mmHg"
}
if i > 0:
changes_detected.update({
"heart_rate": f"Changed by {hr_change} bpm",
"respiratory_rate": f"Changed by {rr_change} breaths/min",
"oxygen_saturation": f"Changed by {o2_change}%",
"blood_pressure": f"Systolic changed by {curr['sbp'] - prev['sbp']}, Diastolic changed by {curr['dbp'] - prev['dbp']}"
})
# Convert timestamp to proper format
timestamp = pd.to_datetime(curr['charttime']).strftime("%Y-%m-%d %H:%M:%S") if pd.notna(curr.get('charttime')) else datetime.now().strftime("%Y-%m-%d %H:%M:%S")
critical_vitals = {
"timestamp": timestamp,
"vitals": {
"temperature": curr['temperature'],
"heart_rate": curr['heartrate'],
"respiratory_rate": curr['resprate'],
"oxygen_saturation": curr['o2sat'],
"blood_pressure": {
"systolic": curr['sbp'],
"diastolic": curr['dbp']
},
"heart_rhythm": curr.get('rhythm', 'Unknown')
},
"risk_score": risk_score,
"model_prediction": "High Risk of Cardiac Event",
"changes_detected": changes_detected
}
summary = generate_vitals_summary(OPEN_API_KEY, critical_vitals)
warning_message = {
"role": "assistant",
"content": f"⚠️ URGENT: Cardiac Event Warning!\n\nRisk Score: {risk_score:.4f}\n\nPatient Summary:\n{summary}"
}
chat_display.markdown(f"{warning_message['content']}\n")
break
else:
warning_display.empty()
# Update progress bar
progress_bar.progress((i + 1) / len(patient_data))
# Add a small delay to simulate real-time monitoring
time.sleep(1)
def process_vitals_rule_based(patient_data):
"""Process vitals using rule-based monitoring"""
# Create placeholders for vital signs display
vitals_display = st.empty()
warning_display = st.empty()
chat_display = st.sidebar.empty()
# Add a progress bar
progress_bar = st.progress(0)
for i in range(len(patient_data)):
curr = patient_data[i]
# Display current vitals in a formatted way
vitals_display.markdown(f"""
### Current Vital Signs
- Heart Rate: {curr['heartrate']} bpm
- Respiratory Rate: {curr['resprate']} breaths/min
- O2 Saturatio
[truncated — 4308 more characters]
```