# Project export: Burnout Monitor

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 10.0
- Tagline: Track, anticipate, and proactively respond to burnout in healthcare professionals, using a friendly video monitoring system and a custom-trained stress prediction model.
- Devpost: https://devpost.com/software/burnout-monitor
- GitHub: https://github.com/AarhamWasit/calhacks
- Video: https://www.youtube.com/embed/NvZFrUz4Ax4?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 3 GitHub contributor(s) — AarhamWasit (2 commits), Shubhankar Tripathy (2 commits), Uzair (1 commits)

## Devpost submission (written by the team)

### Overview

Problem The demanding nature of healthcare work often leads to burnout, impacting both the individuals themselves and the quality of patient care. Unfortunately, burnout frequently goes unnoticed until it reaches a critical stage, affecting not only the mental health of the professionals but also potentially jeopardizing patient safety and outcomes. Our Solution The app not only efficiently detects emotional states but identifies early indicators of burnout among medical professionals through a custom neural network. It offers a user-friendly interface that presents weekly logs of burnout risk levels and a user friendly web dashboard of their associated emotional trends. This app builds upon Hume's video emotion detection technology to predict and prevent burnout among healthcare workers. Existing approaches use frequent surveillance to profile medical practitioners' mental states, jeopardizing privacy. Our application protects privacy by recording infrequent but regular interactions with patients, with full knowledge of those involved.

### How we built it

At the core of our project was Hume's emotion detection API's ability to recognize 52 diverse, expressive nuances in facial expressions. We used the results of HumeBatchClient and HumeStreamClient's facial expression model to train a custom ML model to detect burnout on a carefully curated training dataset. We collected images labeled based on levels of burnout from open health research 1 2, friends, Hume's sample images, and synthetic data to train a regression model in scikit-learn. The model took as inputs the 52 emotions extracted from video frames as well as their manually selected labels, and optimized to learn the relationship between these emotions and eventual burnout. Our project wouldn't be possible without CockroachDB's serverless database and their intuitive SQL Shell, which allowed us to define and work with a PostgreSQL schema. We communicate with the database from the Flask backend as well as the python script used to collect video and query the Hume batch API endpoints.

### Challenges we ran into

One of the biggest challenges was the amount of iteration our project went through before a finalized idea. We took great care to find a socially relevant, unique problem with an original solution. We struggled to determine the ground truth for what counts as someone showing signs of being "burnt out". We found it effective to use some synthetic data generated by dall-e 3 along with manually labeled images.

### Accomplishments we're proud of

Successfully trained a classification model and deployed it on Hume's custom model platform using a self-labeled dataset for identifying potential burnout signs.

### What we learned

Explored HumeStreamClient for WebSocket integration in Flask. We learned about nuances of WebSockets vs other HTTP connections. None of us have ever worked with front-end before. We successfully made a working flask, html, and Javascript website to demo our machine learning app.

### What's next

We want to incorporate spoken data to improve our prediction and data visualization. Conversational Emotion Recognition (CER) is a task to predict the emotion of dialogue in the context of a conversation. We want to improve the accuracy of our model by expanding on training data, parameters, and compute. Add a centralized alert system that creates actionable suggestions based on likely burnout for workers.

## README (from the GitHub repository)

# Burnout Monitor

A Cal Hacks 10 submission

## Problem

The demanding nature of healthcare work often leads to burnout, impacting both the individuals themselves and the quality of patient care. Unfortunately, burnout frequently goes unnoticed until it reaches a critical stage, affecting not only the mental health of the professionals but also potentially jeopardizing patient safety and outcomes.

## Our Solution


The app not only efficiently detects emotional states but identifies early indicators of burnout among medical professionals through a custom neural network. 

It offers a user-friendly interface that presents weekly logs of burnout risk levels and a user friendly web dashboard of their associated emotional trends.

This app builds upon Hume's video emotion detection technology to predict and prevent burnout among healthcare workers. Existing approaches use frequent surveillance to profile medical practitioners' mental states, jeopardizing privacy. Our application protects privacy by recording infrequent but regular interactions with patients, with full knowledge of those involved. 

## How we built it

At the core of our project was Hume's emotion detection API's ability to recognize 52 diverse, expressive nuances in facial expressions. We used the results of `HumeBatchClient` and `HumeBatchClient`'s facial expression model to train a custom ML model to detect burnout on a carefully curated training dataset.

We collected images labeled based on levels of burnout from open health research [1](https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-023-04003-y#Sec46) [2](https://www.aapl.org/docs/pdf/VIDEO%20RECORDING%20GUIDELINE%202013.pdf), friends, Hume's sample images, and synthetic data to train a regression model in scikit-learn. The model took as inputs the 52 emotions extracted from video frames as well as their manually selected labels, and optimized to learn the relationship between these emotions and eventual burnout.

Our project wouldn't be possible without CockroachDB's serverless database and their intuitive SQL Shell, which allowed us to define and work with schemas.

We communicate with the database from the Flask backend as well as the python script used to collect video and query the Hume batch API endpoints.

## Challenges we ran into

- One of the biggest challenges was the amount of iteration our project went through before a finalized idea. We took great care to find a socially relevant, unique problem with an original solution.
- We struggled to determine the ground truth for what counts as someone showing signs of being "burnt out". We found it effective to use some synthetic data generated by dall-e 3 and manually labeled images 

## Accomplishments that we're proud of

- Successfully trained a classification model and deployed it on Hume's custom model platform using a self-labeled dataset for identifying potential burnout signs.

## What we learned

- Explored `HumeStreamClient` for WebSocket integration in Flask. We learned about nuances of WebSockets vs other HTTP connections.

## What's next for Burnout Monitor

- We want to incorporate spoken data to improve our prediction and data visualization. Conversational Emotion Recognition (CER) is a task to predict the emotion of dialogue in the context of a conversation.
- We want to improve the accuracy of our model by expanding on training data, parameters, and compute. 
- Add a centralized alert system that creates actionable suggestions based on likely burnout for workers.


## Detected evidence (automated analysis)

Indexed codebase: 13 recognized source files, 38 KB.
- CSS (language) — detected in the code
- Express (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- OpenAI (technology) — detected in the code
- PostgreSQL (technology) — detected in the code
- Python (language) — detected in the code
- Flask (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (22 of 22)

```
.DS_Store
finaltest/.env
finaltest/a.py
finaltest/chat.py
finaltest/main.py
finaltest/requirements.txt
finaltest/testing.py
LICENSE
README.md
uzair/.env
uzair/.ignore
uzair/app.py
uzair/call_hume_api_custom_model.py
uzair/model.pkl
uzair/model.py
uzair/package.json
uzair/requirements.txt
uzair/static/css/style.css
uzair/static/js/script.js
uzair/templates/data_visualization.html
uzair/templates/index.html
uzair/uzair_code.py
```

### Dependencies

- finaltest/requirements.txt: colorama, git@+https://github.com/aarnphm/whispercpp.git, gTTS, hume, hume[stream], numpy, openai, opencv-python, playsound, psycopg2, pvrecorder, pynput, python-dotenv, whispercpp
- uzair/package.json: express@^4.18.2, multer@^1.4.5-lts.1

### Recent commits (newest first)

- mewmew
- Update README.md
- Update README.md
- meowwww
- Rename call_hume_api_custom_model to call_hume_api_custom_model.py
- Create call_hume_api_custom_model
- meow3
- meow 2
- meow
- newnew
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### uzair/package.json

```
{
  "dependencies": {
    "express": "^4.18.2",
    "multer": "^1.4.5-lts.1"
  }
}

```

### finaltest/requirements.txt

```
whispercpp
openai
pynput
playsound
pvrecorder
numpy
git+https://github.com/aarnphm/whispercpp.git
gTTS
opencv-python
hume
hume[stream]
colorama
psycopg2
python-dotenv
```

### uzair/app.py

```python
import os
import time
import asyncio
import uzair_code
from flask import Flask, render_template, request, jsonify
from concurrent.futures import ThreadPoolExecutor
from hume.models.config import FaceConfig
from hume import HumeStreamClient
import uzair_code

app = Flask(__name__)
app.config['UPLOAD_FOLDER'] = 'static/uploads/'

# Ensure the upload directory exists
os.makedirs(app.config['UPLOAD_FOLDER'], exist_ok=True)

executor = ThreadPoolExecutor(max_workers=5)

# Asynchronous analysis function


async def analyze_frame(file_path):
    client = HumeStreamClient(
        "wwO3WtzawTu07HFAWCORQrGWDtRcE4BkcCKCITHTRVYGOgMa")
    config = FaceConfig(identify_faces=True)
    async with client.connect([config]) as socket:
        result = await socket.send_file(file_path)
        return result

# Callback for asynchronous task


# Global variable to store the latest prediction
latest_prediction = None


def frame_analysis_callback(future):
    global latest_prediction
    result = future.result()
    emotions = result["face"]["predictions"][0]["emotions"]
    emotions_vector = [emotion["score"] for emotion in emotions]
    latest_prediction = uzair_code.predict2([emotions_vector])


@app.route('/get-latest-prediction')
def get_latest_prediction():
    if latest_prediction is not None:
        return jsonify({"prediction": latest_prediction})
    else:
        return jsonify({"error": "No prediction available"}), 404


@app.route('/')
def index():
    return render_template('index.html')


@app.route('/upload', methods=['POST'])
def upload_file():
    if 'file' not in request.files:
        return jsonify({"error": "No file part"}), 400

    file = request.files['file']
    if file.filename == '':
        return jsonify({"error": "No selected file"}), 400

    if file:
        filename = os.path.join(app.config['UPLOAD_FOLDER'], file.filename)
        file.save(filename)
        time.sleep(9)  # Simulate some processing delay
        result = str(round(uzair_code.predict(filename)*100, 2))
        return jsonify({"result": result})


@app.route('/data-visualization')
def data_visualization():
    return render_template('data_visualization.html')


@app.route('/save-frame', methods=['POST'])
def save_frame():
    if 'frame' not in request.files:
        return jsonify({"error": "No frame received"}), 400

    frame = request.files['frame']
    if frame:
        frame_path = os.path.join(app.config['UPLOAD_FOLDER'], 'temp.jpg')
        frame.save(frame_path)

        # Use executor to run async function
        future = executor.submit(asyncio.run, analyze_frame(frame_path))
        future.add_done_callback(frame_analysis_callback)

        return jsonify({"message": "Frame saved and analysis started"}), 200
    else:
        return jsonify({"error": "Failed to save frame"}), 500


if __name__ == '__main__':
    app.run(debug=True)

```

### finaltest/main.py

```python
import threading
import asyncio
import os
import cv2
import time
import traceback
import websockets
import numpy as np

from pynput import keyboard
from pvrecorder import PvRecorder
from whispercpp import Whisper
from chat import message, store_emotions
from playsound import playsound
from hume import HumeStreamClient, HumeClientException
from hume.models.config import FaceConfig
from gtts import gTTS
from dotenv import load_dotenv
import psycopg2

load_dotenv()


# Configurations
# paste your API Key here
HUME_API_KEY = "D9H6vIAdB3GQf6E3i58HGM6iPgucEUCU90D2nGXs8SS9UsbW"
HUME_FACE_FPS = 1 / 3  # 3 FPS

TEMP_FILE = 'temp.jpg'
TEMP_WAV_FILE = 'temp.wav'

# Initialize whisper model, pyttsx3 engine, and pv recorder
w = Whisper.from_pretrained("tiny.en")
recorder = PvRecorder(device_index=-1, frame_length=512)

# Global variables
recording = False
recording_data = []

# Webcam setup
cam = cv2.VideoCapture(0)


async def webcam_loop():
    while True:
        try:
            client = HumeStreamClient(HUME_API_KEY)
            config = FaceConfig(identify_faces=True)
            async with client.connect([config]) as socket:
                print("(Connected to Hume API!)")
                while True:
                    print("po")
                    if not recording:
                        _, frame = cam.read()
                        cv2.imwrite(TEMP_FILE, frame)
                        result = await socket.send_file(TEMP_FILE)
                        global rec_id
                        store_emotions(result, rec_id)
                        await asyncio.sleep(1 / 3)
        except websockets.exceptions.ConnectionClosedError:
            print("Connection lost. Attempting to reconnect in 1 seconds.")
            time.sleep(1)
        except HumeClientException:
            print(traceback.format_exc())
            break
        except Exception:
            print(traceback.format_exc())


def start_asyncio_event_loop(loop, asyncio_function):
    asyncio.set_event_loop(loop)
    loop.run_until_complete(asyncio_function)


def recording_loop():
    global recording_data, recording
    while recording:
        frame = recorder.read()
        recording_data.append(frame)

    recorder.stop()
    print("(Recording stopped...)")

    recording_data = np.hstack(recording_data).astype(
        np.int16).flatten().astype(np.float32) / 32768.0
    transcription = w.transcribe(recording_data)
    response = message(transcription)
    tts = gTTS(text=response, lang='en')
    tts.save(TEMP_WAV_FILE)
    playsound(TEMP_WAV_FILE)
    os.remove(TEMP_WAV_FILE)


rec_id = 0


def on_press(key):
    global recording, recording_data, recorder
    if key == keyboard.Key.space:
        if recording:
            recording = False
        else:
            recording = True
            recording_data = []
            recorder.start()
            print("(Recording started...)")

            conn = psycopg2.connect(os.environ["DATABASE_URL"])
            print("a")
            with conn.cursor() as cur:
                cur.execute(
                    "INSERT INTO \"Recordings\" DEFAULT VALUES RETURNING *")
                res = cur.fetchone()[0]
                global rec_id
                rec_id = res
                print(rec_id)
                print("RES", res)
                conn.commit()

            threading.Thread(target=recording_loop).start()


new_loop = asyncio.new_event_loop()

threading.Thread(target=start_asyncio_event_loop,
                 args=(new_loop, webcam_loop())).start()


with keyboard.Listener(on_press=on_press) as listener:
    print("Speak to Joaquin!")
    print("(Press spacebar to speak. To finish speaking, press spacebar again)")
    listener.join()

```

### finaltest/a.py

```python
import psycopg2
import os
from dotenv import load_dotenv


load_dotenv()


conn = psycopg2.connect(os.environ["DATABASE_URL"])
print("a")
with conn.cursor() as cur:
    cur.execute("INSERT INTO \"Recordings\" DEFAULT VALUES")
    cur
    conn.commit()

```

### finaltest/testing.py

```python
import cv2


def main():
    # Start video capture (0 for default camera)
    cap = cv2.VideoCapture(0)

    # Check if the camera opened successfully
    if not cap.isOpened():
        print("Error: Camera could not be opened.")
        return

    try:
        while True:
            # Capture frame-by-frame
            ret, frame = cap.read()

            # If frame is read correctly, ret is True
            if not ret:
                print("Can't receive frame. Exiting ...")
                break

            # Display the resulting frame
            cv2.imshow('Camera Test', frame)

            # Press 'q' to quit the window
            if cv2.waitKey(1) & 0xFF == ord('q'):
                break
    finally:
        # When everything done, release the capture
        cap.release()
        cv2.destroyAllWindows()


if __name__ == "__main__":
    main()

```

### uzair/call_hume_api_custom_model.py

```python
import requests
import base64

def call_hume_api(image_path):
    # Load image and convert to base64
    with open(image_path, "rb") as image_file:
        encoded_string = base64.b64encode(image_file.read()).decode('utf-8')
    
    # Prepare the data for the API call
    data = {
        'image': encoded_string
    }
    
    # Define the HUME API endpoint and your custom model endpoint
    url = "https://api.hume.ai/v1/models/9915e721-bd93-429d-ae45-27098417283d/versions/e4ac03fd-185d-40ae-98e9-b2c4fc00da42:predict"
    
    # Define the headers for the API call (if necessary, e.g., for authentication)
    headers = {
        'Authorization': 'Bearer jxYLjpcsWArNEAoTuOjdRU0QwKkDzDo0Ry047WrXYSGNlVo3',
        'Content-Type': 'application/json',
    }
    
    # Make the API call
    response = requests.post(url, json=data, headers=headers)
    
    # Check for a valid response
    if response.status_code == 200:
        # Parse the response (assuming JSON response)
        response_data = response.json()
        return response_data
    else:
        print(f"Failed to call API. Status code: {response.status_code}")
        print(response.text)  # Print error message if any

# Example usage:
result = call_hume_api('path_to_your_image.jpg')
print(result)

```

### uzair/model.py

```python
from sklearn.linear_model import LogisticRegression
import os
from dotenv import load_dotenv
from hume import HumeBatchClient
from hume.models.config import FaceConfig
import pandas as pd
import pickle

load_dotenv()


def get_emotion_vector(file_path):
    client = HumeBatchClient(os.environ["HUME_API_KEY"], timeout=600)

    print(file_path)

    configs = [FaceConfig()]
    job = client.submit_job(None, configs, files=[file_path])

    job.await_complete(timeout=600)
    results = job.get_predictions()

    emotions = results[0]["results"]["predictions"][0]["models"]["face"]["grouped_predictions"][0]["predictions"][0]["emotions"]

    emotions_vector = [emotion["score"] for emotion in emotions]

    print(pd.DataFrame(emotions).sort_values(
        by="score", ascending=False).head(5)["name"])

    return emotions_vector


def train_model(file_paths, labels):
    # train a model to predict label from jpegs files

    vectors = [get_emotion_vector(file_path) for file_path in file_paths]

    model = LogisticRegression()
    model.fit(vectors, labels)

    with open('model.pkl', 'wb') as f:
        pickle.dump(model, f)


def predict(file_path):
    # load
    with open('model.pkl', 'rb') as f:
        model = pickle.load(f)
        return model.predict_proba([get_emotion_vector(file_path)])[0][1]


if __name__ == "__main__":
    file_paths = [
        f"faces/{i}.jpg" for i in ["103", "101", "104", "150"]
    ]
    labels = [0, 0, 1, 1]
    train_model(file_paths, labels)
    print(predict("faces/0.png"))

```

### uzair/uzair_code.py

```python
from sklearn.linear_model import LogisticRegression
import os
from dotenv import load_dotenv
from hume import HumeBatchClient
from hume.models.config import FaceConfig
import pandas as pd
import pickle

load_dotenv()


def get_emotion_vector(file_path):
    client = HumeBatchClient(os.environ["HUME_API_KEY"], timeout=600)

    print(file_path)

    configs = [FaceConfig()]
    job = client.submit_job(None, configs, files=[file_path])

    job.await_complete(timeout=600)
    results = job.get_predictions()

    emotions = results[0]["results"]["predictions"][0]["models"]["face"]["grouped_predictions"][0]["predictions"][0]["emotions"]

    emotions_vector = [emotion["score"] for emotion in emotions]

    print(pd.DataFrame(emotions).sort_values(
        by="score", ascending=False).head(5)["name"])

    return emotions_vector


def train_model(file_paths, labels):
    # train a model to predict label from jpegs files

    vectors = [get_emotion_vector(file_path) for file_path in file_paths]

    model = LogisticRegression()
    model.fit(vectors, labels)

    with open('model.pkl', 'wb') as f:
        pickle.dump(model, f)


def predict2(vec):
    with open('model.pkl', 'rb') as f:
        model = pickle.load(f)
        return model.predict_proba(vec)[0][1]


def predict(file_path):
    # load
    with open('model.pkl', 'rb') as f:
        model = pickle.load(f)

        print("breaker")

        return model.predict_proba([get_emotion_vector(file_path)])[0][1]


if __name__ == "__main__":
    file_paths = [
        f"faces/{i}.jpg" for i in ["103", "101", "104", "150"]
    ]
    labels = [0, 0, 1, 1]
    train_model(file_paths, labels)
    print(predict("faces/0.png"))

```

### finaltest/chat.py

```python
import openai
import numpy as np
import re
from colorama import Fore, Style

import os
import psycopg2

openai.api_key = "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"

SYSTEM_INSTRUCTIONS_COMEDIAN = """You are now playing the role of an AI assistant designed to support medical personnel, including doctors, nurses, and other healthcare workers. Your responses should be empathetic, informative, and encouraging, aimed at providing support and guidance, especially to those who might be nearing burnout. Your tone should be gentle and understanding, similar to a well-trained counselor or mentor in the healthcare field. Your primary goal is to offer support, resources, and understanding to help alleviate stress and burnout symptoms.

In your first response, you will:

Introduce yourself as an AI assistant.
Describe the purpose of this experience.
Ask the user about their current state or feelings, and if there's anything specific they need support with.
Throughout the conversation:

You will provide empathetic responses and offer practical advice or resources related to managing stress and burnout in the healthcare industry.
Ask open-ended questions to understand the user's situation better, such as their specific challenges, feelings, and needs.
Offer suggestions on stress management techniques, resources for professional support, and self-care tips tailored to healthcare professionals.
If the user seems disengaged or overwhelmed, gently probe with more specific questions about their work environment, recent experiences, or personal coping mechanisms.
Your responses will be concise yet informative, consisting of 2-4 sentences, with the last sentence always being a question to encourage further dialogue. The interaction should be user-centered, focusing on their needs and emotions. Remember, you are there to offer support and guidance, not to provide medical or psychological treatment.

Remember, in your first response, you will (i) introduce yourself, (ii) describe the purpose of this experience, and (iii) ask about the user’s current state or specific needs for support."""

EMOTIONS = np.array([
    "admiring", "adoring", "appreciative", "amused", "angry", "anxious", "awestruck", "uncomfortable", "bored", "calm",
    "focused", "contemplative", "confused", "contemptuous", "content", "hungry", "determined", "disappointed",
    "disgusted", "distressed", "doubtful", "euphoric", "embarrassed", "disturbed", "entranced", "envious", "excited",
    "fearful", "guilty", "horrified", "interested", "happy", "enamored", "nostalgic", "pained", "proud", "inspired",
    "relieved", "smitten", "sad", "satisfied", "desirous", "ashamed", "negatively surprised", "positively surprised",
    "sympathetic", "tired", "triumphant"
])

conversation = [{
    "role": "system",
    "content": SYSTEM_INSTRUCTIONS_COMEDIAN
}, {
    'role':
    'user',
    'content':
    "The user walks into your comedy club. As a comedian named Joaquin, what is the first thing you say to them?"
}]

emotion_history = []


def create_message(user_message=None, user_emotion=None):

    return f"The user says, '{user_message}'. Initially the user looked {user_emotion[0]}, then {user_emotion[1]}."


def find_max_emotion(predictions):

    def get_adjective(score):
        if 0.26 <= score < 0.35:
            return "slightly"
        elif 0.35 <= score < 0.44:
            return "somewhat"
        elif 0.44 <= score < 0.53:
            return "moderately"
        elif 0.53 <= score < 0.62:
            return "quite"
        elif 0.62 <= score < 0.71:
            return "very"
        elif 0.71 <= score <= 3:
            return "extremely"
        else:
            return ""

    if len(predictions) == 0:
        return ["calm", "bored"]

    def process_section(section):
        emotion_predictions = []
        for frame_dict in section:
            if 'predictions' not in frame_dict['face']:
                continue
            frame_emo_dict = frame_dict['face']["predictions"][0]["emotions"]
            emo_dict = {x["name"]: x["score"] for x in frame_emo_dict}
            emo_frame = sorted(emo_dict.items())
            emo_frame = np.array([x[1] for x in emo_frame])
            emotion_predictions.append(emo_frame)
        if len(emotion_predictions) == 0:
            return 'calm'
        # Assuming 'emotion_predictions' is a 2D array
        mean_predictions = np.array(emotion_predictions).mean(axis=0)
        # Get the index of the highest value
        top_index = np.argmax(mean_predictions)

        # Add adjectives to the top emotion based on the prediction score
        top_emotion_adjective = f"{get_adjective(mean_predictions[top_index])} {EMOTIONS[top_index]}"
        return top_emotion_adjective

    # Split predictions into 2 sections
    section_size = len(predictions) // 2
    sections = [
        predictions[i * section_size:(i + 1) * section_size] for i in range(2)]

    # Get top emotion for each section
    top_emotions = [process_section(section) for section in sections]
    return top_emotions


def store_emotions(result, rec_id):

    print("b")

    emotion_history.append(result)


def estimate_burnout(predictions):
    print("f")
    emotion_predictions = []
    for frame_dict in predictions:

        print("g")
        if 'predictions' not in frame_dict['face']:
            continue
        frame_emo_dict = frame_dict['face']["predictions"][0]["emotions"]
        emo_dict = {x["name"]: x["score"] for x in frame_emo_dict}
        emo_frame = sorted(emo_dict.items())
        emo_frame = np.array([x[1] for x in emo_frame])
        emotion_predictions.append(emo_frame)
    if len(emotion_predictions) == 0:
        return 'calm'

    print(emotion_predictions)


def message(transcription):
    global emotion_history
    user_emotions = find_max_emotion(emotion_history)
    message = create_message(transcription, user_emotions)
    print(Fore.GREEN + "PROMPT:", message + Style.RESET_ALL)
    conversation.append({"
[truncated — 789 more characters]
```

[4 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]