# Project export: Om: Optimized Meditation This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2025 - Tagline: Optimizing mindfulness through adaptive meditation: Using biometric wearables and a visual learning model to personalize meditation, enhance resilience, and tailor wellness products to user needs - Devpost: https://devpost.com/software/om-optimized-meditation - GitHub: https://github.com/rahulrejeev/treehacks-2025.git - Video: https://www.youtube.com/embed/fEaOdKsXo84?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — Rahul Rejeev (15 commits), neil pattanaik (13 commits) ## Devpost submission (written by the team) ### Overview TL;DR High-stress professions, such as military service, contribute to cognitive impairment, increased impulsivity, and long-term health risks, yet existing stress management techniques like self-guided meditation lack structure and real-time adaptability. Guided meditation is effective, but not always practical through a human. Our AI-driven system personalizes meditation questioning sequences in real-time by integrating biometric wearables and visual analysis to assess physiological and cognitive responses. By dynamically adjusting meditation sessions and providing personalized wellness recommendations, our solution offers a scalable, effective, and data-driven approach to stress mitigation. ### Inspiration / Problem Statement / What We Learned We have seen firsthand the long-term consequences of high-stress environments. Veterans, first responders, and individuals in high-stakes professions often experience mental health disorders, PTSD, and substance use disorder (SUD) at disproportionately high rates. We wanted to explore an effective, preventative measure to mitigate the long-term impact of chronic stress before it manifests into severe mental health diagnoses. The Effects of High-Stress Environments High-stress professions lead to significant cognitive, emotional, and physiological challenges: Cognitive Disruption – Stress reduces attention span by 30%, depleting working memory and impairing decision-making. Risky Coping Mechanisms – Weakened prefrontal cortex (PFC) function increases impulsivity and SUD rates. Physiological Impact – Heightened amygdala reactivity causes muscle tension, anxiety, and long-term health risks. The Limitations of Self-Guided Meditation While meditation is a proven tool for stress reduction, self-guided meditation is often ineffective in high-stress situations because it: Lacks external structure, making engagement difficult. Relies on personal discipline, leading to inconsistent practice. Fails to override cognitive overload, rendering it ineffective for immediate relief. As a result, self-guided meditation struggles to: Actively regulate stress responses. Enhance cognitive resilience. Optimize decision-making in real-time. The Power of Personalized Meditation The U.S. Army’s STRONG Project found that personalized meditation can: Reduce acute stress markers by 33% during survival training. Improve working memory capacity in 72% of participants after four weeks. Cut burnout-related medical discharges by 19% in high-deployment units. The Problem with Human-Guided Meditation Despite its benefits, human-guided meditation has scalability issues: Limited Availability – Trained instructors are scarce. High Cost – Widespread adoption is impractical. Lack of Real-Time Adaptation – Traditional methods do not integrate biofeedback. Inability to Personalize at Scale – Generic meditation techniques fail to address individual needs. ### What it does We developed an AI-driven system that personalizes meditation in real-time by integrating biometric data and visual analysis: Biometric Wearable Data: Tracks heart rate variability, respiratory rate, electrodermal activity, blood volume pulse, and body temperature to assess physiological responses. AI-Powered Visual Analysis: Uses video to monitor facial muscle tension, eye movement, head positioning, postural sway, and micro-expressions, providing insights into cognitive engagement and emotional states. AI-Powered Personalization & Feedback: Real-Time Effectiveness Analysis – AI evaluates biometric and visual markers to determine meditation efficacy. If the user remains at baseline levels, the questioning series will continue under our meditation framework. If an adverse signal is detected (ie increased HRV, increased body temperature, and increased eye tracking), the question series will further explore that area. Personalized Guidance – Sessions adapt dynamically to individual stress levels and cognitive patterns. Customized Wellness Recommendations – AI suggests relaxation strategies and wellness products like Ekkomi® teas based on user needs. ### How we built it To enhance accuracy and effectiveness, we: Integrated biometric wearables and AI-powered visual analysis to assess user stress levels. Used Perplexity AI to research and refine meditation personalization based on real-time physiological and cognitive patterns. Applied machine learning for dynamic session adjustments and wellness recommendations based on meditation response patterns. ### Challenges we ran into Developing Mathematical Models for Facial Recognition – We wrote custom mathematical formulas to analyze facial expressions and movements: Blink Detection – Used the eye aspect ratio (EAR) formula to determine eye closures. Facial Tension Analysis – Normalized the eyebrow-eye distance to quantify muscle tension. Head Movement Tracking – Measured facial center shifts to detect posture adjustments. Micro-Expression Recognition – Applied differential eyebrow position analysis to identify subtle emotional changes. Optimizing Real-Time Processing – Running API calls concurrently with audio playback in Streamlit required efficient request handling for Eleven Labs, Gemini, and Amazon S3 to minimize latency. This integration of mathematical modeling and real-time processing ensures accurate, adaptive meditation guidance. ### Accomplishments we're proud of Developed a multi-modal platform that leverages multiple data mediums to iterate in real time. Reasoning and developing context over multiple prompts worked effectively. Slept an average of 4 hours over 36 hours per team member. Consumed 500 mg of caffeine and too many calories. ### What's next Deploy and test the platform on a team member's app, Novus link. Have access to 250+ users. Wedge into the market going B2B with companies that have existing meditation apps, and eventually deploy a B2C application. ## README (from the GitHub repository) # treehacks-2025 our novel idea to cure health we take bio-wearable data and visual cues in order to cross validate effective prompting for meditation ## Detected evidence (automated analysis) Indexed codebase: 8 recognized source files, 54 KB. - Flask (technology) — detected in the code - Python (language) — detected in the code - Streamlit (technology) — detected in the code - AWS (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (14 of 14) ``` .DS_Store .gitattributes .gitignore daily_reflection.py helpers.py libs.py meditation_processor.py meditation.py README.md requirements.txt shape_predictor_68_face_landmarks.dat shape_predictor_68_face_landmarks.dat.bz2 test.py trigger_recognition.py ``` ### Dependencies - requirements.txt: boto3, dlib, elevenlabs, flask, google-genai, imutils, mediapipe, numpy, opencv-python, Pillow, python-dotenv, pyttsx3, streamlit, streamlit-webrtc@>=0.47.0, terra-python ### Recent commits (newest first) - Update README.md - Remove .env file and add to .gitignore - final2 - final - add test - some changes for flow - some changes for flow - some changes for flow - f - t - t - better prompting - fixed pag - ? - ? - added prompting - refactor afetr audio - fucking fixed audio - g - g ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` streamlit python-dotenv opencv-python google-genai Pillow flask mediapipe numpy imutils dlib pyttsx3 boto3 terra-python streamlit-webrtc>=0.47.0 elevenlabs ``` ### helpers.py ```python import numpy as np def eye_aspect_ratio(eye): A = np.linalg.norm(eye[1] - eye[5]) B = np.linalg.norm(eye[2] - eye[4]) C = np.linalg.norm(eye[0] - eye[3]) return (A + B) / (2.0 * C) def average_point(points): return np.mean(points, axis=0) ``` ### libs.py ```python from dataclasses import dataclass from typing import List, Tuple from collections import namedtuple QuestionDataWrapper = namedtuple('QuestionDataWrapper', ["session_id", "question", "agg_feedback"]) @dataclass class MeditationSession: questions_responses: List[QuestionDataWrapper] session_id: str @dataclass class User: user_id: str sessions: List[MeditationSession] ``` ### daily_reflection.py ```python import os import time import google.generativeai as genai from dotenv import load_dotenv load_dotenv() genai.configure(api_key=os.environ["GOOGLE_API_KEY"]) def upload_to_gemini(path, mime_type=None): """Uploads the given file to Gemini. See https://ai.google.dev/gemini-api/docs/prompting_with_media """ file = genai.upload_file(path, mime_type=mime_type) print(f"Uploaded file '{file.display_name}' as: {file.uri}") return file def wait_for_files_active(files): """Waits for the given files to be active. Some files uploaded to the Gemini API need to be processed before they can be used as prompt inputs. The status can be seen by querying the file's "state" field. This implementation uses a simple blocking polling loop. Production code should probably employ a more sophisticated approach. """ print("Waiting for file processing...") for name in (file.name for file in files): file = genai.get_file(name) while file.state.name == "PROCESSING": print(".", end="", flush=True) time.sleep(10) file = genai.get_file(name) if file.state.name != "ACTIVE": raise Exception(f"File {file.name} failed to process") print("...all files ready") print() # Create the model generation_config = { "temperature": 1, "top_p": 0.95, "top_k": 40, "max_output_tokens": 8192, "response_mime_type": "text/plain", } model = genai.GenerativeModel( model_name="gemini-2.0-flash", generation_config=generation_config, ) # TODO Make these files available on the local file system # You may need to update the file paths files = [ upload_to_gemini("dailyvlog.mov", mime_type="video/quicktime"), ] # Some files have a processing delay. Wait for them to be ready. wait_for_files_active(files) chat_session = model.start_chat( history=[ { "role": "user", "parts": [ files[0], ], }, { "role": "user", "parts": [ "We will be making a recap of the day for a windddown meditation session. We want to recap all the things that happened today. So recap what happened ha" ], }, ] ) response = chat_session.send_message("INSERT_INPUT_HERE") print(response.text) ``` ### trigger_recognition.py ```python import os from dotenv import load_dotenv load_dotenv() import streamlit as st import cv2 import time from datetime import datetime import base64 import google.generativeai as genai from PIL import Image import io # Configure Gemini API GOOGLE_API_KEY = os.getenv('GOOGLE_API_KEY') if not GOOGLE_API_KEY: raise ValueError("Please set GOOGLE_API_KEY in your .env file") genai.configure(api_key=GOOGLE_API_KEY) # List available models to debug print("Available models:", genai.list_models()) # Initialize Gemini Vision model with the correct version model = genai.GenerativeModel('gemini-2.0-flash-001') # Updated model name # Predefined list of triggers TRIGGER_LIST = ['iphone', 'waterbottle', 'chairs'] def classify_frame(frame): """ This function sends a frame to the Google Gemini API and analyzes the content for objects matching the trigger list. """ try: # Convert frame to PIL Image frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) pil_image = Image.fromarray(frame_rgb) # Create a byte stream for the image byte_stream = io.BytesIO() pil_image.save(byte_stream, format='JPEG') byte_stream.seek(0) # More specific prompt for better object detection prompt = f""" Look at this image carefully. You are an object detection system. Make sure whole object is visible. Your task is to check if any of these specific objects are present: {', '.join(TRIGGER_LIST)}. Respond in this exact format: OBJECTS_FOUND: - [object name] Only list objects that are exactly matching the provided list. If no objects from the list are found, respond with: OBJECTS_FOUND: None """ # Generate content using Gemini with generation config generation_config = genai.types.GenerationConfig( temperature=0.1, # Lower temperature for more focused responses top_p=0.1, top_k=32, ) # Start timing the Gemini API call start_time = time.perf_counter() response = model.generate_content( contents=[prompt, Image.open(byte_stream)], generation_config=generation_config ) # Wait for response to complete response.resolve() end_time = time.perf_counter() # Log the elapsed time print(f"Gemini API call took {end_time - start_time:.4f} seconds") # Debug print the raw response print(f"Raw Gemini response: {response.text}") # Parse the response response_text = response.text.lower() detected_objects = [] # Check if we got a response and it contains our marker if 'objects_found:' in response_text: # Split at our marker and get the list part objects_part = response_text.split('objects_found:')[1].strip() # If not "none", process the items if 'none' not in objects_part: # Split by newlines and process each line items = [item.strip('- []').strip() for item in objects_part.split('\n') if item.strip()] detected_objects = [item for item in items if item in TRIGGER_LIST] # Debug print print(f"Detected objects: {detected_objects}") return detected_objects except Exception as e: print(f"Error in classification: {str(e)}") import traceback print(f"Full traceback: {traceback.format_exc()}") return [] # Streamlit UI st.title("Real-Time Trigger Recognition with Google Gemini") st.write("This demo captures video frames and uses Gemini to detect specific objects.") # Initialize session state for running flag if it doesn't exist if 'running' not in st.session_state: st.session_state.running = False # Single button to toggle video state if not st.session_state.running: if st.button("Start Video"): st.session_state.running = True st.rerun() else: if st.button("Stop Video"): st.session_state.running = False st.rerun() # Add a frame counter to track processing frame_counter = 0 # Main video processing loop if st.session_state.running: # Initialize video capture cap = cv2.VideoCapture(0) # Create placeholders video_placeholder = st.empty() log_placeholder = st.empty() status_placeholder = st.empty() # Initialize logs logs = [] try: while st.session_state.running: ret, frame = cap.read() if not ret: st.write("Failed to capture video frame.") break frame_counter += 1 # Process every 30th frame (approximately every 1 second at 30fps) if frame_counter % 30 == 0: status_placeholder.text("Processing frame...") print(f"Processing frame {frame_counter}") detected_objects = classify_frame(frame) # Log any triggers for obj in detected_objects: timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S") log_entry = f"{timestamp}: Detected {obj}" logs.append(log_entry) # Console log with clear marker print(f"🔔 TRIGGER DETECTED: {obj} at {timestamp}") status_placeholder.text("Waiting for next frame...") # Display the frame frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) video_placeholder.image(frame_rgb, channels="RGB") # Update logs if logs: log_placeholder.text("\n".join(logs[-10:])) # Brief sleep to control processing rate [truncated — 382 more characters] ``` ### meditation_processor.py ```python import streamlit as st import cv2 import numpy as np import dlib from imutils import face_utils import tempfile import time import os import av from streamlit_webrtc import webrtc_streamer, WebRtcMode import datetime import base64 # --- Helper Functions --- def eye_aspect_ratio(eye): A = np.linalg.norm(eye[1] - eye[5]) B = np.linalg.norm(eye[2] - eye[4]) C = np.linalg.norm(eye[0] - eye[3]) return (A + B) / (2.0 * C) def average_point(points): return np.mean(points, axis=0) # Add this helper function to process the video def process_video(video_path): st.write("Processing video...") # --- Initialize dlib's face detector and landmark predictor --- detector = dlib.get_frontal_face_detector() predictor = dlib.shape_predictor("shape_predictor_68_face_landmarks.dat") cap = cv2.VideoCapture(video_path) # --- Metrics and Variables --- total_frames = 0 closed_frames = 0 blink_count = 0 eye_closed = False prev_face_center = None total_head_movement = 0 micro_expression_events = 0 prev_eyebrow_position = None tension_score_accum = 0 tension_count = 0 # --- Thresholds and Parameters --- EAR_THRESHOLD = 0.2 EYEBROW_EYE_NORM_THRESHOLD = 0.15 MICRO_EXPR_THRESHOLD = 3.0 # Create a placeholder for video frames frame_placeholder = st.empty() while cap.isOpened(): ret, frame = cap.read() if not ret: break total_frames += 1 gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) faces = detector(gray) for face in faces: shape = predictor(gray, face) shape_np = face_utils.shape_to_np(shape) # --- Eye Movement & Blink Rate --- rightEye = shape_np[36:42] leftEye = shape_np[42:48] rightEAR = eye_aspect_ratio(rightEye) leftEAR = eye_aspect_ratio(leftEye) ear = (rightEAR + leftEAR) / 2.0 if ear < EAR_THRESHOLD: closed_frames += 1 if not eye_closed: blink_count += 1 eye_closed = True else: eye_closed = False # Draw eye landmarks for (x, y) in np.concatenate((rightEye, leftEye), axis=0): cv2.circle(frame, (x, y), 2, (0, 255, 0), -1) # --- Facial Muscle Tension Dynamics --- left_eyebrow = shape_np[17:22] right_eyebrow = shape_np[22:27] left_eye_center = average_point(shape_np[36:42]) right_eye_center = average_point(shape_np[42:48]) left_distance = np.linalg.norm(average_point(left_eyebrow) - left_eye_center) right_distance = np.linalg.norm(average_point(right_eyebrow) - right_eye_center) avg_eyebrow_eye_distance = (left_distance + right_distance) / 2.0 face_width = face.right() - face.left() normalized_distance = avg_eyebrow_eye_distance / face_width tension = normalized_distance < EYEBROW_EYE_NORM_THRESHOLD tension_score_accum += (1 if tension else 0) tension_count += 1 for (x, y) in np.concatenate((left_eyebrow, right_eyebrow), axis=0): cv2.circle(frame, (x, y), 2, (255, 0, 0), -1) # --- Head Movement & Postural Sway --- face_center = average_point(shape_np) cv2.circle(frame, (int(face_center[0]), int(face_center[1])), 3, (0, 0, 255), -1) if prev_face_center is not None: movement = np.linalg.norm(face_center - prev_face_center) total_head_movement += movement prev_face_center = face_center # --- Micro-Expression Recognition --- current_eyebrow_position = average_point(np.concatenate((left_eyebrow, right_eyebrow), axis=0)) if prev_eyebrow_position is not None: diff = np.linalg.norm(current_eyebrow_position - prev_eyebrow_position) if diff > MICRO_EXPR_THRESHOLD: micro_expression_events += 1 prev_eyebrow_position = current_eyebrow_position cv2.putText(frame, f"EAR: {ear:.2f}", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2) cv2.putText(frame, f"Blinks: {blink_count}", (10, 60), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2) cv2.putText(frame, f"Head Move: {total_head_movement:.2f}", (10, 90), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 0, 0), 2) cv2.putText(frame, f"Micro-Expr: {micro_expression_events}", (10, 120), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (255, 0, 0), 2) frame_placeholder.image(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB), channels="RGB") time.sleep(0.03) cap.release() fps = cap.get(cv2.CAP_PROP_FPS) if cap.get(cv2.CAP_PROP_FPS) > 0 else 30 duration_minutes = (total_frames / fps) / 60 blink_rate = blink_count / duration_minutes if duration_minutes > 0 else blink_count focus_score = ((total_frames - closed_frames) / total_frames * 100) if total_frames > 0 else 0 avg_tension = (tension_score_accum / tension_count * 100) if tension_count > 0 else 0 st.write("### Meditation Analysis Summary") st.write(f"**Total Frames Processed:** {total_frames}") st.write(f"**Blinks Detected:** {blink_count}") st.write(f"**Blink Rate:** {blink_rate:.2f} blinks/min") st.write(f"**Focus Score (Frames with eyes open):** {focus_score:.2f}%") st.write(f"**Facial Tension Detected:** {avg_tension:.2f}% of frames (lower normalized eyebrow-eye distance indicates tension)") st.write(f"**Total Head Movement:** {total_head_movement:.2f} pixels displacement") st.write(f"**Micro-Expression Events Detected:** {micro_expression_events}") st.write("**Note:** Thresholds and heuristics used in this demo may need calibration for [truncated — 3015 more characters] ``` ### test.py ```python import boto3 from dotenv import load_dotenv import os import json from time import sleep load_dotenv() try: AWS_ACCESS_KEY=os.getenv('AWS_ACCESS_KEY') AWS_SECRET_ACCESS_KEY=os.getenv('AWS_SECRET_ACCESS_KEY') AWS_REGION=os.getenv('AWS_REGION') AWS_BUCKET_NAME=os.getenv('AWS_BUCKET_NAME') s3 = boto3.client('s3', aws_access_key_id=AWS_ACCESS_KEY, aws_secret_access_key=AWS_SECRET_ACCESS_KEY, region_name=AWS_REGION) except Exception as e: print(f"Error: {e}") raise e # Define the five JSON telemetry payloads as Python dictionaries payloads = [ # 1. Calm { "user": { "user_id": "75fe4fd8-28fa-49bb-a26e-4ad4c9d92ee5", "provider": "FITBIT", "active": True, "scopes": None, "reference_id": "", "created_at": None, "last_webhook_update": None }, "data": [ { "metadata": { "start_time": "2025-02-16T14:00:00.000000+00:00", "end_time": "2025-02-16T14:10:00.000000+00:00", "emotional_state": "calm" }, "heart_data": { "heart_rate_data": { "summary": { "avg_hr_bpm": 65, "resting_hr_bpm": 60, "max_hr_bpm": 70, "min_hr_bpm": 60, "avg_hrv_rmssd": 90 }, "detailed": None } }, "oxygen_data": { "avg_saturation_percentage": 98.0, "vo2max_ml_per_min_per_kg": None, "vo2_samples": None, "saturation_samples": None }, "blood_pressure_data": None, "hydration_data": None, "temperature_data": None, "glucose_data": None, "ketone_data": None, "device_data": None, "measurements_data": None } ], "type": "body" }, # 2. Restless { "user": { "user_id": "75fe4fd8-28fa-49bb-a26e-4ad4c9d92ee5", "provider": "FITBIT", "active": True, "scopes": None, "reference_id": "", "created_at": None, "last_webhook_update": None }, "data": [ { "metadata": { "start_time": "2025-02-16T14:15:00.000000+00:00", "end_time": "2025-02-16T14:25:00.000000+00:00", "emotional_state": "restless" }, "heart_data": { "heart_rate_data": { "summary": { "avg_hr_bpm": 80, "resting_hr_bpm": 75, "max_hr_bpm": 90, "min_hr_bpm": 75, "avg_hrv_rmssd": 60 }, "detailed": None } }, "oxygen_data": { "avg_saturation_percentage": 97.0, "vo2max_ml_per_min_per_kg": None, "vo2_samples": None, "saturation_samples": None }, "blood_pressure_data": None, "hydration_data": None, "temperature_data": None, "glucose_data": None, "ketone_data": None, "device_data": None, "measurements_data": None } ], "type": "body" }, # 3. Angry { "user": { "user_id": "75fe4fd8-28fa-49bb-a26e-4ad4c9d92ee5", "provider": "FITBIT", "active": True, "scopes": None, "reference_id": "", "created_at": None, "last_webhook_update": None }, "data": [ { "metadata": { "start_time": "2025-02-16T14:30:00.000000+00:00", "end_time": "2025-02-16T14:40:00.000000+00:00", "emotional_state": "angry" }, "heart_data": { "heart_rate_data": { "summary": { "avg_hr_bpm": 105, "resting_hr_bpm": 100, "max_hr_bpm": 110, "min_hr_bpm": 95, "avg_hrv_rmssd": 40 }, "detailed": None } }, "oxygen_data": { "avg_saturation_percentage": 95.0, "vo2max_ml_per_min_per_kg": None, "vo2_samples": None, "saturation_samples": None }, "blood_pressure_data": None, "hydration_data": None, "temperature_data": None, "glucose_data": None, "ketone_data": None, "device_data": None, "measurements_data": None } ], "type": "body" }, # 4. Somewhat Relaxed { "user": { "user_id": "75fe4fd8-28fa-49bb-a26e-4ad4c9d92ee5", "provider": "FITBIT", "active": True, "scopes": None, "reference_id": "", "created_at": None, "last_webhook_update": None }, "data": [ { "metadata": { "start_time": "2025-02-16T14:45:00.000000+00:00", "end_time": "2025-02-16T14:55:00.000000+ [truncated — 3756 more characters] ``` ### meditation.py ```python import json import os import time import boto3 import cv2 import dlib from google import genai from dotenv import load_dotenv import numpy as np import streamlit as st from imutils import face_utils from helpers import eye_aspect_ratio, average_point import logging from elevenlabs.client import ElevenLabs import base64 import re logger = logging.Logger('logger') logger.setLevel(logging.DEBUG) # Create a stream handler (console output) handler = logging.StreamHandler() handler.setLevel(logging.DEBUG) # Create a formatter and attach it to the handler formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') handler.setFormatter(formatter) # Add the handler to your logger logger.addHandler(handler) prompts_asked = [] visual_cues_history = [] telemetry_history = [] load_dotenv() # Load environment variables try: GOOGLE_API_KEY=os.getenv('GOOGLE_API_KEY') AWS_ACCESS_KEY=os.getenv('AWS_ACCESS_KEY') AWS_SECRET_ACCESS_KEY=os.getenv('AWS_SECRET_ACCESS_KEY') AWS_REGION=os.getenv('AWS_REGION') AWS_BUCKET_NAME=os.getenv('AWS_BUCKET_NAME') ELEVEN_LABS_API_KEY=os.getenv('ELEVEN_LABS_KEY') VOICE_ID=os.getenv('VOICE_ID') s3 = boto3.client('s3', aws_access_key_id=AWS_ACCESS_KEY, aws_secret_access_key=AWS_SECRET_ACCESS_KEY, region_name=AWS_REGION) elevenlabs_client = ElevenLabs(api_key=ELEVEN_LABS_API_KEY) except Exception as e: raise e # Config gemini_client = genai.Client(api_key=GOOGLE_API_KEY) TIME_INTERVAL = 40 # seconds ELEVEN_LABS_API = f"https://api.elevenlabs.ai/v1/text-to-speech/{VOICE_ID}" # Video config # Load dlib's face detector and landmark predictor. detector = dlib.get_frontal_face_detector() predictor_path = "shape_predictor_68_face_landmarks.dat" if not os.path.exists(predictor_path): st.error(f"Missing {predictor_path} file!") st.stop() predictor = dlib.shape_predictor(predictor_path) # --- Session State Initialization --- def main_loop(): questions = 0 while not run_live: time.sleep(0.1) initialize() while questions < 5: curr = time.time() handle_audio() while time.time() - curr < TIME_INTERVAL: frame, rgb_frame = update_camera() render_frame(rgb_frame) main(frame, rgb_frame) update_scipt() questions += 1 camera.release() FRAME_WINDOW.image(image="./landing.jpeg", use_container_width=True) def extract_readable_script(text): """ Given a text string in the format with bold section titles and minute markers, remove those formatting markers and return the clean script. """ # Regex explanation: # \*\*.*?\*\* -> Matches any text enclosed in ** (non-greedy) # \(Minute\s*\d+\s*-\s*\d+\) -> Matches minute markers like (Minute 1-3) # pattern = r'\*\*.*?\*\*|\(Minute\s*\d+\s*-\s*\d+\)' # # Remove the matched patterns from the text. # cleaned_text = re.sub(pattern, '', text) # # Optionally, remove extra blank lines or surrounding whitespace. # # This splits the text by lines, strips them, and rejoins non-empty lines. # cleaned_lines = [line.strip() for line in cleaned_text.splitlines() if line.strip()] # return "\n".join(cleaned_lines) return text def initialize(): st.session_state.metrics = { 'total_frames': 0, 'blink_count': 0, 'closed_frames': 0, 'tension_score_accum': 0, 'tension_count': 0, 'total_head_movement': 0, 'prev_face_center': None, 'prev_eyebrow_position': None, 'micro_expression_events': 0 } st.session_state.last_update = time.time() st.session_state.current_script = "Alright, let's begin. Find a comfortable position, either sitting or lying down, and gently close your eyes. Take a deep breath in, and as you exhale, let go of any tension you might be holding. Now, what do you notice right now? What sensations are present in your body, and what sounds do you hear around you? Just observe without judgment." prompts_asked.append(st.session_state.current_script) # --- Custom CSS for styling --- st.markdown("""