# Project export: TalkTuah

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: Talk smarter. Talk smoother. TalkTuah—your AI-powered conversation coach.
- Devpost: https://devpost.com/software/talktuah
- GitHub: https://github.com/IbrahimKhanGH/TreeHacks2
- Video: https://www.youtube.com/embed/XnNf8KjoK-c?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) — IbrahimKhanGH (15 commits)

## Devpost submission (written by the team)

### Inspiration

I was at the Columbia hackathon last week when I had the opportunity to see social Stockfish (an AI that predicts conversation flow, like a chess engine for chats) get built next to me. It blew my mind. Shoutout to @eddybuild and @cadenbuild for their work on the chess glasses (the reason I bought the meta ray bands lmao) and social Stockfish---it was a huge spark for me. I wanted to create something similarly badass, a glimpse into the future of AI-powered wearable social intelligence. So, I jumped into a solo project to challenge myself at one of the biggest stages. The result? TalkTuah---a real-time conversation coach that listens, analyzes, and guides you through any interaction. What It Does Real-Time Listening: Captures conversation and transcribes it on the fly. Engagement & Body Language Tracking: Keeps an eye on how people are reacting. Smart Suggestions: Provides AI-generated tips and responses right when you need them. Wearable Output: Whispers prompts through Meta Ray-Bans (yes the glasses---but no official SDK!). It's perfect for dates, interviews, or just hanging out. TalkTuah subtly nudges you toward your goal without taking over the conversation. How I Built It Speechmatics for live transcription. OpenAI + Grok to generate and refine conversation strategies. BlackHole 2ch for seamless audio routing (in and out). ElevenLabs for text-to-speech, whispering cues through the Ray-Bans. Meta Ray-Bans have no official SDK, so I had to piece together custom workarounds to make it all function in real time. Challenges No SDK for Meta Ray-Bans: The biggest headache by far. Low Latency: Had to make sure everything responded fast enough for real conversation. Natural Feel: Too many AI interruptions get weird, so timing was everything. Next Steps Speed & Smoothness: Optimize real-time performance even more. Advanced Body Language Detection: Refine how TalkTuah interprets nonverbal cues. Mobile App Companion: Bring these features to people without the glasses.

## README (from the GitHub repository)

# TreeHacks2


## Detected evidence (automated analysis)

Indexed codebase: 10 recognized source files, 44 KB.
- Python (language) — detected in the code
- OpenAI (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (12 of 12)

```
.gitignore
.vscode/settings.json
ai_service.py
audio_system.py
conversation_buffer.py
elevenlabs_output.py
main.py
posture_analyzer.py
posture_recognizer.py
README.md
screen_capture.py
transcription_handler.py
```

### Dependencies

No dependency index available.

### Recent commits (newest first)

- pre demo pus
- this somehow worked for evan
- CV implementation bismillah
- i dont think this is working lmfao
- something is slightly working ill take it
- evan test it aberely works;
- ELEVEN LABS BEFOR EPOKER ALHAMDULILLAH
- semi working diarization
- assistnat triggerrrr
- before we do some ai:
- i pray to god this work
- luh calm audio and capture alhamdulillah
- luh calm STT w glasses (not working screen capture
- this took tooooo long
- first commit

## Key source files (fetched from GitHub, selected and truncated for size)

### main.py

```python
import asyncio
from audio_system import AudioSystem
from transcription_handler import TranscriptionHandler
from conversation_buffer import ConversationBuffer

def main():
    """Main program entry point"""
    # Get conversation context
    print("\n=== CONVERSATION SETUP ===")
    goal = input("What's your goal for this conversation? ")
    context = input("\nAny other relevant context? (relationship, background, current situation, etc.)\n> ")
    
    # Initialize systems
    audio_system = AudioSystem()
    conversation = ConversationBuffer(goal=goal, audience=context, audio_system=audio_system)
    
    # Generate and play opening message
    print("\nGenerating opening message...")
    asyncio.run(conversation.start_conversation())
    
    # Start transcription
    transcription = TranscriptionHandler(audio_system, conversation)
    transcription.start()

if __name__ == "__main__":
    main()
```

### ai_service.py

```python
import asyncio
from typing import List, Dict
from openai import AsyncOpenAI

class AIService:
    """
    Handles interactions with the Groq API using OpenAI's client.
    """
    def __init__(self, api_key: str):
        self.client = AsyncOpenAI(
            base_url="https://api.groq.com/openai/v1",
            api_key=api_key  # This should be your GROQ_API_KEY
        )

    async def get_ai_response(self, messages: List[Dict[str, str]]) -> str:
        """
        Sends messages to Groq and retrieves the response.
        """
        try:
            # Extract goal type from messages to determine length
            goal_type = messages[0]["content"].lower()
            
            # Set max tokens based on context
            if "initial strategy" in goal_type:
                max_tokens = 50  # Longer for strategy
            elif "transition" in goal_type:
                max_tokens = 25  # Medium for transitions
            else:
                max_tokens = 15  # Short for quick suggestions
            
            stream = await self.client.chat.completions.create(
                model="mixtral-8x7b-32768",
                messages=messages,
                max_tokens=max_tokens,
                temperature=0.7,  # More consistent
                presence_penalty=0.5,  # Avoid repetition
                frequency_penalty=0.3,  # More varied
                stream=True
            )
            
            collected_message = ""
            async for chunk in stream:
                if chunk.choices[0].delta.content:
                    collected_message += chunk.choices[0].delta.content
            
            # Clean up response
            response = collected_message.strip()
            if not response.endswith((".", "!", "?")):
                response += "."
                
            return response
            
        except Exception as e:
            print(f"Groq API error: {e}")
            return "" 
```

### posture_analyzer.py

```python
import cv2
import mediapipe as mp
import numpy as np
from posture_recognizer import PostureRecognizer
import time

class PostureAnalyzer:
    def __init__(self):
        """Initialize the posture analyzer with webcam capture"""
        # Initialize video capture from OBS Virtual Camera
        self.cap = cv2.VideoCapture(0)  # Try 0, 1, or 2 if not found
        self.running = False
        
        # Initialize posture recognizer
        self.recognizer = PostureRecognizer()
        
        print("PostureAnalyzer initialized - looking for OBS Virtual Camera...")
        
    def start(self):
        """Start the analysis loop"""
        print("Starting posture analysis...")
        self.running = True
        start_time = time.time()
        max_duration = 30  # 30 seconds maximum duration
        
        try:
            while self.running:
                # Check if maximum duration exceeded
                if time.time() - start_time > max_duration:
                    print("Maximum analysis duration reached")
                    break
                    
                ret, frame = self.cap.read()
                if not ret:
                    print("Failed to grab frame from camera")
                    continue
                    
                # Process frame
                joints, face_landmarks = self.recognizer.process_frame(frame)
                
                # Get posture analysis
                posture = self.recognizer.recognize_posture()
                
                # Draw debug visualization
                debug_frame = self.recognizer.draw_debug(frame)
                
                # Add posture text to frame - update wording to be more positive
                cv2.putText(
                    debug_frame,
                    f"Facing: {posture.replace('away', 'towards')}",  # Replace negative wording
                    (10, 30),
                    cv2.FONT_HERSHEY_SIMPLEX,
                    1,
                    (0, 255, 0),
                    2
                )
                
                # Show the frame
                cv2.imshow('Posture Analysis', debug_frame)
                
                # Break loop on 'q' press
                if cv2.waitKey(1) & 0xFF == ord('q'):
                    break
                    
        except Exception as e:
            print(f"Analysis error: {e}")
        finally:
            self.stop()
            
    def stop(self):
        """Stop the analysis and clean up"""
        self.running = False
        self.cap.release()
        cv2.destroyAllWindows()

def main():
    """Test function"""
    analyzer = PostureAnalyzer()
    analyzer.start()

if __name__ == "__main__":
    main() 
```

### elevenlabs_output.py

```python
import os
import io
import numpy as np
from pydub import AudioSegment
from elevenlabs.client import ElevenLabs

class ElevenLabsOutput:
    """
    Handles text-to-speech conversion using ElevenLabs.
    """
    def __init__(self):
        self.api_key = os.getenv('ELEVENLABS_API_KEY')
        if not self.api_key:
            raise ValueError("ELEVENLABS_API_KEY not found in environment variables")
        self.client = ElevenLabs(api_key=self.api_key)

    def speak(self, text: str, audio_system):
        """
        Converts text to speech and plays it through the audio system.

        Args:
            text (str): The text to be spoken.
            audio_system (AudioSystem): The audio system to play the speech.
        """
        try:
            # Clean up text before speaking
            text = text.strip()
            if text.startswith("Start by"):
                text = text.replace("Start by", "").strip()
            if not text.endswith((".", "!", "?")):
                text = text + "."
                
            print(f"\n[SPEAKING] {text}")
            
            audio_bytes = self.client.generate(
                text=text,
                voice="Rachel",
                model="eleven_monolingual_v1",  # Faster model
                optimize_streaming_latency=4,    # Max optimization
                voice_settings={
                    "stability": 0.5,           # Lower stability for faster speech
                    "similarity_boost": 0.5,     # Lower similarity for faster processing
                    "style": 0.0,               # Neutral style for speed
                    "speaking_rate": 1.5        # Speed up speech (1.0 is normal, 2.0 is double)
                }
            )
            
            # Convert generator to bytes
            audio_data = b''.join(chunk for chunk in audio_bytes)
            
            # Create BytesIO object for AudioSegment processing
            fp = io.BytesIO(audio_data)
            
            # Process audio
            audio_segment = AudioSegment.from_mp3(fp)
            
            # Speed up the audio by modifying the frame rate
            audio_segment = audio_segment.speedup(playback_speed=1.2)  # 20% faster
            
            # Set audio properties
            audio_segment = audio_segment.set_frame_rate(48000)
            audio_segment = audio_segment.set_channels(1)
            audio_segment = audio_segment.set_sample_width(4)
            
            # Reduce volume and add shorter fades
            audio_segment = audio_segment - 30
            audio_segment = audio_segment.fade_in(20).fade_out(20)  # Shorter fades
            
            # Normalize
            normalized = np.array(audio_segment.get_array_of_samples()).astype(np.float32)
            normalized = normalized / np.max(np.abs(normalized))
            
            # Play through existing audio system
            audio_system.output_stream.write(normalized.tobytes())
            
        except Exception as e:
            print(f"Error with ElevenLabs: {e}") 
```

### transcription_handler.py

```python
import speechmatics
from speechmatics.models import ConnectionSettings, TranscriptionConfig, AudioSettings
from speechmatics.client import WebsocketClient
from dotenv import load_dotenv
import os
from ai_service import AIService
from conversation_buffer import ConversationBuffer
import asyncio

# Load environment variables
load_dotenv()

# API Configuration
API_KEY = os.getenv('SPEECHMATICS_API_KEY')
LANGUAGE = "en"

class TranscriptionHandler:
    def __init__(self, audio_system, conversation: ConversationBuffer):
        self.audio_system = audio_system
        self.conversation = conversation
        self.ai_service = AIService(api_key=os.getenv('OPENAI_API_KEY'))
        self.ws = self._setup_client()

    def _setup_client(self):
        """Setup Speechmatics client and handlers"""
        ws = WebsocketClient(
            ConnectionSettings(
                url="wss://eu2.rt.speechmatics.com/v2",
                auth_token=API_KEY
            )
        )

        ws.add_event_handler(
            event_name=speechmatics.models.ServerMessageType.AddPartialTranscript,
            event_handler=lambda msg: self._handle_transcript(msg)
        )
        
        ws.add_event_handler(
            event_name=speechmatics.models.ServerMessageType.AddTranscript,
            event_handler=lambda msg: self._handle_transcript(msg)
        )

        return ws

    def _handle_transcript(self, msg):
        """Handle incoming transcripts"""
        is_final = msg.get('message') == 'AddTranscript'
        results = msg.get('results', [])
        transcript = ""
        
        # Combine all pieces into one transcript
        if results:
            for result in results:
                if result.get('alternatives'):
                    transcript += " " + result['alternatives'][0].get('content', '')
            transcript = transcript.strip()
            
        if not transcript:
            return
            
        # Get speaker from last result
        speaker = 'UU'
        if results and results[-1].get('alternatives'):
            speaker = results[-1]['alternatives'][0].get('speaker', 'UU')
            
        print(f"\n[FINAL] [{speaker}] {transcript}")
        
        if speaker != 'UU':
            self.conversation.add_final(transcript, speaker)

    def start(self):
        """Start transcription"""
        config = TranscriptionConfig(
            language=LANGUAGE,
            enable_partials=True,
            max_delay=0.7,  # Faster response time
            operating_point="enhanced",
            diarization="speaker",
            speaker_diarization_config={
                "max_speakers": 2  # Only allowed property for real-time
            }
        )

        audio_settings = AudioSettings(
            sample_rate=48000,
            chunk_size=512,
            encoding="pcm_f32le"
        )

        print("\nStarting transcription... (Press Ctrl+C to stop)")
        print("Say 'Hey Assistant' to trigger a response!")
        print("=" * 50)

        try:
            self.ws.run_synchronously(self.audio_system, config, audio_settings)
        except KeyboardInterrupt:
            print("\nStopping...")
        except Exception as e:
            print(f"\nError: {e}")
        finally:
            self.audio_system.close()
```

### audio_system.py

```python
import pyaudio
import sounddevice as sd
import numpy as np
from gtts import gTTS
import io
from pydub import AudioSegment

# Audio Configuration Constants
CHUNK = 512      # Buffer size for audio processing
FORMAT = pyaudio.paFloat32  # 32-bit float audio format
CHANNELS = 1      # Mono audio
RATE = 48000      # Sample rate in Hz

class AudioSystem:
    """
    Handles all audio input/output operations including:
    - Capturing audio from BlackHole 2ch
    - Playing audio through Aggregate Device
    - Processing and normalizing audio data
    """
    def __init__(self):
        self.p = pyaudio.PyAudio()
        self._list_available_devices()
        self.input_stream = self.setup_input()
        self.output_stream = self.setup_output()
        print("\nAudio system initialized")

    def _list_available_devices(self):
        """Lists all available audio devices for debugging"""
        print("\nAvailable Audio Devices:")
        devices = sd.query_devices()
        for i, device in enumerate(devices):
            print(f"{i}: {device['name']} (in={device['max_input_channels']}, out={device['max_output_channels']})")

    def setup_input(self):
        """Sets up the input stream from BlackHole 2ch"""
        devices = sd.query_devices()
        blackhole_2ch = None
        for i, device in enumerate(devices):
            if 'BlackHole 2ch' in device['name']:
                blackhole_2ch = i
                print(f"Found BlackHole 2ch (input) at index {i}")
                break
        
        if blackhole_2ch is None:
            raise Exception("BlackHole 2ch not found!")

        return self.p.open(
            format=FORMAT,
            channels=CHANNELS,
            rate=RATE,
            input=True,
            input_device_index=blackhole_2ch,
            frames_per_buffer=CHUNK
        )

    def setup_output(self):
        """Sets up the output stream to the Aggregate Device"""
        devices = sd.query_devices()
        aggregate_device = None
        for i, device in enumerate(devices):
            if 'TreeHacks Audio' in device['name']:
                aggregate_device = i
                print(f"Found Aggregate Device (output) at index {i}")
                break
        
        if aggregate_device is None:
            raise Exception("Aggregate Device not found! Please set it up in Audio MIDI Setup.")

        return self.p.open(
            format=FORMAT,
            channels=CHANNELS,
            rate=RATE,
            output=True,
            output_device_index=aggregate_device,
            frames_per_buffer=CHUNK
        )

    def play_test_sound(self):
        """Plays a test notification sound"""
        try:
            print("\nPlaying test sound...")
            audio = AudioSegment.from_mp3("notification.mp3")
            audio = self._process_audio(audio)
            self.output_stream.write(audio.tobytes())
            print("Test sound played through Aggregate Device.")
        except Exception as e:
            print(f"Error playing test sound: {e}")

    def speak(self, text):
        """Converts text to speech and plays it"""
        try:
            print(f"\n[SPEAKING] {text}")
            tts = gTTS(text=text, lang='en')
            fp = io.BytesIO()
            tts.write_to_fp(fp)
            fp.seek(0)
            
            audio = AudioSegment.from_mp3(fp)
            audio = self._process_audio(audio)
            self.output_stream.write(audio.tobytes())
        except Exception as e:
            print(f"Error speaking: {e}")

    def _process_audio(self, audio):
        """Process audio for output"""
        audio = audio.set_frame_rate(RATE)
        audio = audio.set_channels(CHANNELS)
        audio = audio.set_sample_width(4)
        audio = audio - 30  # Reduce volume
        audio = audio.fade_in(50).fade_out(50)
        
        normalized = np.array(audio.get_array_of_samples()).astype(np.float32)
        normalized = normalized / np.max(np.abs(normalized))
        return normalized

    def read(self, size=-1):
        """Reads audio data from the input stream"""
        try:
            data = self.input_stream.read(CHUNK, exception_on_overflow=False)
            audio_data = np.frombuffer(data, dtype=np.float32)
            
            if np.max(np.abs(audio_data)) > 0.01:
                print("Receiving audio input...")
            
            return audio_data.tobytes()
        except Exception as e:
            print(f"Error reading from input stream: {e}")
            return b''

    def close(self):
        """Cleanly closes all audio streams"""
        self.input_stream.stop_stream()
        self.input_stream.close()
        self.output_stream.stop_stream()
        self.output_stream.close()
        self.p.terminate()
```

### screen_capture.py

```python
import cv2
import numpy as np
import os
import time
from Quartz import (
    CGWindowListCreateImage,
    CGRectNull,
    kCGWindowListOptionOnScreenOnly,
    kCGNullWindowID,
    kCGWindowImageDefault,
    CGImageGetWidth,
    CGImageGetHeight,
    CGImageGetDataProvider,
    CGDataProviderCopyData,
    CGImageGetBytesPerRow
)
import pyvirtualcam

class ScreenCapture:
    """
    Captures and processes screen content for computer vision analysis.
    Specifically designed to capture the left portion of the screen where
    video call participants typically appear.
    """
    
    def __init__(self, width=1280, height=720, fps=30.0):
        """
        Initialize screen capture parameters.
        
        Args:
            width (int): Target width of capture
            height (int): Target height of capture
            fps (float): Frames per second for capture
        """
        self.width = width
        self.height = height
        self.fps = fps
        self.running = False
        self.cam = None

        # Crop parameters as percentages of screen
        self.CROP_TOP = 0.16      # Start 16% from top
        self.CROP_BOTTOM = 0.96   # End at 96% from top
        self.CROP_LEFT = 0.02     # Start 2% from left
        self.CROP_RIGHT = 0.33    # End at 33% from left
        
        # Output dimensions for preview
        self.PREVIEW_WIDTH = 600
        self.PREVIEW_HEIGHT = 1080
        
        print("ScreenCapture initialized with dimensions:", width, "x", height)

    def capture_frame(self):
        """
        Captures and processes a single frame from the screen.
        
        Returns:
            numpy.ndarray: Processed frame ready for analysis, or None if capture fails
        """
        try:
            # Capture full screen
            print("Attempting to capture screen...")
            screenshot = CGWindowListCreateImage(
                CGRectNull,
                kCGWindowListOptionOnScreenOnly,
                kCGNullWindowID,
                kCGWindowImageDefault
            )
            
            if screenshot is None:
                print("Failed to capture screen - screenshot is None")
                return None
                
            print("Screen captured successfully")
            
            # Convert to numpy array
            width = int(CGImageGetWidth(screenshot))
            height = int(CGImageGetHeight(screenshot))
            bytes_per_row = int(CGImageGetBytesPerRow(screenshot))
            bytes_per_pixel = 4
            
            data_provider = CGImageGetDataProvider(screenshot)
            data = CGDataProviderCopyData(data_provider)
            
            if data is None:
                print("Failed to get image data")
                return None
                
            # Process image data
            data = np.frombuffer(data, dtype=np.uint8)
            data = data.reshape(height, bytes_per_row // bytes_per_pixel, bytes_per_pixel)
            frame = cv2.cvtColor(data[:, :width], cv2.COLOR_RGBA2RGB)
            
            # Calculate crop dimensions
            height_margin_top = int(height * self.CROP_TOP)
            height_margin_bottom = int(height * self.CROP_BOTTOM)
            left_start = int(width * self.CROP_LEFT)
            left_end = int(width * self.CROP_RIGHT)
            
            # Crop frame
            left_frame = frame[height_margin_top:height_margin_bottom, left_start:left_end]
            
            # Resize for preview
            output_frame = cv2.resize(left_frame, (self.PREVIEW_WIDTH, self.PREVIEW_HEIGHT))
            print("Frame processed successfully")
            
            return output_frame
            
        except Exception as e:
            print(f"Detailed capture error: {str(e)}")
            import traceback
            traceback.print_exc()
            return None

    def start(self):
        """Starts continuous frame capture and streams to virtual camera"""
        print("Starting screen capture...")
        self.running = True
        
        try:
            # Initialize virtual camera
            with pyvirtualcam.Camera(width=self.PREVIEW_WIDTH, 
                                   height=self.PREVIEW_HEIGHT, 
                                   fps=20) as cam:
                print(f'Virtual camera created: {cam.device}')
                
                while self.running:
                    frame = self.capture_frame()
                    if frame is not None:
                        # Convert to RGB format required by pyvirtualcam
                        frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
                        cam.send(frame_rgb)
                        cam.sleep_until_next_frame()
                        
                        # Optional preview window
                        cv2.imshow('Screen Capture Preview', frame)
                        if cv2.waitKey(1) & 0xFF == ord('q'):
                            break
                    
        except KeyboardInterrupt:
            print("\nCapture stopped by user")
        finally:
            cv2.destroyAllWindows()

    def stop(self):
        """Stops the screen capture process"""
        self.running = False
        cv2.destroyAllWindows()

def main():
    """Main entry point for testing screen capture"""
    print("Testing screen capture...")
    capture = ScreenCapture()
    capture.start()

if __name__ == "__main__":
    main() 
```

### posture_recognizer.py

```python
import mediapipe as mp
import numpy as np
import cv2

class PostureRecognizer:
    def __init__(self):
        # Initialize pose detection
        self.mp_pose = mp.solutions.pose
        self.pose = self.mp_pose.Pose(
            min_detection_confidence=0.3,  # Lower threshold for demo
            min_tracking_confidence=0.3
        )
        
        # Initialize face detection
        self.mp_face = mp.solutions.face_mesh
        self.face_mesh = self.mp_face.FaceMesh(
            max_num_faces=1,
            min_detection_confidence=0.3,
            min_tracking_confidence=0.3
        )
        
        self.joints = {}
        self.face_landmarks = None
        self.engagement_history = []  # Track last few scores
        
    def process_frame(self, frame):
        """Process frame for both pose and face"""
        rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        
        # Process pose
        pose_results = self.pose.process(rgb_frame)
        face_results = self.face_mesh.process(rgb_frame)
        
        # Handle pose results
        if pose_results.pose_landmarks:
            self.joints = {}
            for idx, landmark in enumerate(pose_results.pose_landmarks.landmark):
                joint_name = self.mp_pose.PoseLandmark(idx).name
                self.joints[joint_name] = {
                    'x': landmark.x,
                    'y': landmark.y,
                    'z': landmark.z,
                    'visibility': landmark.visibility
                }
        
        # Handle face results
        self.face_landmarks = face_results.multi_face_landmarks[0] if face_results.multi_face_landmarks else None
        
        return self.joints, self.face_landmarks
    
    def calculate_face_direction(self):
        """Calculate if face is turned away"""
        if not self.face_landmarks:
            return False
            
        # Get nose and ear landmarks
        nose = self.face_landmarks.landmark[4]  # Nose tip
        left_ear = self.face_landmarks.landmark[234]  # Left ear
        right_ear = self.face_landmarks.landmark[454]  # Right ear
        
        # Calculate ear difference (indicates head turn)
        ear_diff = abs(left_ear.x - right_ear.x)
        
        # If one ear is much more visible than the other, head is turned
        return ear_diff > 0.15
    
    def calculate_torso_angle(self):
        """Calculate angle of torso relative to camera"""
        try:
            left_shoulder = self.joints['LEFT_SHOULDER']
            right_shoulder = self.joints['RIGHT_SHOULDER']
            
            # Calculate shoulder width as seen by camera
            shoulder_width = abs(right_shoulder['x'] - left_shoulder['x'])
            
            # Convert to angle (rough approximation)
            # When facing camera directly, shoulder width is maximum
            # As person turns, apparent width decreases
            angle = np.arccos(shoulder_width / 0.3) * 180 / np.pi  # 0.3 is approx max shoulder width
            return angle
        except:
            return 0
    
    def recognize_posture(self):
        """Recognize posture and face direction"""
        try:
            # Calculate measurements
            torso_angle = self.calculate_torso_angle()
            shoulder_width = abs(self.joints['RIGHT_SHOULDER']['x'] - self.joints['LEFT_SHOULDER']['x'])
            face_turned = self.calculate_face_direction()
            
            # 1. Check if person is turned away (body or face)
            if torso_angle > 30 or face_turned:
                return "Uninterested (Facing Away)"
            
            # 2. Check engagement level
            if shoulder_width < 0.2:  # Person is far from camera
                return "Too Far (Move Closer)"
            
            return "Engaged"
            
        except KeyError:
            return "Insufficient Data"
    
    def draw_debug(self, frame):
        """Draw debug information on frame"""
        # Draw pose landmarks
        if self.joints:
            for joint in self.joints.values():
                x = int(joint['x'] * frame.shape[1])
                y = int(joint['y'] * frame.shape[0])
                cv2.circle(frame, (x, y), 5, (0, 255, 0), -1)
        
        # Draw face landmarks
        if self.face_landmarks:
            for landmark in self.face_landmarks.landmark:
                x = int(landmark.x * frame.shape[1])
                y = int(landmark.y * frame.shape[0])
                cv2.circle(frame, (x, y), 1, (0, 0, 255), -1)
        
        return frame 
    
    def get_engagement_score(self) -> float:
        """Calculate a simple engagement score (0-1)"""
        try:
            score = 1.0  # Start with perfect score
            
            # Simple checks that reduce score
            if not self.face_landmarks:
                score *= 0.5  # Face not visible
            
            if not self.joints:
                score *= 0.5  # Body not visible
            
            # Add some noise for demo purposes
            import random
            score *= random.uniform(0.8, 1.0)
            
            # Smooth score with history
            self.engagement_history.append(score)
            self.engagement_history = self.engagement_history[-5:]  # Keep last 5 scores
            return sum(self.engagement_history) / len(self.engagement_history)
            
        except Exception as e:
            print(f"Error calculating engagement: {e}")
            return 0.5  # Default to neutral
    
    def get_engagement_state(self) -> str:
        """Get a human-readable engagement state"""
        score = self.get_engagement_score()
        
        if score > 0.8:
            return "Highly Engaged"
        elif score > 0.6:
            return "Engaged"
        elif score > 0.4:
            return "Neutral"
        else:
            return "Low Engagement" 
```

### conversation_buffer.py

```python
import asyncio
import time
from typing import List, Dict
import os
from dotenv import load_dotenv
from ai_service import AIService
from elevenlabs_output import ElevenLabsOutput
from screen_capture import ScreenCapture
from posture_recognizer import PostureRecognizer

# Load environment variables
load_dotenv()

class ConversationBuffer:
    """
    Manages conversation context and handles AI analysis timing.
    Buffers transcripts and triggers AI analysis at appropriate intervals.
    """
    def __init__(self, goal: str = "", audience: str = "", user_speaker_id: str = "", audio_system=None):
        self.goal = goal
        self.audience = audience
        self.context = []
        self.current_phase = "greeting"
        self.last_analysis_time = time.time()  # Initialize with current time
        self.MIN_ANALYSIS_INTERVAL = 5.0  # Increase to 5 seconds
        self.ai_service = AIService(api_key=os.getenv('GROQ_API_KEY'))
        self.phase_progress = {
            "rapport": 0,
            "interest": 0,
            "value": 0,
            "ask": 0
        }
        self.last_speaker = None
        self.speaker_patterns = {
            "user": [],    # Store user's speech patterns
            "other": []    # Store other person's patterns
        }
        self.user_speaker_id = user_speaker_id  # Track who's wearing the glasses
        self.other_speaker_id = None
        self.ai_suggestions = []  # Track AI's previous suggestions
        self.audio_system = audio_system  # Store audio system reference
        self.audio_output = ElevenLabsOutput()
        self.current_thread = None
        self.thread_start_time = None
        self.thread_timeout = 10.0  # Seconds before a thread is considered complete
        self.response_in_progress = False
        self.conversation_state = {
            "stage": "initial",
            "rapport_level": 0.0,
            "resistance_signals": [],
            "positive_signals": [],
            "last_strategy": None,
            "successful_angles": [],
            "progress": 0,
            "completion_signals": 0,  # Track how many completion indicators we've seen
            "max_suggestions": 5      # Limit total suggestions
        }
        self.conversation_history = []
        self.conversation_phases = {
            "greeting": {
                "complete": False,
                "triggers": ["hi", "hey", "hello", "how's it going"],
                "min_exchanges": 2  # Need at least 2 back-and-forth
            },
            "rapport": {
                "complete": False,
                "triggers": ["weekend", "weather", "plans", "been up to"],
                "min_exchanges": 3
            },
            "interest": {
                "complete": False,
                "triggers": ["interesting", "tell me more", "what about you"],
                "min_exchanges": 2
            },
            "transition": {
                "complete": False,
                "triggers": ["speaking of", "by the way", "actually"],
                "min_exchanges": 1
            },
            "goal": {
                "complete": False,
                "triggers": [],  # Custom based on goal
                "min_exchanges": 2
            }
        }
        self.exchanges_in_phase = 0
        
        # Add screen capture initialization
        self.screen_capture = ScreenCapture()
        self.screen_capture_task = None

        # Initialize posture recognizer
        self.recognizer = PostureRecognizer()

    def add_partial(self, text: str, speaker: str):
        """
        Store partial transcripts without triggering analysis.
        
        Args:
            text (str): Partial transcript text
            speaker (str): Speaker identifier
        """
        if not text.strip():
            return
            
        self.context.append({
            "type": "partial",
            "speaker": speaker,
            "text": text,
            "timestamp": time.time()
        })

    def add_final(self, text: str, speaker: str):
        """
        Add final transcript and potentially trigger analysis.
        
        Args:
            text (str): Final transcript text
            speaker (str): Speaker identifier
        """
        if not text.strip():
            return
            
        # Initialize speaker IDs
        if not self.user_speaker_id:
            self.user_speaker_id = speaker
            print(f"\n[System] 🎯 You are {speaker}")
        
        elif speaker != self.user_speaker_id and not self.other_speaker_id:
            self.other_speaker_id = speaker
            print(f"\n[System] 🎤 Other person is {speaker}")
        
        # Append to context
        self.context.append({
            "type": "final",
            "speaker": speaker,
            "text": text,
            "timestamp": time.time(),
            "is_speaker_change": speaker != self.last_speaker
        })
        
        self.last_speaker = speaker
        
        # Create task to check intervention
        asyncio.create_task(self._check_and_analyze())

        print(f"Added final transcript: '{text}' from Speaker: {speaker}")

    async def _check_and_analyze(self):
        """Checks if we should intervene and analyzes if needed"""
        if self.response_in_progress:
            print("DEBUG: Skipping analysis - response in progress")
            return
        
        should_intervene = await self.should_intervene()
        print(f"DEBUG: Should intervene result: {should_intervene}")
        
        if should_intervene:
            self.response_in_progress = True
            print("DEBUG: AI is triggering analysis based on conversation state.")
            try:
                await self.analyze_conversation()
            finally:
                self.response_in_progress = False
                self.last_analysis_time = time.time()

    async def analyze_conversation(self):
        """Analyzes conversation and outputs audio response"""
        print(
[truncated — 10796 more characters]
```