# Project export: Obscurify

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: UC Berkeley AI Hackathon 2025
- Tagline: Automatically blur sensitive details when sharing your screen during meetings or calls.
- Devpost: https://devpost.com/software/obscurify
- GitHub: https://github.com/AlexTalreja/obscurify
- Video: https://www.youtube.com/embed/b2IIkba9XXU?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 3 GitHub contributor(s) — ChittebbayiPenugonda (29 commits), Selina (20 commits), AlexTalreja (4 commits)

## Devpost submission (written by the team)

### Inspiration

With the spread of online video calling and thus, screen-sharing, we noticed how often passwords, API keys, or even faces of other people flash on a call before the user can react. We wanted a security net for screen-sharing that would serve to help prevent those accidents automatically, thus alleviating the pressure of relying on human reflexes.

### What it does

Obscurify detects sensitive information on your screen and automatically blurs it when you want to screenshare. It works on both text and images, able to identify faces, passwords, names, addresses, and more. Obscurify can retrieve information from pre-defined lists of sensitive data and can allow the user to conditionally filter blurring for different categories of information with a simple checkbox panel. Fortunately, and with much difficultly, it runs fast enough for real-time conferencing.

### How we built it

Uses a Letta agent to retrieve personal information to hide from external curated block-word lists Uses tkinter for lightweight UI that allows users to choose additional items to censor Frames retrieved are split between two paths: PyTesseract OCR → regex match for private information Gemini Vision + OpenCV → face & semantic object detection Coordinates from both paths are merged and passed to an OpenCV blur renderer, which paints the boxes and publishes a proxy virtual screen/window that video calling apps like Zoom can pick up Adjustments for performance including image preprocessing, calculated delay, and threaded pipelines

### Challenges we ran into

Achieving a workflow that processed frames fast enough to still have usage in live video conferences. We initially set up a flow that used two separate models per frame to render annotation boxes for faces and text, which caused the project to run extremely slowly. However, by adding threadpooling/asynchronous OCR, image downscaling, and using a faster model in combination with regex, we were able to significantly improve the frame rate. Also, we added image preprocessing to address problems with OCR accuracy on variable-resolution screens and different UIs.

### Accomplishments we're proud of

We are happy that we got a demo that works quick enough for its use case, as it is difficult to use an LLM like Gemini for such a niche task and be efficient for real-time streaming. If we were to have more time to work on Obscurify, we would seek to target a broader scope of censored information (profanity, inappropriate imagery, etc), interactive toggled blurs (using image segmentation), and using a custom pre-trained model to be more suited for this purpose, and thus more efficient.

## README (from the GitHub repository)

<img src="obscurify_logo.png" style="width:30%;" alt="Obscurify logo">
<h1> Obscurify </h1>
<h3> Keeping your secrets ... secret</h3>
Obscurify detects sensitive information on your screen and automatically blurs it when you want to screenshare
<h2> What it does </h2>
<ul>
<li>Works on both text and images (faces, passwords, names, addresses, etc.).</li>
<li>Uses an agent (with Letta) to retrieve external personal information to hide.</li>
<li>Lets users customize additional items they’d like to hide with a simple checkbox panel.</li>
<li>Uses multiple paths like regex, gemini, and opencv to identify sensitive information.</li>
<li>Runs fast enough for real-time conferencing. </li>
</ul>
<h2> To try yourself:</h2>
run the python file obscurify_fast.py with your own GEMINI_API_KEY and LETTA_TOKEN


## Detected evidence (automated analysis)

Indexed codebase: 15 recognized source files, 152 KB.
- Python (language) — detected in the code
- Google Gemini (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (15 of 15)

```
alternates/gemini_cv.py
alternates/gemini_full_img.py
alternates/gemini_identify.py
alternates/gemini.py
alternates/letta_agent2.py
alternates/letta.py
alternates/new_lemma.py
alternates/obscurify_fast.py
alternates/og_improved.py
alternates/og.py
alternates/old.py
alternates/prev_letta.py
alternates/screen_mirror.py
obscurify_agent_regex.py
README.md
```

### Dependencies

No dependency index available.

### Recent commits (newest first)

- obscurify with regex main
- obscurify with regex main
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Rename obscurify_logo.png.png to obscurify_logo.png
- added logo
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- more cleaning
- cleaning
- letta support
- uses letta for direct password discovery
- frontend checkbox thingy

## Key source files (fetched from GitHub, selected and truncated for size)

### obscurify_agent_regex.py

```python
#!/usr/bin/env python3
"""
screen_stream_overlay_async.py  (v7.11 – detailed prompt, fixed filters)

Workflow
========
1. A Tkinter window appears first with 7 check-boxes:
     • Passwords / API keys
     • Emails
     • Phone numbers
     • Credit-card numbers
     • Addresses
     • Names
     • Faces
2. Click **Start** — the dashboard closes and capture begins.
3. Only checked categories are blurred. Unchecked categories are guaranteed
   to remain visible because the Gemini prompt forbids blurring them.
"""

from __future__ import annotations
import argparse, platform, subprocess, sys, time, re, os, json, ast
from concurrent.futures import ThreadPoolExecutor, Future
import tkinter as tk

import cv2, numpy as np
from mss import mss
import pytesseract, mediapipe as mp
from dotenv import load_dotenv
import google.generativeai as genai
import letta_agent2 

# ───────────── OCR & regex helpers ──────────────────────────────────────────
TESS_PATH   = r"C:\Program Files\Tesseract-OCR\tesseract.exe"
pytesseract.pytesseract.tesseract_cmd = TESS_PATH
TESS_CONFIG = "--oem 3 --psm 11 -l eng"

API_KEY_RE = re.compile(r"(AKIA|ASIA|SK|sk_live_)[A-Za-z0-9]{16,}", re.I)
PHRASE_RE  = re.compile(r"(password|secret|apikey|token)", re.I)
EMAIL_RE   = re.compile(r"[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]{2,}")
LETTAPW_LIST = getattr(letta_agent2, "parsed_list", [])          # list from letta_agent2.py
PW_LIST_RE   = (
    re.compile("|".join(re.escape(pw) for pw in LETTAPW_LIST), re.I)
    if LETTAPW_LIST else None
)

# ───────────── Gemini setup ────────────────────────────────────────────────
load_dotenv()
genai.configure(api_key=os.getenv("GEMINI_API_KEY"))
model = genai.GenerativeModel("gemini-1.5-flash-latest")

# ───────────── dashboard (blocking) ────────────────────────────────────────
CATS = {
    "Passwords / API keys": "pw",
    "Emails":               "email",
    "Phone numbers":        "phone",
    "Credit-card numbers":  "cc",
    "Addresses":            "addr",
    "Names":                "name",
    "Faces":                "face",
}

def choose_filters() -> set[str]:
    root = tk.Tk()
    root.title("Select categories to BLUR")
    vars_: dict[str, tk.BooleanVar] = {}
    tk.Label(root, text="Blur the checked categories:", font=("Arial", 12, "bold")).pack()
    for label in CATS:
        var = tk.BooleanVar(value=True)
        vars_[label] = var
        tk.Checkbutton(root, text=label, variable=var, anchor="w").pack(fill="x")
    selected: set[str] = set()
    def _start():
        nonlocal selected
        selected = {CATS[lbl] for lbl, v in vars_.items() if v.get()}
        root.destroy()
    tk.Button(root, text="Start", command=_start, width=10).pack(pady=8)
    root.mainloop()
    return selected

FILTER_CODES = choose_filters()
print("Blurring:", FILTER_CODES or "nothing (all text visible)")

# ───────────── prompt builder ───────────────────────────────────────────────
_BULLETS = {
    "pw":   "• Passwords, passphrases, API tokens, secrets",
    "email":"• E-mail addresses",
    "phone":"• Personal phone numbers (7–15 digits, any locale)",
    "cc":   "• Credit-card or bank numbers (≈12–19 digits, with/without dashes)",
    "addr": "• Street / mailing addresses (number + street + city/state/zip)",
    "name": "• Personal names **when paired with other private data** on the line",
}
_DESC = {
    "pw": "passwords/API keys", "email": "emails", "phone": "phone numbers",
    "cc": "credit-card numbers", "addr": "addresses", "name": "names"
}

def make_prompt(lines: list[str]) -> str:
    checked  = FILTER_CODES & _BULLETS.keys()
    unchecked = (set(_BULLETS) - checked)
    bullets = "\n".join(_BULLETS[c] for c in checked) or "• (none — blur no text categories)"
    unchecked_list = ", ".join(_DESC[c] for c in unchecked) or "none"

    return f"""You are a security assistant reading raw OCR lines from a live
screen.  The text may contain minor recognition errors (e.g. '8' instead of 'B').
Mentally correct obvious typos, but **only blur** a line if you are at least
≈80 % certain it contains sensitive data belonging to the *checked* categories
below.  

If a line matches an **unchecked** category (e.g. {unchecked_list}), you must
**leave it visible** — do not blur it.

Checked categories to BLUR:
{bullets}

Ignore generic tech terms like “login”, “user”, “id”, etc. unless strong evidence
shows an actual secret (digits, token patterns, etc.).

If no line needs blurring, return [].

Lines (index : text):
{chr(10).join(f"{i}: {t}" for i, t in enumerate(lines))}

Respond with JSON only, e.g. [1,3]  — no extra commentary."""

# ───────────── CLI / capture parameters ─────────────────────────────────────
ap = argparse.ArgumentParser()
ap.add_argument("-w","--window"); ap.add_argument("--rtmp")
ap.add_argument("--fps", type=int, default=30); ap.add_argument("--interval", type=int, default=1)
ap.add_argument("--hold", type=int, default=40); ap.add_argument("--pad", type=int, default=4)
ap.add_argument("--scale", type=float, default=1.5); ap.add_argument("--face-blur", type=int, default=75)
ap.add_argument("--color", default="255,0,0")
args = ap.parse_args()

box_color   = tuple(map(int,args.color.split(",")))
frame_delay = 1/args.fps
face_k      = max(3, args.face_blur | 1)
text_k      = 35
mp_face = mp.solutions.face_detection.FaceDetection(0,0.6) if "face" in FILTER_CODES else None

# ───────────── helpers ─────────────────────────────────────────────────────
def preprocess(gray):
    sharp = cv2.addWeighted(gray,1.5,cv2.GaussianBlur(gray,(0,0),1.2),-0.5,0)
    bw = cv2.adaptiveThreshold(sharp,255,cv2.ADAPTIVE_THRESH_MEAN_C,
                               cv2.THRESH_BINARY,35,11)
    return cv2.morphologyEx(bw,cv2.MORPH_CLOSE,
            cv2.getStructuringElement(cv2.MORPH_RECT,(2,2)),1)

def robust_json_extract(txt:str)->list[int]:
    try:
        out = ast.literal_eval(txt.strip())
        return [int(x) for x in out] if isinstance(out,list) else []
    except Exception:
        m =
[truncated — 5564 more characters]
```

### alternates/letta_agent2.py

```python
from letta_client import Letta, MessageCreate, TextContent
import json, os
from dotenv import load_dotenv

load_dotenv()

client = Letta(token=os.environ.get('LETTA_TOKEN2'))
agentID = "agent-5f15a39e-ae95-4383-ba6a-06a187bbf8e7"

# directory = "/Users/alextalreja/Desktop/Berkeley2025/berkeley25/corporate_files"

msg = MessageCreate(
    role="user",
    content=[TextContent(text=f"List all employee passwords from our company data. Return them as an array of comma separated strings. Do not include any other text, symbols, or visuals besides the array.")]
)

resp = client.agents.messages.create(
    agent_id="agent-5f15a39e-ae95-4383-ba6a-06a187bbf8e7",
    messages=[msg],
)

import re

# ---------------------------------------------------------------
# 1️⃣  Grab the raw text from the Letta response
# ---------------------------------------------------------------
latest_msg = resp.messages[-1]                 # the assistant’s reply
raw_text   = (
    latest_msg.content[0].text                 # usual case: first TextContent
    if isinstance(latest_msg.content, list)
    else latest_msg.content                    # fallback if .content is already str
)

# ---------------------------------------------------------------
# 2️⃣  Regex-parse every string between double quotes
#     (handles escaped quotes \" correctly)
# ---------------------------------------------------------------
pattern = re.compile(
    r'"((?:\\.|[^"\\])*)"'                     # match quoted string, ignore escapes
)
parsed_list = [re.sub(r'\\"', '"', s) for s in pattern.findall(raw_text)]

print("full parsed list: ", parsed_list)

# ---------------------------------------------------------------
# 3️⃣  Use the resulting list
# ---------------------------------------------------------------
print("Total parsed:", len(parsed_list))
if parsed_list:
    print("First element:", parsed_list[0])
    print("Last element :", parsed_list[-1])
else:
    print("No quoted strings found!")

# parsed_list is now a regular Python list you can work with

```

### alternates/gemini_cv.py

```python
import json

import cv2
import ultralytics
from google import genai
from google.genai import types
from PIL import Image
from ultralytics.utils.downloads import safe_download
from ultralytics.utils.plotting import Annotator, colors

ultralytics.checks()

# Initialize the Gemini client with your API key
client = genai.Client(api_key="AIzaSyDw4gALLTTQZvGargegcUVLp4TiBa-Cbns")


def inference(image, prompt, temp=0.5):
    """
    Performs inference using Google Gemini 2.5 Pro Experimental model.

    Args:
        image (str or genai.types.Blob): The image input, either as a base64-encoded string or Blob object.
        prompt (str): A text prompt to guide the model's response.
        temp (float, optional): Sampling temperature for response randomness. Default is 0.5.

    Returns:
        str: The text response generated by the Gemini model based on the prompt and image.
    """
    response = client.models.generate_content(
        model="gemini-2.5-pro-exp-03-25",
        contents=[prompt, image],  # Provide both the text prompt and image as input
        config=types.GenerateContentConfig(
            temperature=temp,  # Controls creativity vs. determinism in output
        ),
    )

    return response.text  # Return the generated textual response

def read_image(filename=None):
    if filename is not None:
        image_name = filename
    else:
        image_name = "bus.jpg"  # or "zidane.jpg"

    # Download the image
    safe_download(f"https://github.com/ultralytics/notebooks/releases/download/v0.0.0/{image_name}")

    # Read image with opencv
    image = cv2.cvtColor(cv2.imread(f"/content/{image_name}"), cv2.COLOR_BGR2RGB)

    # Extract width and height
    h, w = image.shape[:2]

    # # Read the image using OpenCV and convert it into the PIL format
    return Image.fromarray(image), w, h

def clean_results(results):
    """Clean the results for visualization."""
    return results.strip().removeprefix("```json").removesuffix("```").strip()


# OBJECT DETECTION
# Define the text prompt
prompt = """
Detect the 2d bounding boxes of objects in image.
"""

# Fixed, plotting function depends on this.
output_prompt = "Return just box_2d and labels, no additional text."

image, w, h = read_image("gemini-image1.jpg")  # Read img, extract width, height

results = inference(image, prompt + output_prompt)  # Perform inference

cln_results = json.loads(clean_results(results))  # Clean results, list convert

annotator = Annotator(image)  # initialize Ultralytics annotator

for idx, item in enumerate(cln_results):
    # By default, gemini model return output with y coordinates first.
    # Scale normalized box coordinates (0–1000) to image dimensions
    y1, x1, y2, x2 = item["box_2d"]  # bbox post processing,
    y1 = y1 / 1000 * h
    x1 = x1 / 1000 * w
    y2 = y2 / 1000 * h
    x2 = x2 / 1000 * w

    if x1 > x2:
        x1, x2 = x2, x1  # Swap x-coordinates if needed
    if y1 > y2:
        y1, y2 = y2, y1  # Swap y-coordinates if needed

    annotator.box_label([x1, y1, x2, y2], label=item["label"], color=colors(idx, True))

Image.fromarray(annotator.result())  # display the output

# REASONING
# Define the text prompt
prompt = """
Detect the 2d bounding box around:
highlight the area of morning light +
notebook on PC table
potted plant near mirror.
"""

# Fixed, plotting function depends on this.
output_prompt = "Return just box_2d and labels, no additional text."

image, w, h = read_image("gemini-image2.jpg")  # Read image and extract width, height

results = inference(image, prompt + output_prompt)

# Clean the results and load results in list format
cln_results = json.loads(clean_results(results))

annotator = Annotator(image)  # initialize Ultralytics annotator

for idx, item in enumerate(cln_results):
    # By default, gemini model return output with y coordinates first.
    # Scale normalized box coordinates (0–1000) to image dimensions
    y1, x1, y2, x2 = item["box_2d"]  # bbox post processing,
    y1 = y1 / 1000 * h
    x1 = x1 / 1000 * w
    y2 = y2 / 1000 * h
    x2 = x2 / 1000 * w

    if x1 > x2:
        x1, x2 = x2, x1  # Swap x-coordinates if needed
    if y1 > y2:
        y1, y2 = y2, y1  # Swap y-coordinates if needed

    annotator.box_label([x1, y1, x2, y2], label=item["label"], color=colors(idx, True))

Image.fromarray(annotator.result())  # display the output


# OCR
# Define the text prompt
prompt = """
Extract the text from the image
"""

# Fixed, plotting function depends on this.
output_prompt = """
Return just box_2d which will be location of detected text areas + label"""

image, w, h = read_image("gemini-image3.png")  # Read image and extract width, height

results = inference(image, prompt + output_prompt)

# Clean the results and load results in list format
cln_results = json.loads(clean_results(results))

annotator = Annotator(image)  # initialize Ultralytics annotator

for idx, item in enumerate(cln_results):
    # By default, gemini model return output with y coordinates first.
    # Scale normalized box coordinates (0–1000) to image dimensions
    y1, x1, y2, x2 = item["box_2d"]  # bbox post processing,
    y1 = y1 / 1000 * h
    x1 = x1 / 1000 * w
    y2 = y2 / 1000 * h
    x2 = x2 / 1000 * w

    if x1 > x2:
        x1, x2 = x2, x1  # Swap x-coordinates if needed
    if y1 > y2:
        y1, y2 = y2, y1  # Swap y-coordinates if needed

    annotator.box_label([x1, y1, x2, y2], label=item["label"], color=colors(idx, True))

Image.fromarray(annotator.result())  # display the output

```

### alternates/old.py

```python
#!/usr/bin/env python3
"""
screen_stream_overlay.py  (v2 – with automatic redaction)

* Captures the desktop or a single window.
* Detects human faces + sensitive text (API keys, “password”, etc.).
* Blurs those regions in-line, then previews locally or streams to RTMP.
"""

import argparse, platform, subprocess, sys, time, re
from pathlib import Path

import cv2, numpy as np
from mss import mss
import pytesseract
import mediapipe as mp
# Path to the Tesseract executable (adjust if yours lives elsewhere)
pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"

# ------------------------------------------------------------------ #
# 1. command-line arguments
# ------------------------------------------------------------------ #
parser = argparse.ArgumentParser(description="Screen capture + auto-redact")
parser.add_argument("-w", "--window", help="Exact window title (Windows only)")
parser.add_argument("--rtmp", help="RTMP URL (omit => local preview)")
parser.add_argument("--fps", type=int, default=30, help="Capture frame-rate")
parser.add_argument("--interval", type=int, default=3,
                    help="Run text OCR every N frames (perf tweak)")
parser.add_argument("--box", type=int, default=50, help="Demo overlay square")
parser.add_argument("--color", default="255,0,0", help="Overlay BGR color")
args = parser.parse_args()
box_color = tuple(map(int, args.color.split(",")))
frame_delay = 1.0 / args.fps

# ------------------------------------------------------------------ #
# 2. detectors
# ------------------------------------------------------------------ #
mp_face = mp.solutions.face_detection.FaceDetection(
    model_selection=0, min_detection_confidence=0.4)

API_KEY_RE = re.compile(r"(AKIA|ASIA|SK|sk_live_)[A-Za-z0-9]{16,}")
PHRASE_RE  = re.compile(r"(password|secret|apikey|token)", re.I)

def find_sensitive_text(gray_img) -> list[tuple[int, int, int, int]]:
    """Return [x1,y1,x2,y2] boxes whose text matches the regexes."""
    boxes = []
    data = pytesseract.image_to_data(gray_img, output_type=pytesseract.Output.DICT)
    for i, txt in enumerate(data["text"]):
        if not txt or len(txt) < 4:
            continue
        if API_KEY_RE.search(txt) or PHRASE_RE.search(txt):
            x, y, w, h = (data[k][i] for k in ("left", "top", "width", "height"))
            boxes.append((x, y, x + w, y + h))
    return boxes

def blur_region(img, x1, y1, x2, y2, k=35):
    sub = img[y1:y2, x1:x2]
    if sub.size:
        img[y1:y2, x1:x2] = cv2.GaussianBlur(sub, (k, k), 0)

# ------------------------------------------------------------------ #
# 3. pick capture region
# ------------------------------------------------------------------ #
def pick_monitor(sct: mss, title: str | None):
    if title and platform.system() == "Windows":
        import win32gui
        hwnd = win32gui.FindWindow(None, title)
        if not hwnd:
            sys.exit(f'Window "{title}" not found.')
        l, t, r, b = win32gui.GetWindowRect(hwnd)
        return {"left": l, "top": t, "width": r - l, "height": b - t}
    elif title:
        sys.exit("Window capture by title is Windows-only.")
    return sct.monitors[0]

# ------------------------------------------------------------------ #
# 4. main loop
# ------------------------------------------------------------------ #
with mss() as sct:
    monitor = pick_monitor(sct, args.window)
    W, H = monitor["width"], monitor["height"]

    # optional ffmpeg pipe
    ffmpeg = None
    if args.rtmp:
        ffmpeg = subprocess.Popen([
            "ffmpeg", "-loglevel", "error", "-y",
            "-f", "rawvideo", "-pix_fmt", "bgr24", "-s", f"{W}x{H}",
            "-r", str(args.fps), "-i", "-", "-c:v", "libx264",
            "-preset", "veryfast", "-pix_fmt", "yuv420p",
            "-f", "flv", args.rtmp
        ], stdin=subprocess.PIPE)

    frame_i = 0
    cached_text_boxes, cached_face_boxes = [], []

    try:
        while True:
            t0 = time.time()

            # capture
            frame = sct.grab(monitor)
            img = np.ascontiguousarray(np.array(frame)[:, :, :3])

            # -------------------------------------------------- #
            # 4A. face detection every frame
            results = mp_face.process(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
            cached_face_boxes = []
            if results.detections:
                for det in results.detections:
                    bb = det.location_data.relative_bounding_box
                    x1 = int(bb.xmin * W); y1 = int(bb.ymin * H)
                    x2 = int((bb.xmin + bb.width) * W)
                    y2 = int((bb.ymin + bb.height) * H)
                    cached_face_boxes.append((x1, y1, x2, y2))

            # 4B. OCR every N frames (tweak with --interval)
            if frame_i % args.interval == 0:
                gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
                cached_text_boxes = find_sensitive_text(gray)

            # 4C. apply blur
            for x1, y1, x2, y2 in cached_face_boxes + cached_text_boxes:
                blur_region(img, x1, y1, x2, y2)

            # demo overlay square (unchanged from original)
            cv2.rectangle(img, (0, 0), (args.box, args.box), box_color, -1)

            # output
            if ffmpeg:
                ffmpeg.stdin.write(img.tobytes())
            else:
                cv2.imshow("Preview – press Q to quit", img)
                if cv2.waitKey(1) & 0xFF in (ord('q'), ord('Q')):
                    break

            # FPS limiter
            dt = time.time() - t0
            if dt < frame_delay:
                time.sleep(frame_delay - dt)
            frame_i += 1

    except KeyboardInterrupt:
        pass
    finally:
        if ffmpeg:
            ffmpeg.stdin.close(); ffmpeg.wait()
        cv2.destroyAllWindows()

```

### alternates/screen_mirror.py

```python
#!/usr/bin/env python3
"""
screen_stream_overlay.py  (v5.2 – strong face-blur)

* Captures the desktop or a single window.
* Detects human faces & sensitive text.
* Blurs:
    • Faces  with a large kernel  (--face-blur, default 75)
    • Text   with a modest kernel (35)
* Holds every blur for --hold frames after last detection (anti-flicker).
"""

import argparse, platform, subprocess, sys, time, re
from pathlib import Path

import cv2, numpy as np
from mss import mss
import pytesseract
import mediapipe as mp
import openai
import json
import os
from dotenv import load_dotenv

# Path to the Tesseract executable (adjust if yours lives elsewhere)
pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"

load_dotenv() 
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") 
# ────────────────────────────────────────────────────────────────
# 1. command-line arguments
# ────────────────────────────────────────────────────────────────
parser = argparse.ArgumentParser(description="Screen capture + auto-redact")
parser.add_argument("-w", "--window", help="Exact window title (Windows only)")
parser.add_argument("--rtmp", help="RTMP URL (omit => local preview)")
parser.add_argument("--fps", type=int, default=30, help="Capture frame-rate")
parser.add_argument("--interval", type=int, default=1,
                    help="OCR every N frames (keep at 1 for zero-leak)")
parser.add_argument("--hold", type=int, default=15,
                    help="Frames to KEEP a box after last seen (anti-flicker)")
parser.add_argument("--pad", type=int, default=4,
                    help="Extra pixel padding around every box")
parser.add_argument("--face-blur", type=int, default=75,
                    help="Gaussian-kernel size for faces (odd; higher = stronger)")
parser.add_argument("--box", type=int, default=50, help="Demo overlay square")
parser.add_argument("--color", default="255,0,0", help="Overlay BGR color")
args = parser.parse_args()

box_color   = tuple(map(int, args.color.split(",")))
frame_delay = 1.0 / args.fps
face_k      = 100       
text_k      = 35                               # legacy value

# ────────────────────────────────────────────────────────────────
# 2. detectors
# ────────────────────────────────────────────────────────────────
mp_face = mp.solutions.face_detection.FaceDetection(
    model_selection=0, min_detection_confidence=0.4)

API_KEY_RE = re.compile(r"(AKIA|ASIA|SK|sk_live_)[A-Za-z0-9]{16,}")
PHRASE_RE  = re.compile(r"(password|secret|apikey|token)", re.I)

pytesseract.pytesseract.tesseract_cmd = (
    r"C:\Program Files\Tesseract-OCR\tesseract.exe"  # adjust if needed
)

# OPEN AI STUFF

openai.api_key = OPENAI_API_KEY
GPT_MODEL = "gpt-4o-mini"          # fast & cheap enough for frame-level use
GPT_FUNC = {
    "name": "mark_sensitive",
    "description": "Return indexes of sensitive strings.",
    "parameters": {
        "type": "object",
        "properties": {
            "indexes": {
                "type": "array",
                "items": {"type": "integer"},
                "description": "0-based indexes that MUST be redacted."
            }
        },
        "required": ["indexes"],
    },
}
def find_sensitive_text(gray_img) -> list[tuple[int,int,int,int]]:
    """
    1. OCR the frame exactly as before (pytesseract).
    2. Ask GPT which snippets are sensitive.
    3. Return their bounding boxes.
    """
    data = pytesseract.image_to_data(
        gray_img, output_type=pytesseract.Output.DICT
    )

    # Collect candidate strings ≥ 4 chars to keep the prompt short
    texts, idx_map = [], []
    for i, txt in enumerate(data["text"]):
        if txt and len(txt) >= 4:
            idx_map.append(i)     # map from compact index -> tesseract index
            texts.append(txt)

    if not texts:
        return []

    # --------  Call GPT  ----------------------------------------------------
    prompt = (
        "You are a security assistant looking at raw OCR output.\n"
        "For any string that is *definitely* sensitive (API keys, "
        "passwords, tokens, credit-card numbers, SSNs, private addresses, "
        "personal phone numbers, etc.) you must return its index. For testing purposes now, treat the literal word password as sensitive\n\n"
        "### Strings (index : text)\n" +
        "\n".join(f"{i}: {t}" for i, t in enumerate(texts)) + "\n\n"
        "Respond *only* with JSON that matches the function schema."
    )
    try:
        rsp = openai.chat.completions.create(
            model=GPT_MODEL,
            messages=[{"role": "system", "content": "You are a helpful assistant."},
                      {"role": "user", "content": prompt}],
            functions=[GPT_FUNC],
            function_call={"name": "mark_sensitive"},
            temperature=0.0,
        )
        sensitive = json.loads(rsp.choices[0].message.function_call.arguments)["indexes"]
        print(sensitive)
    except Exception as e:
        print("OpenAI failure, falling back to old regex:", e)
        sensitive = []   # (or call your old regex here as a backup)

    # --------  Convert indexes → bounding boxes  ----------------------------
    boxes = []
    for compact_idx in sensitive:
        i = idx_map[compact_idx]
        x, y, w, h = (data[k][i] for k in ("left", "top", "width", "height"))
        boxes.append((x, y, x + w, y + h))
    return boxes


# ────────────────────────────────────────────────────────────────
# helpers
# ────────────────────────────────────────────────────────────────
def expand_box(b, pad, W, H):
    x1, y1, x2, y2 = b
    return (max(0, x1 - pad), max(0, y1 - pad),
            min(W, x2 + pad), min(H, y2 + pad))

def blur_region(img, x1, y1, x2, y2, k):
    sub = img[y1:y2, x1:x2]
    if sub.size:
        k = max(3, k | 1)  # odd, ≥3
        img[y1:y2, x1:x2] = cv2.GaussianBlur(sub, (k, k), 0)

def iou(b1, b2):
    x1 = max(b1[0], b2[0]); y1 = max(b1[1], b2[1])
    x2 = min(b1[2], b2[2]); y2 = min(b1[3], b2[3])
    inter = max
[truncated — 3820 more characters]
```

### alternates/gemini_identify.py

```python
#!/usr/bin/env python3
"""
screen_stream_overlay.py  (v5.2 – strong face-blur)

* Captures the desktop or a single window.
* Detects human faces & sensitive text.
* Blurs:
    • Faces  with a large kernel  (--face-blur, default 75)
    • Text   with a modest kernel (35)
* Holds every blur for --hold frames after last detection (anti-flicker).
"""

import argparse, platform, subprocess, sys, time, re
from pathlib import Path

import cv2, numpy as np
from mss import mss
import pytesseract
import mediapipe as mp
import openai
import json
import os
from dotenv import load_dotenv
from google import genai

# Path to the Tesseract executable (adjust if yours lives elsewhere)
pytesseract.pytesseract.tesseract_cmd = r"C:\Program Files\Tesseract-OCR\tesseract.exe"
client = genai.Client(api_key="")

load_dotenv() 
# ────────────────────────────────────────────────────────────────
# 1. command-line arguments
# ────────────────────────────────────────────────────────────────
parser = argparse.ArgumentParser(description="Screen capture + auto-redact")
parser.add_argument("-w", "--window", help="Exact window title (Windows only)")
parser.add_argument("--rtmp", help="RTMP URL (omit => local preview)")
parser.add_argument("--fps", type=int, default=30, help="Capture frame-rate")
parser.add_argument("--interval", type=int, default=1,
                    help="OCR every N frames (keep at 1 for zero-leak)")
parser.add_argument("--hold", type=int, default=15,
                    help="Frames to KEEP a box after last seen (anti-flicker)")
parser.add_argument("--pad", type=int, default=4,
                    help="Extra pixel padding around every box")
parser.add_argument("--face-blur", type=int, default=75,
                    help="Gaussian-kernel size for faces (odd; higher = stronger)")
parser.add_argument("--box", type=int, default=50, help="Demo overlay square")
parser.add_argument("--color", default="255,0,0", help="Overlay BGR color")
args = parser.parse_args()

box_color   = tuple(map(int, args.color.split(",")))
frame_delay = 1.0 / args.fps
face_k      = 100       
text_k      = 35                               # legacy value

# ────────────────────────────────────────────────────────────────
# 2. detectors
# ────────────────────────────────────────────────────────────────
mp_face = mp.solutions.face_detection.FaceDetection(
    model_selection=0, min_detection_confidence=0.4)

API_KEY_RE = re.compile(r"(AKIA|ASIA|SK|sk_live_)[A-Za-z0-9]{16,}")
PHRASE_RE  = re.compile(r"(password|secret|apikey|token)", re.I)

pytesseract.pytesseract.tesseract_cmd = (
    r"C:\Program Files\Tesseract-OCR\tesseract.exe"  # adjust if needed
)

# OPEN AI STUFF

openai.api_key = os.getenv("OPENAI_API_KEY")
GPT_MODEL = "gpt-4o-mini"          # fast & cheap enough for frame-level use
GPT_FUNC = {
    "name": "mark_sensitive",
    "description": "Return indexes of sensitive strings.",
    "parameters": {
        "type": "object",
        "properties": {
            "indexes": {
                "type": "array",
                "items": {"type": "integer"},
                "description": "0-based indexes that MUST be redacted."
            }
        },
        "required": ["indexes"],
    },
}
def find_sensitive_text(gray_img) -> list[tuple[int,int,int,int]]:
    """
    1. OCR the frame exactly as before (pytesseract).
    2. Ask GPT which snippets are sensitive.
    3. Return their bounding boxes.
    """
    data = pytesseract.image_to_data(
        gray_img, output_type=pytesseract.Output.DICT
    )

    # Collect candidate strings ≥ 4 chars to keep the prompt short
    texts, idx_map = [], []
    for i, txt in enumerate(data["text"]):
        if txt and len(txt) >= 4:
            idx_map.append(i)     # map from compact index -> tesseract index
            texts.append(txt)

    if not texts:
        return []

    # --------  Call GPT  ----------------------------------------------------
    prompt = (
        "You are a security assistant looking at raw OCR output.\n"
        "For any string that is *definitely* sensitive (API keys, "
        "passwords, tokens, credit-card numbers, SSNs, private addresses, "
        "personal phone numbers, etc.) you must return its index. For testing purposes now, treat the literal word password as sensitive\n\n"
        "### Strings (index : text)\n" +
        "\n".join(f"{i}: {t}" for i, t in enumerate(texts)) + "\n\n"
        "Respond *only* with JSON that matches the function schema."
    )
    try:
        # rsp = openai.chat.completions.create(
        #     model=GPT_MODEL,
        #     messages=[{"role": "system", "content": "You are a helpful assistant."},
        #               {"role": "user", "content": prompt}],
        #     functions=[GPT_FUNC],
        #     function_call={"name": "mark_sensitive"},
        #     temperature=0.0,
        # )
        # sensitive = json.loads(rsp.choices[0].message.function_call.arguments)["indexes"]
        # print(sensitive)
        response = client.models.generate_content(
            model="gemini-2.5-flash",
            contents=prompt,
        )
        sensitive = json.loads(response.text)
        print(sensitive)
    except Exception as e:
        print("OpenAI failure, falling back to old regex:", e)
        sensitive = []   # (or call your old regex here as a backup)

    # --------  Convert indexes → bounding boxes  ----------------------------
    boxes = []
    for compact_idx in sensitive:
        i = idx_map[compact_idx]
        x, y, w, h = (data[k][i] for k in ("left", "top", "width", "height"))
        boxes.append((x, y, x + w, y + h))
    return boxes


# ────────────────────────────────────────────────────────────────
# helpers
# ────────────────────────────────────────────────────────────────
def expand_box(b, pad, W, H):
    x1, y1, x2, y2 = b
    return (max(0, x1 - pad), max(0, y1 - pad),
            min(W, x2 + pad), min(H, y2 + pad))

def blur_region(img, x1, y1, x2, y2, k):
    sub = img[y1:y2, x1:x2]
    i
[truncated — 4065 more characters]
```

[8 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]