# Project export: Iris - Eyes for the Blind

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: UC Berkeley AI Hackathon 2025
- Tagline: Imagine walking into the room, unaware of your surroundings, with a feeling of anxiety and mystery. Not anymore; introducing Iris, the new eyes for the Blind.
- Devpost: https://devpost.com/software/iris-hardware-hack
- GitHub: https://github.com/the-akze/berkAI-python-cv-etc
- Demo: https://github.com/the-akze/berkAI-hardware-arduino
- Video: https://www.youtube.com/embed/hzFLwks1W-c?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) — Akze (5 commits)

## Devpost submission (written by the team)

No Devpost description available.

## README (from the GitHub repository)

see repository of hardware (Arduino) portion of project:
[https://github.com/the-akze/berkAI-hardware-arduino](https://github.com/the-akze/berkAI-hardware-arduino)

## Detected evidence (automated analysis)

Indexed codebase: 5 recognized source files, 5 KB.
- Flask (technology) — detected in the code
- Python (language) — detected in the code
- PyTorch (technology) — detected in the code
- Google Gemini (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (8 of 8)

```
.gitignore
cam_script.py
google_ai_interface.py
main.py
README.md
requirements.txt
systemprompt.txt
tts.py
```

### Dependencies

- requirements.txt: absl-py@==1.4.0, aiohappyeyeballs@==2.6.1, aiohttp@==3.12.13, aiosignal@==1.3.2, altgraph@==0.17.3, annotated-types@==0.7.0, anyio@==4.9.0, appnope@==0.1.3, argon2-cffi@==23.1.0, argon2-cffi-bindings@==21.2.0, arrow@==1.3.0, asttokens@==2.4.1, astunparse@==1.6.3, async-lru@==2.0.4, async-timeout@==4.0.3, attrs@==23.2.0, Authlib@==1.3.0, autopep8@==2.0.2, av@==13.1.0, Babel@==2.14.0, beautifulsoup4@==4.12.3, bidict@==0.23.1, bleach@==6.1.0, bleak@==0.22.3, bpy@==4.0.0, Brotli@==1.1.0, CacheControl@==0.12.11, cachetools@==5.3.0, certifi@==2025.1.31, cffi@==1.15.1, cfgv@==3.4.0, chardet@==5.2.0, charset-normalizer@==3.1.0, click@==8.1.3, cloup@==3.0.5, comm@==0.2.1, contourpy@==1.0.7, cryptography@==40.0.2, cssselect2@==0.7.0, cvzone@==1.5.6, cycler@==0.11.0, Cython@==3.0.11, debugpy@==1.8.0, decorator@==5.1.1, defusedxml@==0.7.1, diagrams@==0.24.4, diffusers@==0.31.0, distlib@==0.3.9, dnspython@==2.4.2, einops@==0.8.0, exceptiongroup@==1.2.0, executing@==2.0.1, fastjsonschema@==2.19.1, ffmpeg@==1.4, ffmpeg-python@==0.2.0, filelock@==3.16.1, firebase-admin@==6.1.0, Flask@==2.2.3, Flask-PyMongo@==2.3.0, Flask-SocketIO@==5.5.1, flatbuffers@==23.3.3, fonttools@==4.38.0, fqdn@==1.5.1, frozenlist@==1.7.0, fsspec@==2024.10.0, future@==0.18.3, gast@==0.4.0, glcontext@==3.0.0, google-api-core@==2.11.0, google-api-python-client@==2.86.0, google-auth@==2.17.3, google-auth-httplib2@==0.1.0, google-auth-oauthlib@==1.0.0, google-cloud-core@==2.3.2, google-cloud-firestore@==2.11.0, google-cloud-storage@==2.8.0, google-crc32c@==1.5.0, google-genai@==1.21.1, google-pasta@==0.2.0, google-resumable-media@==2.4.1, googleapis-common-protos@==1.59.0, graphviz@==0.20.3, grpcio@==1.54.0, grpcio-status@==1.54.0, h11@==0.16.0, h5py@==3.8.0, httpcore@==1.0.9, httplib2@==0.22.0, httpx@==0.28.1, huggingface-hub@==0.26.2, identify@==2.6.10, idna@==3.4, importlib_metadata@==8.5.0, inquirerpy@==0.3.4, ipykernel@==6.29.0, ipython@==8.20.0, iso8601@==1.1.0, isoduration@==20.11.0, isosurfaces@==0.1.2, itsdangerous@==2.1.2, jax@==0.4.8, jedi@==0.19.1, Jinja2@==3.1.2, json5@==0.9.14, jsonpointer@==2.4, jsonschema@==4.21.1, jsonschema-specifications@==2023.12.1, jupyter_client@==8.6.0, jupyter_core@==5.7.1, jupyter_server@==2.12.5, jupyter_server_terminals@==0.5.2, jupyter-events@==0.9.0, jupyter-lsp@==2.2.2, jupyterlab@==4.0.12, jupyterlab_pygments@==0.3.0, jupyterlab_server@==2.25.2, keras@==2.12.0, kiwisolver@==1.4.4, libclang@==16.0.0, lmnt@==1.2.0, lxml@==5.3.0, macholib@==1.16.2, manim@==0.18.1, ManimPango@==0.6.0, mapbox_earcut@==1.0.2, Markdown@==3.4.3, markdown-it-py@==3.0.0, MarkupSafe@==2.1.2, matplotlib@==3.6.3, matplotlib-inline@==0.1.6, mdurl@==0.1.2, mediapipe@==0.9.3.0, mediapipe-silicon@==0.9.2.1, mido@==1.3.2, mistune@==3.0.2, ml-dtypes@==0.1.0, moderngl@==5.12.0, moderngl-window@==2.4.6, MouseInfo@==0.1.3, mpmath@==1.3.0, msgpack@==1.0.5, multidict@==6.5.0, multipledispatch@==1.0.0, mutagen@==1.47.0, nbclient@==0.9.0, nbconvert@==7.14.2, nbformat@==5.9.2, nest-asyncio@==1.6.0, networkx@==3.4.2, nodeenv@==1.9.1, notebook_shim@==0.2.3, numpy@==1.26.4, numpy-stl@==3.2.0, oauthlib@==3.2.2, opencv-contrib-python@==4.7.0.68, opencv-python@==4.11.0.86, opt-einsum@==3.3.0, outcome@==1.3.0.post0, overrides@==7.7.0, packaging@==23.1, pandas@==2.2.3, pandocfilters@==1.5.1, parso@==0.8.3, pexpect@==4.9.0, pfzy@==0.3.4, pillow@==11.0.0, platformdirs@==4.2.0, pre_commit@==4.2.0, prometheus-client@==0.19.0, prompt-toolkit@==3.0.43, propcache@==0.3.2, proto-plus@==1.22.2, protobuf@==3.20.3, psutil@==5.9.8, ptyprocess@==0.7.0, pure-eval@==0.2.2, pyasn1@==0.5.0, pyasn1-modules@==0.3.0, PyAutoGUI@==0.9.53, pycairo@==1.27.0, pycodestyle@==2.10.0, pycparser@==2.21, pycryptodomex@==3.20.0, pydantic@==2.11.7, pydantic_core@==2.33.2, pydub@==0.25.1, pygame@==2.3.0, PyGetWindow@==0.0.9, pyglet@==2.0.18, Pygments@==2.17.2, pyinstaller@==5.10.1, pyinstaller-hooks-contrib@==2023.2, PyJWT@==2.6.0, pymongo@==4.6.1, PyMsgBox@==1.0.9, pynput@==1.7.6, pyobjc@==9.1.1, pyobjc-core@==10.3.2, pyobjc-framework-Accessibility@==9.1.1, pyobjc-framework-Accounts@==9.1.1, pyobjc-framework-AddressBook@==9.1.1, pyobjc-framework-AdServices@==9.1.1, pyobjc-framework-AdSupport@==9.1.1, pyobjc-framework-AppleScriptKit@==9.1.1, pyobjc-framework-AppleScriptObjC@==9.1.1, pyobjc-framework-ApplicationServices@==9.1.1, pyobjc-framework-AppTrackingTransparency@==9.1.1, pyobjc-framework-AudioVideoBridging@==9.1.1, pyobjc-framework-AuthenticationServices@==9.1.1, pyobjc-framework-AutomaticAssessmentConfiguration@==9.1.1, pyobjc-framework-Automator@==9.1.1, pyobjc-framework-AVFoundation@==9.1.1, pyobjc-framework-AVKit@==9.1.1, pyobjc-framework-AVRouting@==9.1.1, pyobjc-framework-BackgroundAssets@==9.1.1, pyobjc-framework-BusinessChat@==9.1.1, pyobjc-framework-CalendarStore@==9.1.1, pyobjc-framework-CallKit@==9.1.1, pyobjc-framework-CFNetwork@==9.1.1, pyobjc-framework-ClassKit@==9.1.1, pyobjc-framework-CloudKit@==9.1.1, pyobjc-framework-Cocoa@==10.3.2, pyobjc-framework-Collaboration@==9.1.1, pyobjc-framework-ColorSync@==9.1.1, pyobjc-framework-Contacts@==9.1.1, pyobjc-framework-ContactsUI@==9.1.1, pyobjc-framework-CoreAudio@==9.1.1, pyobjc-framework-CoreAudioKit@==9.1.1, pyobjc-framework-CoreBluetooth@==10.3.2, pyobjc-framework-CoreData@==9.1.1, pyobjc-framework-CoreHaptics@==9.1.1, pyobjc-framework-CoreLocation@==9.1.1, pyobjc-framework-CoreMedia@==9.1.1, pyobjc-framework-CoreMediaIO@==9.1.1, pyobjc-framework-CoreMIDI@==9.1.1, pyobjc-framework-CoreML@==9.1.1, pyobjc-framework-CoreMotion@==9.1.1, pyobjc-framework-CoreServices@==9.1.1, pyobjc-framework-CoreSpotlight@==9.1.1, pyobjc-framework-CoreText@==9.1.1, pyobjc-framework-CoreWLAN@==9.1.1, pyobjc-framework-CryptoTokenKit@==9.1.1, pyobjc-framework-DataDetection@==9.1.1, pyobjc-framework-DeviceCheck@==9.1.1, pyobjc-framework-DictionaryServices@==9.1.1, pyobjc-framework-DiscRecording@==9.1.1, pyobjc-framework-DiscRecordingUI@==9.1.1, pyobjc-framework-DiskArbitration@==9.1.1, pyobjc-framework-DVDPlayback@==9.1.1, pyobjc-framework-EventKit@==9.1.1, pyobjc-framework-ExceptionHandling@==9.1.1, pyobjc-framework-ExecutionPolicy@==9.1.1, pyobjc-framework-ExtensionKit@==9.1.1, pyobjc-framework-ExternalAccessory@==9.1.1, pyobjc-framework-FileProvider@==9.1.1, pyobjc-framework-FileProviderUI@==9.1.1, pyobjc-framework-FinderSync@==9.1.1, pyobjc-framework-FSEvents@==9.1.1, pyobjc-framework-GameCenter@==9.1.1, pyobjc-framework-GameController@==9.1.1, pyobjc-framework-GameKit@==9.1.1, pyobjc-framework-GameplayKit@==9.1.1, pyobjc-framework-HealthKit@==9.1.1, pyobjc-framework-ImageCaptureCore@==9.1.1, pyobjc-framework-IMServicePlugIn@==9.1.1, pyobjc-framework-InputMethodKit@==9.1.1, pyobjc-framework-InstallerPlugins@==9.1.1, pyobjc-framework-InstantMessage@==9.1.1, pyobjc-framework-Intents@==9.1.1, pyobjc-framework-IntentsUI@==9.1.1, pyobjc-framework-IOBluetooth@==9.1.1, pyobjc-framework-IOBluetoothUI@==9.1.1, pyobjc-framework-IOSurface@==9.1.1, pyobjc-framework-iTunesLibrary@==9.1.1, pyobjc-framework-KernelManagement@==9.1.1, pyobjc-framework-LatentSemanticMapping@==9.1.1, pyobjc-framework-LaunchServices@==9.1.1, pyobjc-framework-libdispatch@==10.3.2, pyobjc-framework-libxpc@==9.1.1, pyobjc-framework-LinkPresentation@==9.1.1, pyobjc-framework-LocalAuthentication@==9.1.1, pyobjc-framework-LocalAuthenticationEmbeddedUI@==9.1.1, pyobjc-framework-MailKit@==9.1.1, pyobjc-framework-MapKit@==9.1.1, pyobjc-framework-MediaAccessibility@==9.1.1, pyobjc-framework-MediaLibrary@==9.1.1, pyobjc-framework-MediaPlayer@==9.1.1, pyobjc-framework-MediaToolbox@==9.1.1, pyobjc-framework-Metal@==9.1.1, pyobjc-framework-MetalFX@==9.1.1, pyobjc-framework-MetalKit@==9.1.1, pyobjc-framework-MetalPerformanceShaders@==9.1.1, pyobjc-framework-MetalPerformanceShadersGraph@==9.1.1, pyobjc-framework-MetricKit@==9.1.1, pyobjc-framework-MLCompute@==9.1.1, pyobjc-framework-ModelIO@==9.1.1, pyobjc-framework-MultipeerConnectivity@==9.1.1, pyobjc-framework-NaturalLanguage@==9.1.1, pyobjc-framework-NetFS@==9.1.1, pyobjc-framework-Network@==9.1.1, pyobjc-framework-NetworkExtension@==9.1.1, pyobjc-framework-NotificationCenter@==9.1.1, pyobjc-framework-OpenDirectory@==9.1.1, pyobjc-framework-OSAKit@==9.1.1, pyobjc-framework-OSLog@==9.1.1, pyobjc-framework-PassKit@==9.1.1, pyobjc-framework-PencilKit@==9.1.1, pyobjc-framework-PHASE@==9.1.1, pyobjc-framework-Photos@==9.1.1, pyobjc-framework-PhotosUI@==9.1.1, pyobjc-framework-PreferencePanes@==9.1.1, pyobjc-framework-PushKit@==9.1.1, pyobjc-framework-Quartz@==9.1.1, pyobjc-framework-QuickLookThumbnailing@==9.1.1, pyobjc-framework-ReplayKit@==9.1.1, pyobjc-framework-SafariServices@==9.1.1, pyobjc-framework-SafetyKit@==9.1.1, pyobjc-framework-SceneKit@==9.1.1, pyobjc-framework-ScreenCaptureKit@==9.1.1, pyobjc-framework-ScreenSaver@==9.1.1, pyobjc-framework-ScreenTime@==9.1.1, pyobjc-framework-ScriptingBridge@==9.1.1, pyobjc-framework-SearchKit@==9.1.1, pyobjc-framework-Security@==9.1.1, pyobjc-framework-SecurityFoundation@==9.1.1, pyobjc-framework-SecurityInterface@==9.1.1, pyobjc-framework-ServiceManagement@==9.1.1, pyobjc-framework-SharedWithYou@==9.1.1, pyobjc-framework-SharedWithYouCore@==9.1.1, pyobjc-framework-ShazamKit@==9.1.1, pyobjc-framework-Social@==9.1.1, pyobjc-framework-SoundAnalysis@==9.1.1, pyobjc-framework-Speech@==9.1.1, pyobjc-framework-SpriteKit@==9.1.1, pyobjc-framework-StoreKit@==9.1.1, pyobjc-framework-SyncServices@==9.1.1, pyobjc-framework-SystemConfiguration@==9.1.1, pyobjc-framework-SystemExtensions@==9.1.1, pyobjc-framework-ThreadNetwork@==9.1.1, pyobjc-framework-UniformTypeIdentifiers@==9.1.1, pyobjc-framework-UserNotifications@==9.1.1, pyobjc-framework-UserNotificationsUI@==9.1.1, pyobjc-framework-VideoSubscriberAccount@==9.1.1, pyobjc-framework-VideoToolbox@==9.1.1, pyobjc-framework-Virtualization@==9.1.1, pyobjc-framework-Vision@==9.1.1, pyobjc-framework-WebKit@==9.1.1, pyparsing@==3.0.9, pyperclip@==1.8.2, PyQt5@==5.15.11, PyQt5_sip@==12.17.0, PyQt5-Qt5@==5.15.16, PyRect@==0.2.0, pyrr@==0.10.3, PyScreeze@==0.1.28, pyserial@==3.5, PySocks@==1.7.1, python-dateutil@==2.8.2, python-dotenv@==1.0.0, python-engineio@==4.12.2, python-json-logger@==2.0.7, python-socketio@==5.13.0, python-utils@==3.9.1, pytube@==15.0.0, pytweening@==1.0.4, pytz@==2025.1, PyYAML@==6.0, pyzmq@==25.1.2, referencing@==0.33.0, regex@==2024.9.11, reportlab@==4.2.5, requests@==2.32.3, requests-oauthlib@==1.3.1, rfc3339-validator@==0.1.4, rfc3986-validator@==0.1.1, rich@==13.7.1, rich-click@==1.8.0, rpds-py@==0.17.1, rsa@==4.9, rubicon-objc@==0.4.6, safetensors@==0.4.5, scipy@==1.10.1, screeninfo@==0.8.1, selenium@==4.23.0, Send2Trash@==1.8.2, serial@==0.0.97, simple-websocket@==1.1.0, six@==1.16.0, skia-pathops@==0.8.0.post2, sniffio@==1.3.0, sortedcontainers@==2.4.0, sounddevice@==0.4.6, soupsieve@==2.5, spotify2ytmusic@==0.9.30, srt@==3.5.3, stack-data@==0.6.3, svgelements@==1.9.6, svglib@==1.5.1, sympy@==1.13.1, tenacity@==8.5.0, tensorboard@==2.12.2, tensorboard-data-server@==0.7.0, tensorboard-plugin-wit@==1.8.1, tensorflow-estimator@==2.12.0, tensorflow-macos@==2.12.0, termcolor@==2.2.0, terminado@==0.18.0, timm@==1.0.11, tinycss2@==1.2.1, tomli@==2.0.1, torch@==2.5.1, torchvision@==0.20.1, tornado@==6.4, tqdm@==4.66.5, traitlets@==5.14.1, trio@==0.26.0, trio-websocket@==0.11.1, types-python-dateutil@==2.8.19.20240106, typing_extensions@==4.14.0, typing-inspection@==0.4.1, tzdata@==2025.1, uri-template@==1.3.0, uritemplate@==4.1.1, urllib3@==2.2.2, virtualenv@==20.31.2, watchdog@==5.0.3, wcwidth@==0.2.13, webcolors@==1.13, webencodings@==0.5.1, websocket-client@==1.8.0, websockets@==15.0.1, Werkzeug@==2.2.3, wrapt@==1.14.1, wsproto@==1.2.0, yarl@==1.20.1, yt-dlp@==2024.8.6, ytmusicapi@==1.8.1, zipp@==3.20.2, zstandard@==0.23.0

### Recent commits (newest first)

- updated system prompt, added tts functionality, fixed color & resize issue, full thing works with voice speaking out loud about camera scene
- moved something to .env
- add requirements.txt
- add README
- flow of getting descriptions of camera feed works

## Key source files (fetched from GitHub, selected and truncated for size)

### requirements.txt

```
absl-py==1.4.0
aiohappyeyeballs==2.6.1
aiohttp==3.12.13
aiosignal==1.3.2
altgraph==0.17.3
annotated-types==0.7.0
anyio==4.9.0
appnope==0.1.3
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
asttokens==2.4.1
astunparse==1.6.3
async-lru==2.0.4
async-timeout==4.0.3
attrs==23.2.0
Authlib==1.3.0
autopep8==2.0.2
av==13.1.0
Babel==2.14.0
beautifulsoup4==4.12.3
bidict==0.23.1
bleach==6.1.0
bleak==0.22.3
bpy==4.0.0
Brotli==1.1.0
CacheControl==0.12.11
cachetools==5.3.0
certifi==2025.1.31
cffi==1.15.1
cfgv==3.4.0
chardet==5.2.0
charset-normalizer==3.1.0
click==8.1.3
cloup==3.0.5
comm==0.2.1
contourpy==1.0.7
cryptography==40.0.2
cssselect2==0.7.0
cvzone==1.5.6
cycler==0.11.0
Cython==3.0.11
debugpy==1.8.0
decorator==5.1.1
defusedxml==0.7.1
diagrams==0.24.4
diffusers==0.31.0
distlib==0.3.9
dnspython==2.4.2
einops==0.8.0
exceptiongroup==1.2.0
executing==2.0.1
fastjsonschema==2.19.1
ffmpeg==1.4
ffmpeg-python==0.2.0
filelock==3.16.1
firebase-admin==6.1.0
Flask==2.2.3
Flask-PyMongo==2.3.0
Flask-SocketIO==5.5.1
flatbuffers==23.3.3
fonttools==4.38.0
fqdn==1.5.1
frozenlist==1.7.0
fsspec==2024.10.0
future==0.18.3
gast==0.4.0
glcontext==3.0.0
google-api-core==2.11.0
google-api-python-client==2.86.0
google-auth==2.17.3
google-auth-httplib2==0.1.0
google-auth-oauthlib==1.0.0
google-cloud-core==2.3.2
google-cloud-firestore==2.11.0
google-cloud-storage==2.8.0
google-crc32c==1.5.0
google-genai==1.21.1
google-pasta==0.2.0
google-resumable-media==2.4.1
googleapis-common-protos==1.59.0
graphviz==0.20.3
grpcio==1.54.0
grpcio-status==1.54.0
h11==0.16.0
h5py==3.8.0
httpcore==1.0.9
httplib2==0.22.0
httpx==0.28.1
huggingface-hub==0.26.2
identify==2.6.10
idna==3.4
importlib_metadata==8.5.0
inquirerpy==0.3.4
ipykernel==6.29.0
ipython==8.20.0
iso8601==1.1.0
isoduration==20.11.0
isosurfaces==0.1.2
itsdangerous==2.1.2
jax==0.4.8
jedi==0.19.1
Jinja2==3.1.2
json5==0.9.14
jsonpointer==2.4
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
jupyter-events==0.9.0
jupyter-lsp==2.2.2
jupyter_client==8.6.0
jupyter_core==5.7.1
jupyter_server==2.12.5
jupyter_server_terminals==0.5.2
jupyterlab==4.0.12
jupyterlab_pygments==0.3.0
jupyterlab_server==2.25.2
keras==2.12.0
kiwisolver==1.4.4
libclang==16.0.0
lmnt==1.2.0
lxml==5.3.0
macholib==1.16.2
manim==0.18.1
ManimPango==0.6.0
mapbox_earcut==1.0.2
Markdown==3.4.3
markdown-it-py==3.0.0
MarkupSafe==2.1.2
matplotlib==3.6.3
matplotlib-inline==0.1.6
mdurl==0.1.2
mediapipe==0.9.3.0
mediapipe-silicon==0.9.2.1
mido==1.3.2
mistune==3.0.2
ml-dtypes==0.1.0
moderngl==5.12.0
moderngl-window==2.4.6
MouseInfo==0.1.3
mpmath==1.3.0
msgpack==1.0.5
multidict==6.5.0
multipledispatch==1.0.0
mutagen==1.47.0
nbclient==0.9.0
nbconvert==7.14.2
nbformat==5.9.2
nest-asyncio==1.6.0
networkx==3.4.2
nodeenv==1.9.1
notebook_shim==0.2.3
numpy==1.26.4
numpy-stl==3.2.0
oauthlib==3.2.2
opencv-contrib-python==4.7.0.68
opencv-python==4.11.0.86
opt-einsum==3.3.0
outcome==1.3.0.post0
overrides==7.7.0
packaging==23.1
pandas==2.2.3
pandocfilters==1.5.1
parso==0.8.3
pexpect==4.9.0
pfzy==0.3.4
pillow==11.0.0
platformdirs==4.2.0
pre_commit==4.2.0
prometheus-client==0.19.0
prompt-toolkit==3.0.43
propcache==0.3.2
proto-plus==1.22.2
protobuf==3.20.3
psutil==5.9.8
ptyprocess==0.7.0
pure-eval==0.2.2
pyasn1==0.5.0
pyasn1-modules==0.3.0
PyAutoGUI==0.9.53
pycairo==1.27.0
pycodestyle==2.10.0
pycparser==2.21
pycryptodomex==3.20.0
pydantic==2.11.7
pydantic_core==2.33.2
pydub==0.25.1
pygame==2.3.0
PyGetWindow==0.0.9
pyglet==2.0.18
Pygments==2.17.2
pyinstaller==5.10.1
pyinstaller-hooks-contrib==2023.2
PyJWT==2.6.0
pymongo==4.6.1
PyMsgBox==1.0.9
pynput==1.7.6
pyobjc==9.1.1
pyobjc-core==10.3.2
pyobjc-framework-Accessibility==9.1.1
pyobjc-framework-Accounts==9.1.1
pyobjc-framework-AddressBook==9.1.1
pyobjc-framework-AdServices==9.1.1
pyobjc-framework-AdSupport==9.1.1
pyobjc-framework-AppleScriptKit==9.1.1
pyobjc-framework-AppleScriptObjC==9.1.1
pyobjc-framework-ApplicationServices==9.1.1
pyobjc-framework-AppTrackingTransparency==9.1.1
pyobjc-framework-AudioVideoBridging==9.1.1
pyobjc-framework-AuthenticationServices==9.1.1
pyobjc-framework-AutomaticAssessmentConfiguration==9.1.1
pyobjc-framework-Automator==9.1.1
pyobjc-framework-AVFoundation==9.1.1
pyobjc-framework-AVKit==9.1.1
pyobjc-framework-AVRouting==9.1.1
pyobjc-framework-BackgroundAssets==9.1.1
pyobjc-framework-BusinessChat==9.1.1
pyobjc-framework-CalendarStore==9.1.1
pyobjc-framework-CallKit==9.1.1
pyobjc-framework-CFNetwork==9.1.1
pyobjc-framework-ClassKit==9.1.1
pyobjc-framework-CloudKit==9.1.1
pyobjc-framework-Cocoa==10.3.2
pyobjc-framework-Collaboration==9.1.1
pyobjc-framework-ColorSync==9.1.1
pyobjc-framework-Contacts==9.1.1
pyobjc-framework-ContactsUI==9.1.1
pyobjc-framework-CoreAudio==9.1.1
pyobjc-framework-CoreAudioKit==9.1.1
pyobjc-framework-CoreBluetooth==10.3.2
pyobjc-framework-CoreData==9.1.1
pyobjc-framework-CoreHaptics==9.1.1
pyobjc-framework-CoreLocation==9.1.1
pyobjc-framework-CoreMedia==9.1.1
pyobjc-framework-CoreMediaIO==9.1.1
pyobjc-framework-CoreMIDI==9.1.1
pyobjc-framework-CoreML==9.1.1
pyobjc-framework-CoreMotion==9.1.1
pyobjc-framework-CoreServices==9.1.1
pyobjc-framework-CoreSpotlight==9.1.1
pyobjc-framework-CoreText==9.1.1
pyobjc-framework-CoreWLAN==9.1.1
pyobjc-framework-CryptoTokenKit==9.1.1
pyobjc-framework-DataDetection==9.1.1
pyobjc-framework-DeviceCheck==9.1.1
pyobjc-framework-DictionaryServices==9.1.1
pyobjc-framework-DiscRecording==9.1.1
pyobjc-framework-DiscRecordingUI==9.1.1
pyobjc-framework-DiskArbitration==9.1.1
pyobjc-framework-DVDPlayback==9.1.1
pyobjc-framework-EventKit==9.1.1
pyobjc-framework-ExceptionHandling==9.1.1
pyobjc-framework-ExecutionPolicy==9.1.1
pyobjc-framework-ExtensionKit==9.1.1
pyobjc-framework-ExternalAccessory==9.1.1
pyobjc-framework-FileProvider==9.1.1
pyobjc-framework-FileProviderUI==9.1.1
pyobjc-framework-FinderSync==9.1.1
pyobjc-framework-FSEvents==9.1.1
pyobjc-framework-GameCenter==9.1.1
pyobjc-framework-GameController==9.1.1
pyobjc-framework-GameKit==9.1.1
pyobjc-framework-GameplayKit==9.1.1
pyobjc-framework-Healt
[truncated — 4838 more characters]
```

### main.py

```python
import time
import cam_script
import google_ai_interface
from PIL import Image
import base64
from io import BytesIO
import tts
import cv2

def loop():
    print("start")

    print("getting frame")
    frame = cam_script.get_camera_image()
    if type(frame) == type(None):
        print("couldn't get frame")
        return

    print("resizing image... ", end="")
    rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    print(cam_script.width, cam_script.height)
    resized = Image.fromarray(rgb_frame).resize((320, int(320 * float(cam_script.height) / float(cam_script.width))), Image.Resampling.LANCZOS)
    print("resized!")

    print("making b64... ", end="")
    buffered = BytesIO()
    resized.save(buffered, format="JPEG")
    img_str = base64.b64encode(buffered.getvalue())
    print("made!")

    print("asking google ai abt image...")
    response = google_ai_interface.generate_response_from_image(img_str, "jpeg")
    print(response)

    tts.play_tts(response)


def main():
    print("initialize")
    while True:
        loop()
        print("waiting before next loop")
        time.sleep(3)

if __name__ == "__main__":
    main()
```

### cam_script.py

```python
import cv2

cap = cv2.VideoCapture(0)  # 0 is the default camera


if not cap.isOpened():
    print("Cannot open camera")
    exit()

width = cap.get(cv2.CAP_PROP_FRAME_WIDTH)
height = cap.get(cv2.CAP_PROP_FRAME_HEIGHT)

def get_camera_image(show_window=False):
    ret, frame = cap.read()
    if not ret:
        print("Failed to grab frame")
        return None

    if show_window:
        cv2.imshow('Video', frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            return None

    return frame

if __name__ == "__main__":
    while True:
        if type(get_camera_image(True)) == type(None):
            break

    cap.release()
    cv2.destroyAllWindows()
```

### tts.py

```python
import asyncio
from lmnt.api import Speech
from playsound import playsound
from io import BytesIO
import base64

async def text_to_audio(text: str, voice: str = 'leah') -> bytes:
    async with Speech() as speech:
        synthesis = await speech.synthesize(text, voice)
        print("t1", type(synthesis))
        print("t2", type(synthesis['audio']))
        return synthesis['audio']

def text_to_audio_sync(text: str, voice: str = 'leah') -> bytes:
    return asyncio.run(text_to_audio(text, voice))

def play_tts(text):
    with open("./temp/audio.mp3", "wb") as audio_file:
        audio_file.write(text_to_audio_sync(text))

    playsound("./temp/audio.mp3")

if __name__ == "__main__":
    play_tts("Hello there, how are you? This is an AI voice that is created using an AI voice creating thing that creates AI voices.")
```

### google_ai_interface.py

```python
from google import genai
from google.genai import types
import base64
from os import getenv

systemprompt = ""

with open("./systemprompt.txt", "r") as systemprompt_file:
  systemprompt = systemprompt_file.read()

def generate_response_from_image(b64img, img_ext="png"):
  client = genai.Client(
      vertexai=True,
      project=getenv("GOOGLE_AI_PROJECT_NAME"),
      location="us-central1",
  )

  image1 = types.Part.from_bytes(
      data=base64.b64decode(b64img),
      mime_type="image/" + img_ext,
  )

  si_text1 = systemprompt

  model = "gemini-2.5-flash"
  contents = [
    types.Content(
      role="user",
      parts=[
        image1,
        types.Part.from_text(text="Describe the scene and any relevant updates, if applicable.")
      ]
    )
  ]

  generate_content_config = types.GenerateContentConfig(
    temperature = 1,
    top_p = 1,
    seed = 0,
    max_output_tokens = 65535,
    safety_settings = [types.SafetySetting(
      category="HARM_CATEGORY_HATE_SPEECH",
      threshold="OFF"
    ),types.SafetySetting(
      category="HARM_CATEGORY_DANGEROUS_CONTENT",
      threshold="OFF"
    ),types.SafetySetting(
      category="HARM_CATEGORY_SEXUALLY_EXPLICIT",
      threshold="OFF"
    ),types.SafetySetting(
      category="HARM_CATEGORY_HARASSMENT",
      threshold="OFF"
    )],
    system_instruction=[types.Part.from_text(text=si_text1)],
    thinking_config=types.ThinkingConfig(
      thinking_budget=-1,
    ),
  )

  for chunk in client.models.generate_content_stream(
    model = model,
    contents = contents,
    config = generate_content_config,
    ):

    return chunk.text

if __name__ == "__main__":
    from time import sleep
    import base64

    img1data = ""
    img2data = ""
    img3data = ""

    with open("test1.png", "rb") as test_img1:
      img_data = test_img1.read()
      img1data = base64.b64encode(img_data)

    with open("test2.png", "rb") as test_img2:
      img_data = test_img2.read()
      img2data = base64.b64encode(img_data)

    with open("test3.png", "rb") as test_img3:
      img_data = test_img3.read()
      img3data = base64.b64encode(img_data)

    generate_response_from_image(img1data)
    print("\ngenerated 1\n")
    sleep(3)

    generate_response_from_image(img2data)
    print("\ngenerated 2\n")
    sleep(3)

    generate_response_from_image(img3data)
    print("\ngenerated 3\n")
    sleep(3)
```