# Project export: ForgetMeNot This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 11.0 - Tagline: OH, SHOOT, you missed your partner's birthday, and now you're a terrible person. Depending on just how upset or late in the day you are, you can get personalized gift ideas! - Devpost: https://devpost.com/software/forgetmenot-a1hbm5 - GitHub: https://github.com/s-pat6/calhacks - Video: https://www.youtube.com/embed/HzHKUKFaNh0?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 4 GitHub contributor(s) — Karmanyaah Malhotra (14 commits), Raghav (14 commits), NINJAQKk (11 commits), tanneberger (7 commits) ## Devpost submission (written by the team) ### Inspiration Everyone's heard or experienced the misfortune of forgetting a birthday. In fact right as the hackathon was starting, we realized we had forgotten one of our friend's birthdays in the rush about CalHacks. That's why we created ForgetMeNot, a way to remember when your friends birthdays are, and to make them great gifts at the last minute, so that way you can gaslight anyone into thinking you spent weeks preparing! ### What it does ForgetMeNot serves to recognize when the person you are looking at is upset, and tries to figure out the cause. Whether it be an anniversary, birthday, or they're just a little under the weather. No matter the cause, ForgetMeNot has the solution for you! Whether it be a custom love letter, teasing birthday collage, last minute flower delivery, or even a custom 3d printed gift, just for them! No matter the reason, or situation, there's something on hand. ### How we built it Firstly, we used Deepface for facial recognition and emotional analysis. We would match them with the identities and dates of those in a scraped past year's google calendar and your phone's contact list and photos. By analysis for matches with occasions, we could determine whether it was because of a missed occasion, or maybe just something small. Based the level of anger or frustration they showed on their face, and the amount of time left in the day, we came up with a doom's day esq countdown clock, so you could figure out how long you had to come up with your definitely not last minute gift. Depending on whether this was shorter or longer, we used Deepgram's emotionally intelligent AI to give suggestions about whether to comfort your partner, or gaslight them to buy time. It not only generated the text, but was able to say it aloud to one's earbuds to allow them more discrete advice. Using Groq's chat completion features and context window system, we were able to feed in the data from the emotional analysis and facial recognition, and sent it to the Llama 3 8b generative AI model. After prompting, it was able to provide concise and heartfelt messages to replicate a love letter. Meme Video Generation -OpenCV and ffmpeg (Continued to apply emotional analysis to determine which photos to use) Scraped flowers for multiple occasions from BeautifulSoup to highlight and recommend flowers that can be bought for 4 hour delivery. The flowers were recommended by looking at the occasion. Through utilizing reflex's redirect button, we were able to send our users directly to the site to complete their purchase. A similar process was applied for assembling and showcasing the 3d print files. We used a 3D slicer to computer the time taken for files to be printed. ### Challenges we ran into Some of the challenges we ran into, were around adopting a new framework for our front end. Reflex made some tasks simpler, however it also made it difficult to learn and deploy more advanced and interactive Front End Elements, such as a 3d Web based STL viewer like Three.js Another challenge was that we had initially not received AR glasses, however we bootstrapped our own setup with wireless headphones, a camera, and our own glasses. ### Accomplishments we're proud of The product itself was genuinely entertaining, and we were able to laugh at, and find uses for the majority of the content and gifts ideas and products that came out of ForgetMeNot. Furthermore, there was a diverse range of tools used in building this, such as *facial recognition, text-to-speech, NLP, webscraping, AI based recommendations, and a brand new front end framework. * Despite this, we were able to successfully integrate the majority of the technologies we set out to incorporate. ### What we learned We gained a greater grasp of facial recognition and tuning, as well as exploring Reflex and Deepgram, which were completely new tools to us. ### What's next From here, ForgetMeNot can be expanded to design custom STL files for your partner instead of recommending from a pre-existing collection. Furthermore, integrations with a google photos API and google calendar would allow a more dynamic recognition of events, while reducing the set-up time for users ## README (from the GitHub repository) USE PNG PNGPNPNGPNPNG Run python -m http.server 4000 and nix develop; source .venv/bin/activate; reflex run ## Detected evidence (automated analysis) Indexed codebase: 41 recognized source files, 9638 KB. - Flask (technology) — detected in the code - Python (language) — detected in the code - TensorFlow (technology) — detected in the code ## Codebase structure (from repository index) ### Files (52 of 52) ``` .gitignore assets/filigree_double_wall_tealight_holder_116mm.stl calhacks3/__init__.py calhacks3/backend/__init__.py calhacks3/backend/backend.py calhacks3/backend/models.py calhacks3/calhacks3.py calhacks3/components/__init__.py calhacks3/components/flower_data.csv calhacks3/components/flower_scraper.py calhacks3/components/form_field.py calhacks3/components/gender_badges.py calhacks3/components/images/image.json calhacks3/face_recog/__init__.py calhacks3/video_gen/__init__.py calhacks3/views/__init__.py calhacks3/views/camera.py calhacks3/views/deploysafetymeasures.py calhacks3/views/email.py calhacks3/views/flower.py calhacks3/views/loveletter.py calhacks3/views/navbar.py calhacks3/views/printphotos.py calhacks3/views/table.py calhacks3/views/timer.py calhacks3/voice/main.py flake.lock flake.nix haarcascades/data/haarcascade_eye_tree_eyeglasses.xml haarcascades/data/haarcascade_eye.xml haarcascades/data/haarcascade_frontalcatface_extended.xml haarcascades/data/haarcascade_frontalcatface.xml haarcascades/data/haarcascade_frontalface_alt_tree.xml haarcascades/data/haarcascade_frontalface_alt.xml haarcascades/data/haarcascade_frontalface_alt2.xml haarcascades/data/haarcascade_frontalface_default.xml haarcascades/data/haarcascade_fullbody.xml haarcascades/data/haarcascade_lefteye_2splits.xml haarcascades/data/haarcascade_license_plate_rus_16stages.xml haarcascades/data/haarcascade_lowerbody.xml haarcascades/data/haarcascade_profileface.xml haarcascades/data/haarcascade_righteye_2splits.xml haarcascades/data/haarcascade_russian_plate_number.xml haarcascades/data/haarcascade_smile.xml haarcascades/data/haarcascade_upperbody.xml photodatabase/Kim/meta.json photodatabase/meta.json photodatabase/Spandan/meta.json photodatabase/Tassilo/meta.json README.md requirements.txt rxconfig.py ``` ### Dependencies - requirements.txt: absl-py@==2.1.0, aiohappyeyeballs@==2.4.3, aiohttp@==3.10.10, aiosignal@==1.3.1, annotated-types@==0.7.0, anyio@==4.6.2.post1, apispec@==6.6.1, astunparse@==1.6.3, attrs@==24.2.0, bech32@==1.2.0, blinker@==1.8.2, certifi@==2024.8.30, charset-normalizer@==3.4.0, click@==8.1.7, colorama@==0.4.6, contourpy@==1.3.0, cosmpy@==0.9.2, cycler@==0.12.1, deepface@==0.0.93, deepgram-sdk@==3.7.4, ecdsa@==0.19.0, ffmpeg@==1.4, ffmpeg-python@==0.2.0, filelock@==3.16.1, fire@==0.7.0, Flask@==3.0.3, Flask-Cors@==5.0.0, flatbuffers@==24.3.25, fonttools@==4.54.1, frozenlist@==1.4.1, fsspec@==2024.9.0, future@==1.0.0, gast@==0.6.0, gdown@==5.2.0, google-pasta@==0.2.0, googleapis-common-protos@==1.65.0, groq@==0.11.0, grpcio@==1.67.0, gunicorn@==23.0.0, h11@==0.14.0, h5py@==3.12.1, httpcore@==1.0.6, httpx@==0.27.2, idna@==3.10, iniconfig@==2.0.0, iterators@==0.2.0, itsdangerous@==2.2.0, janus@==1.0.0, Jinja2@==3.1.4, joblib@==1.4.2, jsonschema@==4.23.0, jsonschema-specifications@==2024.10.1, keras@==3.6.0, kiwisolver@==1.4.7, libclang@==18.1.1, lz4@==4.3.3, markdown-it-py@==3.0.0, MarkupSafe@==3.0.2, matplotlib@==3.9.2, mdurl@==0.1.2, ml-dtypes@==0.4.1, mpmath@==1.3.0, msgpack@==1.1.0, mtcnn@==1.0.0, multidict@==6.1.0, namex@==0.0.8, networkx@==3.4.1, opencv-python@==4.10.0.84, opt_einsum@==3.4.0, optree@==0.13.0, packaging@==24.1, pandas@==2.2.3, pillow@==11.0.0, pluggy@==1.5.0, propcache@==0.2.0, protobuf@==4.25.5, psutil@==6.1.0, py-cpuinfo@==9.0.0, PyAudio@==0.2.14, pycryptodome@==3.21.0, pydantic@==2.8.2, pydantic_core@==2.20.1, Pygments@==2.18.0, pyparsing@==3.2.0, PySocks@==1.7.1, pytest@==8.3.3, pytest-asyncio@==0.24.0, python-dateutil@==2.9.0.post0, pytz@==2024.2, PyYAML@==6.0.2, referencing@==0.35.1, reflex@>=0.4.6, requests@==2.32.3, retina-face@==0.0.17, rich@==13.9.2, rpds-py@==0.20.0, setuptools@==75.2.0, six@==1.16.0, sniffio@==1.3.1, soupsieve@==2.6, sympy@==1.13.1, tensorboard@==2.17.1, tensorboard-data-server@==0.7.2, tensorflow@==2.17.0, termcolor@==2.5.0, tf_keras@==2.17.0, tqdm@==4.66.5, typing_extensions@==4.12.2, tzdata@==2024.2, urllib3@==2.2.3, uv@==0.4.24, uvicorn@==0.30.6, websockets@==12.0, Werkzeug@==3.0.4, wrapt@==1.16.0, yarl@==1.15.5 ### Recent commits (newest first) - eworks - karmanyaah updates - Merge branch 'main' of https://github.com/s-pat6/calhacks - better gaslight - Merge branch 'main' of https://github.com/s-pat6/calhacks - raghav - uhhh - Merge branch 'main' of https://github.com/s-pat6/calhacks - print out ai speech - make camera freeze reset - other deps - deps - yooooo - Merge branch 'main' of https://github.com/s-pat6/calhacks - gitignore - Merge branch 'main' of https://github.com/s-pat6/calhacks - Merge branch 'main' of https://github.com/s-pat6/calhacks - Merge branch 'main' of https://github.com/s-pat6/calhacks - flowers, cleaned up UI, some hover transitions - voice & speech integration ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` absl-py==2.1.0 astunparse==1.6.3 blinker==1.8.2 certifi==2024.8.30 charset-normalizer==3.4.0 click==8.1.7 deepface==0.0.93 filelock==3.16.1 fire==0.7.0 Flask==3.0.3 Flask-Cors==5.0.0 flatbuffers==24.3.25 gast==0.6.0 gdown==5.2.0 google-pasta==0.2.0 grpcio==1.67.0 gunicorn==23.0.0 h5py==3.12.1 idna==3.10 itsdangerous==2.2.0 Jinja2==3.1.4 joblib==1.4.2 keras==3.6.0 libclang==18.1.1 lz4==4.3.3 markdown-it-py==3.0.0 MarkupSafe==3.0.2 mdurl==0.1.2 ml-dtypes==0.4.1 mtcnn==1.0.0 namex==0.0.8 opencv-python==4.10.0.84 opt_einsum==3.4.0 optree==0.13.0 packaging==24.1 pillow==11.0.0 protobuf==4.25.5 Pygments==2.18.0 PySocks==1.7.1 requests==2.32.3 retina-face==0.0.17 rich==13.9.2 soupsieve==2.6 tensorboard==2.17.1 tensorboard-data-server==0.7.2 tensorflow==2.17.0 termcolor==2.5.0 tf_keras==2.17.0 tqdm==4.66.5 typing_extensions==4.12.2 urllib3==2.2.3 Werkzeug==3.0.4 wrapt==1.16.0 reflex>=0.4.6 aiohappyeyeballs==2.4.3 aiohttp==3.10.10 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.6.2.post1 apispec==6.6.1 attrs==24.2.0 bech32==1.2.0 certifi==2024.8.30 charset-normalizer==3.4.0 click==8.1.7 colorama==0.4.6 contourpy==1.3.0 cosmpy==0.9.2 cycler==0.12.1 ecdsa==0.19.0 ffmpeg==1.4 ffmpeg-python==0.2.0 filelock==3.16.1 fonttools==4.54.1 frozenlist==1.4.1 fsspec==2024.9.0 future==1.0.0 googleapis-common-protos==1.65.0 grpcio==1.67.0 h11==0.14.0 httpcore==1.0.6 httpx==0.27.2 idna==3.10 iniconfig==2.0.0 iterators==0.2.0 Jinja2==3.1.4 jsonschema==4.23.0 jsonschema-specifications==2024.10.1 kiwisolver==1.4.7 MarkupSafe==3.0.2 matplotlib==3.9.2 mpmath==1.3.0 msgpack==1.1.0 multidict==6.1.0 networkx==3.4.1 opencv-python==4.10.0.84 packaging==24.1 pandas==2.2.3 pillow==11.0.0 pluggy==1.5.0 propcache==0.2.0 protobuf==4.25.5 psutil==6.1.0 py-cpuinfo==9.0.0 pycryptodome==3.21.0 pydantic==2.8.2 pydantic_core==2.20.1 pyparsing==3.2.0 pytest==8.3.3 pytest-asyncio==0.24.0 python-dateutil==2.9.0.post0 pytz==2024.2 PyYAML==6.0.2 referencing==0.35.1 requests==2.32.3 rpds-py==0.20.0 setuptools==75.2.0 six==1.16.0 sniffio==1.3.1 sympy==1.13.1 tqdm==4.66.5 typing_extensions==4.12.2 tzdata==2024.2 urllib3==2.2.3 uv==0.4.24 uvicorn==0.30.6 websockets==12.0 yarl==1.15.5 PyAudio==0.2.14 janus==1.0.0 deepgram-sdk==3.7.4 groq==0.11.0 ``` ### calhacks3/voice/main.py ```python import asyncio import os from deepgram import ( DeepgramClient, SpeakOptions, ) from groq import Groq import pyaudio import wave from play_sounds import play_file_async async def speak(txt="Hello and welcome to ForgetMeNot. I'm here to guide you through human emotions.", file='output.wav', block=True): SPEAK_OPTIONS = {'text': txt} filename = file try: # STEP 1: Create a Deepgram client using the API key from environment variables deepgram = DeepgramClient(api_key='d06a398656d51aa46a048e0e346314756189b101') # STEP 2: Configure the options (such as model choice, audio configuration, etc.) options = SpeakOptions( model="aura-athena-en", encoding="linear16", container="wav" ) # STEP 3: Call the save method on the speak property response = deepgram.speak.v("1").save(filename, SPEAK_OPTIONS, options) chunk = 1024 * 16 await play_file_async('./' + filename, block=block) #f = wave.open(r'./'+filename, 'rb') #p = pyaudio.PyAudio() #stream = p.open(format=p.get_format_from_width(f.getsampwidth()), # channels=f.getnchannels(), # rate=f.getframerate(), # output=True) #data = f.readframes(chunk) #while data: # stream.write(data) # data=f.readframes(chunk) # await asyncio.sleep(0) #stream.stop_stream() #stream.close() #p.terminate() #print(response.to_json(indent=4)) except Exception as e: print(f"Exception: {e}") def generate(content='Reaffirm the user', system='You are a voice assistant for an app called ForgetMeNot. You are assisting a user who forgot it was their anniversary with their significant other. Based on the emotions of the significant other, provide a course of action for the user to first identify how the other person is feeling, then make up for forgetting by presenting gifts.', tokens=30, assistant=''): client = Groq( api_key='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' ) chat_completion = client.chat.completions.create( messages=[ { 'role': 'system', 'content': system, }, { 'role': 'user', 'content': content, }, { 'role': 'assistant', 'content': assistant } ], model='llama3-8b-8192', temperature=0.5, max_tokens=tokens, ) return chat_completion.choices[0].message.content async def generate_and_speak(file='output.wav', content='Reaffirm the user', system='You are a voice assistant for an app called ForgetMeNot. You are assisting a user who forgot it was their anniversary with their significant other. Based on the emotions of the significant other, provide a course of action for the user to first identify how the other person is feeling, then make up for forgetting by presenting gifts.', tokens=30, assistant='', block = True): print('context: ' + system) print('content: ' + content) text = generate(content, system, tokens, assistant) print('text: ' + text) await speak(text, file, block) return text def main(): speak() generate() if __name__ == "__main__": main() ``` ### rxconfig.py ```python import reflex as rx config = rx.Config( app_name="calhacks3", ) ``` ### calhacks3/calhacks3.py ```python import asyncio import reflex as rx from .views.navbar import navbar from .views.email import email_gen_ui from .views.table import main_table from .views.camera import camera_feed, layout_with_video_and_another_component, deploy_safety_measures from .views.timer import countdown_clock from .backend.backend import State from .voice.main import generate_and_speak, speak import os from .face_recog import get_emotions from json import dump def process_library_emotions(directory): for root, dirs, files in os.walk(directory): # Recursively go through directories fs = [] #print(f"Processing directory: {os.path.basename(root)}") for file in files: if file.endswith('.png'): file_path = os.path.join(root, file) #print(f"Processing file: {file_path}") # Your file processing logic here fs.append(get_emotions(file_path)[0]) fs[-1]['path'] = file_path #print(fs) dump(fs, open(os.path.join(root, 'meta.json'), 'w')) process_library_emotions('./photodatabase') process_library_emotions('./nicephotodatabase') async def camera_task(): from .face_recog import stuff try: await stuff() except asyncio.CancelledError: print('camera task ended') return speak() def index() -> rx.Component: return rx.vstack( navbar(), layout_with_video_and_another_component(), countdown_clock(100), deploy_safety_measures(), # rx.flex( # rx.box(main_table(), width=["100%", "100%", "100%", "60%"]), # # email_gen_ui(), # spacing="6", # width="100%", # flex_direction=["column", "column", "column", "row"], # ), height="100vh", bg="#000000", width="100%", padding_x=["1.5em", "1.5em", "3em"], padding_y=["1em", "1em", "2em"], margin_top="5em", ) app = rx.App( stylesheets=[ "https://fonts.googleapis.com/css2?family=Rubik+Bubbles&display=swap", ], theme=rx.theme( appearance="dark", # Set to "dark" if you want a dark-themed appearance has_background=True, radius="large", accent_color="plum", colors={ "primary": "#ff006c", # Custom pink for primary elements "secondary": "#000000", # Black for secondary elements "accent": "#FF007F", # Pink for accent elements "background": "#000000", # Black background color "text": "#FFFFFF", # White text for contrast }, ), styles={ "body": { "backgroundColor": "#000000", # This forces the background color to black "color": "#FFFFFF", # Text color set to white for contrast } } ) app.add_page( index, # on_load=State.load_entries, title="ForgetMeNot", description="CalHacks 11.0", ) app.register_lifespan_task(camera_task) ``` ### calhacks3/backend/models.py ```python import reflex as rx class Customer(rx.Model, table=True): # type: ignore """The customer model.""" customer_name: str email: str age: int gender: str location: str job: str salary: int ``` ### calhacks3/components/gender_badges.py ```python import reflex as rx def _badge(text: str, color_scheme: str): return rx.badge( text, color_scheme=color_scheme, radius="full", variant="soft", size="3" ) def gender_badge(gender: str): badge_mapping = { "Male": ("♂️ Male", "blue"), "Female": ("♀️ Female", "pink"), "Other": ("Other", "gray"), } return _badge(*badge_mapping.get(gender, ("Other", "gray"))) ``` ### calhacks3/components/form_field.py ```python import reflex as rx def form_field( label: str, placeholder: str, type: str, name: str, icon: str, default_value: str = "", ) -> rx.Component: return rx.form.field( rx.flex( rx.hstack( rx.icon(icon, size=16, stroke_width=1.5), rx.form.label(label), align="center", spacing="2", ), rx.form.control( rx.input( placeholder=placeholder, type=type, default_value=default_value ), as_child=True, ), direction="column", spacing="1", ), name=name, width="100%", ) ``` ### calhacks3/views/navbar.py ```python import reflex as rx def navbar(): return rx.flex( rx.spacer(), # Spacer to push content from the left rx.hstack( rx.image(src="/ForgetMeNotLogo.png", width="60px", height="60px"), # Logo rx.text("ForgetMeNot", font_family="Rubik Bubbles", font_size="36px", font_weight="thin", # Correct use of font_weight color="#ff006c"), # Add text with custom font align="center", spacing="3", ), rx.spacer(), # Spacer to push content to the right align="center", spacing="2", flex_direction="row", # Horizontal alignment for the flex container width="100%", background_color='#000000', # Set background color to black padding="10px", # Padding for better spacing inside the navbar position="fixed", # Keep the navbar fixed at the top top="0px", # Position it at the very top left="0px", # Ensure it spans the full width right="0px", # Ensures the navbar stretches from left to right z_index="1000", # Keep the navbar on top of other content margin="0", # Remove any potential default margin ) ``` ### calhacks3/views/timer.py ```python import reflex as rx def countdown_clock(time): return rx.box( rx.html("""
"""), width="100%", align="center", padding="20px", ) ``` ### calhacks3/views/email.py ```python import reflex as rx from ..backend.backend import State def email_box(): return rx.box( rx.scroll_area( rx.icon_button( rx.icon("copy"), variant="soft", color_scheme="gray", size="2", on_click=[ rx.set_clipboard(State.email_content_data), rx.toast.info("Copied to clipboard"), ], cursor="pointer", position="absolute", bottom="1px", right="1px", z_index="10", ), rx.text(State.email_content_data, line_height="1.75"), type="auto", scrollbars="vertical", height="100%", position="relative", ), width="100%", height=["400px", "400px", "550px"], ) def options(): return rx.vstack( rx.vstack( rx.heading(f"Length limit: {State.length}", size="5"), rx.slider( min=500, max=1500, default_value=1000, step=25, size="2", on_change=State.set_length, ), width="100%", ), rx.vstack( rx.heading("Choose a tone", size="5"), rx.select( items=[ "😊 Formal", "😎 Casual", "😀 Friendly", "🧐 Convincing", "🙏 Humble", "😰 Urgent", "😂 Humorous", ], default_value="😊 Formal", size="3", on_change=State.set_tone, ), width="100%", ), spacing="5", width="100%", ) def email_gen_ui(): return ( rx.card( rx.flex( email_box(), rx.divider(), options(), flex_direction=[ "column-reverse", "column-reverse", "column-reverse", "column", ], padding=["0.5em", "0.5em", "1em"], spacing="5", ), width=["100%", "100%", "100%", "40%"], ), ) ``` [11 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]