# Project export: Munch This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2024 - Tagline: The day old problem of not knowing where to grab food with friends solved by SOTA LLM embedding, summary & sentiment analysis models with a novel multi-party aggregation technique. - Devpost: https://devpost.com/software/munch-b1i5mr - GitHub: https://github.com/rohandavidi/munch.git - Team: 1 GitHub contributor(s) — siroda (3 commits) ## Devpost submission (written by the team) ### Overview Why? In the dawn of various modern technological advancements, there exists a growing criticism that these ecosystems of innovations are breaking apart the relations we hold in the real world. We sough to utilize these very tools to build an application that brings people together to break bread. What? Munch is an online search platform that leverages a graphical representation of user preferences and natural language to create tailored dining recommendations that perfectly blend the culinary tastes of two or more individuals. Whether you're planning a romantic first date, meticulously planning a large group dinner, or simply meeting up with old friends, Munch is here to streamline your decision making process. Our platform analyzes each member of your party's previous restaurant ratings and preferences, and learns from your group's similarities and dissimilarities to curate a selection of dining options in your local area. How? To build Munch, we used: • Reflex — an open-source framework for quickly building aesthetic, interactive web applications in pure Python. • Instructor Embeddings — a Hugging face embedding model that relies on insight into the task at hand for developing rich embeddings. • GPT-4 — used the LLM to summarize restaurant review info and data to feed to embedding models. • Google Maps API — web-scraped API holding data on ratings, reviews, price levels, and locations of restaurants throughout the Bay Area. • SciPy and NumPy — used these technologies to make computations (e.g., for cosine similarities) more efficient and easier to work with • Our very own multi-party aggregation algorithm — when determining the optimal way to combine preferences for a number of users, a key aspect to the computation is the way in which the different parties are aggregated. We create our own proprietary iterative algorithm that uses cosine similarity and embeddings to find restaurants that align with the top few restaurant choices for each of the users, ensuring that as few group members as possible end up dining at a restaurant NOT similar to one of their original top choices. At the same time, our algorithm ensures that each group member will dine at a restaurant where they have not dined recently, ensuring group bonding over novel dining experiences. The algorithm scales to a large number of group members, a large embedding size, and a large restaurant pool. Despite? Through our building journey 🛠️, we encountered a few technical challenges: Creating a reliable, thorough database of user reviews was very challenging, as different APIs resulted in a variety of inconsistent metrics. Moreover, the way in which we had to select our member aggregation methods required considerable thinking (which led us to our proprietary solution) from many less-successful algorithm trials. So What? Munch isn't just about finding a place to eat—it's about forging connections and creating lasting memories over a plate of your community's perfect blend. In a digital age where technology often isolates, Munch is as a testament to the power of technology to facilitate, build, and strengthen real-world interactions. By prioritizing meaningful, human interactions over digital distractions, we counter the paradox of online connectivity leading to real-world isolation. Our platform, devoid of any social media pressures (e.g. posting to your timeline, being able to view other's recent activity, etc.), encourages users to be present. Through Munch, we strive to make every dining experience a shared adventure worth savoring. Next? We hope to expand the site beyond use amongst friends to the making of new ones. By safely integrating location data, we hope to provide suggestions for people living near each other of their common tastes and potential restaurants and cafe's they both might enjoy. We can further reinforce our algorithm by taking in user-inputted textual data for an even further fine grained understanding of their taste. ## README (from the GitHub repository) No README available. ## Detected evidence (automated analysis) Indexed codebase: 7 recognized source files, 21 KB. - OpenAI (technology) — detected in the code - Python (language) — detected in the code - Hugging Face (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (15 of 15) ``` .DS_Store .gitignore colab.research.google.com: 2.webloc colab.research.google.com: 3.webloc colab.research.google.com: 4.webloc colab.research.google.com:.webloc requirements.txt rxconfig.py treehacks_2024/__init__.py treehacks_2024/.DS_Store treehacks_2024/components/navbar.py treehacks_2024/connect_database.py treehacks_2024/get_review_summary.py treehacks_2024/styles.py treehacks_2024/treehacks_2024.py ``` ### Dependencies - requirements.txt: dotenv, firebase-admin, openai, reflex ### Recent commits (newest first) - final push - summary data fetching - first commit ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` reflex dotenv openai firebase-admin ``` ### rxconfig.py ```python import reflex as rx config = rx.Config( app_name="treehacks_2024", db_url="sqlite:///reflex.db", ) ``` ### treehacks_2024/connect_database.py ```python import firebase_admin from firebase_admin import credentials from firebase_admin import db cred = credentials.Certificate('/Users/rohandavidi/Desktop/treehacks_2024/treehacks_2024/secret_key.json') firebase_admin.initialize_app(cred, { 'databaseURL': "https://treehacks2024-1c2ab-default-rtdb.firebaseio.com" }) ref = db.reference('/') ``` ### treehacks_2024/styles.py ```python import reflex as rx bg_dark_color = "#111" bg_medium_color = "#222" border_color = "#fff3" accent_light = "#6649D8" accent_color = "#5535d4" accent_dark = "#4c2db3" icon_color = "#fff8" text_light_color = "#fff" shadow_light = "rgba(17, 12, 46, 0.15) 0px 48px 100px 0px;" shadow = "rgba(50, 50, 93, 0.25) 0px 50px 100px -20px, rgba(0, 0, 0, 0.3) 0px 30px 60px -30px, rgba(10, 37, 64, 0.35) 0px -2px 6px 0px inset;" message_style = dict(display="inline-block", p="4", border_radius="xl", max_w="30em") input_style = dict( bg=bg_medium_color, border_color=border_color, border_width="1px", p="4", ) icon_style = dict( font_size="md", color=icon_color, _hover=dict(color=text_light_color), cursor="pointer", w="8", ) sidebar_style = dict( border="double 1px transparent;", border_radius="10px;", background_image=f"linear-gradient({bg_dark_color}, {bg_dark_color}), radial-gradient(circle at top left, {accent_color},{accent_dark});", background_origin="border-box;", background_clip="padding-box, border-box;", p="2", _hover=dict( background_image=f"linear-gradient({bg_dark_color}, {bg_dark_color}), radial-gradient(circle at top left, {accent_color},{accent_light});", ), ) base_style = { rx.chakra.Avatar: { "shadow": shadow, "color": text_light_color, "bg": border_color, }, rx.chakra.Button: { "shadow": shadow, "color": text_light_color, "_hover": { "bg": accent_dark, }, }, rx.chakra.Menu: { "bg": bg_dark_color, "border": f"red", }, rx.chakra.MenuList: { "bg": bg_dark_color, "border": f"1.5px solid {bg_medium_color}", }, rx.chakra.MenuDivider: { "border": f"1px solid {bg_medium_color}", }, rx.chakra.MenuItem: { "bg": bg_dark_color, "color": text_light_color, }, rx.chakra.DrawerContent: { "bg": bg_dark_color, "color": text_light_color, "opacity": "0.9", }, rx.chakra.Hstack: { "align_items": "center", "justify_content": "space-between", }, rx.chakra.Vstack: { "align_items": "stretch", "justify_content": "space-between", }, } ``` ### treehacks_2024/get_review_summary.py ```python import requests from dotenv import load_dotenv import os import json from openai import OpenAI # Get API Keys load_dotenv() google_API_KEY = os.getenv("google_API_KEY") openai_API_KEY = os.getenv("openai_API_KEY") # Define the request payload payload = { 'textQuery': 'Bucca di Beppo in Bay Area' } # Define the request headers headers_text = { 'Content-Type': 'application/json', 'X-Goog-Api-Key': google_API_KEY, 'X-Goog-FieldMask': 'places.id,places.displayName' } # Define the API endpoint url_text = 'https://places.googleapis.com/v1/places:searchText' # Make the POST request response = requests.post(url_text, json=payload, headers=headers_text) # Check if the request was successful if response.status_code != 200: print('Error:', response.status_code) # Get IDs from textSearch results restaurant_dict = response.json() idList = [] topk = 1 for i in range(topk): idList.append(restaurant_dict['places'][i]['id']) # Get Reviews from details search results headers_details = { 'Content-Type': 'application/json', 'X-Goog-Api-Key': google_API_KEY, 'X-Goog-FieldMask': 'id,displayName,price_level,rating,reviews' } restaurant_data = {} for id in idList: url_cur = 'https://places.googleapis.com/v1/places/' + id response = requests.get(url_cur, headers = headers_details) if response.status_code != 200: print('Error:', response.status_code) restaurant_data[id] = response.json() # Add GPT Summary of reviews for embedding client = OpenAI( api_key=openai_API_KEY ) for id in restaurant_data: cur_reviews = restaurant_data[id]['reviews'] prompt_string = "Using these reviews write a summary of the restaurant incorporating the majority of the details listed and shared amongst the reviews:" for i in range(len(cur_reviews)): prompt_string += "\n " + str(i) + ") " cur_reviews[i] completion = client.chat.completions.create( model="gpt-3.5-turbo", messages=[ {"role": "system", "content": "You are a summarizer who takes reviews of restaurants and determines a holistic review using these texts to represent the restaurant as a whole."}, {"role": "user", "content": prompt_string} ] ) review_summary = completion.choices[0].message print(review_summary) restaurant_data[id]['review_summary'] = review_summary ``` ### treehacks_2024/treehacks_2024.py ```python """Welcome to Reflex! This file outlines the steps to create a basic app.""" from rxconfig import config import requests import reflex as rx import json from treehacks_2024 import styles import numpy as np import firebase_admin from firebase_admin import db, credentials from dotenv import load_dotenv import os import pickle from scipy.spatial.distance import cosine # Open the pickle file in binary mode for reading with open('/Users/rohandavidi/Desktop/treehacks_2024/treehacks_2024/embs_dict_correct.pickle', 'rb') as f: # Load the pickle file as a dictionary embs_dict = pickle.load(f) cred = credentials.Certificate('/Users/rohandavidi/Desktop/treehacks_2024/treehacks_2024/secret_key.json') firebase_admin.initialize_app(cred, { 'databaseURL': "https://treehacks2024-1c2ab-default-rtdb.firebaseio.com" }) ref = firebase_admin.db.reference("/") USER_NAME = "rdange" load_dotenv() google_API_KEY = os.getenv("google_API_KEY") embedding_size = 768 #embeddings_normalization = restaurant_embeddings['normalization'] class State(rx.State): """The app state.""" new_rating_data: dict = {} finding_data: dict = {} match_found: str = "..." def finding_submit(self, form_data: dict): self.finding_data = form_data three_users = len(self.finding_data['user_3']) > 1 data = ref.get() user_a_ratings = data[self.finding_data['user_1']]['ratings'] user_b_ratings = data[self.finding_data['user_2']]['ratings'] user_c_ratings = None if three_users: user_c_ratings = data[self.finding_data['user_3']]['ratings'] print(user_a_ratings) print(user_b_ratings) def find_highest_cosine_similarity(restaurant_embeddings, target_embedding, excluded_indices): similarities = [1 - cosine(target_embedding, restaurant_embeddings[i]) if i not in excluded_indices else -np.inf for i in range(len(restaurant_embeddings))] return np.argmax(similarities) def recommend_restaurant(user_top_k_restaurants, restaurant_embeddings, visited_restaurants, t, ind_to_rest, max_iters=10): R, d = restaurant_embeddings.shape N = len(user_top_k_restaurants) k = len(user_top_k_restaurants[0]) # Convert visited_restaurants to recent_visits for the first t items recent_visits = [user_visits[:t] for user_visits in visited_restaurants] # Initialize neighboring lists for each user neighboring_lists = [[] for _ in range(N)] common_restaurants_found = [] for iteration in range(1, max_iters + 1): print("--------") print("ITERATION " + str(iteration)) for user_idx in range(N): # Start with recent visits for the user excluded_indices = set(recent_visits[user_idx]) # Add already found neighbors for the user excluded_indices.update(neighboring_lists[user_idx]) print("EXCLUDED INDICES: " + str(excluded_indices)) for r_idx in user_top_k_restaurants[user_idx]: # Ensure we don't recommend the same restaurant by adding it to excluded indices # excluded_indices.add(r_idx) # Find the restaurant with the highest cosine similarity highest_similarity_idx = find_highest_cosine_similarity(restaurant_embeddings, restaurant_embeddings[r_idx], excluded_indices) if highest_similarity_idx not in neighboring_lists[user_idx]: neighboring_lists[user_idx].append(highest_similarity_idx) print("NEIGHBORING LIST FOR USER " + str(user_idx) + ": " + str(neighboring_lists[user_idx])) # Check if there's a common restaurant in all neighboring lists common_restaurants = set(neighboring_lists[0]) for neighbor_list in neighboring_lists[1:]: common_restaurants.intersection_update(neighbor_list) # Process common restaurants found in this iteration common_restaurants_temp = [] num_left = 3 - len(common_restaurants_found) for r_common in common_restaurants: if r_common not in common_restaurants_found: common_restaurants_temp.append(r_common) # hit exactly 3 total if len(common_restaurants_temp) == num_left: sorted_temp = sorted(common_restaurants_temp, key=lambda r: sum(visited_restaurants[user_idx].index(r) if r in visited_restaurants[user_idx] else float('inf') for user_idx in range(N)), reverse=True) common_restaurants_found.extend(sorted_temp) # Return the top 3 common restaurants if we have found 3 return common_restaurants_found # hit >3, so only take 3 if len(common_restaurants_temp) > num_left: sorted_temp = sorted(common_restaurants_temp, key=lambda r: sum(visited_restaurants[user_idx].index(r) if r in visited_restaurants[user_idx] else float('inf') for user_idx in range(N)), reverse=True) common_restaurants_found.extend(sorted_temp[:num_left]) return common_restaurants_found # hit <3 if common_restaurants_temp: # print(list(common_restaurants_temp)) # Return the common restaurant maximizing the sum of indices in visited_restaurants sorted_temp = sorted(common_restaurants_temp, key=lambda r: sum(visited_restaurants[user_idx].index(r) if r in visited_restaurants[user_idx] else float('inf') for user_idx in range(N)), reverse=True) # best_common_restaurant = max(common_restaurants, key=lambda r: sum(visited_res [truncated — 10279 more characters] ``` ### treehacks_2024/components/navbar.py ```python import reflex as rx from treehacks_2024 import styles ```