# Project export: Scope Search This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 11.0 - Tagline: Effortless, intuitive email search at your fingertips. - Devpost: https://devpost.com/software/scope-sbenyg - GitHub: https://github.com/sfkunal/ScopeSearch - Team: 3 GitHub contributor(s) — Kunal Srivastava (34 commits), Andrew Shen (26 commits), ayaanrahim (13 commits) ## Devpost submission (written by the team) ### Inspiration Email search sucks... A lot... We've all had negative experiences with email. To be specific, it's quite hard to find specific emails. Email search is a cross-platform pain point, regardless of mobile or web: the core problem stems from the fact that most email search systems are reactive, simplistic, and keyword-based. They rely on older indexing and retrieval methods rather than understanding the true intent behind a query. To figure out what we could do to make some progress, we talked to over 50 hackers this weekend before we got to building. We learned users typically don’t remember exact email keywords but instead rely on contextual clues—like who the email was from, the approximate date, or a related attachment. Most email clients force users to manually filter by these parameters, which introduces friction. We hypothesized that creating a retrieval system that had semantic understanding would likely provide a much better UX. Additionally, we could use a language model to provide responses to queries in natural language. What if your inbox could understand everyday language? You could ask, "When’s my next meeting?" or "Show me the latest sales report," and get the right answer right away. Our idea turns email search into a happier, easier experience. With embeddings, vector databases, and language models, we’re not just making search better — we’re changing how people deal with emails, helping them save time and work more efficiently. ### What it does Scope streamlines email search by making it fast and intuitive. After logging in, Scope retrieves, preprocesses, and cleans your emails. It then embeds these emails as high-dimensional vectors in ChromaDB, a vector database designed for efficient semantic search. When you input a natural language query, Scope processes it using a language model to understand intent rather than just keywords. It semantically searches the vector database and returns the most relevant emails alongside a concise, natural language response using Groq's super-speed inference times. We even multi-modally embed images contained in emails for a heightened understanding. Whether you're looking for a specific message or general information, Scope provides quick, accurate results—without manual filtering. Engage with your email like you would with a human. It just makes more sense. ### How we built it We built Scope using Electron to support both cross-platform desktop apps and a website, ensuring accessibility across devices. The frontend is crafted with JavaScript, Electron, React, and designed in Figma for a clean, intuitive UI. For the backend, we chose Python, Flask, and Groq. Groq provides a ~40x speed improvement in inference over OpenAI, allowing for faster query processing. We used ChromaDB for its built-in, customizable embeddings and efficient vector search, enabling us to handle semantic queries seamlessly. Our v0 started with 100 synthetically generated emails, and the final product includes Google login integration, allowing users to sign in with their Gmail and search their inbox with natural language. ### Challenges we ran into One of the key challenges we faced was figuring out the best way to preprocess emails. Emails are complex, often containing HTML, tags, images, and various types of data. We needed a way to extract relevant content without pulling in unnecessary noise. Additionally, since emails come in many different formats, we required a flexible, non-discriminative preprocessing approach that could handle any structure effectively. Integrating with Google was another hurdle, especially since it was our first time using Google Cloud Platform. We had to dive deep into understanding access tokens and credentials to obtain READ privileges for emails. This also involved building communication routes between the frontend and backend to allow users to log in with their Gmail accounts and stay logged in seamlessly. Additionally, the GMail API rate-limits us, forcing us only to index 500 emails at a time. Lastly, choosing the most effective embedding model was quite challenging. We experimented with several, including sentence transformers, word transformers, multimodal models, and models from OpenAI and Hugging Face. Through rigorous testing and quantitative evaluation, we were able to identify the most effective model for our needs, ensuring accurate and efficient email search results. ### Accomplishments we're proud of FAST email search and LLM inference: We achieved rapid email retrieval by optimizing our embedding model and leveraging Groq's 40x faster inference, significantly improving response times over traditional methods. FAST email search and LLM inference: We achieved rapid email retrieval by optimizing our embedding model and leveraging Groq's 40x faster inference, significantly improving response times over traditional methods. Built-in integration with Gmail: Implementing seamless Google login with full email READ access was a major win, allowing users to authenticate easily and use Scope with their Gmail accounts to search emails without friction. Built-in integration with Gmail: Implementing seamless Google login with full email READ access was a major win, allowing users to authenticate easily and use Scope with their Gmail accounts to search emails without friction. Customizing ChromaDB setup: We tailored ChromaDB to fit our specific use case, optimizing its embeddings and vector search for fast and accurate semantic query results, ensuring our users get relevant email matches. Customizing ChromaDB setup: We tailored ChromaDB to fit our specific use case, optimizing its embeddings and vector search for fast and accurate semantic query results, ensuring our users get relevant email matches. Clean, intuitive, beautiful user interface: Our frontend design is sleek and user-friendly, thanks to careful collaboration between Figma and React, making Scope feel simple yet powerful for all users. Clean, intuitive, beautiful user interface: Our frontend design is sleek and user-friendly, thanks to careful collaboration between Figma and React, making Scope feel simple yet powerful for all users. Turning user research into a working product in under 40 hours: Through rapid iteration and focusing on user pain points, we were able to transform our insights into a fully functional solution, bringing real impact in a short time frame. Turning user research into a working product in under 40 hours: Through rapid iteration and focusing on user pain points, we were able to transform our insights into a fully functional solution, bringing real impact in a short time frame. Scoping hard and finishing v0 early: By setting clear, realistic goals, we delivered our v0 ahead of schedule, which gave us valuable extra time to refine and improve the product before final submission. Scoping hard and finishing v0 early: By setting clear, realistic goals, we delivered our v0 ahead of schedule, which gave us valuable extra time to refine and improve the product before final submission. ### What we learned Working with new technologies like Groq, ChromaDB, and Google API taught us how to integrate cutting-edge tools and optimize them for our specific needs, significantly enhancing our product's performance. Working with new technologies like Groq, ChromaDB, and Google API taught us how to integrate cutting-edge tools and optimize them for our specific needs, significantly enhancing our product's performance. User research is crucial for identifying real problems. By speaking with users, we were able to hone in on their actual pain points and design a solution that directly addressed them. User research is crucial for identifying real problems. By speaking with users, we were able to hone in on their actual pain points and design a solution that directly addressed them. We gained a deep understanding of high-dimensional vector spaces and how they can be leveraged to retrieve semantic meaning, improving the accuracy of our email search. We gained a deep understanding of high-dimensional vector spaces and how they can be leveraged to retrieve semantic meaning, improving the accuracy of our email search. Fast-paced development is both rewarding and risky. By keeping our development goals top-of-mind, we were able to reduce bugs, iterate quickly, and still deliver a polished product under tight time constraints. Fast-paced development is both rewarding and risky. By keeping our development goals top-of-mind, we were able to reduce bugs, iterate quickly, and still deliver a polished product under tight time constraints. ### What's next for Scope Integrate with email server protocols instead of relying solely on the Google API, allowing Scope to support a wider range of email services and offer greater flexibility. Integrate with email server protocols instead of relying solely on the Google API, allowing Scope to support a wider range of email services and offer greater flexibility. Build out mail extensions for a seamless experience, enabling users to access Scope through a global keyboard shortcut, similar to Spotlight search, for even faster email querying. Build out mail extensions for a seamless experience, enabling users to access Scope through a global keyboard shortcut, similar to Spotlight search, for even faster email querying. We're excited to take Scope to the next level! Over the coming weeks, we'll be refining the product, setting up meetings with potential stakeholders, and working to identify a strong product-market fit. We're excited to take Scope to the next level! Over the coming weeks, we'll be refining the product, setting up meetings with potential stakeholders, and working to identify a strong product-market fit. Boom. ## README (from the GitHub repository) # Scope Search 📧🔭👁️ > Kunal Srivastava, Ayaan Rahim, Connor Chan, Andrew Shen --- ## Inspiration 🌟 > Email search sucks... A lot... We've all had negative experiences with email. To be specific, **it's quite hard to find specific emails**. Email search is a cross-platform pain point, regardless of mobile or web: the core problem stems from the fact that most email search systems are reactive, simplistic, and keyword-based. They rely on older indexing and retrieval methods rather than understanding the true intent behind a query. To figure out what we could do to make some progress, we talked to over 50 hackers this weekend before we got to building. We learned users typically don’t remember exact email keywords but instead rely on contextual clues—like who the email was from, the approximate date, or a related attachment. Most email clients force users to manually filter by these parameters, which introduces friction. We hypothesized that creating a retrieval system that had semantic understanding would likely provide a much better UX. Additionally, we could use a language model to provide responses to queries in natural language. What if your inbox could understand everyday language? You could ask, "When’s my next meeting?" or "Show me the latest sales report," and get the right answer right away. Our idea turns email search into a happier, easier experience. With embeddings, vector databases, and language models, we’re not just making search better — we’re changing how people deal with emails, helping them **save time and work more efficiently.** ## What it does 🕵️📧⁉️ Scope streamlines email search by making it fast and intuitive. After logging in, Scope retrieves, preprocesses, and cleans your emails. It then embeds these emails as high-dimensional vectors in ChromaDB, a vector database designed for efficient semantic search. When you input a natural language query, Scope processes it using a language model to understand intent rather than just keywords. It semantically searches the vector database and returns the most relevant emails alongside a concise, natural language response using Groq's super-speed inference times. We even multi-modally embed images contained in emails for a heightened understanding. Whether you're looking for a specific message or general information, Scope provides quick, accurate results—without the manual filtering. Engage with your email like you would with a human. *It just makes more sense*. ## How we built it 💻💖 We built Scope using Electron to support both cross-platform desktop apps and a website, ensuring accessibility across devices. The frontend is crafted with JavaScript, Electron, React, and designed in Figma for a clean, intuitive UI. For the backend, we chose Python, Flask, and Groq. Groq provides a ~40x speed improvement in inference over OpenAI, allowing for faster query processing. We used ChromaDB for its built-in, customizable embeddings and efficient vector search, enabling us to handle semantic queries seamlessly. Our v0 started with 100 synthetically generated emails, and the final product includes Google login integration, allowing users to sign in with their Gmail and search their inbox with natural language. ## Challenges we ran into 🏔️🧗 One of the key challenges we faced was figuring out the best way to preprocess emails. Emails are complex, often containing HTML, tags, images, and various types of data. We needed a way to extract relevant content without pulling in unnecessary noise. Additionally, since emails come in many different formats, we required a flexible, non-discriminative preprocessing approach that could handle any structure effectively. Integrating with Google was another hurdle, especially since it was our first time using Google Cloud Platform. We had to dive deep into understanding access tokens and credentials to obtain READ privileges for emails. This also involved building communication routes between the frontend and backend to allow users to log in with their Gmail accounts and stay logged in seamlessly. Additionally, the GMail API rate-limits us, forcing us to only index 500 emails at a time Lastly, choosing the most effective embedding model was quite challenging. We experimented with several, including sentence transformers, word transformers, multimodal models, and models from OpenAI and Hugging Face. Through rigorous testing and quantitative evaluation, we were able to identify the most effective model for our needs, ensuring accurate and efficient email search results. ## ⭐Accomplishments⭐ that we're proud of - **FAST email search and LLM inference**: We achieved rapid email retrieval by optimizing our embedding model and leveraging Groq's 40x faster inference, significantly improving response times over traditional methods. - **Built-in integration with Gmail**: Implementing seamless Google login with full email READ access was a major win, allowing users to authenticate easily and use Scope with their Gmail accounts to search emails without friction. - **Customizing ChromaDB setup**: We tailored ChromaDB to fit our specific use case, optimizing its embeddings and vector search for fast and accurate semantic query results, ensuring our users get relevant email matches. - **Clean, intuitive, beautiful user interface**: Our frontend design is sleek and user-friendly, thanks to careful collaboration between Figma and React, making Scope feel simple yet powerful for all users. - **Turning user research into a working product in under 40 hours**: Through rapid iteration and focusing on user pain points, we were able to transform our insights into a fully functional solution, bringing real impact in a short time frame. - **Scoping hard and finishing v0 early**: By setting clear, realistic goals, we delivered our v0 ahead of schedule, which gave us valuable extra time to refine and improve the product before final submission. ## What we learned 📝🎓 - Working with new technologies like Groq, ChromaDB, and Google API taught us how to integrate cutting-edge tools and optimize them for our specific needs, significantly enhancing our product's performance. - User research is crucial for identifying real problems. By speaking with users, we were able to hone in on their actual pain points and design a solution that directly addressed them. - We gained a deep understanding of high-dimensional vector spaces and how they can be leveraged to retrieve semantic meaning, improving the accuracy of our email search. - Fast-paced development is both rewarding and risky. By keeping our development goals top-of-mind, we were able to reduce bugs, iterate quickly, and still deliver a polished product under tight time constraints. ## What's next for Scope 🌌👆 - Integrate with email server protocols instead of relying solely on the Google API, allowing Scope to support a wider range of email services and offer greater flexibility. - Build out mail extensions for a seamless experience, enabling users to access Scope through a global keyboard shortcut, similar to Spotlight search, for even faster email querying. - We're excited to take Scope to the next level! Over the coming weeks, we'll be refining the product, setting up meetings with potential stakeholders, and working to identify a strong product-market fit. ## Detected evidence (automated analysis) Indexed codebase: 24 recognized source files, 49 KB. - CSS (language) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Flask (technology) — claimed on Devpost, not found in the code - OpenAI (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (33 of 33) ``` .gitignore assets/icon.icns backend/data/emails_dataset.json backend/data/synth_data.json backend/gmail.py backend/main.py backend/README.md frontend/.gitignore frontend/electron.js frontend/package.json frontend/public/index.html frontend/public/manifest.json frontend/public/robots.txt frontend/README.md frontend/search.html frontend/search.js frontend/src/api/queryEmails.js frontend/src/api/queryLanguageModel.js frontend/src/App.css frontend/src/App.js frontend/src/App.test.js frontend/src/components/Greeting.js frontend/src/components/ResultEmail.js frontend/src/components/SearchBar.js frontend/src/components/TypewriterTypography.js frontend/src/EmailModal.js frontend/src/index.css frontend/src/index.js frontend/src/layout.css frontend/src/MainPage.js frontend/src/setupTests.js README.md requirements.txt ``` ### Dependencies - frontend/package.json: @emotion/react@^11.13.3, @emotion/styled@^11.13.0, @fontsource/roboto@^5.1.0, @mui/icons-material@^6.1.4, @mui/material@^6.1.4, @react-oauth/google@^0.12.1, @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^13.5.0, concurrently@^9.0.1, cross-env@^7.0.3, electron@^33.0.1, electron-builder@^25.1.8, electron-is-dev@^3.0.1, react@^18.3.1, react-dom@^18.3.1, react-scripts@5.0.1, wait-on@^8.0.1, web-vitals@^2.1.4 ### Recent commits (newest first) - speed up typography - Update README.md - send date from backend and render in frontend - Update README.md - remove chroma log - gitignore chroma log - Update README.md - Update README.md - Update README.md - Update README.md - hook up fe finally - fixed no body in email results - fix none in metatdatas - hotfix from - fixed attachment - Style login button and icon - merge - clean up - HOOKED UP INTEGRATIONS - fix ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` ��annotated-types==0.7.0 anyio==4.6.2.post1 asgiref==3.8.1 backoff==2.2.1 bcrypt==4.2.0 blinker==1.8.2 build==1.2.2.post1 cachetools==5.5.0 certifi==2024.8.30 charset-normalizer==3.4.0 chroma-hnswlib==0.7.6 chromadb==0.5.15 click==8.1.7 colorama==0.4.6 coloredlogs==15.0.1 Deprecated==1.2.14 distro==1.9.0 durationpy==0.9 fastapi==0.115.2 filelock==3.16.1 Flask==3.0.3 Flask-Cors==5.0.0 flatbuffers==24.3.25 fsspec==2024.9.0 ftfy==6.3.0 google-api-core==2.21.0 google-api-python-client==2.149.0 google-auth==2.35.0 google-auth-httplib2==0.2.0 google-auth-oauthlib==1.2.1 googleapis-common-protos==1.65.0 groq==0.11.0 grpcio==1.67.0 h11==0.14.0 httpcore==1.0.6 httplib2==0.22.0 httptools==0.6.4 httpx==0.27.2 huggingface-hub==0.26.0 humanfriendly==10.0 idna==3.10 importlib_metadata==8.4.0 importlib_resources==6.4.5 itsdangerous==2.2.0 Jinja2==3.1.4 joblib==1.4.2 kubernetes==31.0.0 markdown-it-py==3.0.0 MarkupSafe==3.0.2 mdurl==0.1.2 mmh3==5.0.1 monotonic==1.6 mpmath==1.3.0 networkx==3.4.1 numpy==2.1.2 oauthlib==3.2.2 onnxruntime==1.19.2 open_clip_torch==2.27.1 opentelemetry-api==1.27.0 opentelemetry-exporter-otlp-proto-common==1.27.0 opentelemetry-exporter-otlp-proto-grpc==1.27.0 opentelemetry-instrumentation==0.48b0 opentelemetry-instrumentation-asgi==0.48b0 opentelemetry-instrumentation-fastapi==0.48b0 opentelemetry-proto==1.27.0 opentelemetry-sdk==1.27.0 opentelemetry-semantic-conventions==0.48b0 opentelemetry-util-http==0.48b0 orjson==3.10.7 overrides==7.7.0 packaging==24.1 pillow==11.0.0 posthog==3.7.0 proto-plus==1.24.0 protobuf==4.25.5 pyasn1==0.6.1 pyasn1_modules==0.4.1 pydantic==2.9.2 pydantic_core==2.23.4 Pygments==2.18.0 pyparsing==3.2.0 PyPika==0.48.9 pyproject_hooks==1.2.0 pyreadline3==3.5.4 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 PyYAML==6.0.2 regex==2024.9.11 requests==2.32.3 requests-oauthlib==2.0.0 rich==13.9.2 rsa==4.9 safetensors==0.4.5 scikit-learn==1.5.2 scipy==1.14.1 sentence-transformers==3.2.0 shellingham==1.5.4 six==1.16.0 sniffio==1.3.1 starlette==0.40.0 sympy==1.13.1 tenacity==9.0.0 threadpoolctl==3.5.0 timm==1.0.11 tokenizers==0.20.1 torch==2.5.0 torchvision==0.20.0 tqdm==4.66.5 transformers==4.45.2 typer==0.12.5 typing_extensions==4.12.2 uritemplate==4.1.1 urllib3==2.2.3 uvicorn==0.32.0 watchfiles==0.24.0 wcwidth==0.2.13 websocket-client==1.8.0 websockets==13.1 Werkzeug==3.0.4 wrapt==1.16.0 zipp==3.20.2 ``` ### frontend/package.json ``` { "name": "frontend", "version": "0.1.0", "private": true, "type": "module", "main": "electron.js", "homepage": "./", "dependencies": { "@emotion/react": "^11.13.3", "@emotion/styled": "^11.13.0", "@fontsource/roboto": "^5.1.0", "@mui/icons-material": "^6.1.4", "@mui/material": "^6.1.4", "@react-oauth/google": "^0.12.1", "@testing-library/jest-dom": "^5.17.0", "@testing-library/react": "^13.4.0", "@testing-library/user-event": "^13.5.0", "electron-is-dev": "^3.0.1", "react": "^18.3.1", "react-dom": "^18.3.1", "react-scripts": "5.0.1", "web-vitals": "^2.1.4" }, "scripts": { "start": "react-scripts start", "build": "react-scripts build", "test": "react-scripts test", "eject": "react-scripts eject", "electron:dev": "concurrently \"cross-env BROWSER=none npm start\" \"wait-on http://localhost:3000 && electron .\"", "electron:build": "npm run build && electron-builder -m" }, "build": { "extends": null, "appId": "com.example.your-app-name", "files": [ "build/**/*", "node_modules/**/*", "electron.js" ], "directories": { "buildResources": "assets" }, "mac": { "icon": "resources/icon.icns" } }, "eslintConfig": { "extends": [ "react-app", "react-app/jest" ] }, "browserslist": { "production": [ ">0.2%", "not dead", "not op_mini all" ], "development": [ "last 1 chrome version", "last 1 firefox version", "last 1 safari version" ] }, "devDependencies": { "concurrently": "^9.0.1", "cross-env": "^7.0.3", "electron": "^33.0.1", "electron-builder": "^25.1.8", "wait-on": "^8.0.1" } } ``` ### backend/main.py ```python from flask import Flask, redirect, request, url_for from flask_cors import CORS, cross_origin import chromadb from chromadb.utils.embedding_functions import SentenceTransformerEmbeddingFunction, OpenCLIPEmbeddingFunction from chromadb.utils.data_loaders import ImageLoader from groq import Groq import os from google_auth_oauthlib.flow import InstalledAppFlow import logging import json import re import configparser from gmail import GmailAPI, download_images logging.basicConfig(level=logging.INFO) CONFIG = configparser.ConfigParser() CONFIG.read("TOKENS.ini") GROQ_API_KEY = CONFIG["SECRETS"]["GROQ_API_KEY"] # flask --app backend/main --debug run app = Flask(__name__) CORS(app, resources={r"/*": {"origins": "http://localhost:3000"}}) groq_client = Groq( api_key=GROQ_API_KEY, ) gmail = GmailAPI() os.environ["OAUTHLIB_INSECURE_TRANSPORT"] = '1' embedder = SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2") multimodal_embedder = OpenCLIPEmbeddingFunction() data_loader = ImageLoader() chromadb_client = chromadb.Client() email_collection = chromadb_client.create_collection( "email_collection", metadata={"hnsw:space": "l2"}, embedding_function=embedder, ) # collection = chromadb_client.create_collection( # "emails", # metadata={"hnsw:space": "cosine"}, # embedding_function=embedder, # ) # multimodal_collection = chromadb_client.create_collection( # "emails_and_attachments", # metadata={"hnsw:space": "cosine"}, # embedding_function=multimodal_embedder, # data_loader=data_loader # ) # multimodal_collection.add( # ids=["cat", "dog"], # uris=download_images( # ["https://www.cdc.gov/healthy-pets/media/images/2024/04/Cat-on-couch.jpg", # "https://hips.hearstapps.com/hmg-prod/images/dog-puppy-on-garden-royalty-free-image-1586966191.jpg"] # ) # ) # img_result = multimodal_collection.query( # query_uris=download_images(["https://d6vhjw8wa28ve.cloudfront.net/wp-content/uploads/2021/03/kitten-sitting-on-floor-031621.jpg"]) # A list of strings representing URIs to data # ) # print(img_result) # txt_result = multimodal_collection.query( # query_texts=["Kitten"] # ) # print(txt_result) # txt_result = multimodal_collection.query( # query_texts=["Puppy"] # ) # print(txt_result) def synth_initialize(): with open("./backend/data/synth_data.json") as fobj: data = json.load(fobj) for id, d in enumerate(data): merged_data = d["subject"]+" "+re.sub(r"[\n]", " ", d["body"]) email_collection.upsert( documents=[merged_data], # we embed for you, or bring your own metadatas=[{ "from": d["from"], "to": d["to"], "subject": d["subject"], "body": d["body"], }], # filter on arbitrary metadata! ids=[str(id)], # must be unique for each doc ) def initialize_live_data(): try: print("STARTS") emails, image_urls = gmail.get_emails(count=100) print(len(emails), len(image_urls)) print("ENDS") default = lambda x: "" if x is None else x for id, (email, image_urls) in enumerate(zip(emails, image_urls)): try: merged_data = email.subject + " " + re.sub(r"[\n]", " ", default(email.body)) email_collection.upsert( documents=[merged_data], metadatas=[{ "from": default(email.from_), "to": default(email.to), "subject": default(email.subject), "body": default(email.body), "date": default(email.date), }], ids=[str(id)] ) # print("ATTACHMENTS:", len(email.attachments)) for iid, attachment in enumerate(email.attachments): merged_data = email.subject + " " + re.sub(r"[\n]", " ", default(attachment.body)) email_collection.upsert( documents=[merged_data], metadatas=[{ "from": default(email.from_), "to": default(email.to), "subject": default(email.subject), "body": default(attachment.body), "content_type": default(attachment.content_type), "email_ref": str(id), "date": default(email.date), }], ids=[str(id) + "A" + str(iid)] ) for image_url in image_urls: try: uris = download_images([image_url]) if len(uris) > 0: email_collection.add( uris=uris, metadatas=[{"email": str(id)}], ids=[str(id) + "I" + str(iid) for iid in range(len(uris))] ) except Exception as e: logging.error(f"Error downloading image: {e}") except Exception as e: logging.error(f"Error processing email {id}: {e}") print("READY TO QUERY") return True except Exception as e: logging.error(f"Critical error in initialize_live_data: {e}") return False @app.get('/result') def return_result(): query = request.args.get("query") results = email_collection.query( query_texts=[query], n_results=5, ) # remove duplicate email if the attachment is present with the associated email # remove any attachment with same email_ref below # NOTE: There might be multiple attachment [truncated — 2927 more characters] ``` ### frontend/src/App.js ```javascript import './layout.css'; import MainPage from "./MainPage.js"; function App() { return (
); } export default App; ``` ### frontend/src/index.js ```javascript import React from 'react'; import ReactDOM from 'react-dom/client'; import './index.css'; import App from './App.js'; // import reportWebVitals from './reportWebVitals.js'; const root = ReactDOM.createRoot(document.getElementById('root')); root.render( ); // If you want to start measuring performance in your app, pass a function // to log results (for example: reportWebVitals(console.log)) // or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals // reportWebVitals(); ``` ### frontend/search.js ```javascript const searchInput = document.getElementById('search-input'); searchInput.addEventListener('keyup', (event) => { if (event.key === 'Enter') { // Perform search action here // console.log('Searching for:', searchInput.value); // You can send this value to the main process to perform actions } else if (event.key === 'Escape') { window.close(); } }); // Clear the input when the window is shown window.addEventListener('focus', () => { searchInput.value = ''; }); ``` ### frontend/search.html ```html Search ``` ### frontend/electron.js ```javascript import { app, BrowserWindow, globalShortcut, screen } from 'electron'; import { join } from 'path'; import isDev from 'electron-is-dev'; function createWindow() { const win = new BrowserWindow({ fullscreen: true, webPreferences: { nodeIntegration: true, contextIsolation: false, }, }); win.loadURL( isDev ? 'http://localhost:3000' : `file://${join(__dirname, '../build/index.html')}` ); } let searchWindow = null; function createSearchWindow() { const { width, height } = screen.getPrimaryDisplay().workAreaSize; searchWindow = new BrowserWindow({ width: 600, height: 60, frame: false, transparent: true, show: false, alwaysOnTop: true, webPreferences: { nodeIntegration: true, contextIsolation: false } }); searchWindow.loadFile('search.html'); searchWindow.setPosition(Math.round(width/2 - 300), Math.round(height/2 - 30)); searchWindow.on('blur', () => { searchWindow.hide(); }); } app.whenReady().then(() => { createWindow(); createSearchWindow(); globalShortcut.register('Command+Shift+Space', () => { if (!searchWindow || searchWindow.isDestroyed()) { createSearchWindow(); } if (searchWindow.isVisible()) { searchWindow.hide(); // console.log("Search window hidden"); } else { searchWindow.show(); // console.log("Search window shown"); } }); }); app.on('window-all-closed', () => { if (process.platform !== 'darwin') { app.quit(); } }); app.on('activate', () => { if (BrowserWindow.getAllWindows().length === 0) { createWindow(); } }); app.on('will-quit', () => { globalShortcut.unregisterAll(); }); ``` ### backend/gmail.py ```python import os.path from pathlib import Path from flask import url_for import requests from google.auth.transport.requests import Request from google.oauth2.credentials import Credentials from google.oauth2 import id_token from google_auth_oauthlib.flow import InstalledAppFlow from googleapiclient.discovery import build from googleapiclient.errors import HttpError import base64 import re from dataclasses import dataclass @dataclass class AttachmentData: body: str content_type: str @dataclass class EmailData: date: str from_: str to: str subject: str body: str content_type: str message_id: str attachments: list[AttachmentData] TEMP_IMG_PATH = Path("./backend/data/images/") def header_value(header: dict, header_name: str): try: ind = [e["name"] for e in header].index(header_name) except ValueError: return None else: return header[ind]["value"] # (processed content, image links) def process_html(html: str) -> tuple[str, list[str]]: if html is None: return None html = re.sub(r"[\r\n\t]", " ", html) image_urls = re.findall(r'https?://[^\s]+)".*/?>', html) html = re.sub(r".*?", " ", html) html = re.sub(r"<.*?>", r" ", html) html = html.replace(" ", " ") image_urls = [image_url for image_url in image_urls if image_url.split( ".")[-1].lower() in ["png", "jpg", "gif", "tif", "bmp", "tiff"]] return html, image_urls def preprocess_emails(txt: str) -> str: txt = re.sub(r"?", r"", txt) txt = re.sub(r"&#?....;", r"", txt) return txt # html = re.sub(r"http(?s:.*?)$", r"", html) def decode_body(body: dict): if body.get("data", None) is not None: value = base64.urlsafe_b64decode(body["data"]).decode("utf-8") value = re.sub(r"[\r\n]", " ", value) return value def decode_and_save_attachments(body: dict, filename: str) -> str: uri = TEMP_IMG_PATH / filename if body.get("data", None) is not None: img = base64.urlsafe_b64decode(body["data"]) with open(uri, "wb") as fobj: fobj.write(img) return str(uri) def download_images(urls: list[str]) -> list[str]: result = [] for url in urls: try: img_data = requests.get(url).content local_path = TEMP_IMG_PATH / Path(url).parts[-1] with open(local_path, "wb") as fobj: fobj.write(img_data) result.append(str(local_path)) except: pass return result class GmailAPI: # If modifying these scopes, delete the file token.json. SCOPES = ["https://www.googleapis.com/auth/gmail.readonly"] def __init__(self): self.auth_state = None self.creds = None self.service = None def login(self) -> str: # print("LOGIN?") auth_url = "" # The file token.json stores the user's access and refresh tokens, and is # created automatically when the authorization flow completes for the first # time. if os.path.exists("token.json"): self.creds = Credentials.from_authorized_user_file( "token.json", self.SCOPES) self.instantiate() # If there are no (valid) credentials available, let the user log in. if not self.creds or not self.creds.valid: if self.creds and self.creds.expired and self.creds.refresh_token: self.creds.refresh(Request()) self.instantiate() else: flow = InstalledAppFlow.from_client_secrets_file( "credentials.json", GmailAPI.SCOPES ) flow.redirect_uri = url_for('callback', _external=True) auth_url, self.auth_state = flow.authorization_url( access_type='offline', prompt='select_account' ) print(self.auth_state) return auth_url def login_callback(self, auth_resp): flow = InstalledAppFlow.from_client_secrets_file( "credentials.json", scopes=GmailAPI.SCOPES, state=self.auth_state) flow.redirect_uri = url_for('callback', _external=True) flow.fetch_token(authorization_response=auth_resp) self.creds = flow.credentials self.instantiate() def instantiate(self): try: # Call the Gmail API with open("token.json", "w") as token: token.write(self.creds.to_json()) self.service = build("gmail", "v1", credentials=self.creds) except HttpError as error: print(f"Error in loading Gmail API service: {error}") def get_email(self): if self.creds and self.creds.valid: try: profile = self.service.users().getProfile(userId='me').execute() email_address = profile.get('emailAddress') return email_address except HttpError as error: print(f"An error occurred: {error}") return None else: print("No valid credentials found.") return None def get_emails(self, count=100) -> tuple[list[EmailData], list[str]]: current_user = self.service.users().messages().list( userId="me", maxResults=count).execute() messages = current_user.get("messages", []) image_uris = [] data = [] for message in messages: image_uris.append([]) payload = self.service.users().messages().get( userId="me", id=message["id"], format="full").execute()["payload"] headers = payload["headers"] body = payload["body"] print(headers) data.append( EmailData( date=header_value(headers, "Date"), from_=header_value(headers, "From"), [truncated — 3225 more characters] ``` ### frontend/src/setupTests.js ```javascript // jest-dom adds custom jest matchers for asserting on DOM nodes. // allows you to do things like: // expect(element).toHaveTextContent(/react/i) // learn more: https://github.com/testing-library/jest-dom import '@testing-library/jest-dom'; ``` [13 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]