# Project export: MailMorph This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 11.0 - Tagline: Transforms how emails are organized and handled. - Devpost: https://devpost.com/software/mailmorph - GitHub: https://github.com/code4hari/Email - Team: 1 GitHub contributor(s) — code4hari (19 commits) ## Devpost submission (written by the team) ### Inspiration Two days ago, as we were standing in the long line to check-in to Calhacks, we were competing to see who had the most emails in their inbox. Surprisingly enough someone had more than 150k unread emails! As fate would have it when it was our turn to check-in we were franticly searching through our inboxes for our tickets. That's when we realized there are so many ways e-mail can be improved on to make life easier, one major one being organization of emails. But, no one in their right mind is going to manually categorize and organize more than 150k e-mails. That's where MailMorph comes in! ### What it does MailMorph is a smart e-mail tool which employs the use of AI and AI-Agents to make life with e-mail easier in five main areas. First, MailMorph can automatically create user specified categories and organize the e-mail in their inbox into the categories. Second, MailMorph can automatically identify mails sitting in the sent folder of you inbox and recommend you to follow up with the other person, and automatically create a draft email for you. Third, MailMorph can automatically identify mails coming into your inbox with important events or dates and automatically add it to your calendar. Fourth, MailMorph can create a daily digest of your emails of that day and summarize it all for you, as well as highlighting the most important details. Last but not least, MailMorph can identify emails with pdf files which need to be filled, and auto fill them for you if there is sufficient information in the email itself. ### How we built it For MailMorph, we combined the capabilities of Gemini AI and AutoGen AI Agents to build a powerful email and PDF automation solution. To integrate with Gmail and Google Calendar, we secured access to their APIs, set up user authentication, and configured credentials to extract the necessary data. Depending on the use case, we retrieved key information: for categorizing emails, we identified relevant details; for adding events to calendars, we extracted dates and times; and for summaries, we collected the full content. We ensured PII redaction before sending any data to the LLMs and used prompt engineering with Gemini AI to generate accurate outputs. On the backend, we structured individual APIs to handle different functionalities, allowing the frontend—built with React, Node.js, and Swift—to communicate seamlessly with the Python backend through API calls. For PDF automation, we leveraged AutoGen AI to process both fillable and scanned PDFs. With fillable PDFs, the agent identified fields and matched them with data from a JSON dataset before generating code to fill the PDF. For scanned PDFs, we incorporated OCR to detect fields and injected fillable inputs at the appropriate locations, following the same logic as the fillable PDFs. This architecture enabled us to efficiently manage emails, automate PDFs, and integrate everything into a smooth, user-friendly interface. ### Challenges we ran into One of the biggest challenges was setting up access to the Gmail API and Google Calendar API, as it involved navigating OAuth authentication and configuring user credentials securely. Another hurdle was ensuring PII redaction before sending any sensitive information to the LLMs, which required careful preprocessing to avoid leaking personal data. Implementing OCR for scanned PDFs was also complex, especially when accurately detecting fields and injecting fillable elements into the correct locations. Additionally, integrating multiple technologies—React, Node.js, Swift, and Python—into a seamless frontend-backend system took considerable effort to ensure smooth communication between all components. ### Accomplishments we're proud of We are proud of the fact that we got most of our features to perform in less than one second. This is huge for users with very large inboxes since they won't have to wait copious amounts of time. Also, it was our first time integrating flask API endpoints with node.js front end so figuring out how the endpoints worked and integrating everything and seeing it all come together was definitely something we were proud about, ### What we learned Throughout the development of MailMorph, we learned the importance of efficient API integration and data security when working with sensitive user information. We also gained hands-on experience in prompt engineering for LLMs to optimize outputs based on different tasks. Another key takeaway was learning how to leverage AI agents to automate complex tasks like processing PDFs and identifying contextual information in emails. Furthermore, working with multiple programming languages and frameworks helped us improve our collaboration and technical integration skills. ### What's next For now MailMorph is limit to Gmail, so in the future we look forward to implementing in outlook, yahoo, and other email services. Also, we look forward to adding more functionalities such as auto flight check-in, better spam detection, voice activated email, etc. Also, we look forward to implementing a mobile application to make an even more convenient way to organize e-mail. ## README (from the GitHub repository) # Email ## Detected evidence (automated analysis) Indexed codebase: 4 recognized source files, 37 KB. - JavaScript (language) — detected in the code - Python (language) — detected in the code - CSS (language) — claimed on Devpost, not found in the code - Flask (technology) — claimed on Devpost, not found in the code - Google Gemini (technology) — claimed on Devpost, not found in the code - HTML (language) — claimed on Devpost, not found in the code - Node.js (technology) — claimed on Devpost, not found in the code - React (technology) — claimed on Devpost, not found in the code - Tailwind CSS (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (6 of 6) ``` .gitignore app.py LandingPage.js LoginPage.js README.md requirements.txt ``` ### Dependencies No dependency index available. ### Recent commits (newest first) - Add files via upload - Add files via upload - Add files via upload - Add files via upload - Delete token.pickle - Delete sample_email.json - Delete linear_updates.py - Delete email_summary.py - Delete email_response.py - Delete email_catagory.py - Delete api_sum.py - Delete api_res.py - Delete api_liup.py - Delete api_cat.py - Update api_sum.py - Update api_res.py - Update api_liup.py - Update api_cat.py - Add files via upload - Initial commit ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` ��annotated-types==0.7.0 blinker==1.8.2 blis==1.0.1 cachetools==5.5.0 catalogue==2.0.10 certifi==2024.8.30 chardet==5.2.0 charset-normalizer==3.4.0 click==8.1.7 cloudpathlib==0.20.0 colorama==0.4.6 confection==0.1.5 cymem==2.0.8 dateparser==1.2.0 en_core_web_sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.8.0/en_core_web_sm-3.8.0-py3-none-any.whl#sha256=1932429db727d4bff3deed6b34cfc05df17794f4a52eeb26cf8928f7c1a0fb85 Faker==30.6.0 FITX==1.0.1 Flask==3.0.3 Flask-Cors==5.0.0 google-ai-generativelanguage==0.6.10 google-api-core==2.21.0 google-api-python-client==2.149.0 google-auth==2.35.0 google-auth-httplib2==0.2.0 google-auth-oauthlib==1.2.1 google-generativeai==0.8.3 googleapis-common-protos==1.65.0 greenlet==3.0.3 grpcio==1.67.0 grpcio-status==1.67.0 httplib2==0.22.0 idna==3.10 itsdangerous==2.2.0 Jinja2==3.1.4 joblib==1.4.2 langcodes==3.4.1 language_data==1.2.0 marisa-trie==1.2.1 markdown-it-py==3.0.0 MarkupSafe==3.0.2 mdurl==0.1.2 murmurhash==1.0.10 nltk==3.9.1 numpy==2.0.2 oauthlib==3.2.2 packaging==24.1 pdf2image==1.17.0 pdfrw==0.4 phonenumbers==8.13.47 pillow==11.0.0 playwright==1.47.0 preshed==3.0.9 proto-plus==1.24.0 protobuf==5.28.2 pyasn1==0.6.1 pyasn1_modules==0.4.1 pydantic==2.9.2 pydantic_core==2.23.4 pyee==12.0.0 Pygments==2.18.0 pyparsing==3.2.0 PyPDF2==3.0.1 python-dateutil==2.9.0.post0 python-dotenv==1.0.1 python-stdnum==1.20 pytz==2024.2 regex==2024.9.11 reportlab==4.2.5 requests==2.32.3 requests-oauthlib==2.0.0 rich==13.9.2 rsa==4.9 scikit-learn==1.5.2 scipy==1.14.1 scrubadub==2.0.1 setuptools==75.2.0 shellingham==1.5.4 six==1.16.0 smart-open==7.0.5 spacy-legacy==3.0.12 spacy-loggers==1.0.5 srsly==2.4.8 textblob==0.15.3 thinc==8.3.2 threadpoolctl==3.5.0 tqdm==4.66.5 typer==0.12.5 typing_extensions==4.12.2 tzdata==2024.2 tzlocal==5.2 uritemplate==4.1.1 urllib3==2.2.3 wasabi==1.1.3 weasel==0.4.1 Werkzeug==3.0.4 wrapt==1.16.0 ``` ### app.py ```python import os import pickle import time from flask import Flask, jsonify from google_auth_oauthlib.flow import InstalledAppFlow from google.auth.transport.requests import Request from googleapiclient.discovery import build from google.oauth2.credentials import Credentials import scrubadub from dotenv import load_dotenv from datetime import datetime import google.generativeai as genai load_dotenv() app = Flask(__name__) # Google API scope SCOPES = ["https://www.googleapis.com/auth/gmail.modify"] def authenticate_gmail(): creds = None if os.path.exists("token.pickle"): with open("token.pickle", "rb") as token: creds = pickle.load(token) if not creds or not creds.valid: if creds and creds.expired and creds.refresh_token: creds.refresh(Request()) else: flow = InstalledAppFlow.from_client_secrets_file( "credentials.json", SCOPES ) creds = flow.run_local_server(port=0) with open("token.pickle", "wb") as token: pickle.dump(creds, token) return build("gmail", "v1", credentials=creds) def get_todays_emails(service): today = datetime.now().strftime('%Y/%m/%d') query = f'after:{today}' results = service.users().messages().list(userId='me', q=query).execute() messages = results.get('messages', []) return messages import base64 def fetch_email_content(service, email_id): email = service.users().messages().get(userId='me', id=email_id, format='full').execute() payload = email.get('payload', {}) headers = payload.get('headers', []) # Fetch subject subject = next((header['value'] for header in headers if header['name'] == 'Subject'), '') # Extract the body of the email (checking for multipart) body = get_email_body(payload) return {'subject': subject, 'body': body, 'id': email_id} def get_email_body(payload): """Helper function to retrieve the email body content, handling multipart emails.""" if 'parts' in payload: parts = payload['parts'] for part in parts: if part['mimeType'] == 'text/plain': return base64.urlsafe_b64decode(part['body']['data']).decode('utf-8') else: # Single-part email body body_data = payload.get('body', {}).get('data', '') if body_data: return base64.urlsafe_b64decode(body_data).decode('utf-8') return "No plain text content found." def mask_pii(text): scrubber = scrubadub.Scrubber() masked_text = scrubber.clean(text) return masked_text def summarize_email_content(email_texts): genai.configure(api_key=os.environ["GEMINI_API_KEY"]) prompt = ( f"First summarize all the emails recieved today in one paragraph atleast 500 words. Then summarize each individual email recieved today also identifying which email you are summarizing. Finally, print the inportant details from all the emails you think should be highlights, and specify which emails these important details came from:\n" f"---\n" + '\n'.join(email_texts) + "\n---" ) model = genai.GenerativeModel("gemini-1.5-flash") response = model.generate_content(prompt) if response and response.text: return response.text.strip() else: print("Failed to get summary from LLM.") return None def process_todays_emails(service): emails = get_todays_emails(service) email_texts = [] for email_data in emails: email = fetch_email_content(service, email_data['id']) masked_content = mask_pii(email['subject'] + '\n' + email['body']) email_texts.append(masked_content) if email_texts: summary = summarize_email_content(email_texts) if summary: print("Today's email summary:") print(summary) return summary else: print("No important content found.") return "No important content found." else: print("No emails received today.") return "No emails received today." @app.route('/api/summarize_emails', methods=['POST']) def summarize_emails_api(): service = authenticate_gmail() summary = process_todays_emails(service) return jsonify({"message": "Email summarization complete", "summary": summary}), 200 import os import pickle import time from flask import Flask, request, jsonify from google_auth_oauthlib.flow import InstalledAppFlow from google.auth.transport.requests import Request from googleapiclient.discovery import build from google.oauth2.credentials import Credentials import scrubadub from dotenv import load_dotenv load_dotenv() app = Flask(__name__) # Google API scope SCOPES = ["https://www.googleapis.com/auth/gmail.modify"] def authenticate_gmail(): creds = None if os.path.exists("token.pickle"): with open("token.pickle", "rb") as token: creds = pickle.load(token) if not creds or not creds.valid: if creds and creds.expired and creds.refresh_token: creds.refresh(Request()) else: flow = InstalledAppFlow.from_client_secrets_file( "credentials.json", SCOPES ) creds = flow.run_local_server(port=0) with open("token.pickle", "wb") as token: pickle.dump(creds, token) return build("gmail", "v1", credentials=creds) def get_unread_emails(service, categories): query = 'is:unread AND ' + ' AND '.join([f'-label:{category}' for category in categories]) results = service.users().messages().list(userId='me', q=query).execute() messages = results.get('messages', []) return messages def fetch_email_content(service, email_id): email = service.users().messages().get(userId='me', id=email_id, format='full').execut [truncated — 15863 more characters] ``` ### LoginPage.js ```javascript import React, { useState } from 'react'; const LoginPage = () => { const [emailSummary, setEmailSummary] = useState(''); const [categories, setCategories] = useState(''); const [isSummarizing, setIsSummarizing] = useState(false); const [isCategorizing, setIsCategorizing] = useState(false); const [isScheduling, setIsScheduling] = useState(false); const [isRecommending, setIsRecommending] = useState(false); const handleApiError = (error, message) => { console.error(message, error); alert(`${message}. Please try again.`); }; const handleSummarizeEmails = async () => { setIsSummarizing(true); const myHeaders = new Headers(); myHeaders.append('Content-Type', 'application/json'); const categoriesArray = categories.split(',').map((c) => c.trim()); const raw = JSON.stringify({ categories: categoriesArray, }); const requestOptions = { method: 'POST', headers: myHeaders, body: raw, redirect: 'follow', }; try { const response = await fetch('http://127.0.0.1:5000/api/categorize_emails', requestOptions); const result = await response.text(); console.log(result); setEmailSummary(result); } catch (error) { handleApiError(error, 'Error summarizing emails'); } finally { setIsSummarizing(false); } }; const handleCategorizeEmails = async () => { setIsCategorizing(true); const myHeaders = new Headers(); myHeaders.append('Content-Type', 'application/json'); const categoriesArray = categories.split(',').map((c) => c.trim()); const raw = JSON.stringify({ categories: categoriesArray }); const requestOptions = { method: 'POST', headers: myHeaders, body: raw, redirect: 'follow', }; try { const response = await fetch('http://127.0.0.1:5000/api/categorize_emails', requestOptions); const result = await response.text(); console.log(result); alert('Emails categorized successfully!'); } catch (error) { handleApiError(error, 'Error categorizing emails'); } finally { setIsCategorizing(false); } }; const handleScheduleEmails = async () => { setIsScheduling(true); const requestOptions = { method: 'POST', redirect: 'follow', }; try { const response = await fetch('http://172.210.16.114:6060/api/linear_updates', requestOptions); const result = await response.text(); console.log(result); alert('Emails scheduled successfully!'); } catch (error) { handleApiError(error, 'Error scheduling emails'); } finally { setIsScheduling(false); } }; const handleFollowUpRecommendations = async () => { setIsRecommending(true); const requestOptions = { method: 'POST', redirect: 'follow', }; try { const response = await fetch('http://172.210.16.114:6060/api/process_emails', requestOptions); const result = await response.text(); console.log(result); alert('Follow-up recommendations generated successfully!'); } catch (error) { handleApiError(error, 'Error getting follow-up recommendations'); } finally { setIsRecommending(false); } }; return (

Email Management Dashboard

{/* Email Summarization */}
{emailSummary && (

Email Summary

{emailSummary}

)}
{/* Email Categorization */}
setCategories(e.target.value)} className="mb-2 p-2 border rounded w-full" />
{/* Automatic Scheduler */}
{/* Follow-up Recommender */}
{/* Email List */}
{[ { name: 'William Smith', subject: 'Meeting Tomorrow', time: '4 months ago', preview: 'Hi, let\'s have a meeting tomorrow to discuss the project. I\'ve been reviewing the project...' }, { name: 'Alice Smith', subject: 'Re: Project Update', time: '4 months ago', preview: 'Thank you for the project update. It looks great! I\'ve gone through the report, and the...' }, { name: 'Bob Johnson', subject: 'Weekend Plans', time: '5 months ago', preview: 'Hey team, I hope this email finds you well. I wanted to touch base about our plans for the...' } ].map((email, index) => (
{email.name} {email.time}
{email.subject}
{email.preview}
))}
); }; const LandingPage = () => { const iconRef = useRef(null); useEffect(() => { const icons = iconRef.current.children; let index = 0; const interval = setInterval(() => { icons[index].style.opacity = '0'; index = (index + 1) % icons.length; icons[index].style.opacity = '1'; }, 2000); return () => clearInterval(interval); }, []); return (
i love email lol
Log in
{[Heart, Globe, Shield, BarChart, File, Rss].map((Icon, index) => ( ))}

a new way to
love email again.